[Case 02]

Validating Design Effectiveness in Digital Education – A/B Testing with 180 Students

Digital Education / Usability

UX in Sustainability Digital Resource

Validating Educational Tools through A/B Testing

[Project Overview]

This project focused on evaluating a digital educational resource about environmental sustainability. Two alternative versions of the interface were created to assess which design students preferred and found more effective. The goal was to validate its pedagogical effectiveness and optimize usability through an in-depth UX study, combining A/B testing and direct classroom observation with middle school students.

[Problem Statement]

Although the resource addressed a relevant topic, there was no empirical evidence regarding which interface design was more effective for students. The company needed concrete data to guide improvements in the resource. The challenge was to design and conduct research that combined methodological rigor with real classroom dynamics, ensuring students’ preferences and performance shaped the final design.

[Industry]

Digital Education / Usability

[My Role]

UX Researcher, Usability Tester

[Platforms]

Web (PC / Tablet)

[Timeline]

February 2023 – November 2023

[Process]

[01] Research & Personas

Creation of personas based on middle school students.

Review of literature on usability and gamification in digital education.

[01] Research & Personas

Creation of personas based on middle school students.

Review of literature on usability and gamification in digital education.

[01] Research & Personas

Creation of personas based on middle school students.

Review of literature on usability and gamification in digital education.

[02] Experimental Design

Development of two design variations of the resource.

Implementation of A/B testing protocol in two Portuguese schools, involving 9 classes and more than 180 students.

[02] Experimental Design

Development of two design variations of the resource.

Implementation of A/B testing protocol in two Portuguese schools, involving 9 classes and more than 180 students.

[02] Experimental Design

Development of two design variations of the resource.

Implementation of A/B testing protocol in two Portuguese schools, involving 9 classes and more than 180 students.

[03] Usability Testing

Direct observation in classroom context.

Collection of quantitative and qualitative data (student preferences, engagement, and performance).

[03] Usability Testing

Direct observation in classroom context.

Collection of quantitative and qualitative data (student preferences, engagement, and performance).

[03] Usability Testing

Direct observation in classroom context.

Collection of quantitative and qualitative data (student preferences, engagement, and performance).

[04] Analysis & Recommendations

Statistical comparison of results to measure design effectiveness.

Formulation of UX recommendations for the development team, based on students’ choices.

[04] Analysis & Recommendations

Statistical comparison of results to measure design effectiveness.

Formulation of UX recommendations for the development team, based on students’ choices.

[04] Analysis & Recommendations

Statistical comparison of results to measure design effectiveness.

Formulation of UX recommendations for the development team, based on students’ choices.

[Outcome]

Empirical validation of design effectiveness in a real classroom context.
Clear student preference for Version B, which offered better navigation and more engaging visuals.
Demonstration of A/B testing as a viable methodology for evaluating digital educational tools.

[Key Learnings]

Student choice validates design

Direct input from real users (students) is a reliable guide for interface effectiveness.

Student choice validates design

Direct input from real users (students) is a reliable guide for interface effectiveness.

Student choice validates design

Direct input from real users (students) is a reliable guide for interface effectiveness.

Data-driven design

Empirical testing ensures product decisions are supported by evidence, not assumptions.

Data-driven design

Empirical testing ensures product decisions are supported by evidence, not assumptions.

Data-driven design

Empirical testing ensures product decisions are supported by evidence, not assumptions.

UX in education

Small interface adjustments can significantly improve engagement and learning outcomes.

UX in education

Small interface adjustments can significantly improve engagement and learning outcomes.

UX in education

Small interface adjustments can significantly improve engagement and learning outcomes.

Select this text to see the highlight effect