One of Terracotta's core features is that it will support within-subject experiment designs. A within-subject design is when participants are assigned to multiple conditions at different times during the experiment. Instead of a between-subject design, where half the participants might get treatment "A" and the other half get "B," a within-subject design might have half the participants get treatments "A-then-B," and the other half get "B-then-A". There are a couple big advantages to within-subject designs (also called crossover designs):
- When each participant receives all treatments, there is less concern about the research causing inequities. For example, imagine that version B turns out to be better than version A for student learning. In a between-subject design, this might cause students who received version A to have worse outcomes, caused by experimental treatment. But in a within-subject design, all students received the same treatments, just staggered in time.
- Because each participant in a within-subject design receives both treatments, each participant is almost like their own "mini-experiment." In effect, the research study has more statistical power to infer differences between treatments A and B.
But while within-subject designs have some big advantages, they're also much more complicated, particularly when embedded in a classroom. For example, time and ordering become important -- the researcher needs to decide when the students receiving treatment A need to switch to treatment B, and vice versa. Also, the experimenter needs to have multiple outcome measures, at least one for every condition. And finally, the researcher needs to have multiple experimental treatments, at least as many treatments as experimental conditions, in order to have a balanced design.
We've been thinking a lot about how to make Terracotta so that the complexities of within-subject experiments are intuitive and easy-to-manage. We've decided to organize things around the idea of an "exposure set." In the illustration below, imagine that you're making an experiment with three conditions, Text, Video, and Video & Text. Because there are three conditions, there will be three exposure sets. Each exposure set has a different arrangement of students-to-conditions:
Animation of how condition assignment changes across three different exposure sets. For example, in exposure set 1, students in Group A will receive Video, students in Group B will receive Text, and students in Group C will receive Video & Text. In exposure set 2, students in Group A will receive Text, students in Group B will receive Text & Video, and students in Group C will receive Video.
An exposure set is similar to the concept of a period in a crossover design, but whereas a period strictly contains only one treatment, we imagine that an exposure set could contain multiple treatments. For example, an AABB/BBAA design would have 4 periods (A,A,B,B or B,B,A,A), but in Terracotta, it would only have 2 exposure sets (AA,BB or BB,AA). Within any exposure set, the researcher can add multiple class assignments, and Terracotta will handle the hard work of ensuring that the right students see the right versions of each assignment (according to the student's treatment condition in that exposure set).
Finally, when mapping outcomes for a within-subject design, the researcher only needs to specify one outcome score for each exposure set, for each student (although it will be possible to add more).
