Sugata Mitra and Emma Crawley have published a paper on the SOLE experiments based in Newcastle, UK titled Effectiveness of Self-Organised Learning by Children: Gateshead Experiments.
The paper seems to have omitted a basic design in such studies for three of the experiments reported whilst the fourth one introduces an unnecessary complication.
If participants in an experiment are given all conditions of an experiment (called a repeated measures design) one needs to avoid order of treatment effects. We can do this by counterbalancing the treatments.
For their experiment 1 Mitra and Crawley (M&C) could have divided their 23 pupils into 2 groups, randomly assign the first group to SOLE (self-organised learning environment) treatment then individual treatment and randomly assign the second group to individual treatment then SOLE treatment. This would have avoided puzzlement of M&C about “the strange result of a test result improving over time with no formal inputs in the interim”.
Similarly the three tests in M&C’s experiment 2 could have been counterbalanced along with individual vs SOLE treatment. Hence there would have been less of a need for M&C to have invoked an “anomalous expansion of understanding” concept to explain why individual testing produced better performance than SOLE testing. The same criticism goes for experiment 3.
Experiment 4 seems the equivalent of a design variable car crash. Individual and group work are counterbalanced but the difficulty level of the reading text are not examined fully. That is a grade 3 text is used in one trial and a grade 5 in another trial. A better design would have included both grades in both trials. Group work has been shown to be more effective if the task is more complex so the best one can say is that M&C’s experiment 4 maybe shows that this is quite strong even with a weak experimental design!
M&C acknowledge their confusing results via this statement in the discussion section:
“Children seem to enjoy working on a ‘hard’ question, when there is no competition and when they are in groups, using the Internet. There is some indication that the opposite is also true, namely, that children do better individually at easy things than they do in groups.”
Finally the journal that Mitra and Crawley went with has a publisher with a less than stellar reputation.
Thanks for reading.