Study abroad has never had a randomised trial

Ian Moore, Carole Torgerson and Nadin Beckmann went looking for a randomised controlled trial of study abroad: one where chance, and not the student, decides who goes.

They searched nine databases, pulled 2,548 records, and screened down to six studies. Every one was quasi-experimental, comparing groups that sorted themselves. Their limitations section says it: "the lack of any extant randomised controlled trial (RCT) to include in the review means that ultimately, determining strong causal effect from the review is extremely difficult." Almost nobody quotes that sentence.

One thing to be careful about first. I run a company selling software to people learning a language without moving abroad, so an argument against study abroad flatters me.

I'm going to give the other side its strongest version, with its own numbers, and keep the parts that cut against me.

Three syntheses, zero randomised trials

Three teams have tried to pool this literature. Their study counts differ because their inclusion rules do. Their design counts are identical.

Randomised trials found, by synthesis

0

studies in Moore, Torgerson and Beckmann (2021), from 2,548 records screened. All quasi-experimental

0

studies in Yang's (2016) meta-analysis, drawn from 66 screened. All comparison-group designs

0

studies in Kim, Lee and Lee (2024), described by its own abstract as "all based on quasi-experimental designs"

Thirty-one study slots across the three syntheses, with overlap between them, and not one random allocation.

Moore's team is explicit about why the trial doesn't exist, and the reason is decent. Randomly assigning a 20-year-old to a year in Seville or a year in Sheffield raises real ethical problems, and no university will run it. A good reason for the gap still leaves the evidence missing.

Table 5 appraises all seven publications on four criteria. On grouping strategy, all seven read identically.

On pre-specified outcomes, all seven say "No", so no author committed to which measure counted before seeing the data. Six of the seven report no attrition figures (how many people dropped out).

The six studies' samples are small too. Moore's mapping table adds up to 471 people across six studies, a median (middle value) of 69.5, a smallest of 18.

Jochum's (2014) study produced a between-group effect of d = 0.56 on oral proficiency. (d is an effect size, a standard measure of how big a difference is.) It ran nine students abroad against nine at home. Nine.

Moore's team rated it "of limited quality", partly because participants were tested up to six weeks after departure.

The plane ticket genuinely buys oral fluency

Now the other side, at full strength. Kim, Lee and Lee (2024) pooled 14 between-group studies of L2 fluency and found a medium effect for study abroad: Hedges' g = 0.65, SE 0.10, 95% CI [0.44, 0.87].

The CI, or confidence interval, is the range the true effect probably sits in. That interval sits clear of zero, and I'd like to keep it standing.

Yang (2016) got d = 0.75 across 11 studies, four above d = 0.8. Yang concluded the data supported the view that study abroad "could lead to greater L2 linguistic attainment compared to at-home classroom learning".

And Moore's team catalogued every weakness above, then still concluded that sojourning "can have a positive impact on linguistic ability and should, within reason, continue to be seen as a catalyst for language development."

The study-abroad gain is oral fluency, and it's consistent. In Llanes and Muñoz (2013), the students abroad got more fluent across a semester at d = 0.77, while those at home moved at d = 0.10.

Segalowitz and Freed (2004) found the study-abroad group gained on five of eight oral measures and the at-home group on none. Something happens to your mouth when you spend three months using it.

And Moore's team ranks general proficiency the best-evidenced domain in the review: "the strongest causal inference may be made pertaining to the notion that SA does enhance general proficiency."

One thing a confidence interval can't do. Kim's [0.44, 0.87] says the pooled effect is unlikely to be zero. It says nothing about whether that effect is the country or the person.

All 14 studies share one confound (a hidden difference between the groups). Pooling them just estimates the confounded quantity precisely.

Across nine outcomes, three favour going abroad

What no one has assembled is the full scoreboard. Moore's synthesis walks through nine outcome domains.

Moore et al. 2021 · outcome by outcome

Domain	Studies	Direction of the evidence
Oral fluency	3	Favours abroadSA d = 0.77 vs AH d = 0.10 in Llanes and Muñoz
General oral proficiency	2	Favours abroadJochum d = 0.56 between groups, on n = 18
General proficiency	2 (3 papers)	Favours abroad for 3 months, then stops+6.50 marks, then context drops out of the regression
Oral accuracy	2	No effectSA d = 0.15 vs AH d = 0.24
Writing accuracy	2	No effectNeither study found a direct effect of the sojourn
Syntactic complexity	2	No effectWritten SA d = 0.01 vs AH d = 0.17
Writing fluency	2	MixedSerrano favours abroad, Llanes and Muñoz favours home
Vocabulary	2	MixedIn writing, AH d = 0.89 vs SA d = 0.39
Reading	1	UninformativeCeiling effects, pre-test scores 9.12 to 9.51 out of 10

Read the vocabulary row twice. On written lexical richness, Llanes and Muñoz found the at-home adults at d = 0.89 against the study-abroad adults at d = 0.39. Serrano and colleagues (2011) found the opposite on the same measure.

Nobody should quote a vocabulary number from this literature in either direction, and that includes me.

Moore's discussion says the findings indicated sojourning "to have little benefit on the development of writing, reading and grammar". That fits the gap that shows up the first time a stranger talks to you at speed.

Does a longer stay abroad keep paying off?

Only for about three months, going by the most useful number in the review. It comes from Moore's highest-rated study.

Hessel (2016) followed 143 German learners of English in three groups: one semester abroad, a full year abroad, and a control group who applied for ERASMUS and were turned down.

Over the first three months, both abroad groups pulled ahead. On a C-test (a gap-fill test), short-stay students improved by 6.50 marks more than the at-home students, and long-stay students by 6.24.

Over the next six months, the long-stay group improved at d = 0.45 and the at-home students at d = 0.17. On the gain scores, "the regression analysis indicated that learning context no longer predicted L2 proficiency development."

Hessel's thesis puts it plainer: progress among the ERASMUS group slowed and "the between-group differences were no longer significant."

Six extra months bought no measurable advantage over staying home, in the study Moore's team rated highest. Two things make that sturdier than a lone result.

Moore's mapping section says European studies generally find "the most substantial gains are seen in the first three months of the study abroad period and that the rate of the gain slows, the longer an individual is abroad." And Yang reports short-term study abroad was more effective than long-term.

Hessel's thesis carries a second finding Moore's review leaves out. I want to report it carefully, because I first read it the way that flattered me.

Hessel ran two hierarchical regressions on who gained how much, each entering baseline proficiency first. (A regression estimates how much each factor predicts the outcome.) The learner-internal model reached R² = .452, so the student's own characteristics account for 45.2% of the variance.

The learner-external model covered English classes, clubs, contact hours and time with co-national friends. It added 10.9% over and above baseline, a "small effect" against the large one on the learner side.

Hessel's 45.2% and 10.9% come from separate models. Dividing one by the other wouldn't be a legitimate ratio, and I'm not going to print one.

The comparison that holds is Hessel's own: none of the individual learner-external predictors "made a significant unique contribution towards predicting the outcome" once baseline was controlled.

And the complication Hessel raises, which I'd rather she hadn't. Her biggest single predictor is entry proficiency at 31.5%, with a negative partial correlation: weaker students gained more.

She calls this "to some extent a methodological artefact" of regression toward the mean on a C-test. (Extreme scores tend to drift back toward average on a retest.) So the largest block in her model is partly a measurement effect, and only partly a story about ambitious people.

Hessel then partly takes the artefact explanation back. She notes the inverse relationship "was far less pronounced for the comparison group in the home environment (r = -.23, p<.05), which renders a purely methodological explanation as simplistic."

That cuts my way, which is why I'm quoting it.

What survives is narrower. In the best study here, what the student walked in with predicted their gain, and what the programme supplied around them did not predict it at any individual level.

The strongest objection comes from rejected applicants

Hessel's design arguably solves the problem I've spent this post complaining about. It's the one place Moore's appraisal says something kind about sampling.

Table 5 records her comparison group as "formed of individuals who wanted to be in treatment group but were unsuccessful in application. Prior pre-test differences considered minimising sampling bias on results."

Rejected applicants wanted to go. If that holds motivation constant, her three-month effect is close to causal. It was "large and highly significant (p =.001), even after the influence of pre-existing participant characteristics" was controlled for.

My best evidence is also what most undermines my framing.

Three reasons Hessel's design still falls short of randomisation. Rejection runs on the merits: places go to the stronger applications, so the accepted group is the upper slice of an already motivated pool.

Rejection is also a treatment. A student told no spends the next year differently from one told yes, and some of that is discouragement rather than geography.

And Hessel controlled for the characteristics she measured. That leaves the 54.8% her best model doesn't explain as the room an unmeasured confounder lives in.

So the fair statement is narrower than I'd prefer. This is the best comparison the field has, and it's genuinely good. It still can't rule out that the accepted students would have pulled ahead anyway.

Where it bites hardest is the six months after the first three, when the same design stopped producing a gap.

My position: fluency comes early, and I have a conflict of interest

So here's what I think, as a claim someone could argue with. Study abroad reliably buys oral fluency, and most of it lands in the first three months.

Set that against the FSI figures, where a Spanish learner is budgeted 552 to 690 class hours and nobody claims to be finished at week 13. Beyond three months the evidence thins fast. For writing, grammar and reading, there's no good evidence it buys anything.

Moore's team names the mechanism without flinching. In the four European studies, the study-abroad group "was formed by individuals who had gained a scholarship or were successful in applying for the programme." In the North American studies, "it is unclear how participants may have differed in each group."

Baseline equivalence controls for where the students started, not the slope they were already on.

Now the part I have to say out loud. Verbamor is an iOS app for people learning Spanish, French, Italian, Portuguese or Russian who aren't moving abroad.

Every sentence above is commercially convenient for me. That's why I quoted Kim's g = 0.65 with its interval, Yang's d = 0.75, and Moore's conclusion that these programmes should keep being funded.

What Verbamor does is narrow. You record a lesson or import audio, and it lifts out each new word and phrase you ran into and schedules them for review.

That helps with vocabulary, the domain where this literature has two studies pointing opposite ways. It doesn't give you a Spanish landlord, and oral fluency comes from talking to people.

If you're going abroad, go. Arrive with vocabulary already in your head. The three-month window in the Hessel data is short, and you don't want to spend it learning the words for "deposit" and "landlord".

If you're staying, the useful page is the hours arithmetic: a semester in Peru buys about 300 hours of Spanish, a number you can compete with. What you can't buy is the pressure of ordering food in a language you half-speak.

The claim to retire is the ambient one: that a year abroad is how you become fluent, unquestioned because it sounds obvious.

Six studies, 471 people, zero randomisations, and an advantage that drops out of the regression at three months. Anyone telling you more is telling you something the literature doesn't contain.

---

Sources