One research team tested 10 rival language apps

Open Duolingo's old marketing and you get "an independent study found that Duolingo trumps university-level language learning." Open Babbel's and you get an efficacy study. Busuu, the same. Rosetta Stone, Pimsleur, Mango, italki, LingQ.

Follow each app's efficacy claim to its source and you arrive at the same place: Roumen Vesselinov and John Grego.

The Vesselinov reports live at comparelanguageapps.com, which credits its design to Vesselinov and lists him as sole contact. I downloaded all 16 PDFs on 3 September 2026 and read them.

I went in expecting a funding problem, and what I found moves the criticism elsewhere.

One team wrote 16 reports for 10 clients

The site's research index, author lines as printed. Vesselinov is on all 16, Grego on 15.

Every study on the hub, oldest first

2009 Rosetta Stone ×3 An effectiveness study, a motivation study, and a comparative motivation study covering Rosetta Stone, Auralog "Tell Me More" and Berlitz.

2012-2018 Duolingo, Language Zen, Babbel, Busuu, Hello English, italki Six studies, six clients, one per year or so. Babbel and Busuu both in 2016.

2019-2025 Rosetta Stone, Pimsleur, Mango, TNT, Busuu ×2, LingQ Seven more. Busuu comes back in 2021 and again in 2025, the largest study of the set.

A note on the report count, because I've seen 14 quoted and couldn't get there. The index lists 16.

Distinct app companies that commissioned work: 10. Rosetta Stone, Duolingo, Language Zen, Babbel, Busuu, Hello English, italki, Pimsleur, Mango, LingQ.

Add Emmersion Learning, whose TrueNorth test the team validated in 2020, for 11 clients. Auralog and Berlitz appear as compared products in 2009 without commissioning anything, so they stay out.

The homepage claims "20+ Years of Research" since 2005. The oldest report I can open is January 2009.

So one research team has produced the efficacy evidence (the evidence that an app works) for most of the language app market over 16 years. And the apps compete.

Do the reports disclose who paid for them?

Yes, and here I correct my own starting assumption. I expected reports that stayed quiet about who paid.

Instead 15 of 16 name their funder, usually on page 5 or 6, under study limitations. The wording barely changes across a decade. Duolingo's 2012 report:

Babbel 2016 (read in full here): "This study was funded by Babbel." Busuu 2016: "funded by busuu." italki 2018, Mango 2019, Pimsleur 2019 and Rosetta Stone 2019 run the same sentence with the name swapped.

LingQ 2023 and Busuu 2025 shift to "The cost for this study was covered by" the client.

Hello English 2017 breaks the pattern in a good way. Its funder wasn't the app. It was the Central Square Foundation, a grant-making organisation.

The 2021 Busuu report goes furthest, filing the funding under "Limitations of the Study" and naming a second problem itself:

Busuu's 2021 wording is a better conflict statement than plenty of published journal work carries.

The rest of this post is critical, and criticism counts only if the credit is real.

The Vesselinov reports also name the structural problem before I get to it. Babbel 2016 asks for help "to require the creators of language learning apps to provide independent efficacy measures."

Busuu 2021 is blunter. It says there are "currently very few opportunities for language app developers to benefit from independent research studies into their efficacy, and so the current best way for them to give confidence to their customers is through commissioning research."

The commissioned researchers are saying it's the only game available.

The exception is the oldest report. The January 2009 Rosetta Stone report, out of Queens College CUNY, carries no funding sentence. I searched the text for fund, sponsor, commission, support and grant and got zero hits.

Its two 2009 companions on motivation both say the work "was commissioned by Rosetta Stone." One out of 16, and the oldest, is the honest way to say it.

Krashen made the Duolingo funding point in January 2014, in the International Journal of Foreign Language Teaching. The study Duolingo cited, he wrote, "is Vesselinov and Grego (2012), funded by Duolingo." He was reading it off the report.

The league table leaves out who paid

The actual problem sits one layer up, in the hub's league tables. The hub also publishes two ranking tables, vocabulary and grammar plus oral proficiency, putting the apps in a single list against each other.

The vocabulary table ranks eight apps by "Study Hours for First College Semester," lowest first.

The vocabulary/grammar table, beginner users, as published

0

hours, LingQ 2023 and Rosetta Stone 2019, top of the table

0

hours, Babbel 2016

0

hours, Duolingo 2012, bottom of the table

Every row in the vocabulary table comes from a study paid for by the company in that row.

The columns are year, app, percent improved, confidence interval, mean hours, range, sample size and a report link. There's no column for who paid, and the words fund and sponsor appear nowhere on either page.

Nothing is hidden: click through and the funder is on page 5.

But the table is the artifact that gets screenshotted. A reader comparing 13 against 34 has no signal that these are 8 separate commissions.

And comparelanguageapps.com invites you to read the table as a bake-off. The 2025 Busuu report says the statistical design "and methodology are comparable for all studies," and LingQ 2023 says the same.

That claim turns 16 client reports into a ranking, and it does the heaviest lifting on the site.

I'm not the first to doubt that comparability claim, and the doubt comes from an unlikely direction.

A 2021 paper in Foreign Language Annals (doi:10.1111/flan.12600) raises it. Four of its five authors list Duolingo as their affiliation.

It notes the reports "were published on company websites as white papers, not peer-reviewed journal articles." And because pretest scores differ, "it is hard to compare the effectiveness across these products."

A peer-reviewed journal says the league table can't carry the weight, and the people saying so work for a company sitting in it.

Duolingo sits at the bottom with 34 hours, on the oldest study in the set, run in 2012 on a product version that no longer exists. LingQ sits at the top with 13, from 2023.

Do a 2012 app and a 2023 app belong in one league table? The table doesn't ask.

Who finished varies by 50 points

Completion rates run from 43.7% to 93.8%, and here's the arithmetic that convinced me. Each report states its initial random sample and its final study sample. Divide one by the other for a completion rate.

Neither the reports nor the tables put those side by side, so I did.

Completion rate by study, my arithmetic on their published samples

highest Hello English 2017, 93.8% 91 of 97, a school study rather than an app-user recruitment. Busuu 2021 is next at 84.4% (119 of 141), then Babbel 2016 at 83.1% (325 of 391), Rosetta Stone 2019 at 81.7% (143 of 175) and italki 2018 at 81.6% (102 of 125).

middle Busuu 2025, 73.7% 1,205 of 1,635. The 2009 Rosetta Stone trio sit at 76.7%, 77.2% and 80.8%; Busuu 2016 at 73.5% (144 of 196), Pimsleur 2019 at 68.3% (82 of 120) and Mango 2019 at 63.8% (95 of 149).

lowest Language Zen 2015, 43.7% 101 of 231. Duolingo 2012 is 44.9% (88 of 196) and LingQ 2023 is 52.6% (101 of 192).

The completion-rate spread is 50.1 points, 43.7% to 93.8%. A Language Zen participant was 9 times likelier to drop out than a Hello English one.

Strip out Hello English, which recruited school students instead of app users, and the other 14 still spread 40.7 points.

Pooled across all 15, 4,330 entered and 3,129 finished: 28% of starters absent.

The completion rate matters for the ranking. LingQ tops the table at 13 hours, having lost 47.4% of its starters. Duolingo sits last at 34, having lost 55.1%. Babbel is mid-table at 21 and kept 83.1%.

The table shows sample size, never what the sample started as.

The oral-proficiency table has a sharper version of the problem. Busuu's 2016 oral row reports 75% improving a level, on 61 people. They were a subsample of the 144, selected for having done "about 16 hours of study or more."

That describes the people who studied most. It's defensible as analysis, and poor as a thing to rank other apps against without a note.

The reports are careful about dropouts. Most compare completers against dropouts on age, gender, education and initial test score, and report no significant difference.

That checks demographics. It says nothing about the variable deciding the result: whether you keep using the app.

Krashen caught the same shape from the other end. Of the 156 he counts as starting, 90 lasted to the end and 88 had usable scores. 66 filled in the exit survey, where 78.8% said they were satisfied.

That's 66 people out of 156.

One more from the Duolingo report, the most quoted of the set. The famous 34 hours comes from a mean gain of 8.1 WebCAPE points per hour. (WebCAPE is the placement test these reports use.)

The same report prints the median, the middle value: 3.9. Redo the division on the median and 270 points, the second-semester cutoff, needs about 69 hours.

Same study, same page, and the answer doubles depending which average you pick.

And every hours figure in the vocabulary table is a mean. Duolingo is the only study printing both, and there the mean flatters by 2.1 times. If that's near typical, the whole column is optimistic by an unknown factor.

Which way does the attrition (people dropping out) cut? These studies keep only people who did at least 2 hours and completed both tests. So survivors are the persistent users.

Heavy attrition should then make an app look more efficient per hour. That predicts high-dropout studies sit near the top.

LingQ lost 47.4% and tops the table at 13 hours, which fits. Duolingo lost 55.1% and sits last at 34, which doesn't.

Two points facing opposite ways isn't a finding. Across 15 studies with differing tests, years and recruitment, I can't separate selection from the rest. Neither can the table.

Ask three questions before you trust a study like this

Vesselinov and Grego publish under their own names, so I'll be exact about which is which.

What's documented: one team produced 16 efficacy reports for 10 competing app companies between 2009 and 2025. Fifteen name their funder, and the oldest carries none I could find.

The hub publishes two tables ranking those apps against each other, neither with a funding column. Completion rates range from 43.7% to 93.8%.

What's inference, and mine: this table is weaker than it looks. It's built from 10 separate commissions, spanning 13 years of product versions and a 41-point spread in who finished.

The funder belongs next to each row, the completion rate beside the sample size.

What I'm not saying: that any result was bought, altered or faked. I have no evidence of that and went looking for none. Funded research is normal here and everywhere else.

These reports state their funding. They describe their methods in enough detail that a stranger can check the arithmetic, which I did. That's more than most marketing claims allow.

So, three questions for anyone reading "scientifically proven" on a pricing page. Who paid? You can answer that in 90 seconds by searching the PDF for funded.

How many started, against how many finished? And how old is the tested product? A 2012 app and the app on your phone share a name and little else.

It's worth knowing what the reports measure, too. Almost all use WebCAPE, a multiple-choice Spanish placement test, some with an oral interview alongside.

That's vocabulary and grammar recognition, some distance from conversation. The strongest published result in this category is still a receptive one.

Verbamor has no efficacy study. Commissioning one costs more than the business makes, so read that as a disclosure.

What it does is narrower than anything in that table. It turns words from lessons you already had into retrieval practice scheduled by FSRS, an open-source algorithm that predicts when you're about to forget each card.

Those are claims about spacing and retrieval, with a research base that has nothing to do with me. I've never measured Verbamor against Babbel, and neither has anyone else.

For the per-hour comparison, see that one with the numbers shown.

---

Sources