The Hello English (HE) efficacy study was independently conducted by ROUMEN VESSELINOV and JOHN GREGO, from September, 2016 to January, 2017.
ROUMEN VESSELINOV has a PhD. in Economics from Queens College, City University of New York.
JOHN GREGO has a PhD. in Statistics from University of South Carolina.
This study was funded by the Central Square Foundation, a grant making organization and policy think tank focused on improving the quality of school education and learning outcomes of children from low-income communities in India.
The English oral proficiency language test used in the study was designed, developed and managed by Language Testing International (LTI), an independent US based language testing company. The Research Team carried out the test data collection and statistical analysis independently.
The study was based on a random representative sample of 97 students in India from grades 8 to 12 from three schools: one government and two private schools.
The research goal for this study was to test whether the introduction of HE as an additional tool for learning English as a foreign language would significantly improve students’ language skills compared to their classmates who did not use the new tool.
The students were randomly assigned to two groups: the HE group, which was given access to the HE app in addition to the regular school instruction and the Control group, which continued with their regular school classes with no access to the app. The participants took one English oral proficiency test in the beginning of the study, and the same test at the end of the study. The improvement in language abilities was measured as the difference in levels between the final and the initial language tests.
HE English Language Oral Proficiency Gain:
- Overall, 73% of the HE students improved their language proficiency by at least one level compared to 42% of the Control group. This difference was statistically significant.
- The 95% confidence interval for the improvement in language proficiency for the HE students was between 60% and 83% compared to an interval of 27% to 58% for the Control group.
- Truly novice English language learners (that is, those who were initially at a Novice-Low level) from the HE group improved the most with 88% gaining one or more levels. Among the more advanced, 45% improved by one or more levels. This is expected and consistent with the results from other language studies.
- From the HE group, 56% of students improved by one level, 15% of students improved by two levels and one student improved by three levels. None of the HE students decreased their language level.
- The language improvement of the HE group compared to the Control group remains significant even after controlling for demographics, school type and results of school based assessments. The HE app worked for everybody in the study regardless of their age, gender etc.
The Hello English efficacy study addressed the question of whether the introduction of an additional learning tool to students from grades 8 to 12 would significantly improve their language skills. The study demonstrates that school-going students who use the HE language app in addition to their regular language classes show greater progress compared to their classmates who do not use the HE app. On average about 73% of the HE users can expect to increase their language knowledge by at least one level compared to only 42% of their classmates. The difference between the two groups is statistically significant. The 95% confidence interval for the progress of HE students is between 60% and 83%. The language improvement of the truly novice English language students (Novice Low level) from the HE group was the most impressive with about 88% of them improving by at least one level. Students with a higher initial level also improved, but at a lower rate (45%). This result is in line with previous language studies (Vesselinov & Grego, 2012, 2016). The progress of more advanced language learners is smaller compared to the progress of the true beginners.
The above findings are promising as they demonstrate that the Hello English app can be used by school students for learning the English language. With the availability of student usage data, we could garner more granular insights on patterns of use thus shedding light not only on whether children learn through the app, but also on how they do so.
For the full report, click here.