Duolingo has taken to its blog to outline how it’s using artificial intelligence to speed up course creation. The language learning platform now says it’s using an automated AI system for the creation of new sentences in courses that previously took hundreds of hours to do manually.
The new system aims to reduce manual editing, help to automate the process with AI, and create a modular system. It consists of three main parts: Generation (creates sentence candidates from pedagogical plans), Evaluation (tags sentences with assessments like "logically coherent" or "grammatically correct"), and Selection (chooses the best sentences that meet specific criteria).
So far, the new system has been used to generate and launch content for many of its Spanish courses, but it will soon expand this.
This comes after CEO Luis von Ahn caused a stir by claiming Duolingo would stop using contractors to do work AI can handle. This caused a backlash which made the company confirm it’s still dependent on its employees and AI won’t replace what they do.
One of the most critical parts of the system is the evaluators which ensure high-quality content. The evaluators are mostly AI prompts that perform objective and subjective assessments of the generated content. Engineers at Duolingo came to realize that AI is better at checking its own output than generating it which is why evaluators are important. Duolingo isn’t totally excluding humans from this process, the company learned that human-curated data is essential to steer LLM’s correctly.
While it’s quite an impressive step, it’s not without issues. Observability was found to be crucial as the automated system’s behavior was tricky to manage and understand. One example of failure was when the system created an entire French course without using the French words for “and” or “or” - essentially making the material useless as these are very common terms. The company said this was a surprise but has since been addressed.
Now that the system is mostly working, the company is expanding it to other languages like Italian and Chinese. It’ll be interesting to see whether it helps the company make new language courses quicker as there has been a noticeable lack of new languages added for English speakers in recent years.