This is a submission for the Cloudflare AI Challenge.

What I Built

A simple virtual foreign language learning companion specialized in holding conversations, helping with translations, and improving diction.

Demo

Please check out the demo at slater.sean.app.

My Code

Journey

Pain points and motivation

I want to learn Mandarin, but I don’t have a Mandarin-speaking friend to practice with. I’m also not confident enough with my foreign language skills to speak with a Mandarin-speaking person online.

Duolingo doesn’t teach me how to write or speak the things I want to express. For example, I want to say This is my favorite coffee shop. I know how to construct the sentence, but I don’t know the Mandarin word for favorite. So, I end up saying – Zhe tiao wo de favorite kafei guan.

Google Translate lacks certain language nuances. It also doesn’t keep track of the things I looked up.

Keeping notes of the words I learn takes effort – I have to write down the English word, hanzi, pinyin, and how it is used in a sentence.

Therefore, I wanted to create an experience where I can converse freely in a foreign language without fear of judgement, where I can quickly translate (and save as “learned”) words, and where I can practice my diction (Chinese pronunciations are really hard).

This is my first time developing an LLM app. I learned a lot in a span of two days (I learned of this challenge a little bit late). The next steps would be, developing this further for personal use, and to extend the use-cases for other languages.

Multiple Models and/or Triple Task Types

Slater has three modes: conversation mode, translation mode, and speaking/diction mode.

Meta LLama 2 for general conversations, but instructed to limit and focus its scope to language learning.

Meta Meta M2M-100 for text translations between English and Mandarin Chinese.

OpenAI Whisper for speech-to-text. However, currently figuring out how to make it work with Mandarin Chinese.