Notes

Why Bengali text-to-speech was dropping vowels

By Ramees Muhammed ·

When a speech engine gets a word slightly wrong, you notice. When it silently leaves out a sound, you often don’t, unless you listen to it all day, the way screen reader users do.

This is the story of a bug like that in Bengali, in eSpeak NG, the open-source speech engine behind my Android app and my browser tool, Try eSpeak NG. It’s a small bug with a lesson for anyone who handles text in Indian scripts.

What was wrong

Some Bengali words lost a vowel. For example:

  • পড়া (pora, “reading”) was read as “por”.
  • পড়ে (pore) was also read as “por”, so two different words sounded the same.
  • বড়ো (boro, “big”) was read as “bor”.
  • হয়েছে (hoyeche, “has happened”) lost its “ye” and was read as “hoche”.

The pattern: the vowel after three letters went missing. These are ড় (ro), ঢ় (rho) and য় (yo). Each of them is written as a base letter with a small dot underneath, called a nukta.

No error, no crash, no strange noise. The word just came out shorter. A sighted person reading along on screen might never notice. A blind person listening to a news article hears words that don’t quite exist.

One letter, two ways to store it

The cause is in Unicode, the standard that gives every character a number.

ড় can be stored in two ways:

  1. As one character, “Bengali letter RRA”.
  2. As two characters: ড (DDA) followed by the nukta sign.

They look identical on screen, and Unicode treats them as the same text. But a program that compares characters sees two different things.

Here’s the surprise. Unicode’s standard way of tidying text, called normalisation form C (NFC), is what most software uses to make text consistent. And NFC does not join these letters into one character. It deliberately splits them into two. Unicode lists ড়, ঢ় and য় as “composition exclusions” for compatibility reasons. So any text that has passed through normalisation, Wikipedia for example, uses the two-character form. Other text, such as many subtitle files, uses the one-character form. Both are common, and the same word can arrive either way.

eSpeak NG’s Bengali rules handled the one-character form well. For the two-character form, the rule that reads a vowel sign looks for a consonant just before it. What it found was the nukta. The nukta isn’t a consonant, so the vowel sign had nothing to attach to and was dropped.

The same problem affected Assamese and Bishnupriya Manipuri, which use the same script. Their rules had a second issue: they defined these letters as two-character groups that the engine’s rule lookup never matches.

How I found it: by being wrong first

I didn’t find this by listening. I found it while building the Assamese page of Try eSpeak NG, which runs the engine in the browser.

The Assamese voice seemed to crash on some text. My first conclusion was a memory bug in the engine. It wasn’t. When I looked properly, the “crash” was something else. The one-character ড় made the Assamese voice hand the word over to the Bengali voice, and my web tool hadn’t loaded the Bengali dictionary. The app never crashed.

But while proving that, I compared the two encodings of the same words, and the missing vowels showed up. The real bug was quieter, and it was in Bengali, a language with hundreds of millions of speakers.

I’m glad I checked before “fixing” the crash. A fix for the wrong cause would have changed code that worked and left the real problem in place.

The fix

The Hindi and Punjabi rules in eSpeak NG already had the answer. Before any reading rules run, they replace the two-character form with the one-character form. Every rule after that sees a single consonant, whichever way the text was typed.

I applied the same idea to Bengali, Assamese and Bishnupriya Manipuri. I also fixed the Assamese and Bishnupriya letter groups. Now both encodings read the same: পড়া is “pora” again, and হয়েছে is “hoyeche”.

One rule needed a decision. Bengali had a special rule for য় at the end of a word that only applied to the one-character form, so the same word could be read two different ways. I made both forms read the same. I’m not a Bengali speaker, so I asked in the pull request for a Bengali speaker to confirm that choice. That review is still open.

The fix is live in Try eSpeak NG’s Bengali page and proposed to the eSpeak NG project as pull request #2582, with tests for all three languages.

What app teams can learn from this

Test both encodings. If your app compares, searches, sorts or speaks Indian-language text, test letters with a nukta in both forms. This applies to Bengali, Assamese, Hindi, Marathi, Punjabi and others. Text from a keyboard, a web page and a database may not match, even when it looks the same.

Normalisation isn’t a cure-all. Many developers assume NFC makes text uniform. It does, but uniform in a way that may not be the one your code expects. Know which form your data is in.

Silent failures need listeners. No automated check flagged this. Every word “worked”. Only listening showed the problem. If you build voice or accessibility features for Indian languages, test with people who use them every day.

Check the cause before the fix. My first explanation was wrong. Proving it before changing anything is what led to the real bug.

If you speak Bengali and use a screen reader, I’d love your help checking this. Contact me through this site.

I also offer screen reader accessibility audits for websites, Android and iOS apps: how my audits work.

Back to all notes