LightsOut Factoryautomation kits
Menu
English

Our Czech FAQ said the opposite of the English. Every check we had passed it.

Languages

On 2 November our Czech and Slovak FAQ told buyers that support helps when a kit does install on a supported setup. The English says when it does not. Both are fixed now, but the line had passed every check we run on a translation, because nothing in it was broken: the right script, the right numbers, the right names, every tag in place. Only a reader could see it, so we asked one.

How we asked

The reader is the model that translates our sites, the same open Qwen model on our own machine, asked a different question. It gets one English line, its translation and a few sentences about the site: its tone, the form of address in each language, and the metaphor we mean (a lights-out factory runs in the dark because nobody needs to be there). Then: would a reader of the translation be misled? A wrong fact or number, the opposite meaning, a different object or action, key information missing or added count; an idiom rendered by its sense, a freer wording or a product name left in English does not. It answers in JSON, one line at a time.

What it found

Seven pages of our store in 42 languages: the home page, prices, questions, the bundle and three kit pages, 6,216 translated lines. It flagged 126, about 2%. Among them:

What it got wrong

We read every flag in the 16 languages we can check ourselves: 24 of them. 9 were real errors, 5 we could argue either way, and 10 were false alarms. It complained about grammar that keeps the meaning (a Macedonian adjective's gender, an Italian pronoun), about a Romanian form of address our brief never asked for, about a list of nouns it read as verbs, about a correct Slovak term for model weights, and about a Macedonian idiom it did not know ("година дена", a year's time).

So the reader is not a judge. It is a filter in front of a second try: a flagged line was translated again with the problem named, and the new line had to pass both the checks and the reader before it replaced the old one. On the first six pages it flagged 91 lines. We left alone the 26 of the kinds it gets wrong (tone, grammar, idioms, names left in English) and sent 65 back: 60 came back with a translation that passed both and replaced the old one, and 5 are still as they were.

On app strings

Web pages are hard reading: long lines, our metaphors, technical words. App strings are short and plain, so we measured the reader on them too, with the 70-string example app that ships with Polyglot. On 72 planted errors of the obvious kind in six languages ("can be undone" for "cannot be undone", online for offline, monthly for a one-time payment) it flagged all 72. On 21 subtler ones it flagged 17. It missed "Remind me at %1$s" written as "remind me in %1$s" in German, French and Spanish, a time read as a wait, and a German line that dropped "your changes are saved". On the example's 484 real translations in nine languages it raised one false alarm, a French table cell it called ambiguous, and none on 21 rewordings that were free but faithful. Short, plain text is where it reads best.

Where it lives now

Polyglot 1.3.0 does this with --review. A translation that passes the checks is read once more; one that would mislead is asked for again with the problem named; one that still reads wrong is kept and listed for a person, who keeps it or stores their own wording with polyglot accept. polyglot review reads translations you already have, an agency's or an older run's. A verdict is remembered with its translation, so each new string costs one more request, once. It does not replace a native speaker for your store listing, your paywall and your terms, and our own pages are proof of why both are worth having. How the checks themselves came to be is in an earlier note.

Updated: