Runtime UI translation with a small LLM and an editable cache

The worked example is mmxc, a self-hosted personal-finance app: a Go server rendering html/template pages, which already calls an OpenAI-compatible LLM endpoint for statement import and a question assistant. Its interface runs in French and German this way. The code shown is Go, but nothing in the pattern depends on it.

The problem with message catalogs

Classic i18n extracts every literal into a catalog keyed by stable ids (accounts.title), and each language gets its own copy with translated values. The cost has two parts:

• Extraction, a one-off: every user-visible string is wrapped in a lookup call. In mmxc that was about 400 template strings, 36 Go messages and a dozen JavaScript strings.
• Maintenance, which never ends: every change to the English must reach every language file. That needs translators.

The extraction can’t be avoided. The maintenance can, if the catalog is filled by a model on demand instead of by people up front. The catalog survives as a plain file a person can still correct, so the design differs from the classic one only in who writes the values.

Architecture

Five parts:

• One lookup function, used wherever text is rendered: T(source, context). For the source language it returns the source unchanged and never calls a model.
• A cache per language: a map in memory, backed by one JSON file on disk.
• A model call on a miss: one string per request, temperature 0, JSON output, to a small translation model (translategemma:4b). It is configured separately from the larger general model the rest of the app uses.
• Guards that decide whether an answer may enter the cache.
• A fallback: on any failure, the English source is shown. The model can make the interface worse in one way only, by leaving a string untranslated.

In a template:

<th>{{.T "Balance" "running account balance after this transaction"}}</th>

The lookup, condensed. The lock is not held during the model call, so two requests can translate the same string at once. The second one to finish finds the first one’s entry and uses it, which keeps the cache consistent:

func (t *translator) get(lang, source, guidance string) string {
	if lang == "" || lang == "en" {
		return source
	}
	key := translationKey(source, guidance, lang)

	t.mu.Lock()
	t.ensureCache(lang) // load, or reload if the file changed on disk
	if e, ok := t.caches[lang][key]; ok {
		t.mu.Unlock()
		return e.Translation
	}
	t.mu.Unlock()

	if !t.modelConfigured() || !t.aiAvailable() {
		return source
	}
	result, err := t.callTranslate(lang, source, guidance)
	if err != nil ||
		!validateTranslationPlaceholders(source, result) ||
		!plausibleTranslationLength(source, result) {
		return source // not cached: retried on the next render
	}

	t.mu.Lock()
	if e, ok := t.caches[lang][key]; ok { // another request got there first
		t.mu.Unlock()
		return e.Translation
	}
	t.caches[lang][key] = cacheEntry{source, guidance, result}
	snapshot := maps.Clone(t.caches[lang])
	t.mu.Unlock()

	t.saveCacheToDisk(lang, snapshot)
	return result
}

Keys: the source text plus context

The key is not an invented id. It is a hash of the English source, the context note and the language:

func translationKey(source, guidance, lang string) string {
	h := sha256.Sum256([]byte(source + "\x00" + guidance + "\x00" + lang))
	return hex.EncodeToString(h[:8])
}

• No id catalog. Code reads as plain English, and nobody has to name or deduplicate keys.
• Editing the English re-translates. A changed source hashes to a new key and is translated on its next render. With id keys, an edited English string keeps its old translation without any warning.
• Identical strings share a translation, which is almost always right. When it isn’t, the context note separates them.
• The NUL separators stop different splits of the fields, such as ("ab", "c") and ("a", "bc"), from producing the same key.
• Consequence: an edited string leaves its old entry behind in the file, unused.

Context for short labels

Sentences translate well with no help. One-word labels don’t, because a word on its own has several meanings. Entries from mmxc’s French cache:

The optional second argument works like gettext’s msgctxt. It is a short note on where the string appears, sent to the model as context and included in the key, so one English word used in two senses gets two cache entries. In mmxc, about 200 of the 466 distinct template strings have one. Context that applies to every string goes in the prompt once instead: the kind of application, and that domain terms win when a word is ambiguous (here, financial and accounting terms).

Context chooses a meaning. It cannot make two strings differ from each other. A confirmation dialog with the buttons Void and Cancel came out as Annuler and Annuler. Each was correct on its own, but together they made the destructive button look exactly like the one that dismisses the dialog. More context did not fix it; a longer source did. Void transaction translates to Annuler la transaction.

The model translates strings, not screens. When two strings on one screen must stay distinct, make the sources distinct.

The prompt: send the string as data

Failure mode. A prompt made of prose instructions ending in English source string: Translation works for sentences. For a one-word label it breaks: the model cannot tell where the instructions end and the string begins, so it translates the app description or the context line instead. The result is valid text, so it gets cached, and a section heading renders as a paragraph of French.

Fix. The source and its context travel as labelled JSON fields, and the reply comes back as JSON (the API’s response_format: json_object). A bare-text reply invites alternatives, explanations and preambles. The full prompt:

Translate the "source" field of the JSON object below into French.
You are translating MMXC, a personal-finance web app for accounts, budgets,
payees, categories, statements, transactions, deposits, withdrawals,
transfers, balances, and reconciliation.
Use financial/accounting terminology where a word is ambiguous.
Keep UI labels, buttons, table headings, validation messages, and short
status text concise and natural for software UI.
Preserve product names and technical names such as MMXC, MMEX, LLM, AI, URL,
PDF, CSS, JSON, and YYYY-MM-DD unless the target language convention clearly
localizes them.
Preserve format placeholders (%s, %d, %.2f, etc.) and named placeholders
({n}, {cat}, etc.) exactly as they appear.
The "context" field says where the string appears. Use it only to pick the
right sense of an ambiguous word: never translate it, never answer it, and
never add words that appear only in it.
The translation renders the source alone, so it stays about as short as the
source.
Reply with one JSON object: {"translation": "<the translated source>"}

{"context":"running account balance after this transaction","source":"Balance"}

Build the payload with a JSON encoder, not string concatenation, so quotes and backslashes in the source can’t break it. Then parse the reply into a struct and reject an empty translation.

Output guards

The cache makes every answer permanent, so each answer is checked before it is kept.

Format placeholders must match. Translated strings often go on to fmt.Sprintf. If the model drops or reorders a verb, Go writes %!d(MISSING) into the page. So the translation must contain the same sequence of verbs as the source:

var placeholderRE = regexp.MustCompile(`%[-+ #0]?[0-9]*(?:\.[0-9]+)?[sdfevqtbxXoOpTU]`)

func validateTranslationPlaceholders(source, translation string) bool {
	return slices.Equal(placeholderRE.FindAllString(source, -1),
		placeholderRE.FindAllString(translation, -1))
}

Length must be plausible. A translation may be at most four times the length of the source plus 24 characters. A model that echoes the prompt or the context exceeds that by an order of magnitude. Normal growth, such as German compounds or French articles, stays well inside it, and the constant slack covers one-word labels.

func plausibleTranslationLength(source, translation string) bool {
	return len([]rune(translation)) <= 4*len([]rune(source))+24
}

A rejection is never cached. The English is shown and the string is tried again on the next render.

A dead endpoint costs one probe, not one timeout per string. Before calling the model, the translator reads the app’s cached health status for the LLM endpoint (a GET /models with a short timeout and a TTL). When the endpoint is down, every miss goes straight to English.

Not covered: named placeholders such as {n} are protected by the prompt only. Extending the check to them is a small change.

The cache file is the catalog

Each language is one JSON file. Each entry stores what it translated, so the file explains itself to whoever edits it:

"b313de732b26d617": {
  "source": "Balance",
  "guidance": "running account balance after this transaction",
  "translation": "Solde"
}

To correct a translation, edit the value. Making edits take effect without a restart takes four details:

• Cheap change detection. A page makes hundreds of lookups, so the file is stated at most once a second per language. It is reloaded when its modification time or size changes.
• The file wins over memory. Deleting an entry from the file has the string translated again.
• Atomic saves, with their own stamp. A save writes a temp file and renames it into place, so a concurrent reload never reads half a file. The save then records the new file’s stamp, so the app does not mistake its own write for an outside edit.
• Broken JSON keeps the old cache. If a save happens halfway through a hand edit and the file no longer parses, the translations already in memory stay in use until it does.

A corrected file can be copied to another installation as it is. That is the same workflow as a hand-written catalog, with the model writing the first draft.

Strings outside templates

• JavaScript can’t call the translator. The page head renders the strings the scripts need into a map (window.MMXC_T = {"Loaded {n} models.": "…"}), and the scripts look them up with a small i18n(key) helper. Strings missing from the map stay in English.
• Native confirm() and alert() ask the question in the app’s language but label their buttons in the browser’s. Replace them with <dialog> elements whose buttons go through T and name the action.
• Dates use a display layout per language. Form inputs stay ISO so their values round-trip.
• Money follows the currency’s own format (in mmxc, stored in the database), not the UI language.
• Not translated: user data (account, category and payee names), and dynamic err.Error() text. Translating errors would put a model call on the error path.

Costs and limits

• The first render of a page in a new language is slow: one synchronous model call per new string. A whole language takes a few hundred calls, once. Batching the misses on a page into one request is the obvious improvement, and mmxc doesn’t do it yet.
• No review step. Each installation translates for itself, and nothing ships in the binary. Quality depends on the model and on how good the context notes are.
• Orphaned entries pile up as the English changes. They do no harm, but a cleanup pass would need the set of live keys.
• Right-to-left languages get translated text, but the layout is not mirrored.

When to use it

It fits when the application already has an LLM endpoint, each installation has few users, there are no translators, and the interface text has no legal weight. Self-hosted tools and internal apps are the typical case.

It doesn’t fit when translations must be reviewed before users see them. There, keep the same pipeline but run it at build time: generate the catalog as a draft, have a person review it, commit it, and ship it.

Checklist for reusing the pattern

1. One lookup function, T(source, context), at every rendering site. The source language returns the source unchanged.
2. Key = hash(source, context, language), with separators.
3. Describe the application’s domain once in the prompt. Give a context note to every short or ambiguous label and every string with placeholders.
4. Send the source and context as JSON fields. Ask for a JSON reply at temperature 0.
5. Check placeholders and length. Never cache a rejected answer.
6. Fall back to the source on any error. Gate model calls on a cached health check.
7. Store one plain file per language with self-describing entries. Write atomically, reload on change, and let the file win.
8. Configure the translation model separately from the main model.
9. Where two strings on one screen must differ, make their sources differ.
10. Cover the edges: strings in client-side scripts, native browser dialogs, dates, and number formats.
11. Batch the misses on a page into one request.

Source: translate.go and docs/i18n.md in the mmxc repository.