Engineering Blog · Open Benchmark

Kazakh OCR Benchmark: Testing Tesseract Accuracy on 9 Letters

Three model sets, four samples, zero neural post-processing. Full test numbers, edge cases, and reproducible evaluation scripts.

Can standard open-source OCR handle Kazakh documents without fine-tuning? We ran an open benchmark across the nine distinct letters with reproducible scripts.

SummaryWe conducted an open engineering benchmark of the offline Tesseract 5.5.0 OCR engine across four synthetic Kazakh document samples without neural post-processing. On clean printed text error rate remains below 0.3%, whereas complex tables and bold headers confuse every fifth Kazakh letter. The kaz language model is mandatory to recognize any of the nine specific Kazakh characters.

Job · engineering lead managing document workflows with Kazakh text

When users upload Kazakh receipts and invoices into an automated pipeline, I want understand raw OCR error rates across the nine unique letters, so that automate standard document processing while keeping human review on edge cases.

What Was Measured and Why We Ran This Open Benchmark

629
Kazakh letters analyzed across 4 document samplesAitomaton Open Benchmark · 2026-10-13

How it relates to your taskClear printed text achieves reliable recognition, establishing a verified baseline for document ingestion.

0.05%
CER on clean printed sample with eng+rus+kaz modelsAitomaton Open Benchmark · 2026-10-13

How it relates to your taskClear printed text achieves reliable recognition, establishing a verified baseline for document ingestion.

4.1%
CER on invoice table sample where letter error rate peakedAitomaton Open Benchmark · 2026-10-13

How it relates to your taskClear printed text achieves reliable recognition, establishing a verified baseline for document ingestion.

What exactly did this Tesseract OCR benchmark measure?

We evaluated offline Tesseract 5.5.0 CLI across four synthetic Kazakh document samples, measuring CER, WER, and letter substitution errors on nine distinct characters.

Why does Kazakh text fail completely without the kaz model?

Without kaz traineddata, Tesseract defaults to Cyrillic or Latin heuristics, corrupting all 271 Kazakh characters in test samples into Russian phonetic lookalikes.

What pushes you away from the old way

Documents arrive in Kazakh while default OCR pipelines lack validation on 9 distinct letters.

What pulls you to the new one

Clean printed Kazakh achieves under 0.3% error rate, sufficient to automate intake pipelines.

What worries you

Misreading crucial amounts or names in complex table invoices.

What keeps you with the habit

Relying on generic Cyrillic models until an invoice header fails unexpectedly.

Evaluation Methodology and Test Setup

We evaluated offline Tesseract CLI version 5.5.0 across four document samples rendered at 300 DPI using DejaVu Sans fonts. Every test run compared character-level alignments against verified ground truth files to count exact substitutions, deletions, and insertions.

Benchmark Results Across Four Synthetic Document Samples

Why do table layouts and bold headings degrade OCR accuracy?

Grid borders and heavy font weights distort character boundaries, causing Tesseract to misinterpret uppercase Kazakh letters as Cyrillic equivalents.

Clean Printed Text

Error rates stay minimal at CER 0.05% with only one mistake across 271 Kazakh letters. Clean font documents can safely enter automated processing without manual re-typing.

Bilingual Kazakh-Russian

Code-switching text achieves CER 0.08%. Combining rus+kaz language models preserves character fidelity across mixed vocabularies.

Invoice & Grid Tables

Table borders and uppercase bold titles elevate error rate to 4.1%, confusing every fifth Kazakh character. This structure demands automated dictionary correction.

Which Kazakh Characters Experience the Highest Error Rates

Letter-level error breakdowns demonstrate that characters with subtle diacritics encounter the highest risk of misclassification:

  • Қ frequently degrades into Cyrillic К.
  • Ұ transforms into standard У.
  • Ң is misinterpreted as Cyrillic Н.

Comparing Fast, Best, and Web Tesseract Model Variants

How documents progress through OCR evaluation

  1. InboundDocument Render

    Synthetic Kazakh text is rasterized at 300 DPI into PNG samples.

  2. RecordTesseract Extraction

    Engine processes image using configured language packs.

  3. Follow-upCharacter Alignment

    jiwer aligns characters to flag letter substitutions.

  4. OutcomeReport Generation

    Results are written into results.csv and per-letter.csv logs.

How does this offline test compare with browser-based DocScan?

This test measures bare offline Tesseract 5.5.0 CLI without pre-scaling or neural post-filtering, providing an objective baseline of raw engine capabilities.

What this gives your business

  1. NumberWithout kaz pack: all 271 Kazakh letters fail
    What it means for youStandard Cyrillic models cannot recognize the 9 unique letters
    What aitomaton does about itInstall kaz.traineddata in tessdata directory
    ValueEnables recognition of Kazakh characters
  2. NumberTable layouts: 21% letter error rate
    What it means for youGrid lines and bold font weights degrade OCR accuracy
    What aitomaton does about itUse tessdata_best model and omit eng pack
    ValueReduces misread characters in enterprise receipts
  3. NumberDiacritic confusion: Қ to К, Ұ to У
    What it means for youSubtle glyph accents are dropped in complex layouts
    What aitomaton does about itDictionary-based automated post-processing
    ValueEnsures verified accuracy before filing

What This Synthetic Benchmark Does Not Measure

Does this benchmark guarantee accuracy across all enterprise scans?

No. The dataset is synthetic and focused on font rendering. Low-resolution scans and mobile camera photos require dedicated image binarization and human verification.

How to Reproduce the Benchmark on Your Own Documents

  1. Step 1: Setup EnvironmentInstall Tesseract 5.5, Pillow, numpy, and jiwer in a Python virtual environment.
  2. Step 2: Generate SamplesRun render.py to generate 300 DPI test images from gt ground truth files.
  3. Step 3: Execute OCRRun score.py to execute Tesseract CLI and calculate CER and WER metrics.
  4. Step 4: Inspect Letter MetricsReview per-letter.csv to evaluate specific substitution patterns for your domain.

Frequently Asked Questions

What exactly did this Tesseract OCR benchmark measure?

We evaluated offline Tesseract 5.5.0 CLI across four synthetic Kazakh document samples, measuring CER, WER, and letter substitution errors on nine distinct characters.

Why does Kazakh text fail completely without the kaz model?

Without kaz traineddata, Tesseract defaults to Cyrillic or Latin heuristics, corrupting all 271 Kazakh characters in test samples into Russian phonetic lookalikes.

Why do table layouts and bold headings degrade OCR accuracy?

Grid borders and heavy font weights distort character boundaries, causing Tesseract to misinterpret uppercase Kazakh letters as Cyrillic equivalents.

How does this offline test compare with browser-based DocScan?

This test measures bare offline Tesseract 5.5.0 CLI without pre-scaling or neural post-filtering, providing an objective baseline of raw engine capabilities.

Does this benchmark guarantee accuracy across all enterprise scans?

No. The dataset is synthetic and focused on font rendering. Low-resolution scans and mobile camera photos require dedicated image binarization and human verification.

See it on your own data

Three days free trial or a live demo tailored to your workflows.

Updated:

Book a platform demo

We will show AI agents working on your business processes, review your funnel and answer all your questions.