Self-hosted OCR deep dive: PaddleOCR, DocTR and the preprocessing that actually matters

OCR benchmarks are run on clean, flat, well-lit documents. Production OCR is run on a receipt that lived in a wallet for three weeks, was photographed at 11pm under a kitchen light, and arrives rotated 4 degrees with a thumb in the corner. This post is about closing that gap.

Stage one: OpenCV before anything reads a character

Recognition models are only as good as the pixels you feed them. Every document that enters FastVision goes through a deterministic OpenCV pipeline before an engine ever sees it.

• Deskew — estimate rotation from text baselines and correct it; even 2° of skew measurably hurts recognition
• Denoise — non-local means filtering to remove sensor noise without eroding stroke edges
• Contrast normalization — CLAHE to recover text from shadows and uneven lighting
• Adaptive binarization — local thresholding that handles gradients a global threshold would smear

Stage two: two engines, one job

PaddleOCR is our primary engine: its detection-plus-recognition architecture with layout awareness is the strongest open-source stack we've measured on mixed real-world documents. But no single engine wins everywhere, so DocTR rides along as a fallback. When PaddleOCR's aggregate confidence on a page drops below threshold, the worker re-runs the page through DocTR and keeps the better result. You can also pin an engine per request with the engine parameter.

job = fv.ocr.pdf(
    "bank_statement.pdf",
    doc_type="bank_statement",
    engine="doctr",   # pin the fallback engine explicitly
    outputs="json,pdf",
)

Bounding boxes are the product

Plain text answers 'what does it say?' Bounding boxes answer 'where does it say it?' — and that second question is where products live. Field-level review UIs, redaction, table reconstruction, click-to-verify: all of them are built on block coordinates and per-block confidence, which every FastVision result includes.

{
  "text": "TOTAL $38.30",
  "confidence": 0.998,
  "bbox": [412, 1088, 631, 1121]
}

The searchable-PDF trick

Request outputs=pdf and we rebuild your original document with an invisible text layer positioned using those same bounding boxes. The scan looks identical, but Cmd+F, copy-paste and screen readers all work. It's the highest-leverage feature-per-line-of-code in the platform.

All of this runs in mock mode in development — deterministic fake results with the exact production shape — so your integration tests never need a GPU. Flip AI_MODE=real in production and the same code paths load the real models.