Town Records OCR Pipeline
This document describes the proposed OCR workflow for processing scanned Town Records archive documents into structured data suitable for semantic search, Retrieval-Augmented Generation (RAG), and future document intelligence applications.
The workflow is designed for historical documents with inconsistent formats and scan quality, including books, letters, invoices, checks, contracts, church records, meeting minutes, ledgers, school memorandums, and other archival materials.
The goal is not only to extract text, but to preserve structure, quality information, document type, metadata, and provenance throughout the pipeline.
Workflow Summary
Scanned Documents
│
▼
Image Ingestion
│
▼
Image Quality Assessment
│
▼
OCR Engine Evaluation / Selection
│
▼
OCR and Layout Extraction
│
▼
Canonical JSON Generation
│
▼
Document Classification
│
▼
Metadata Extraction
│
▼
Quality Validation
│
▼
Semantic Chunking
│
▼
Embedding Generation
│
▼
Qdrant Vector Database
│
▼
RAG / LLM Search
1. Image Ingestion
Purpose
The first stage loads scanned archive documents from the source archive.
Input
Single image
Folder of images
PDF document
Folder containing multiple document folders
Supported Formats
.jpg
.jpeg
.png
.tif
.tiff
.pdf
Output
Each source item is assigned processing metadata, including:
{
"source_path": "...",
"source_filename": "...",
"batch_id": "...",
"page_number": 1,
"input_type": "image"
}
This metadata is preserved throughout the pipeline for citation, review, and traceability.
2. Image Quality Assessment
Purpose
Each page is evaluated before or during OCR to identify quality issues that may reduce extraction accuracy.
Detected Issues
Blur
Low contrast
Washed-out text
Rotation
Skew
Noise
Bleed-through
Blank pages
Damaged paper
Poor resolution
Output
{
"image_quality": {
"quality_level": "questionable",
"blur_detected": true,
"low_contrast": true,
"rotation_detected": false,
"needs_review": true
}
}
Reason
Poor-quality pages should not be blindly trusted. Quality information helps determine whether a page is safe for RAG, requires alternate OCR, or should be flagged for human review.
3. OCR Engine Evaluation / Selection
Purpose
Multiple OCR and Vision Language Model engines will be evaluated against representative archive samples.
Candidate OCR Engines
Docling
Unlimited-OCR
Qwen Vision
GLM-OCR
Additional engines as identified
Candidate Structuring Engines
Docling native document model
NuExtract
LangExtract
Additional extraction frameworks
Output
The evaluation will identify which engine or combination of engines performs best for:
Printed books
Typewritten documents
Handwritten records
Ledgers and tables
Invoices and checks
Letters
Meeting minutes
Forms
Poor-quality scans
Reason
No single OCR engine is expected to perform best across all document types. The workflow is designed so OCR backends can be swapped or routed without changing downstream systems.
4. OCR and Layout Extraction
Purpose
The selected OCR engine extracts text and layout information from each page.
Input
Scanned page image
Image quality metadata
Selected OCR backend
Output
The OCR stage attempts to extract:
Page text
Markdown
Reading order
Text regions
Bounding boxes
Tables
Images
Captions
Layout structure
OCR confidence or quality indicators
Example region output:
{
"region_id": "page_001_region_003",
"type": "paragraph",
"text": "...",
"bbox": {
"x0": 120,
"y0": 340,
"x1": 980,
"y1": 420
},
"page_number": 1
}
Reason
Plain OCR text is not enough for reliable retrieval. Layout, reading order, and bounding boxes allow the system to preserve document structure and support future citation or review tools.
5. Canonical JSON Generation
Purpose
All OCR output is converted into a standardized canonical document format.
Input
OCR text
Markdown
Layout regions
Bounding boxes
Tables
Images
Source metadata
Quality metadata
Output
Canonical JSON
Markdown file
Plain text file
Processing summary
Canonical JSON Sections
{
"source": {},
"page": {},
"ocr": {},
"layout": {},
"regions": [],
"tables": [],
"images": [],
"classification": {},
"metadata": {},
"quality": {},
"processing": {}
}
Reason
Different OCR engines produce different output formats. The canonical format creates one stable interface for all downstream stages, including classification, metadata extraction, chunking, vector indexing, and RAG.
6. Document Classification
Purpose
Each document or page is classified by document type before indexing.
Example Document Types
Book page
Letter
Invoice
Check
Contract
Church record
Meeting minutes
School memo
Ledger
Form
Unknown
Classification Signals
OCR text
Layout structure
Number density
Table presence
Region patterns
Headers and footers
Common keywords
VLM-based classification when needed
Output
{
"classification": {
"document_type": "meeting_minutes",
"confidence": 0.87,
"signals": [
"contains meeting-related terms",
"paragraph-based layout",
"date detected near top of page"
]
}
}
Reason
Document type affects how content should be chunked, searched, reviewed, and interpreted. A ledger should not be processed the same way as a letter or book page.
7. Metadata Extraction
Purpose
Structured metadata is extracted from OCR text and document layout.
Metadata Examples
Dates
People
Organizations
Locations
Document titles
Subjects
Monetary amounts
Signatures
Page numbers
Topics
Output
{
"metadata": {
"dates": ["1898"],
"people": [],
"organizations": [],
"locations": ["Newington"],
"topics": ["town records"]
}
}
Reason
Metadata enables filtered search and improves retrieval quality. For example, searches can be limited to meeting minutes, church records, specific years, or documents mentioning a particular organization.
8. Quality Validation
Purpose
The pipeline determines whether extracted text is reliable enough for RAG.
Output
{
"quality": {
"ocr_quality": "good",
"safe_for_rag": true,
"needs_review": false
}
}
Pages with poor extraction may be marked as:
{
"quality": {
"ocr_quality": "poor",
"safe_for_rag": false,
"needs_review": true
}
}
Reason
Low-quality OCR should not be embedded into the vector database without review. This prevents unreliable text from producing incorrect search results or unsupported RAG answers.
9. Semantic Chunking
Purpose
Validated canonical documents are divided into semantically meaningful chunks.
Chunking Strategy by Document Type
Books -> paragraph or section chunks
Letters -> header, body, signature chunks
Meeting minutes -> agenda, topic, motion, vote chunks
Ledgers -> row or table-based chunks
Contracts -> clause or section chunks
Forms -> field-group chunks
Output
{
"chunk_id": "doc_001_page_003_chunk_002",
"text": "...",
"document_type": "letter",
"page_number": 3,
"source_path": "...",
"metadata": {},
"bbox_refs": []
}
Reason
Fixed-size chunking can split related information apart. Semantic chunking preserves natural document boundaries and improves retrieval accuracy.
10. Embedding Generation
Purpose
Each semantic chunk is converted into a vector embedding.
Input
Chunk text
Chunk metadata
Source references
Output
{
"chunk_id": "...",
"embedding": "[vector]",
"payload": {
"text": "...",
"document_type": "...",
"page_number": 1,
"source_path": "..."
}
}
Reason
Embeddings allow semantic search over the archive, enabling queries based on meaning rather than exact keyword matches.
11. Qdrant Vector Database Ingestion
Purpose
Embeddings and metadata are stored in Qdrant.
Stored Data
Vector embedding
Chunk text
Document type
Page number
Source path
Dates
People
Organizations
Locations
Quality indicators
Canonical document references
Reason
Qdrant supports semantic similarity search with metadata filtering. This allows searches such as:
Find church records mentioning a person.
Search only meeting minutes.
Find documents from a specific year.
Retrieve documents related to a topic.
12. RAG / LLM Search
Purpose
The final stage enables natural language question answering over the processed archive.
Workflow
User question
│
▼
Query embedding
│
▼
Qdrant retrieval
│
▼
Top matching chunks
│
▼
LLM answer generation
│
▼
Answer with citations
Output
Answer
Relevant source chunks
Document citations
Page references
Confidence / review indicators
Reason
The LLM should answer using retrieved archival context rather than unsupported general knowledge. Source references allow answers to be traced back to the original scanned documents.
Final Outputs
The complete workflow produces:
Canonical JSON files
Markdown files
Plain text files
OCR quality reports
Document classification results
Extracted metadata
Semantic chunks
Qdrant vector collections
RAG answers with citations
Review queue for low-quality pages
Initial Prototype Scope
The first prototype will focus on validating the workflow using a representative sample of archive documents.
The prototype will include:
OCR engine comparison
Canonical JSON generation
Basic image quality scoring
Basic document classification
Semantic chunking
Qdrant ingestion
Simple RAG query interface
Source citations
Review flags for poor OCR
Advanced features such as full document routing, perfect handwriting recognition, large-scale automation, and knowledge graph construction will remain future extensions after the workflow is validated.