Test Plan

Test Levels

Unit Tests

Each pipeline stage is tested in isolation with mocked inputs:

  • Ingestion: file discovery, format validation, batch ID generation, PDF page splitting
  • Quality assessment: correct detection of blur, skew, blank pages, noise on synthetic images
  • OCR adapter: engine output correctly mapped to canonical JSON sections
  • Classification: rule-based classifier returns correct type for known text patterns
  • Metadata extraction: date regex, NER, keyword extraction on known text
  • Chunking: boundaries correct for each document type strategy
  • Embedding: vector dimensions match model, payload structure valid

Integration Tests

End-to-end pipeline runs on a small curated test archive (~25 pages):

  • Full run from ingestion through canonical JSON
  • Full run through quality validation gate
  • Full run through chunking and embedding
  • Engine comparison: same pages through each OCR backend, compare canonical output

Quality Benchmarks

  • OCR accuracy against manually transcribed ground truth
  • Classification accuracy on labeled document type set
  • Retrieval precision@5 and recall@5 on known queries
  • RAG answer accuracy — do citations support the answer?

Manual Review

  • Human inspection of RAG answers for groundedness
  • Review queue workflow: can a reviewer quickly find flagged pages?

Test Environment

  • Python with pytest
  • Test fixtures: synthetic images for quality detection, real archive samples for OCR
  • Temporary directories for pipeline output (no filesystem pollution)

Running Tests

pytest tests/unit/
pytest tests/integration/
pytest tests/benchmarks/