The best way to convert PDFs to markdown for feeding into large language models depends on the PDF type, but the current strong default for mixed and complex documents is a layout-aware parser such as Docling, which detects headings, tables, reading order and figures before rendering clean markdown that preserves document structure rather than a flat text dump. For simple, text-native PDFs, lighter libraries that extract the embedded text layer directly are faster and sufficient, since there is no need for OCR or heavy layout inference when the text is already digital. Scanned or image-based PDFs require an OCR or vision-language model step first to produce text before markdown conversion can happen at all, and quality here directly limits everything downstream. Preserving tables as markdown tables rather than collapsing them into paragraphs matters significantly for retrieval-augmented generation, since a flattened table loses the row-column relationships a model needs to answer numeric questions correctly. Chunking the resulting markdown along heading boundaries, rather than by a fixed character count, typically improves retrieval quality because each chunk stays topically coherent. Testing conversion quality against a sample of the actual document set catches layout edge cases before they affect a full pipeline. Nanobase AI builds PDF-to-markdown ingestion pipelines as part of the RAG systems it deploys for customers.
Why markdown structure, not just text, matters for LLM ingestion
Converting a PDF to plain text loses information that markdown preserves: heading hierarchy, table structure, list nesting and figure placement, all of which a large language model uses to understand document structure during retrieval or summarization. A flat text dump forces a downstream LLM or retrieval system to infer structure it can no longer see directly, while well-formed markdown with proper headings and table syntax gives the model explicit structural signal, which measurably improves both retrieval relevance and answer accuracy in a RAG pipeline built on top of it. This is why the conversion step deserves real engineering attention rather than being treated as a disposable preprocessing detail.
Comparing PDF-to-markdown tools
| Tool | Table handling | OCR fallback | Best fit |
|---|---|---|---|
| Docling | Strong, preserves table structure as markdown tables | Built-in for scanned pages | Complex, mixed-layout documents |
| Direct text-layer extraction (e.g. pdfplumber) | Basic, often loses structure | None, native text only | Simple, clean, text-native PDFs |
| Unstructured-style parsers | Moderate, varies by document | Often integrated | General-purpose, moderate complexity |
| Vision-language model direct-to-markdown | Strong for complex layouts, higher cost | Native (reads image directly) | Highest layout complexity, lower volume |
Docling and similar layout-aware parsers currently represent a strong default as of 2026 for mixed and complex documents, since they detect headings, tables and reading order before rendering markdown, while lighter libraries remain faster and sufficient for simple, text-native PDFs with no complex layout to preserve.
Chunking strategy after conversion
Chunking the resulting markdown along heading boundaries rather than a fixed character count typically improves retrieval quality noticeably, because each chunk stays topically coherent instead of splitting mid-sentence or mid-table at an arbitrary character limit. Tables deserve special handling in chunking: splitting a table across two chunks breaks the relationship between a value and its header, so tables should generally be kept whole within a single chunk even if that chunk runs longer than the target size for surrounding text. A short technique that works well in practice is prefixing each chunk with its parent heading path, so a chunk about "Section 4.2, Payment Terms" retains that context even after being separated from the rest of the document during retrieval.
A layout-aware conversion example
docling convert contract.pdf --to markdown --output ./converted/Running conversion through a layout-aware tool like this on a representative sample of the actual document set, before committing to a full ingestion pipeline, surfaces layout edge cases, unusual table structures, multi-column sections, embedded images with captions, that a quick manual spot-check on one clean document would miss entirely.
Frequently asked questions
Do scanned PDFs need OCR before markdown conversion can happen?
Yes, image-based or scanned PDFs need an OCR or vision-language model pass to produce text first, since there is no embedded text layer to convert directly, and the quality of that OCR step directly limits everything downstream in the markdown output.
Does preserving tables as markdown actually improve RAG answer quality?
Yes, meaningfully so for questions involving numeric or tabular data, since a flattened table loses the row-column relationships a model needs to correctly associate a value with its label; a markdown table keeps that structure explicit for the retrieval and generation steps.
How is conversion quality validated before full deployment?
Testing conversion against a representative sample of the actual document set, checking that headings, tables and reading order render correctly, catches layout edge cases before they affect a full ingestion pipeline; a single clean sample document is not sufficient for this check.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, builds PDF-to-markdown ingestion pipelines as part of the RAG systems it deploys for customers, choosing the conversion tool and chunking strategy based on actual document complexity. See the RAG vs fine-tuning decision guide for how this ingestion step fits a broader retrieval architecture, and try a live demo of a working pipeline.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.