## A Real Test of Three Synthetic PDFs: Where the Default Configuration Fails
This is the first of a small set of research pages that do the work an indie developer would actually do before writing code: look at real posted demand, then actually test the tools that would have to do the job, and report what happened — including the parts that don't work.
### 1. Who has this problem, and what are they actually asking for?
Searching the catalog for "PDF table extraction, PDF to Excel, scanned PDF" style demand turns up **11 distinct published features** built from real, budget-posted freelance tasks (Freelancer.com, public postings, PII-scrubbed). Feature counts and representative source links (not every linked task is listed below):
- **PDF Tables to Spreadsheet** (9 linked tasks) — clients with PDFs containing "6 to 15 data tables, all of which include merged cells", financial reports "running over multiple pages", and survey results across "well over ten" separate PDFs. Source postings: [PDF Tables Spreadsheet](https://www.freelancer.com/projects/google-sheets/PDF-Tables-Spreadsheet), [Accurate PDF-Excel Conversion](https://www.freelancer.com/projects/adobe-acrobat/Accurate-PDF-Excel-Conversion-40632143), [Survey PDF Tables Excel](https://www.freelancer.com/projects/adobe-acrobat/Survey-PDF-Tables-Excel). - **PDF Table Data Extraction and Excel Formatting** (6 tasks) — a mix of clean single-table PDFs and messier ones combining "formal tables with narrative paragraphs and bullet-style lists". Example: [PDF Tables Formatted Excel](https://www.freelancer.com/projects/data-analysis/PDF-Tables-Formatted-Excel-40629929). - **50-Page PDF Document Formatting and Word Conversion** (12 tasks) — adjacent demand: full-document retyping, not just tables. Example: [Accurate PDF Word Conversion](https://www.freelancer.com/projects/microsoft-word/Accurate-PDF-Word-Conversion-40627872). - **High-Accuracy Document Transcription and PDF/Image Data Entry** (8 tasks) — general data-entry demand where PDF is one of several source formats alongside scanned images and handwriting. Example: [Typing Scanned Manuscript](https://www.freelancer.com/projects/typing/Typing-Scanned-Manuscript).
Two things the raw task text makes clear that a pure "there's demand" read would miss:
- **A meaningful share of this demand explicitly wants a human, not a tool.** Multiple postings say things like "no scraping tools or automated helpers", "everything must be keyed in by hand", or read as a freelancer's own service listing rather than a client's request (e.g. postings titled in a "jasa data entry..." / "I offer this service" pattern). Some of this demand is not latent demand for automation — it's demand for cheap human labor that happens to route through a PDF, and a seller advertising their own service is not the same signal as a buyer posting a need. - **A cluster of "PDF Tables to Excel"-style postings under Automated PDF Data Extraction to Spreadsheet share near-identical wording and budget tiers.** Similar phrasing and the same price band across different project titles is consistent with a handful of clients (or a templated posting habit), but this research did not run a dedup/identity check across posting accounts, so **the number of independent buyers behind this cluster is unverified, not "a handful."** Task count is not buyer count, and this research cannot currently tell you what the buyer count actually is.
None of this means the demand is fake — the budget-posted tasks are real, source-linked, and PII-scrubbed. It means a naive "N tasks mention PDF tables, therefore N buyers want a $X/month tool" read would overstate the signal.
### 2. Which of these are actually the same underlying need?
Collapsing the 11 features by what's actually being asked for:
1. **Clean, digitally-native tables → spreadsheet** (e.g. financial reports, survey tables). This is the case existing libraries handle best. 2. **Multi-page tables with repeated headers.** A distinct sub-case — several postings explicitly mention tables "running over multiple pages." 3. **Scanned/image PDFs and handwritten manuscripts → structured output.** A harder sub-case that shows up repeatedly ("scanned images", "typed manuscript... over 100 pages", "handwritten manuscripts"). 4. **Whole-document reformatting** (PDF → editable Word, preserving layout) — adjacent to table extraction but a different job: preserving full-document formatting fidelity, not extracting structured rows/columns.
Sub-cases 1–3 are genuinely the same underlying job at different difficulty levels; sub-case 4 is a neighboring but distinct product.
### 3. Why isn't the existing tooling already good enough?
Rather than assume open-source PDF libraries "basically work" (the generic pitch), we actually installed and ran extraction approaches — `pdfplumber`, `pymupdf`, `camelot` (lattice and stream modes), `tabula-py`, and a PyMuPDF+OCR pipeline for the scanned case — against three constructed test documents with known, exact ground truth (a clean invoice table, a 46-row table split across two pages, and a scanned/rasterized inventory table with zero digital text layer). Not every tool was actually run against every scenario — the full matrix, distinguishing what was tested from what failed for environment reasons (missing Java) or simply wasn't attempted:
| Scenario | pdfplumber | pymupdf | camelot (lattice) | camelot (stream) | tabula-py | OCR pipeline | |---|---|---|---|---|---|---| | Clean single-page table | 100% (54/54), 61.6ms | 100% (54/54), 70.8ms | 100% (54/54), 548.7ms | 3.7% (2/54), 34.3ms | env. failure (no Java) | not applicable | | Multi-page table (2 pages) | 96.0% (265/276), 372ms | 96.0% (265/276), 395ms | 100% (276/276), 1050ms | **not tested for this scenario** | env. failure (no Java) | not tested | | Scanned/rasterized table | silent empty result, 3ms | silent empty result, 4ms | empty + UserWarning, 3ms | empty + UserWarning, 2ms | env. failure (no Java) | 91.7% (22/24), 4444ms |
**What actually happened, precisely:**
- On the clean single-page table, `pdfplumber` and `pymupdf` were both perfect (100% cell accuracy) at 61.6ms and 70.8ms respectively, both faster than `camelot`'s lattice mode (also 100% accurate, 548.7ms). **`camelot`'s stream mode misread the table structure**: it pulled the document's title line ("Monthly Financial Summary - Q3 2026") into the table as an extra row, shifting the extracted shape to 10 rows instead of 9. Measured strictly by absolute row/column position, that produced a 3.7% "accuracy" (2 of 54 cells matched) — but that number reflects **position misalignment from the extra row, not that 96.3% of the text was misread.** We did not re-run a content-aligned comparison (stripping the fake header row and re-matching by content), so no corrected accuracy figure is reported here.
| Row | Col | Expected | Actual (stream mode) | |---|---|---|---| | 0 | 0 | Invoice # | (empty) | | 0 | 2 | Date | Monthly Financial Summary - Q3 2026 | | 0 | 4 | Amount | (empty) |
- On the multi-page table, of the three approaches that actually completed this scenario (`pdfplumber`, `pymupdf`, `camelot` lattice — `camelot` stream mode was not tested here and `tabula-py` failed across all scenarios for lack of a Java runtime), none stitches pages together automatically: each returns two separate page-level table objects that a developer has to detect and merge, including stripping a duplicated header row that appears mid-table if you naively concatenate. `pdfplumber` and `pymupdf` each produced the same 11 truncated cells after that merge — every instance was the same pattern, "Needs Improvement" cut to "Needs Improve". The root cause, confirmed by opening the source PDF directly: the "Rating" column in `test2_multipage.pdf` was generated at a fixed 60-point width (`colWidths=[65, 110, 95, 130, 80, 60]` in `generate_samples.py`), while "Needs Improvement" at 8pt Helvetica needs noticeably more room than that — the text visibly overflows the drawn cell border in the source PDF itself, not a marginal few-point edge case. `pdfplumber` and `pymupdf` extract text by position, so anything drawn past the cell's nominal boundary gets cut off there. `camelot`'s lattice mode got all 276 cells right on this scenario -- plausibly related to how it handles cell boundaries differently, though we did not run a controlled tolerance-parameter comparison to confirm that specific mechanism. - On the scanned table, `pdfplumber` and `pymupdf` each returned zero tables silently (empty result, no error or warning); both `camelot` modes also returned zero tables but printed a `UserWarning` explaining the page is image-based. Adding an OCR step (PyMuPDF render + RapidOCR + manual spatial clustering of the raw text boxes into rows/columns) got the shape right and 22 of 24 cells correct, taking 4.44 seconds. That is markedly slower than the ~60-70ms seen on the clean digital-text scenario, but it is not a controlled same-input comparison — the OCR run processed a different (scanned) document, so we do not report a "Nx slower" multiplier. The OCR output also dropped an internal space in one value ("Unit Cost ($)" → "UnitCost ($)"). - `tabula-py` failed in this environment on all three scenarios because it requires a Java runtime — a real deployment cost (and Docker image bloat) that has nothing to do with extraction quality.
The honest conclusion: **for the clean, single-page digital table tested here, `pdfplumber` and `pymupdf` handled it perfectly at low latency.** For the other scenarios — multi-page stitching, scanned documents, and boundary-overflow truncation — the default configuration of these libraries breaks in specific, mostly-silent, non-obvious ways on the documents we constructed. Whether that generalizes beyond these three synthetic documents is untested.
### 4. What would a piece of software actually need to do?
Based directly on what broke in section 3 on these three documents, a viable product would need to address, at minimum:
- **A modality router** that checks text-layer density before picking a strategy, so scanned pages get routed to OCR instead of silently returning nothing. - **A layout mode selector** (lattice vs. stream, or equivalent) with a sane default and a way to detect when the chosen mode has plausibly failed (e.g. a page title showing up as row 0). - **Boundary padding logic** so text that visibly overflows a too-narrow column (as in the "Rating" column above) isn't silently amputated. This is a hypothesis based on the failure pattern observed above, **not a tested fix** — we did not re-run the multi-page scenario with adjusted boundary parameters to confirm it resolves the 11 truncations. [Camelot's own advanced-usage documentation](https://camelot-py.readthedocs.io/en/stable/user/advanced.html) documents configurable table regions and column-boundary parameters that a real implementation would need to actually test against this failure mode, rather than assuming the library "can't do it" by default. - **Cross-page table stitching** that compares column geometry and header text between consecutive page-level tables to merge them and drop duplicate headers. - **An OCR fallback with grid reconstruction**, not just raw OCR — bounding boxes need to be clustered into rows/columns, which breaks down further on multi-line cells, empty cells, or skewed scans (none of which this test even covered).
None of these are exotic — they're the specific, concrete engineering gaps this test surfaced, not "call an LLM API and hope." Whether they're sufficient on real, more varied target documents is a separate, untested question.
### 5. How would you actually validate this before building it?
- Pull a larger, real sample of PDFs from the segment you'd target (financial reports vs. survey exports vs. scanned archives) rather than one synthetic file per category — every number in section 3 came from three constructed documents and should be re-measured on real target documents before being trusted. - Talk to a handful of the people currently posting "PDF to Excel" tasks and ask directly what accuracy bar and turnaround would actually make them switch from a person. Section 1's manual-labor-preference postings suggest that a cheap human option may already be "good enough" for some of this demand regardless of tooling quality — this is a real open question, not a settled one. - Build and test the boundary-padding fix from section 4 first — it is one candidate worth validating first, since the 11-cell truncation pattern was the most repeatable failure observed on the multi-page scenario — before investing in OCR, which was the slowest path tested (4.44s) on a different input; we do not rank OCR's 91.7% cell accuracy against the other scenarios' numbers, since they were run on different documents and are not directly comparable. - Define a stop condition upfront, but set the actual accuracy/cost thresholds from pilot data on real target documents, not from this experiment's three synthetic files — this research does not establish what accuracy bar or per-document cost real buyers would actually accept.
### 6. So — worth building?
**Not a clean yes or no; it depends on which slice, and this research only tested three synthetic documents.** For the clean, single-page digital table case tested here, off-the-shelf libraries at default settings already handled it perfectly, which limits how much a paid product could charge for that slice alone. The more defensible product surface is the combination of boundary-overflow handling + multi-page stitching + an honest OCR fallback — none of which the libraries did out of the box on these documents, though whether Camelot's documented region/parameter options close that gap is untested here. What this research does **not** support is the generic pitch of "wrap pdfplumber in a SaaS and charge per page" as a validated claim, nor does it support a specific accuracy or pricing threshold as market-validated — both need a real pilot on real target documents (section 5) before any commercial claim.
*Evidence sources: 11 published demand-signal pages built from real, budget-posted, PII-scrubbed Freelancer.com tasks (linked above by feature; independent-buyer count for the near-identical-wording cluster is unverified). Extraction benchmark: pdfplumber 0.11.10, pymupdf 1.28.2, camelot-py 2.0.0, tabula-py 2.10.0, rapidocr-onnxruntime 1.4.4, tested against three constructed documents with exact known ground truth on 2026-09-09; full tool×scenario matrix, scripts, and raw JSON results are archived alongside this research under docs/low-value-2026-09-05/ (pdf-extraction-scripts/, including the README with reproduction steps) and available as a controlled attachment to pilot readers. This article and the underlying benchmark have not yet been read by outside developers; verdicts above should be treated as a working hypothesis pending that review, not a final recommendation.*