Marker 2 (Datalab): document conversion benchmarks, accuracy, cost, deployment checklist

TL;DR, Marker 2 in a sentence

Marker 2 (v2.0.0), released July 20, 2026 by Datalab, is a full rewrite of an open-source document conversion pipeline that trades accuracy, cost and speed across three modes: balanced (GPU + Surya VLM), fast (lighter VLM use), and –disable_ocr (text-layer only). On AI2’s olmOCR-bench (1, 403 PDFs, ~8, 400 unit tests) Datalab reports Marker balanced at 76.0% overall with 2.9 pages/second on a single B200 GPU. The numbers are reproducible via the Marker repository’s open harness, but they depend strongly on hardware, concurrency settings and corpus.

What Marker 2 actually does and what changed

  • Inputs & outputs: accepts PDF, image, PPTX, DOCX, XLSX, HTML and EPUB; emits markdown, JSON, HTML or chunks.
  • Core rebuilds: Surya OCR 2. A 20M-parameter “fast layout” model (CPU-friendly) for columns, tables and reading order. A rebuilt pdftext extractor that Datalab reports is 3× faster than Marker v1’s pdftext.
  • Operational modes (vendor wording retained):
    • balanced, GPU-optimized. “Surya VLM handles layout” and re-OCRs pages with poor embedded text.
    • fast, lightweight detector plus pdftext, minimal VLM calls.
    • –disable_ocr, “No VLM calls at all, pure text-layer extraction.”
  • Device-aware default: “Mode now defaults by device: balanced on GPU, fast on CPU/MPS. Override with –mode.”
  • Packaging/runtime: requires Python 3.10+ and now uses hatchling as the PEP 517 build backend. pip install marker-pdf remains the standard install path. Structured-extraction converters were removed. Datalab points to hosted APIs or a --use_llm workflow for those needs.

Benchmark headlines (what Datalab reported)

All benchmark scores and throughput numbers below are Datalab’s runs on AI2’s olmOCR-bench (Ai2 / Allen Institute for AI). Datalab published the runs and an open harness in the Marker repository that includes competitor runners for MinerU, Docling and LiteParse, so you can reproduce the experiments on your hardware.

  • Marker 2, balanced: 76.0% overall. 83.5% on born-digital PDFs. Sustained 2.9 pages/second (pg/s) on a single B200 GPU. Effective per-page latency reported ~341 ms/page.
  • Marker 2, fast: 66.6% overall. 71.6% born-digital. Sustains 7.4 pg/s. Latency ~134 ms/page.
  • Marker 2, fast –disable_ocr: 43.6% overall. 55.8% born-digital. CPU-only at 23.7 pg/s. Latency ~42 ms/page.
  • MinerU (pipeline backend): 72.7% overall at 0.54 pg/s. Datalab notes MinerU also ships a VLM backend that it did not include in this pipeline table.
  • Docling: 50.3% overall at 2.1 pg/s (64.0% on born-digital).
  • LiteParse: 22.4 overall on CPU. 20.4 with OCR off. With OCR disabled LiteParse reports 1, 721 pg/s, which is much higher throughput but far lower accuracy on scans and complex layouts.

Important measurement notes and limitations (reported by Datalab or absent from their public run output):

  • Datalab reports the sustained numbers on “a single B200 GPU”. The vendor, memory footprint, driver/CUDA versions and full machine image were not specified in the public summary. Those details matter for raw throughput. Check the Marker repo run logs when you reproduce runs.
  • Sustained throughput, for example 2.9 pg/s for balanced, is a concurrent, multiplexed measurement. Datalab contrasts it with a single-stream rate of ~0.3 pg/s on the same hardware and attributes the gap to a model architecture that multiplexes many thin CPU workers against a single Surya inference server. The exact worker count, batch size and RPC concurrency used for the reported sustained number were not listed in the summary. Read them from the harness when you repeat the experiment.
  • All scores above come from AI2’s olmOCR-bench (1, 403 PDFs, ~8, 400 pass/fail unit tests across eight categories). Benchmarks are useful baselines, but results change with corpus mix, preprocessing and tuning.

Where Marker 2 wins, and where full-page VLMs still hold the edge

Marker balanced substantially narrows the gap to full-page VLM systems on many business-relevant document types. Datalab’s table places Marker balanced only 0.4 points behind Gemini Flash 3.5 overall (76.0 vs 76.4). Marker also outperforms Gemini Flash 3.5 on born-digital documents (83.5 vs 79.1).

Full-page VLMs remain better on low-quality scans and math-heavy pages. Datalab’s VLM-tier numbers for comparison include Chandra 2 at 85.8, AI2 olmOCR 2 at 82.4 and dots.ocr 1.5 at 83.9 on the same evaluation table.

Selected per-category behavior from Datalab’s Marker mode explorer (highlights):

  • arXiv math: balanced 83.9. fast 23.4. fast –disable_ocr 0.0.
  • Tables: balanced 73.4. fast 69.0. disable_ocr 46.1.
  • Multi-column text: balanced 76.6. fast 76.0. disable_ocr 67.0.
  • Headers & footers: balanced 95.9. fast 93.2. disable_ocr 92.8.
  • Old scans (general): balanced 43.2. fast 43.2. disable_ocr 14.3.

Practical takeaway: for born-digital, multi-column business reports and many enterprise PDFs, Marker balanced or fast is a strong, cost-aware choice. For archives full of low-quality scans, handwriting or heavy math notation, full-page VLM approaches still win unless you accept some loss in accuracy.

Licensing, packaging and adoption considerations

  • Model weights and commercial thresholds: Marker’s codebase is Apache 2.0, but its model weights are distributed under a modified AI Pubs OpenRAIL-M license. The weights are free for research, personal use and startups under $5M funding/revenue. Above that, commercial use of the weights requires a paid license, as stated by Datalab. MinerU’s license is an Apache-derived license with added commercial limits. MinerU requires a separate commercial license above 100M MAU or $20M monthly revenue and asks online services to disclose that usage. Docling’s code is MIT-licensed, with models tracked in their original packages. LiteParse is open source. LlamaIndex/run-llama position LlamaParse as a paid cloud option for hard documents.
  • Operational surface: Marker’s concurrency model multiplexes many CPU workers against a single Surya inference server. That increases sustained throughput but adds an inference-server operational piece to run, monitor and autoscale. Datalab removed some structured-extraction converters and recommends hosted APIs or --use_llm for those features instead of shipping them in-marker.
  • Packaging/narrow compatibility: requires Python 3.10+. Packaging now uses hatchling as the build backend. The published pip package remains marker-pdf.

How to read and reproduce these numbers (practical guidance)

  • Always run the benchmark on a representative sample of your own corpus. AI2’s olmOCR-bench is a standardized baseline (1, 403 PDFs, ~8, 400 unit tests across eight categories), but real-world mixes vary.
  • Check the Marker repository’s open harness. It includes the runner code and competitor runners to reproduce Datalab’s runs. Look for the exact command-line flags, concurrency settings, warm-up runs and any image downscaling toggles before comparing results.
  • Confirm hardware specifics when reproducing. Datalab reports results on “a single B200 GPU” but does not publish vendor, memory, driver/CUDA or machine-image details in the summary. If you need apples-to-apples comparisons, reproduce the run and surface machine-image, CUDA/cuDNN and driver versions in your logs.
  • Model vs system cost: throughput (pg/s) is not the whole cost story. If you self-host Marker balanced you pay GPU hours, infra and ops. If you use a hosted full-page VLM API you pay per-page API fees. Build a simple cost-per-page model by dividing (infrastructure + ops + licensing) by pages processed, or compare against your expected API bill at projected volumes.

Decision checklist for engineering, product and legal teams

  • Sample 1-2k pages from production that represent your edge cases (math, low-quality scans, tables).
  • Run AI2’s olmOCR-bench and your sample through the Marker harness. The repo includes competitor runners to measure accuracy and throughput on your hardware.
  • Measure end-to-end cost per page: include infra (GPU/CPU), storage, operations and any license fees for model weights or hosted services.
  • Confirm licensing constraints for model weights and hosted APIs. If you exceed thresholds (Marker’s $5M startup clause, MinerU’s MAU/revenue thresholds), plan for procurement or alternate stacks.
  • Test failure modes and operational needs: autoscaling Surya inference servers, cold-start latency and monitoring of OCR drift on new document types.

Worked example, rough sizing for a common goal

If you need to process 1, 000, 000 pages per month, estimate required throughput and nodes as follows:

  • Required steady throughput = 1, 000, 000 pages / (30 days * 24 h * 3600 s) ≈ 0.385 pages/second.
  • If you aim to use Marker balanced on a single B200 GPU (Datalab reports ~2.9 pg/s sustained), one B200-equivalent GPU host provides more than enough sustained throughput for that volume (0.385 / 2.9 ≈ 0.13, i.e., one machine at low utilization).
  • If you choose Marker –disable_ocr (CPU-only) at 23.7 pg/s reported by Datalab, a single CPU host also covers the load with lots of headroom.

Those arithmetic steps are useful because licensing and per-page API costs, not raw throughput, often determine the final choice. Always plug your own infra prices and licensing commitments into this same formula.

Key questions, short, honest answers

  • How accurate is Marker 2 compared to competitors on AI2’s benchmark?

    According to Datalab’s runs on AI2’s olmOCR-bench, Marker balanced scores 76.0% overall, beating MinerU’s pipeline backend (72.7%) and Docling (50.3%), while trailing some full-page VLMs like Chandra 2 (85.8) in math/scan-heavy categories.

  • How fast is Marker 2 in practice?

    Datalab reports Marker balanced sustaining ~2.9 pages/second on a single B200 GPU (concurrent multiplexed measurement). Fast mode reaches 7.4 pg/s and fast –disable_ocr runs on CPU at 23.7 pg/s. Exact latency and throughput depend on hardware, concurrency settings and corpus.

  • Can I reproduce these results for my documents?

    Yes, Datalab provides an open harness in the Marker repository that includes the AI2 olmOCR-bench runner and competitor runners so you can run the same tests or run them on your own corpus to validate performance.

  • Which mode should I try first in production?

    Use device-aware defaults as a starting point: balanced on GPU for mixed/scan-heavy corpora; fast on CPU for born-digital workloads. Validate on a representative sample and model cost per page before committing.

  • Do licensing terms restrict commercial deployment?

    Yes. Marker’s code is Apache 2.0, but model weights use a modified OpenRAIL-M that requires a paid license above $5M funding/revenue (per Datalab). MinerU and others have their own thresholds, and hosted API agreements may add separate costs.

Perspective for product leaders

Marker 2 is a pragmatic, pipe-lined approach that gives engineering teams controllable knobs for accuracy, throughput and cost. For high-volume, born-digital document sets, Marker’s CPU and fast modes are compelling. For mixed or scan-heavy archives, balanced mode narrows the gap with full-page VLMs, but expect higher GPU cost and some operational complexity running the inference server. Licensing cliffs change procurement timelines, so include legal early when you cross those thresholds.

“Scores: olmOCR-bench, as reported by Datalab · verified July 22, 2026.”, embedded widget footnote cited by Datalab

0