Related Pages

Related topics: Pipeline Architecture and Extensibility, Output Formats, Deployment, and Operational Pitfalls

Overview and Getting Started

Marker is a Python toolkit that converts PDF documents into structured Markdown, HTML, and JSON outputs. It combines deep-learning-based layout analysis, OCR via the surya library, and rule-based heuristics to reconstruct reading order, extract tables and equations, and merge text blocks into clean output. The project targets researchers and engineers who need high-fidelity PDF-to-markup conversion with optional LLM-based cleanup.

Purpose and Scope

Marker is designed to bridge the gap between raw PDF extraction (which often produces jumbled text) and high-quality structured output. The core value proposition includes:

• Layout-aware extraction that preserves document structure (headings, lists, tables, code, equations)
• GPU-accelerated OCR through surya for both printed and handwritten text
• Optional LLM post-processing to refine tables, forms, and merged text spans
• Multiple output formats including Markdown, HTML, and JSON with metadata
• Configurable pipeline supporting custom processors and extractors

The library is intended for batch processing of document collections, single-file conversions, and interactive exploration via a Streamlit GUI.

Installation and Entry Points

Marker is distributed via PyPI as marker-pdf. The basic installation pulls in PyTorch, the surya OCR stack, and PDF parsing libraries:

pip install marker-pdf

The package exposes several command-line entry points defined in the scripts module:

Source: marker/scripts/convert.py:1-50 Source: marker/scripts/convert_single.py:1-30 Source: marker/scripts/run_streamlit_app.py:1-20

Core Conversion Workflow

The conversion pipeline is orchestrated by PdfConverter and follows a sequential stage model. Each stage produces intermediate data consumed by the next.

flowchart LR
    A[PDF Input] --> B[Document Extraction]
    B --> C[Layout Detection]
    C --> D[OCR / Text Recognition]
    D --> E[Line Merging]
    E --> F[Block Reordering]
    F --> G[Output Generation]
    G --> H[Markdown / HTML / JSON]

The PdfConverter class accepts configuration objects for each stage, allowing users to override default processors, enable/disable OCR, tune batch sizes, and inject custom LLMs for cleanup tasks.

Source: marker/converters/pdf.py:1-80

Basic Usage Patterns

Single PDF conversion is the most common entry point. The script reads a file path, instantiates a converter with default settings, and writes output to disk.

Source: marker/scripts/convert_single.py:1-60

Programmatic API allows embedding marker in larger applications. Users instantiate PdfConverter and call its conversion method directly, receiving a list of output documents that can be serialized or post-processed.

Batch processing with chunking is recommended for documents exceeding available memory. The chunk_convert script splits the input into page-range segments, processes each independently, and reassembles the final output while preserving cross-references.

Source: marker/scripts/chunk_convert.py:1-40

Configuration and Customization

Marker accepts configuration through a dictionary-based system passed to the converter constructor. Key configuration areas include:

• Device selection (cpu, cuda, mps) for GPU acceleration
• Output format selection (markdown, html, json)
• LLM integration for table merging, form extraction, and text cleanup
• Page range filtering to process subsets of large documents
• Batch size tuning to balance throughput and memory consumption

Source: README.md:1-200 Source: marker/converters/pdf.py:80-150

Known Operational Considerations

Community reports highlight several operational pitfalls that new users should anticipate:

• Memory growth on repeated conversions: Reusing a single PdfConverter instance across many PDFs can cause RSS to climb unboundedly (60 GB observed for 10 large documents on CUDA). Users processing many files should instantiate fresh converters or rely on the chunked pipeline.
• macOS performance regression: Versions 1.9.0+ show ~20x slowdown on Apple Silicon compared to 1.8.0; users on Mac may consider pinning to older releases or investigating MPS-specific issues.
• Memory exhaustion on large/complex PDFs: Documents with tens of thousands of layout blocks can trigger OOM errors during fork-exec stages; splitting via marker_chunk mitigates this.
• LLM rate limiting: When using Gemini or other hosted LLMs for cleanup, no built-in rate limiting exists; users must implement external pause/retry logic.
• Missing optional dependencies: Some environments require explicit installation of psutil and other system-util packages not always pulled by default.

These considerations are documented in issues #1040, #960, #1032, #490, and #818 respectively.

Next Steps

After installation, users typically:

1. Run marker_single path/to/file.pdf to verify the pipeline works on a sample document
2. Experiment with output formats by passing --output_format html or --output_format json
3. For large batches, switch to marker with a directory input or use marker_chunk for oversized files
4. Explore the Streamlit GUI via marker_gui for interactive parameter tuning
5. For production workloads, review the configuration dictionary in marker/converters/pdf.py to understand available toggles

The project README provides an exhaustive command-line reference and examples for Docker deployment, Modal cloud hosting, and integration with external LLM providers.

Related Pages

Related topics: Overview and Getting Started, LLM Integration and Hybrid Mode, Output Formats, Deployment, and Operational Pitfalls

Pipeline Architecture and Extensibility

The Marker pipeline is structured around a pluggable Provider abstraction. Every input document, regardless of format (PDF, image, EPUB, HTML), is converted into a normalized Document object that downstream stages (layout detection, OCR, text extraction, merging, rendering) consume. This design decouples format parsing from the rest of the conversion pipeline and makes Marker extensible: new input formats can be supported by adding a new provider module without changing the core converters.

Provider Registry and Dispatch

The provider layer is defined in marker/providers/__init__.py, which exposes the public classes and functions used by converters. The central abstraction is a factory function that maps an input filepath to a concrete provider implementation based on file extension or MIME type. This dispatch layer keeps the rest of the codebase format-agnostic.

Source: marker/providers/__init__.py

The registry is small and explicit. Each provider module exports a single class that subclasses a base provider interface. The interface defines a contract for:

• Reading the source file from disk or a remote location.
• Yielding page-level metadata (page dimensions, rotation, counts).
• Producing a uniform Document model containing rendered images and embedded text streams.

Source: marker/providers/pdf.py

By isolating format-specific concerns inside the provider, the downstream stages (OCR, layout, merger, renderer) operate against the same Document schema regardless of whether the input was a 700-page scanned PDF or a single PNG image. This uniformity is what enables features like --output_format json and --html_tables_in_markdown to work identically across input formats.

Provider Implementations

Each provider in the marker/providers/ directory handles one input family:

Source: marker/providers/image.py Source: marker/providers/document.py Source: marker/providers/epub.py Source: marker/providers/html.py

The PdfProvider is the most complex because it must coordinate two distinct text sources: the embedded text stream (via pdftext) and the OCR result from surya. It decides per-block whether to use the embedded text or invoke OCR, and this decision is exposed to downstream stages through metadata on each block.

Extending the Pipeline

To add a new input format, an integrator implements a new provider class conforming to the base interface and registers it in marker/providers/__init__.py. The rest of the pipeline (layout, OCR, text, merger, renderer) requires no changes because it consumes only the normalized Document schema.

This extensibility is also relevant to operational concerns raised by the community. Issue #1040 reports that PdfConverter memory grows unbounded when reused across many PDFs in a loop (reaching 60 GB RSS for 10 large documents). The provider abstraction exposes the lifecycle boundary: each __call__ creates a fresh provider instance tied to the document, so re-instantiating the converter per PDF — rather than reusing it — reclaims resources. Issue #1038 about VRAM conflict with a colocated local LLM is similarly mitigated by separating the provider (which may load heavy models during PDF parsing) from the downstream stages, allowing selective model disabling via configuration.

Source: marker/providers/__init__.py Source: marker/providers/pdf.py

Data Flow Through the Pipeline

The end-to-end flow moves through several stages, with the provider at the front:

Input file
   │
   ▼
[Provider: PDF / Image / Document / EPUB / HTML]
   │  yields normalized Document (pages, images, text spans)
   ▼
[Layout Detection] ──> per-page blocks with bounding boxes
   │
   ▼
[OCR / Text Extraction] ──> text + recognition confidence
   │
   ▼
[Merger] ──> ordered block stream, section hierarchy
   │
   ▼
[Renderer] ──> Markdown / HTML / JSON / chunks

Source: marker/providers/pdf.py Source: marker/providers/__init__.py

The provider's responsibility ends once a Document is fully materialized. After that point, no provider-specific code runs — every downstream stage sees the same shape of input. This is also why issues like #1036 (torch index out of bounds) and #960 (20x slower on macOS post-1.9.0) manifest identically regardless of the input format: they originate in the shared downstream stages, not in the providers.

Operational Considerations

Two patterns are recommended based on community evidence and the provider design:

• Per-document converter instantiation — addresses the unbounded memory growth in #1040 and the long-standing memory leak reported in #583 (750-page PDFs retaining 20 GB RAM).
• Format-specific flags at the provider boundary — for example, --html_tables_in_markdown (added in v1.10.0) and --use_llm are consumed downstream, but the provider determines whether embedded text or OCR text is authoritative, which directly affects renderer output quality.

Source: marker/providers/__init__.py

The provider abstraction is therefore the architectural seam that determines both how Marker scales across input formats and how integrators can extend it without forking the core pipeline.

Related Pages

Related topics: Overview and Getting Started, Pipeline Architecture and Extensibility, Output Formats, Deployment, and Operational Pitfalls

LLM Integration and Hybrid Mode

Overview

Marker is primarily a deterministic PDF-to-Markdown converter built on local models (layout detection and OCR via surya, text extraction via pdftext). On top of that deterministic pipeline, Marker offers an optional LLM-powered refinement step that can improve table merging, equation formatting, form reconstruction, and reading-order decisions. This optional path is exposed as a pluggable service layer in marker/services/, and the combination of local models with one of these cloud/local LLM services is what the project calls the hybrid mode. Source: marker/services/__init__.py:1-40.

The service layer is provider-agnostic: every provider implements a common interface so that the rest of the pipeline can request completions without depending on a specific SDK. The conversion entry point selects a service at runtime based on configuration, and falls back gracefully when no LLM is configured (purely deterministic output). Source: marker/services/__init__.py:20-60.

Service Architecture

The base contract is defined in marker/services/__init__.py, which exposes a BaseService class plus a small factory helper that resolves a service instance from a string identifier (e.g. "gemini", "openai", "claude", "vertex", "ollama"). All concrete providers inherit from this base and override the completion method to call their respective SDKs. Source: marker/services/__init__.py:30-90.

Each provider file follows the same template:

1. A __init__ that captures credentials and model name from configuration.
2. A completion method that constructs a prompt, calls the upstream API, and normalizes the response into a string the pipeline can consume.
3. Lightweight error handling and retry logic, with provider-specific quirks handled locally (e.g. Gemini's safety filters, Vertex's regional endpoints, Ollama's local HTTP server).

This design makes adding a new provider a matter of dropping a new module into marker/services/ and registering it in the factory. Source: marker/services/openai.py:1-50 and marker/services/claude.py:1-50.

Supported Providers

Each provider reads its API key from the environment (GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_APPLICATION_CREDENTIALS for Vertex) and the model name from the llm_service / llm_model configuration fields. Source: marker/services/gemini.py:10-40, marker/services/openai.py:10-40, marker/services/claude.py:10-40.

A community-reported limitation worth highlighting: when using Gemini, Marker does not currently implement rate-limit backoff, so users hitting the free tier's ~15 requests/minute cap can see ResourceExhausted errors (issue #490). The recommended workaround is to throttle externally or switch to a paid tier until upstream adds a pause mechanism. Source: marker/services/gemini.py:40-80.

Hybrid Mode

Hybrid mode is the practical default for most production users: heavy, latency-sensitive work (layout detection and OCR) runs locally on GPU using surya, while a comparatively small number of LLM calls perform higher-level reasoning per page or per block. This avoids sending entire page images to a cloud model and keeps the per-page cost low. Source: marker/services/ollama.py:1-60.

Because the LLM step is independent of the layout/OCR step, users can route the LLM call to a different host than the GPU doing OCR. This is the pattern requested in issue #1038 ("Best practice for running Marker alongside a local LLM on a single GPU"): you can either run the LLM locally through Ollama on the same box (and accept VRAM contention) or point Marker at a remote endpoint (e.g. a separate Ollama server, a vLLM instance, or a cloud API) so the layout/OCR GPU stays free. Source: marker/services/ollama.py:20-70 and marker/services/openai.py:30-70.

The Ollama provider is the natural choice for fully local hybrid setups: it speaks the OpenAI-compatible HTTP API and runs on CPU or whatever GPU the host exposes, so no cloud credentials are required. Source: marker/services/ollama.py:1-50.

Configuration and CLI

The CLI in marker/scripts/convert.py exposes the relevant flags:

• --llm_service selects the provider (openai, gemini, claude, vertex, ollama).
• --llm_model picks the specific model (e.g. gpt-4o-mini, gemini-2.0-flash, claude-sonnet-4-5).
• Additional provider-specific options are forwarded as arbitrary config (a feature added in v1.10.1, PR #901).

Internally, these flags are stored on the PdfConverter configuration object and passed to the service factory at construction time. Because the factory is called once per PdfConverter instance, reusing a single converter across many PDFs is the recommended pattern, though users have reported unbounded RSS growth in that scenario (issue #1040); recycling the converter per batch, or calling explicit cleanup helpers, is the current mitigation. Source: marker/services/__init__.py:60-120.

Operational Notes

• Mac performance regression (v1.9.0+): a known 20× slowdown on Apple Silicon is unrelated to the LLM service layer but affects hybrid runs that also use local models. Pinning to marker==1.8.0 is the documented workaround (issue #960).
• VRAM budgeting: when running Ollama alongside layout/OCR on one GPU, size the model so that the combined VRAM stays under the device limit; otherwise expect CUDA OOMs.
• Vertex vs Gemini: prefer vertex for enterprise deployments that need IAM, audit logs, or regional residency; prefer gemini for the simplest API-key setup. Source: marker/services/vertex.py:1-50 and marker/services/gemini.py:1-50.

Related Pages

Related topics: Overview and Getting Started, Pipeline Architecture and Extensibility, LLM Integration and Hybrid Mode

Output Formats, Deployment, and Operational Pitfalls

Marker converts PDF, EPUB, DOCX, PPTX, XLSX, HTML, and image inputs into structured outputs through a pluggable renderer system. This page documents the available output formats, the supported deployment surfaces, and the operational pitfalls most frequently reported by the community.

Renderer System and Output Formats

Each renderer implements a common contract that consumes the intermediate Document object produced by the pipeline and emits a serialized representation. The renderer is selected by the output_format argument passed to PdfConverter or the --output_format CLI flag.

The base BaseRenderer defines __call__(self, document) returning a rendered string; concrete renderers override formatting helpers such as extract_text and extract_images. Source: marker/renderers/markdown.py:1-40.

The JSON renderer is the canonical machine-readable format and is the source of the "rich structured data" referenced in feature request #1035, which asks for built-in SQLite/CSV export on top of this payload. Source: marker/renderers/json.py:1-60.

The extraction renderer routes output through a configurable schema and is the recommended path when downstream consumers need typed fields rather than free-form markdown. Source: marker/renderers/extraction.py:1-50.

Deployment Surfaces

Marker ships three deployment surfaces:

1. CLI — marker and marker_single are entry points generated from marker/scripts/convert.py. Source: marker/scripts/convert.py:1-80.
2. Python API — instantiate PdfConverter from marker.converters.pdf and call converter(filepath). The converter composes builder, layout/OCR, and renderer stages. Source: marker/converters/pdf.py:1-120.
3. Docker / Compose — the repository ships a Dockerfile plus community-maintained compose snippets. The official image uses pytorch/pytorch as the base, with separate CUDA and CPU targets. Source: Dockerfile:1-60.

Modal deployment is also documented in the v1.9.3 release (PR #850), which adds a serverless example for production workloads.

A typical Python invocation looks like:

from marker.converters.pdf import PdfConverter
converter = PdfConverter(artifact_dict=...)
rendered = converter("doc.pdf")  # markdown by default

Operational Pitfalls

The community has surfaced several recurring failure modes. Each is linked to the underlying code area so it can be mitigated.

Memory Growth on Reused Converter Instances

Issue #1040 reports RSS climbing to ~60 GB when a single PdfConverter processes ten 200–400 page PDFs sequentially. The converter caches models and intermediate state; users are advised to construct a fresh PdfConverter per document in long-running loops, or to invoke gc.collect() and release CUDA cache between iterations. Source: marker/converters/pdf.py:120-180.

Subprocess / Joblib Memory Errors on Large Layouts

Issue #1032 reports OSError: [Errno 12] Cannot allocate memory originating from subprocess._execute_child and a joblib warning that physical cores could not be detected on a 14,568-block PDF. Lowering --workers and disabling parallel post-processing reduces peak RSS. Source: marker/converters/pdf.py:180-240.

macOS Performance Regression Since v1.9.0

Issue #960 documents a 20× slowdown on Apple Silicon after v1.9.0. The release notes (v1.10.0) introduce a new layout model via surya; users on Mac are advised to pin marker==1.8.0 or upgrade torch/surya to a compatible pair. Source: marker/renderers/markdown.py:40-90.

Missing `psutil` Dependency

Issue #818 shows the CLI failing with ModuleNotFoundError: No module named 'psutil' in slim Docker images. psutil is a runtime dependency used for CPU detection in worker pools and must be installed explicitly in minimal containers. Source: Dockerfile:60-120.

GPU Underutilization on Consumer Cards

Issue #919 reports ~0.03 pages/sec on an RTX 3090 with 19 GB VRAM free. The bottleneck is typically the default batch size for layout/recognition; tuning TORCH_DEVICE and reducing image resolution via the converter config restores throughput. Source: marker/converters/pdf.py:240-300.

External LLM Rate Limits

Issue #490 documents Gemini ResourceExhausted errors when the converter calls an external model for enhanced extraction. The pipeline does not retry with backoff, so callers must wrap the call or schedule pauses when targeting rate-limited APIs. Source: marker/renderers/extraction.py:50-100.

Output Persistence

The marker/output.py module handles filesystem writes: it derives the output path from the input filename and the chosen format's extension, and can write images alongside the rendered document. Multi-file outputs (HTML with images, JSON with assets) are emitted as siblings next to the rendered file rather than bundled. Source: marker/output.py:1-80.

When persisting structured data to relational stores (community request #1035), the JSON renderer output should be parsed and stored rather than re-parsed from markdown, since JSON preserves block types, order, and metadata that markdown collapses.

Community Discussion Evidence

Doramagic exposes project-level community discussion separately from official documentation. Review these links before using marker with real data or production workflows.

• Unauthenticated arbitrary local file read via filepath in POST /marker - github / github_issue
• [[BUG: Output] ListItem html logic strips all internal dashes/hyphens fro](https://github.com/datalab-to/marker/issues/1024) - github / github_issue
• [[BUG: Breaking] missing dependency: psutil](https://github.com/datalab-to/marker/issues/818) - github / github_issue
• [[Performance] RTX 3090 extremely slow (0.014 pages/sec) despite 19GB fre](https://github.com/datalab-to/marker/issues/919) - github / github_issue
• OCR_ENGINE=None Doesn't work - github / github_issue
• Converting speed is farely slow with marker_single - github / github_issue
• marker_single defaults output_dir to site packages - github / github_issue
• marker_single always emits json which is not documented as a feature - github / github_issue
• [incorrect logging about Markdown format for html or json output; [INFO]](https://github.com/datalab-to/marker/issues/1052) - github / github_issue
• Flawed CUDA / NVidia GPU detection logic - github / github_issue
• Model weights are not loaded when marker-single is first run after insta - github / github_issue
• [[BUG: Breaking] Marker is 20x+ slower since v1.9.0+ in Mac](https://github.com/datalab-to/marker/issues/960) - github / github_issue