# marker - Doramagic AI Context Pack > Positioning: a pre-install experience and judgment asset. It helps the host AI get off to a good start, but it does not mean the project has already been installed, run, or validated. ## Sufficiency Principle - **Sufficiency over compression**: The AI Context Pack should be sufficient for the host AI to understand the project's value, capability boundaries, entrypoints, risks, and evidence sources before starting work; it may be layered, but it does not aim for the shortest possible summary. - **Compression policy**: Compress only noise and duplication, never context that affects judgment or the quality of the work. ## How the Host AI Should Use This You are reading the AI Context Pack that Doramagic compiled for marker. Treat it as pre-work context: help the user understand who it fits, what it can do, how to start, what must be verified after install, and where the risks are. Do not claim that you have already installed, run, or executed the target project. ## Claim Consumption Rules - **Fact source**: Repo Evidence + Claim/Evidence Graph; the Human Wiki only supplies salience, terminology, and narrative structure. - **Minimum status for a fact**: `supported` - `supported`: May be used as a project fact, but the answer must cite the claim_id and evidence path. - `weak`: Usable only as a low-confidence lead; the user must be asked to keep verifying. - `inferred`: Usable only for risk notes or open questions; must not be packaged as a project fact. - `unverified`: Must not be used as fact; state clearly that evidence is insufficient. - `contradicted`: Must show the conflicting sources and must not force a single version on the user's behalf. ## Who It Fits Best - **AI researchers or builders of research-oriented Agents**: The README clearly centers on research, experiment, or paper workflows. Evidence: `README.md` Claim: `clm_0002` supported 0.86 - **Developers already using host AIs such as Claude/Codex/Cursor/Gemini**: The README or plugin config mentions multiple host AIs. Evidence: `README.md` Claim: `clm_0003` supported 0.86 ## What It Can Do - **Command-Line Startup or Install Flow** (Verify after install): The project documentation contains runnable commands; real use requires running them in a local or host environment. Evidence: `README.md` Claim: `clm_0001` supported 0.86 ## How to Start - `pip install marker-pdf` Evidence: `README.md` Claim: `clm_0004` supported 0.86, `clm_0005` supported 0.86 - `pip install marker-pdf[full]` Evidence: `README.md` Claim: `clm_0005` supported 0.86 - `pip install streamlit streamlit-ace` Evidence: `README.md` Claim: `clm_0006` supported 0.86 - `pip install -U uvicorn fastapi python-multipart` Evidence: `README.md` Claim: `clm_0007` supported 0.86 - `git clone https://github.com/VikParuchuri/marker.git` Evidence: `README.md` Claim: `clm_0008` supported 0.86 ## Continue-or-Stop Decision Card - **Current recommendation**: Needs admin / security approval - **Why**: Continuing may involve secrets, accounts, external services, or sensitive context; get admin or security approval first. ### 30-Second Read - **What to do now**: Needs admin / security approval - **Minimum safe next step**: Run Prompt Preview first; if credentials or an enterprise environment are involved, get approval before trialing - **Do not trust yet**: Real output quality cannot be trusted before install. - **Continuing will touch**: Command execution, Local environment or project files, Environment variables / API keys ### What You Can Trust Now - **Target-audience signal: AI researchers or builders of research-oriented Agents** (supported): Backed by a supported claim or project evidence, but that still is not the same as real install results. Evidence: `README.md` Claim: `clm_0002` supported 0.86 - **Target-audience signal: Developers already using host AIs such as Claude/Codex/Cursor/Gemini** (supported): Backed by a supported claim or project evidence, but that still is not the same as real install results. Evidence: `README.md` Claim: `clm_0003` supported 0.86 - **Capability exists: Command-Line Startup or Install Flow** (supported): You can trust that the project contains signals of this capability; whether it fits your specific task still needs trial or after-install verification. Evidence: `README.md` Claim: `clm_0001` supported 0.86 - **There are Quick Start / install-command signals** (supported): You can trust that the docs mention a startup or install entrypoint; do not run it directly in your primary environment because of that. Evidence: `README.md` Claim: `clm_0004` supported 0.86, `clm_0005` supported 0.86 ### What You Cannot Trust Yet - **Real output quality cannot be trusted before install.** (unverified): Prompt Preview can only show how it guides you; it cannot prove result quality in the real project. - **Host AI version compatibility cannot be trusted before install.** (unverified): Host loading rules and version differences across Claude, Cursor, Codex, Gemini, and others must be verified in a real environment. - **That it will not pollute your existing host AI's behavior cannot be trusted directly.** (inferred): Skill, plugin, and AGENTS/CLAUDE/GEMINI instructions may change the host AI's default behavior. - **Safe rollback cannot be assumed by default.** (unverified): Unless the project clearly provides uninstall and recovery instructions, verify in an isolated environment first. - **After a real install, is it compatible with the user's current host AI version?** (unverified): Compatibility can only be verified in the actual host environment. - **Does the project's output quality meet the user's specific task?** (unverified): The pre-install preview can only show flow and boundaries; it cannot replace real evaluation. - **Do the install commands require network access, permissions, or global writes?** (unverified): This affects install risk in both enterprise and personal environments. Evidence: `README.md` ### What Continuing Will Touch - **Command execution**: Package managers, network downloads, the local plugin directory, project config, or the user's home directory. Why: Running the very first command can already change your environment; decide whether it is worth running first. Evidence: `README.md` - **Local environment or project files**: Install results, plugin caches, project config, or local dependency directories. Why: The write scope and rollback path cannot be proven before install and need isolated verification. Evidence: `README.md` - **Environment variables / API keys**: Project entry docs explicitly showing API key, token, secret, or account credential configuration. Why: If a real install needs credentials, use test credentials first and go through a permission/compliance review. Evidence: `README.md`, `marker/settings.py` - **Host AI context**: The AI Context Pack, Prompt Preview, Skill routing, risk rules, and project facts. Why: Importing context affects the host AI's later judgment, so avoid packaging unverified items as facts. ### Minimum Safe Next Steps - **Run Prompt Preview first**: Use a pre-install interactive trial to judge whether the way of working fits; it needs no authorization or environment change. (applies when: Applies to any project, especially when output quality is unknown.) - **Trial-install only in an isolated directory or a test account**: Avoid letting install commands pollute your primary host AI, real projects, or home directory. (applies when: When there are signals of command execution, plugin config, or local writes.) - **Do not use real production credentials**: Once an environment variable / API key enters the host or toolchain, it can create account and compliance risk. (applies when: When environment signals like API, TOKEN, KEY, or SECRET appear.) - **After install, verify just one minimal task**: Verify loading, compatibility, output quality, and rollback first, then decide whether to use it deeply. (applies when: When moving from a trial into a real workflow.) ### Exit Plan - **Preserve the pre-install state**: Record the original host config and project state so you can later judge whether it is recoverable. - **Record the install commands and written paths**: Without clear uninstall instructions, you at least need to know which directories or configs to clean up manually. - **Be ready to revoke test API keys or tokens**: If test credentials leak or are misused, you can cut losses quickly. - **If there is no rollback path, do not enter your primary environment**: No rollback is a blocker before continuing; do not proceed on trust or luck. ## What Can Only Be Previewed - Explain who the project fits and what it can do - Demonstrate a typical conversation flow based on project docs - Help the user decide whether it is worth installing or researching further ## What Must Be Verified After Install - Actually installing the Skill, plugin, or CLI - Running scripts, modifying local files, or accessing external services - Verifying real output quality, performance, and compatibility ## Boundary & Risk Decision Card - **Mistaking the pre-install preview for a real run**: The user may overestimate how much configuration, permission, and compatibility verification the project has already done. Mitigation: Clearly separate prompt_preview_can_do from runtime_required. Claim: `clm_0009` inferred 0.45 - **Command execution will modify the local environment**: Install commands may write to the user's home directory, the host plugin directory, or project configuration. Mitigation: Run in an isolated environment or a test account first. Evidence: `README.md` Claim: `clm_0010` supported 0.86 - **To confirm**: After a real install, is it compatible with the user's current host AI version?. Why: Compatibility can only be verified in the actual host environment. - **To confirm**: Does the project's output quality meet the user's specific task?. Why: The pre-install preview can only show flow and boundaries; it cannot replace real evaluation. - **To confirm**: Do the install commands require network access, permissions, or global writes?. Why: This affects install risk in both enterprise and personal environments. ## Pre-Work Working Context ### Loading Order - First read how_to_use.host_ai_instruction to establish the boundaries of this pre-install judgment asset. - Read claim_graph_summary to confirm facts come from the Claim/Evidence Graph, not the Human Wiki narrative. - Then read intended_users, capabilities, and quick_start_candidates to judge whether the user is a match. - When you need to carry out a concrete task, check role_skill_index first, then evidence_index. - For real install, file modification, network access, performance, or compatibility questions, turn to risk_card and boundaries.runtime_required. ### Task Routes - **Command-Line Startup or Install Flow**: State that this is an after-install capability first, then give a pre-install checklist. Boundary: Must be verified after a real install or run. Evidence: `README.md` Claim: `clm_0001` supported 0.86 ### Context Scale - Total files: 197 - Important-file coverage: 40/197 - Evidence index entries: 63 - Role / Skill entries: 6 ### Handling Insufficient Evidence - **missing_evidence**: State that evidence is insufficient and ask the user for the target file, a README section, or after-install verification records; do not fill in facts. - **out_of_scope_request**: State that the task is beyond the current AI Context Pack's evidence scope and suggest the user check the Human Manual or verify after a real install. - **runtime_request**: Provide a pre-install checklist and command sources, but do not run commands for the user or claim they have been run. - **source_conflict**: Show the conflicting sources side by side, mark them as unverified, and do not force a single version. ## Prompt Recipes ### Fit assessment - Goal: Judge whether this project fits the user's current task. - Expected output: A fit conclusion, key reasons, evidence citations, what can be previewed before install, what must be verified after install, and a next-step recommendation. ```text Based on the AI Context Pack for marker, ask me 3 necessary questions first, then judge whether it fits my task. The answer must cover: who it fits, what it can do, what it cannot do, whether it is worth installing, and where the evidence comes from. Every project fact must cite evidence_refs, source_paths, or a claim_id. ``` ### Pre-install experience - Goal: Let the user feel the core workflow before installing, while avoiding packaging the preview as real capability or a marketing promise. - Expected output: An experience script with boundary labels, an after-install verification checklist, and a cautious recommendation; with no real-run promises or strong marketing language. ```text Treat marker as a pre-install experience asset, not an already-installed tool or a real runtime environment. Output exactly four parts: 1. Ask me 3 necessary questions first. 2. Give an "experience script": use the three labels [Previewable before install], [Must verify after install], and [Insufficient evidence] to show how it might guide the workflow. 3. Give an after-install verification checklist: list which capabilities can only be confirmed after a real install, real host loading, and a real project run. 4. Give a cautious recommendation: only "worth researching/trialing further", "add information before deciding", or "not recommended to continue"; do not endorse the project. Hard boundaries: - Do not claim you have installed, run, executed tests, modified files, or produced real results. - Do not write promise-like phrasing such as "auto-adapts", "guarantees passing", "perfect fit", or "strongly recommend installing". - If you describe how it works after install, you must use a conditional such as "if installed successfully and the host loads the Skill correctly, it might...". - The experience script may only be written as "example lines / hypothetical flow": use "might ask / might suggest / might show", not "has written, has generated, has passed, is running, is generating". - Prompt Preview does not hand out install commands; if the user is ready to trial, only prompt them to read Quick Start and the Risk Card first and to verify in an isolated environment. - Every project fact must come from a supported claim, evidence_refs, or source_paths; inferred/unverified items can only be risks or open questions. ``` ### Role / Skill selection - Goal: Pick the best-matching asset from the project's roles or Skills. - Expected output: A list of candidate roles or Skills, each with an applicable scenario, evidence paths, risk boundary, and whether after-install verification is needed. ```text Read role_skill_index and recommend 3-5 of the most relevant roles or Skills for my target task. For each recommendation, state the applicable scenario, likely output, risk boundary, and evidence_refs. ``` ### Risk pre-check - Goal: Identify environment, permission, rule-conflict, and quality risks before installing or adopting. - Expected output: A checklist of environment, permission, dependency, license, host-conflict, quality risk, and unknown items. ```text Based on risk_card, boundaries, and quick_start_candidates, give me a pre-install risk pre-check list. Do not run commands for me; only explain what I should check, why, and what impact a failure would have. ``` ### Host AI kickoff instruction - Goal: Turn the project context into a host AI instruction for the start of a conversation. - Expected output: A pre-work instruction with clear boundaries and clear evidence citations, suitable to copy to a host AI. ```text Based on the AI Context Pack for marker, generate a pre-work instruction I can paste to my host AI. This instruction must obey not_runtime=true and must not claim the project has been installed, run, or produced real results. ``` ## Role / Skill Index - Indexed 6 role / Skill / project-doc entries. - **Marker** (project_doc): Datalab State of the Art models for Document Intelligence Activation hint: Reference this when the user needs to understand the project's structure, install path, or boundaries. Evidence: `README.md` - **Usage Examples** (project_doc): This directory contains examples of running marker in different contexts. Activation hint: Reference this when the user needs to understand the project's structure, install path, or boundaries. Evidence: `examples/README.md` - **Cla** (project_doc): This Marker Contributor Agreement "MCA" applies to any contribution that you make to any product or project managed by us the "project" , and sets out the intellectual property rights you grant to us in the contributed materials. The term "us" shall mean Endless Labs, Inc. The term "you" shall mean the person or entity identified below. Activation hint: Reference this when the user needs to understand the project's structure, install path, or boundaries. Evidence: `CLA.md` - **An Aggregated Multicolumn Dilated Convolution Network for Perspective-Free Counting** (project_doc): An Aggregated Multicolumn Dilated Convolution Network for Perspective-Free Counting Activation hint: Reference this when the user needs to understand the project's structure, install path, or boundaries. Evidence: `data/examples/markdown/multicolcnn/multicolcnn.md` - **Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity** (project_doc): Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity Activation hint: Reference this when the user needs to understand the project's structure, install path, or boundaries. Evidence: `data/examples/markdown/switch_transformers/switch_trans.md` - **Think Python** (project_doc): How to Think Like a Computer Scientist Activation hint: Reference this when the user needs to understand the project's structure, install path, or boundaries. Evidence: `data/examples/markdown/thinkpython/thinkpython.md` ## Evidence Index - Indexed 63 evidence entries. - **Marker** (documentation): Datalab State of the Art models for Document Intelligence Evidence: `README.md` - **Usage Examples** (documentation): This directory contains examples of running marker in different contexts. Evidence: `examples/README.md` - **License** (source_file): GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Evidence: `LICENSE` - **Output** (source_file): def unwrap outer tag html: str ⋮---- soup = BeautifulSoup html, "html.parser" contents = list soup.contents ⋮---- def json to html block: JSONBlockOutput BlockOutput ⋮---- child html = json to html child for child in block.children child ids = child.id for child in block.children ⋮---- soup = BeautifulSoup block.html, "html.parser" content refs = soup.find all "content-ref" ⋮---- src id = ref.attrs "src" ⋮---- child soup = BeautifulSoup ⋮---- def output exists output dir: str, fname base: str ⋮---- exts = "md", "html", "json" ⋮---- def text from rendered rendered: BaseModel ⋮---- from marker.renderers.chunk import ChunkOutput Has an import from this file ⋮---- def convert if not rgb image:… Evidence: `marker/output.py` - **Replace placeholders** (source_file): def init self, kwargs ⋮---- @staticmethod def convert to md html: str ⋮---- md = MarkdownRenderer markdown = md.md cls.convert html ⋮---- def call self, sample - BenchmarkResult ⋮---- def render self, markdown: str ⋮---- @staticmethod def convert to html md: str ⋮---- block placeholders = inline placeholders = ⋮---- def block sub match ⋮---- content = match.group 1 placeholder = f"1BLOCKMATH{len block placeholders }1" ⋮---- def inline sub match ⋮---- placeholder = f"1INLINEMATH{len inline placeholders }1" ⋮---- md = re.sub r'\${2} . ? \${2}', block sub, md, flags=re.DOTALL md = re.sub r'\$ . ? \$', inline sub, md ⋮---- html = markdown2.markdown md, extras= 'tables' ⋮---- Replace placeholder… Evidence: `benchmarks/overall/methods/__init__.py` - **Init** (source_file): class BaseScorer ⋮---- def init self ⋮---- def call self, sample, gt markdown: List str , method markdown: str - BlockScores Evidence: `benchmarks/overall/scorers/__init__.py` - **Gemini** (source_file): prompt = """ ⋮---- class TableSchema BaseModel ⋮---- table html: str ⋮---- def gemini table rec image: Image.Image ⋮---- client = genai.Client ⋮---- image bytes = BytesIO ⋮---- responses = client.models.generate content ⋮---- output = responses.candidates 0 .content.parts 0 .text Evidence: `benchmarks/table/gemini.py` - **Init** (source_file): class BaseBuilder ⋮---- def init self, config: Optional BaseModel dict = None ⋮---- def call self, data, args, kwargs Evidence: `marker/builders/__init__.py` - **Document** (source_file): class DocumentBuilder BaseBuilder ⋮---- lowres image dpi: Annotated highres image dpi: Annotated disable ocr: Annotated ⋮---- def call self, provider: PdfProvider, layout builder: LayoutBuilder, line builder: LineBuilder, ocr builder: OcrBuilder ⋮---- document = self.build document provider ⋮---- def build document self, provider: PdfProvider ⋮---- PageGroupClass: PageGroup = get block class BlockTypes.Page lowres images = provider.get images provider.page range, self.lowres image dpi highres images = provider.get images provider.page range, self.highres image dpi initial pages = DocumentClass: Document = get block class BlockTypes.Document Evidence: `marker/builders/document.py` - **Init** (source_file): class BaseConverter ⋮---- def init self, config: Optional BaseModel dict = None ⋮---- def call self, args, kwargs ⋮---- def resolve dependencies self, cls ⋮---- init signature = inspect.signature cls. init parameters = init signature.parameters ⋮---- resolved kwargs = {} ⋮---- def initialize processors self, processor cls lst: List Type BaseProcessor - List BaseProcessor ⋮---- processors = ⋮---- simple llm processors = p for p in processors if issubclass type p , BaseLLMSimpleBlockProcessor other processors = p for p in processors if not issubclass type p , BaseLLMSimpleBlockProcessor ⋮---- llm positions = i for i, p in enumerate processors if issubclass type p , BaseLLMSimpleBlockProcessor… Evidence: `marker/converters/__init__.py` - **Extraction** (source_file): logger = get logger ⋮---- class ExtractionConverter PdfConverter ⋮---- pattern: str = r"{\d+\}-{48}\n\n" existing markdown: Annotated ⋮---- def build document self, filepath: str ⋮---- provider cls = provider from filepath filepath layout builder = self.resolve dependencies self.layout builder class line builder = self.resolve dependencies LineBuilder ocr builder = self.resolve dependencies OcrBuilder provider = provider cls filepath, self.config document = DocumentBuilder self.config structure builder cls = self.resolve dependencies StructureBuilder ⋮---- def call self, filepath: str - ExtractionOutput ⋮---- markdown = self.existing markdown ⋮---- renderer = self.resolve dependencies Markd… Evidence: `marker/converters/extraction.py` - **Pdf** (source_file): class PdfConverter BaseConverter ⋮---- override map: Annotated use llm: Annotated default processors: Tuple BaseProcessor, ... = default llm service: BaseService = GoogleGeminiService ⋮---- config = {} ⋮---- processor list = strings to classes processor list ⋮---- processor list = self.default processors ⋮---- renderer = strings to classes renderer 0 ⋮---- renderer = MarkdownRenderer ⋮---- llm service cls = strings to classes llm service 0 llm service = self.resolve dependencies llm service cls ⋮---- llm service = self.resolve dependencies self.default llm service ⋮---- processor list = self.initialize processors processor list ⋮---- @contextmanager def filepath to str self, file input: Uni… Evidence: `marker/converters/pdf.py` - **Init** (source_file): class BaseExtractor ⋮---- max concurrency: Annotated disable tqdm: Annotated ⋮---- def init self, llm service: BaseService, config=None ⋮---- def call self, document: Document, args, kwargs Evidence: `marker/extractors/__init__.py` - **Document** (source_file): logger = get logger ⋮---- class DocumentExtractionSchema BaseModel ⋮---- analysis: str document json: str ⋮---- class DocumentExtractor BaseExtractor ⋮---- page schema: Annotated ⋮---- Input: Evidence: `marker/extractors/document.py` - **Init** (source_file): class BaseProcessor ⋮---- block types: Tuple BlockTypes None = None ⋮---- def init self, config: Optional BaseModel dict = None ⋮---- def call self, document: Document, args, kwargs Evidence: `marker/processors/__init__.py` - **Don't show progress if there are no blocks to process** (source_file): logger = get logger ⋮---- class PromptData TypedDict ⋮---- prompt: str image: Image.Image block: Block schema: BaseModel page: PageGroup additional data: dict None ⋮---- class BlockData TypedDict ⋮---- class BaseLLMProcessor BaseProcessor ⋮---- max concurrency: Annotated image expansion ratio: Annotated use llm: Annotated disable tqdm: Annotated block types = None ⋮---- def init self, llm service: BaseService, config=None ⋮---- def normalize block json self, block: Block, document: Document, page: PageGroup ⋮---- page width = page.polygon.width page height = page.polygon.height block bbox = block.polygon.bbox ⋮---- normalized bbox = ⋮---- block json = { ⋮---- def load blocks self, response:… Evidence: `marker/processors/llm/__init__.py` - **Init** (source_file): class ProviderOutput BaseModel ⋮---- line: Line spans: List Span chars: Optional List List Char = None ⋮---- @property def raw text self ⋮---- def hash self ⋮---- def merge self, other: "ProviderOutput" ⋮---- new output = deepcopy self other copy = deepcopy other ⋮---- ProviderPageLines = Dict int, List ProviderOutput ⋮---- class BaseProvider ⋮---- def init self, filepath: str, config: Optional BaseModel dict = None ⋮---- def len self ⋮---- def get images self, idxs: List int , dpi: int - List Image.Image ⋮---- def get page bbox self, idx: int - PolygonBox None ⋮---- def get page lines self, idx: int - List Line ⋮---- def get page refs self, idx: int - List Reference ⋮---- def enter self ⋮-… Evidence: `marker/providers/__init__.py` - **Initialize the PDF provider with the temp pdf path** (source_file): logger = get logger ⋮---- css = """ ⋮---- class DocumentProvider PdfProvider ⋮---- def init self, filepath: str, config=None ⋮---- temp pdf = tempfile.NamedTemporaryFile delete=False, suffix=".pdf" ⋮---- Initialize the PDF provider with the temp pdf path ⋮---- def del self ⋮---- def convert docx to pdf self, filepath: str ⋮---- result = mammoth.convert to html docx file html = result.value ⋮---- @staticmethod def preprocess base64 images html content ⋮---- pattern = r'data: ^; + ;base64, ^"\' \s + ' ⋮---- def convert image match ⋮---- img data = base64.b64decode match.group 2 ⋮---- output = BytesIO ⋮---- new base64 = base64.b64encode output.getvalue .decode Evidence: `marker/providers/document.py` - **Initialize the PDF provider with the temp pdf path** (source_file): css = ''' ⋮---- class EpubProvider PdfProvider ⋮---- def init self, filepath: str, config=None ⋮---- temp pdf = tempfile.NamedTemporaryFile delete=False, suffix=f".pdf" ⋮---- Initialize the PDF provider with the temp pdf path ⋮---- def del self ⋮---- def convert epub to pdf self, filepath ⋮---- ebook = epub.read epub filepath ⋮---- styles = html content = "" img tags = {} ⋮---- img data = base64.b64encode item.get content .decode "utf-8" ⋮---- soup = BeautifulSoup html content, 'html.parser' ⋮---- src = img.get 'src' ⋮---- normalized src = src.replace '../', '' ⋮---- src = image.get 'xlink:href' ⋮---- html content = str soup full style = ''.join css Evidence: `marker/providers/epub.py` - **Initialize the PDF provider with the temp pdf path** (source_file): class HTMLProvider PdfProvider ⋮---- def init self, filepath: str, config=None ⋮---- temp pdf = tempfile.NamedTemporaryFile delete=False, suffix=".pdf" ⋮---- Initialize the PDF provider with the temp pdf path ⋮---- def del self ⋮---- def convert html to pdf self, filepath: str ⋮---- font css = self.get font css Evidence: `marker/providers/html.py` - **Image** (source_file): class ImageProvider BaseProvider ⋮---- page range: Annotated ⋮---- image count: int = 1 ⋮---- def init self, filepath: str, config=None ⋮---- def len self ⋮---- def get images self, idxs: List int , dpi: int - List Image.Image ⋮---- def get page bbox self, idx: int - PolygonBox None ⋮---- bbox = self.page bboxes idx ⋮---- def get page lines self, idx: int - List Line ⋮---- def get page refs self, idx: int - List Reference Evidence: `marker/providers/image.py` - **Manually assign page bboxes, since we can't get them from pdftext** (source_file): class PdfProvider BaseProvider ⋮---- page range: Annotated pdftext workers: Annotated flatten pdf: Annotated force ocr: Annotated ocr invalid chars: Annotated ocr space threshold: Annotated ocr newline threshold: Annotated ocr alphanum threshold: Annotated image threshold: Annotated strip existing ocr: Annotated disable links: Annotated keep chars: Annotated ⋮---- def init self, filepath: str, config=None ⋮---- Manually assign page bboxes, since we can't get them from pdftext ⋮---- @contextlib.contextmanager def get doc self ⋮---- doc = None ⋮---- doc = pdfium.PdfDocument self.filepath ⋮---- Must be called on the parent pdf, before retrieving pages to render correctly ⋮---- def len self - i… Evidence: `marker/providers/pdf.py` - **Init** (source_file): class BaseRenderer ⋮---- image blocks: Annotated extract images: Annotated bool, "Extract images from the document." = True image extraction mode: Annotated keep pageheader in output: Annotated keep pagefooter in output: Annotated add block ids: Annotated bool, "Whether to add block IDs to the output HTML." = ⋮---- def init self, config: Optional BaseModel dict = None ⋮---- def call self, document ⋮---- def extract image self, document: Document, image id, to base64=False ⋮---- image block = document.get block image id cropped = image block.get image ⋮---- image buffer = io.BytesIO ⋮---- cropped = cropped.convert "RGB" ⋮---- cropped = base64.b64encode image buffer.getvalue .decode ⋮---- @st… Evidence: `marker/renderers/__init__.py` - **Chunk** (source_file): class FlatBlockOutput BaseModel ⋮---- id: str block type: str html: str page: int polygon: List List float bbox: List float section hierarchy: Dict int, str None = None images: dict None = None ⋮---- class ChunkOutput BaseModel ⋮---- blocks: List FlatBlockOutput page info: Dict int, dict metadata: dict ⋮---- def collect images block: JSONBlockOutput - dict str, str ⋮---- images = block.images or {} ⋮---- def assemble html with images block: JSONBlockOutput, image blocks: set str - str ⋮---- child html = assemble html with images child, image blocks for child in block.children child ids = child.id for child in block.children ⋮---- soup = BeautifulSoup block.html, "html.parser" content refs =… Evidence: `marker/renderers/chunk.py` - **Extraction** (source_file): class ExtractionOutput BaseModel ⋮---- analysis: str document json: str original markdown: str ⋮---- class ExtractionRenderer BaseRenderer Evidence: `marker/renderers/extraction.py` - **Html** (source_file): class HTMLOutput BaseModel ⋮---- html: str images: dict metadata: dict ⋮---- class HTMLRenderer BaseRenderer ⋮---- page blocks: Annotated paginate output: Annotated ⋮---- def extract image self, document, image id ⋮---- image block = document.get block image id cropped = image block.get image ⋮---- def insert block id self, soup, block id: BlockId ⋮---- outermost tag = None ⋮---- outermost tag = element ⋮---- wrapper = soup.new tag "span" ⋮---- contents = list soup.contents ⋮---- def extract html self, document, document output, level=0 ⋮---- soup = BeautifulSoup document output.html, "html.parser" ⋮---- content refs = soup.find all "content-ref" ref block id = None images = {} ⋮---- src =… Evidence: `marker/renderers/html.py` - **Json** (source_file): class JSONBlockOutput BaseModel ⋮---- id: str block type: str html: str polygon: List List float bbox: List float children: List "JSONBlockOutput" None = None section hierarchy: Dict int, str None = None images: dict None = None ⋮---- class JSONOutput BaseModel ⋮---- children: List JSONBlockOutput block type: str = str BlockTypes.Document metadata: dict ⋮---- def reformat section hierarchy section hierarchy ⋮---- new section hierarchy = {} ⋮---- class JSONRenderer BaseRenderer ⋮---- image blocks: Annotated page blocks: Annotated ⋮---- def extract json self, document: Document, block output: BlockOutput ⋮---- cls = get block class block output.id.block type ⋮---- children = ⋮---- child outpu… Evidence: `marker/renderers/json.py` - **Sometimes the colspan/rowspan predictions can overflow** (source_file): logger = get logger ⋮---- def escape dollars text ⋮---- def cleanup text full text ⋮---- full text = re.sub r"\n{3,}", "\n\n", full text full text = re.sub r" \n\s {3,}", "\n\n", full text ⋮---- def get formatted table text element ⋮---- text = ⋮---- stripped = content.strip ⋮---- content str = escape dollars str content ⋮---- full text = "" ⋮---- class Markdownify MarkdownConverter ⋮---- def convert div self, el, text, parent tags ⋮---- is page = el.has attr "class" and el "class" 0 == "page" ⋮---- page id = el "data-page-id" pagination item = ⋮---- def convert p self, el, text, parent tags ⋮---- hyphens = r"-—¬" has continuation = el.has attr "class" and "has-continuation" in el "class" ⋮… Evidence: `marker/renderers/markdown.py` - **Ocr Json** (source_file): class OCRJSONCharOutput BaseModel ⋮---- id: str block type: str text: str polygon: List List float bbox: List float ⋮---- class OCRJSONLineOutput BaseModel ⋮---- html: str ⋮---- children: List "OCRJSONCharOutput" None = None ⋮---- class OCRJSONPageOutput BaseModel ⋮---- children: List OCRJSONLineOutput None = None ⋮---- class OCRJSONOutput BaseModel ⋮---- children: List OCRJSONPageOutput block type: str = str BlockTypes.Document metadata: dict None = None ⋮---- class OCRJSONRenderer BaseRenderer ⋮---- image blocks: Annotated page blocks: Annotated ⋮---- def extract json self, document: Document - List OCRJSONPageOutput ⋮---- pages = ⋮---- page equations = equation lines = ⋮---- page lines =… Evidence: `marker/renderers/ocr_json.py` - **Init** (source_file): class BlockTypes str, Enum ⋮---- Line = auto Span = auto Char = auto FigureGroup = auto TableGroup = auto ListGroup = auto PictureGroup = auto Page = auto Caption = auto Code = auto Figure = auto Footnote = auto Form = auto Equation = auto Handwriting = auto TextInlineMath = auto ListItem = auto PageFooter = auto PageHeader = auto Picture = auto SectionHeader = auto Table = auto Text = auto TableOfContents = auto Document = auto ComplexRegion = auto TableCell = auto Reference = auto ⋮---- def str self Evidence: `marker/schema/__init__.py` - **Document** (source_file): class DocumentOutput BaseModel ⋮---- children: List BlockOutput html: str block type: BlockTypes = BlockTypes.Document ⋮---- class TocItem BaseModel ⋮---- title: str heading level: int page id: int polygon: List List float ⋮---- class Document BaseModel ⋮---- filepath: str pages: List PageGroup ⋮---- table of contents: List TocItem None = None debug data path: str None = None ⋮---- def get block self, block id: BlockId ⋮---- page = self.get page block id.page id block = page.get block block id ⋮---- def get page self, page id ⋮---- ignored block types = next block = None ⋮---- page = self.get page block.page id next block = page.get next block block, ignored block types ⋮---- next block = p… Evidence: `marker/schema/document.py` - **Chunk Convert** (source_file): def chunk convert cli ⋮---- parser = argparse.ArgumentParser description="Convert a folder of PDFs to a folder of markdown files in chunks." ⋮---- args = parser.parse args ⋮---- cur dir = os.path.dirname os.path.abspath file script path = os.path.join cur dir, "chunk convert.sh" ⋮---- cmd = f"{script path} {args.in folder} {args.out folder}" Evidence: `marker/scripts/chunk_convert.py` - **Convert** (source_file): logger = get logger ⋮---- def worker init ⋮---- model dict = create model dict ⋮---- model refs = model dict ⋮---- def worker exit ⋮---- def process single pdf args ⋮---- page count = 0 ⋮---- config parser = ConfigParser cli options ⋮---- out folder = config parser.get output folder fpath base name = config parser.get base filename fpath ⋮---- converter cls = config parser.get converter cls config dict = config parser.generate config dict ⋮---- converter = converter cls rendered = converter fpath ⋮---- page count = converter.page count ⋮---- @ConfigParser.common options def convert cli in folder: str, kwargs ⋮---- total pages = 0 in folder = os.path.abspath in folder files = os.path.join in… Evidence: `marker/scripts/convert.py` - **Convert Single** (source_file): logger = get logger ⋮---- @click.command cls=CustomClickPrinter, help="Convert a single PDF to markdown." @click.argument "fpath", type=str @ConfigParser.common options def convert single cli fpath: str, kwargs ⋮---- models = create model dict start = time.time config parser = ConfigParser kwargs ⋮---- converter cls = config parser.get converter cls converter = converter cls rendered = converter fpath out folder = config parser.get output folder fpath Evidence: `marker/scripts/convert_single.py` - **Run Streamlit App** (source_file): def streamlit app cli app name: str = "streamlit app.py" ⋮---- argv = sys.argv 1: cur dir = os.path.dirname os.path.abspath file app path = os.path.join cur dir, app name cmd = ⋮---- def extraction app cli Evidence: `marker/scripts/run_streamlit_app.py` - **Init** (source_file): class BaseService ⋮---- timeout: Annotated int, "The timeout to use for the service." = 30 max retries: Annotated retry wait time: Annotated int, "The wait time between retries." = 3 max output tokens: Annotated ⋮---- def img to base64 self, img: PIL.Image.Image, format: str = "WEBP" ⋮---- image bytes = BytesIO ⋮---- def process images self, images: List PIL.Image.Image - list ⋮---- def format image for llm self, image ⋮---- image = image ⋮---- image parts = self.process images image ⋮---- def init self, config: Optional BaseModel dict = None Evidence: `marker/services/__init__.py` - **Azure Openai** (source_file): logger = get logger ⋮---- class AzureOpenAIService BaseService ⋮---- azure endpoint: Annotated azure api key: Annotated azure api version: Annotated str, "The Azure OpenAI API version to use." = None deployment name: Annotated ⋮---- def process images self, images: List PIL.Image.Image - list ⋮---- images = images ⋮---- max retries = self.max retries ⋮---- timeout = self.timeout ⋮---- client = self.get client image data = self.format image for llm image ⋮---- messages = ⋮---- total tries = max retries + 1 ⋮---- response = client.beta.chat.completions.parse response text = response.choices 0 .message.content total tokens = response.usage.total tokens ⋮---- wait time = tries self.retry wait t… Evidence: `marker/services/azure_openai.py` - **Claude** (source_file): logger = get logger ⋮---- class ClaudeService BaseService ⋮---- claude model name: Annotated claude api key: Annotated str, "The Claude API key to use for the service." = None max claude tokens: Annotated ⋮---- def process images self, images: List Image.Image - List dict ⋮---- def validate response self, response text: str, schema: type T - T ⋮---- response text = response text.strip ⋮---- response text = response text 7: ⋮---- response text = response text :-3 ⋮---- out schema = schema.model validate json response text out json = out schema.model dump ⋮---- escaped str = response text.replace "\\", "\\\\" out schema = schema.model validate json escaped str ⋮---- def get client self ⋮----… Evidence: `marker/services/claude.py` - **The response was not valid JSON** (source_file): logger = get logger ⋮---- class BaseGeminiService BaseService ⋮---- gemini model name: Annotated thinking budget: Annotated ⋮---- def img to bytes self, img: PIL.Image.Image ⋮---- image bytes = BytesIO ⋮---- def get google client self, timeout: int ⋮---- def process images self, images ⋮---- image parts = ⋮---- max retries = self.max retries ⋮---- timeout = self.timeout ⋮---- client = self.get google client timeout=timeout image parts = self.format image for llm image ⋮---- total tries = max retries + 1 temperature = 0 ⋮---- config = { ⋮---- responses = client.models.generate content output = responses.candidates 0 .content.parts 0 .text total tokens = responses.usage metadata.total token c… Evidence: `marker/services/gemini.py` - **Ollama** (source_file): logger = get logger ⋮---- class OllamaService BaseService ⋮---- ollama base url: Annotated ollama model: Annotated str, "The model name to use for ollama." = ⋮---- def process images self, images ⋮---- image bytes = self.img to base64 img for img in images ⋮---- url = f"{self.ollama base url}/api/generate" headers = {"Content-Type": "application/json"} ⋮---- schema = response schema.model json schema format schema = { ⋮---- image bytes = self.format image for llm image ⋮---- payload = { ⋮---- response = requests.post url, json=payload, headers=headers ⋮---- response data = response.json ⋮---- total tokens = ⋮---- data = response data "response" Evidence: `marker/services/ollama.py` - **Openai** (source_file): logger = get logger ⋮---- class OpenAIService BaseService ⋮---- openai base url: Annotated openai model: Annotated str, "The model name to use for OpenAI-like model." = openai api key: Annotated openai image format: Annotated ⋮---- def process images self, images: List Image.Image - List dict ⋮---- images = images ⋮---- img fmt = self.openai image format ⋮---- max retries = self.max retries ⋮---- timeout = self.timeout ⋮---- client = self.get client image data = self.format image for llm image ⋮---- messages = ⋮---- total tries = max retries + 1 ⋮---- response = client.beta.chat.completions.parse response text = response.choices 0 .message.content total tokens = response.usage.total tokens… Evidence: `marker/services/openai.py` - **Vertex** (source_file): class GoogleVertexService BaseGeminiService ⋮---- vertex project id: Annotated vertex location: Annotated gemini model name: Annotated vertex dedicated: Annotated ⋮---- def get google client self, timeout: int ⋮---- http options = {"timeout": timeout 1000} Evidence: `marker/services/vertex.py` - **Image** (source_file): def is blank image image: Image.Image, polygon: Optional List List int = None - bool ⋮---- image = np.asarray image ⋮---- rounded polys = int corner 0 , int corner 1 for corner in polygon ⋮---- gray = cv2.cvtColor image, cv2.COLOR RGB2GRAY gray = cv2.GaussianBlur gray, 7, 7 , 0 ⋮---- binarized = cv2.adaptiveThreshold ⋮---- cleaned = np.zeros like binarized ⋮---- kernel = np.ones 1, 5 , np.uint8 dilated = cv2.dilate cleaned, kernel, iterations=3 b = dilated / 255 Evidence: `marker/utils/image.py` - **Cla** (documentation): This Marker Contributor Agreement "MCA" applies to any contribution that you make to any product or project managed by us the "project" , and sets out the intellectual property rights you grant to us in the contributed materials. The term "us" shall mean Endless Labs, Inc. The term "you" shall mean the person or entity identified below. Evidence: `CLA.md` - **An Aggregated Multicolumn Dilated Convolution Network for Perspective-Free Counting** (documentation): An Aggregated Multicolumn Dilated Convolution Network for Perspective-Free Counting Evidence: `data/examples/markdown/multicolcnn/multicolcnn.md` - **Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity** (documentation): Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity Evidence: `data/examples/markdown/switch_transformers/switch_trans.md` - **Think Python** (documentation): How to Think Like a Computer Scientist Evidence: `data/examples/markdown/thinkpython/thinkpython.md` - **Multicolcnn** (structured_config): { "children": { "id": "/page/0/Page/277", "block type": "Page", "html": " ", "polygon": 0.0, 0.0 , 612.0, 0.0 , 612.0, 792.0 , 0.0, 792.0 , "bbox": 0.0, 0.0, 612.0, 792.0 , "children": { "id": "/page/0/PageHeader/14", "block type": "PageHeader", "html": "", "polygon": 18.119998931884766, 211.199951171875 , 36.2599983215332, 211.199951171875 , 36.2599983215332, 559.2799987792969 , 18.119998931884766, 559.2799987792969 , "bbox": 18.119998931884766, 211.199951171875, 36.2599983215332, 559.2799987792969 , "children": null, "section hierarchy": {}, "images": {} }, { "id": "/page/0/SectionHeader/0", "block type": "SectionHeader", "html": " An Aggregated Multicolumn Dilated Convolution Network for… Evidence: `data/examples/json/multicolcnn.json` - **Switch Trans** (structured_config): { "children": { "id": "/page/0/Page/164", "block type": "Page", "html": " ", "polygon": 0.0, 0.0 , 612.0, 0.0 , 612.0, 792.0 , 0.0, 792.0 , "bbox": 0.0, 0.0, 612.0, 792.0 , "children": { "id": "/page/0/PageHeader/0", "block type": "PageHeader", "html": "", "polygon": 90.0, 41.72613525390625 , 521.8120727539062, 41.72613525390625 , 521.8120727539062, 49.83837890625 , 90.0, 49.83837890625 , "bbox": 90.0, 41.72613525390625, 521.8120727539062, 49.83837890625 , "children": null, "section hierarchy": {}, "images": {} }, { "id": "/page/0/PageHeader/1", "block type": "PageHeader", "html": "", "polygon": 348.43359375, 42.369873046875 , 521.15625, 42.369873046875 , 521.15625, 49.74169921875 , 348.433… Evidence: `data/examples/json/switch_trans.json` - **Thinkpython** (structured_config): { "children": { "id": "/page/0/Page/10", "block type": "Page", "html": " ", "polygon": 0.0, 0.0 , 612.0, 0.0 , 612.0, 792.0 , 0.0, 792.0 , "bbox": 0.0, 0.0, 612.0, 792.0 , "children": { "id": "/page/0/SectionHeader/0", "block type": "SectionHeader", "html": " Think Python ", "polygon": 398.935546875, 265.095703125 , 525.6013793945312, 265.095703125 , 525.6013793945312, 289.6333312988281 , 398.935546875, 289.6333312988281 , "bbox": 398.935546875, 265.095703125, 525.6013793945312, 289.6333312988281 , "children": null, "section hierarchy": { "2": "/page/0/SectionHeader/0" }, "images": {} }, { "id": "/page/0/SectionHeader/1", "block type": "SectionHeader", "html": " How to Think Like a Computer… Evidence: `data/examples/json/thinkpython.json` - **Multicolcnn Meta** (structured_config): { "table of contents": { "title": "An Aggregated Multicolumn Dilated Convolution Network\nfor Perspective-Free Counting", "heading level": null, "page id": 0, "polygon": 117.5888671875, 105.9219970703125 , 477.371826171875, 105.9219970703125 , 477.371826171875, 138.201171875 , 117.5888671875, 138.201171875 }, { "title": "Abstract", "heading level": null, "page id": 0, "polygon": 144.1845703125, 232.4891357421875 , 190.48028564453125, 232.4891357421875 , 190.48028564453125, 244.4443359375 , 144.1845703125, 244.4443359375 }, { "title": "1. Introduction", "heading level": null, "page id": 0, "polygon": 50.016357421875, 512.06591796875 , 128.49609375, 512.06591796875 , 128.49609375, 524.0211181… Evidence: `data/examples/markdown/multicolcnn/multicolcnn_meta.json` - **Switch Trans Meta** (structured_config): { "table of contents": { "title": "Switch Transformers: Scaling to Trillion Parameter Models\nwith Simple and Efficient Sparsity", "heading level": null, "page id": 0, "polygon": 93.0849609375, 101.5679931640625 , 515.77734375, 101.5679931640625 , 515.77734375, 133.84625244140625 , 93.0849609375, 133.84625244140625 }, { "title": "William Fedus\u2217", "heading level": null, "page id": 0, "polygon": 89.2001953125, 151.53192138671875 , 174.6650390625, 151.53192138671875 , 174.6650390625, 164.20635986328125 , 89.2001953125, 164.20635986328125 }, { "title": "Barret Zoph\u2217", "heading level": null, "page id": 0, "polygon": 90.00000762939453, 181.7108154296875 , 165.849609375, 181.710815429687… Evidence: `data/examples/markdown/switch_transformers/switch_trans_meta.json` - **Thinkpython Meta** (structured_config): { "table of contents": { "title": "Think Python", "heading level": null, "page id": 0, "polygon": 398.935546875, 265.095703125 , 525.6013793945312, 265.095703125 , 525.6013793945312, 289.6333312988281 , 398.935546875, 289.6333312988281 }, { "title": "How to Think Like a Computer Scientist", "heading level": null, "page id": 0, "polygon": 267.3017578125, 306.861328125 , 525.6033325195312, 306.861328125 , 525.6033325195312, 323.876953125 , 267.3017578125, 323.876953125 }, { "title": "Think Python", "heading level": null, "page id": 2, "polygon": 398.63671875, 264.90234375 , 525.6013793945312, 264.90234375 , 525.6013793945312, 289.6333312988281 , 398.63671875, 289.6333312988281 }, { "title": "… Evidence: `data/examples/markdown/thinkpython/thinkpython_meta.json` - **Cla** (structured_config): { "signedContributors": { "name": "korakot", "id": 3155646, "comment id": 2143359366, "created at": "2024-06-01T08:25:52Z", "repoId": 712111618, "pullRequestNo": 161 }, { "name": "tosaddler", "id": 13705399, "comment id": 2144014410, "created at": "2024-06-02T20:40:52Z", "repoId": 712111618, "pullRequestNo": 165 }, { "name": "q2333gh", "id": 32679742, "comment id": 2156122900, "created at": "2024-06-08T18:01:39Z", "repoId": 712111618, "pullRequestNo": 176 }, { "name": "q2333gh", "id": 32679742, "comment id": 2156614334, "created at": "2024-06-09T13:48:49Z", "repoId": 712111618, "pullRequestNo": 176 }, { "name": "aniketinamdar", "id": 79044809, "comment id": 2157453610, "created at": "2024-0… Evidence: `signatures/version1/cla.json` - **Byte-compiled / optimized / DLL files** (source_file): private.py .DS Store local.env experiments test data training wandb .dat report.json benchmark data debug data temp.md temp conversion results uploads /cache Evidence: `.gitignore` - **.Pre Commit Config** (source_file): repos: - repo: https://github.com/astral-sh/ruff-pre-commit rev: v0.9.10 hooks: - id: ruff types or: python, pyi args: --fix - id: ruff-format types or: python, pyi Evidence: `.pre-commit-config.yaml` - **Model License** (source_file): AI PUBS OPEN RAIL-M LICENSE MODIFIED Evidence: `MODEL_LICENSE` - **Verify Scores** (source_file): def verify scores file path ⋮---- data = json.load file ⋮---- raw scores = data "scores" k for k in data "scores" marker scores = r "marker" "heuristic" "score" for r in raw scores marker score = sum marker scores / len marker scores ⋮---- def verify table scores file path ⋮---- avg = sum r "marker score" for r in data "marker" / len data ⋮---- parser = argparse.ArgumentParser description="Verify benchmark scores" ⋮---- args = parser.parse args Evidence: `benchmarks/verify_scores.py` - **data/.gitignore** (source_file): latex pdfs references Evidence: `data/.gitignore` - **Latex To Md** (source_file): FILES=$ find latex -name " .tex" for f in $FILES do echo "Processing $f file..." base name=$ basename "$f" .tex out file="references/${base name}.md" pandoc --wrap=none \ --no-highlight \ --strip-comments \ --from=latex \ --to=commonmark x+pipe tables \ "$f" \ -o "$out file" sed -i .bak 's/ / /g' "$out file" sed -i .bak 's/ / /g' "$out file" sed -i .bak 's/ / /g' "$out file" sed -i .bak 's/ / /g' "$out file" sed -i.bak -E 's/ \\cite //g; s/ //g; s/\{ ^} \}//g; s/\\cite\{ ^} \}//g' "$out file" sed -i.bak -E ' s/ \\cite //g; Remove \cite commands inside backticks s/::: //g; Remove the leading ::: for content markers s/\ //g; Remove opening square bracket s/\ //g; Remove closing square bracket… Evidence: `data/latex_to_md.sh` - The remaining 3 evidence entries are in `AI_CONTEXT_PACK.json` or `EVIDENCE_INDEX.json`. ## Rules the Host AI Must Follow - **Treat this asset as pre-work context, not a runtime environment.**: The AI Context Pack contains only an evidence-backed understanding of the project, not the project's executable state. Evidence: `README.md`, `examples/README.md`, `LICENSE` - **When answering the user, distinguish what can be previewed from what can only be verified after install.**: The consumer value of the pre-install experience comes from reducing bad installs and misjudgments, not from pretending to be a real run. Evidence: `README.md`, `examples/README.md`, `LICENSE` ## Questions the User Should Answer First - Which host AI or local environment do you plan to use it in? - Do you just want to experience the workflow first, or are you ready to actually install? - What matters most to you: install cost, output quality, or conflicts with your existing rules? ## Acceptance Checks - Every capability claim can be traced back to a file path in evidence_refs. - AI_CONTEXT_PACK.md does not package previews as a real run. - The user can understand who it fits, what it can do, how to start, and the risk boundaries within 3 minutes. --- ## Doramagic Context Augmentation The following sections strengthen the repository context for a host AI. Human Manual data is a reading route, and pitfall notes become operating constraints. ## Human Manual Outline Usage rule: this is only a reading route and salience signal, not factual authority. Concrete claims must still return to repo evidence or Claim Graph. Host AI hard rules: - Do not treat page titles, section order, summaries, or importance values as factual project evidence. - When explaining the Human Manual outline, state that it is only a reading route or salience signal. - Capability, installation, compatibility, runtime state, and risk claims must cite repo evidence, source paths, or Claim Graph. - **Overview and Getting Started**: importance `high` - source_paths: README.md, marker/converters/pdf.py, marker/scripts/convert_single.py, marker/scripts/convert.py, marker/scripts/chunk_convert.py - **Pipeline Architecture and Extensibility**: importance `high` - source_paths: marker/providers/__init__.py, marker/providers/pdf.py, marker/providers/document.py, marker/providers/image.py, marker/providers/epub.py - **LLM Integration and Hybrid Mode**: importance `high` - source_paths: marker/services/__init__.py, marker/services/gemini.py, marker/services/vertex.py, marker/services/ollama.py, marker/services/claude.py - **Output Formats, Deployment, and Operational Pitfalls**: importance `high` - source_paths: marker/renderers/markdown.py, marker/renderers/html.py, marker/renderers/json.py, marker/renderers/chunk.py, marker/renderers/extraction.py ## Repo Inspection Evidence - repo_clone_verified: true - repo_inspection_verified: true - repo_commit: `ef16c2caa29d76f3ca3126944d2e1be79f560bda` - inspected_files: `README.md`, `pyproject.toml`, `examples/README.md`, `examples/marker_modal_deployment.py` Host AI hard rules: - Without repo_clone_verified=true, do not claim that the source code has been read. - Without repo_inspection_verified=true, do not write README, docs, or package-file conclusions as facts. - Without quick_start_verified=true, do not claim that the Quick Start path has run successfully. ## Doramagic Pitfall Constraints These rules come from Doramagic discovery, validation, or compilation findings. The host AI must treat them as operating constraints, not background notes. - No project-specific pitfall log has English-native content; rely on the base pack's risk card and evidence index. Do not invent missing facts.