# ragas - Doramagic AI Context Pack > Purpose: pre-work context for the user's host AI. This pack does not prove that the project has been installed, run, or validated. ## Project - canonical_name: `vibrantlabsai/ragas` - capability: Supercharge Your LLM Application Evaluations 🚀 - expected_user_outcome: Supercharge Your LLM Application Evaluations 🚀 ## Operating Boundaries - Do not claim that the project has been installed, run, called through an API, or used on local files unless separate evidence proves it. - Project facts must come from repo evidence, Claim Graph, or explicit source references. - When a capability is not verified, mark it as unverified instead of completing it as fact. - publish_status: `publishable` - blocking_gaps: none --- ## Doramagic Context Augmentation The following sections strengthen the repository context for a host AI. Human Manual data is a reading route, and pitfall notes become operating constraints. ## Human Manual Outline Usage rule: this is only a reading route and salience signal, not factual authority. Concrete claims must still return to repo evidence or Claim Graph. Host AI hard rules: - Do not treat page titles, section order, summaries, or importance values as factual project evidence. - When explaining the Human Manual outline, state that it is only a reading route or salience signal. - Capability, installation, compatibility, runtime state, and risk claims must cite repo evidence, source paths, or Claim Graph. - **Overview, Installation, and Architecture**: importance `high` - source_paths: README.md, pyproject.toml, src/ragas/__init__.py, src/ragas/cli.py, src/ragas/_analytics.py - **Evaluation Metrics, Collections, and Cost Tracking**: importance `high` - source_paths: src/ragas/metrics/__init__.py, src/ragas/metrics/collections/__init__.py, src/ragas/metrics/collections/base.py, src/ragas/metrics/collections/faithfulness/metric.py, src/ragas/metrics/collections/context_precision/metric.py - **Testset Generation, Datasets, and Storage Backends**: importance `high` - source_paths: src/ragas/testset/__init__.py, src/ragas/testset/graph.py, src/ragas/testset/graph_queries.py, src/ragas/testset/persona.py, src/ragas/testset/synthesizers/__init__.py - **LLM Adapters, Integrations, Prompt Optimization, and Troubleshooting**: importance `high` - source_paths: src/ragas/llms/__init__.py, src/ragas/llms/base.py, src/ragas/llms/litellm_llm.py, src/ragas/llms/oci_genai_wrapper.py, src/ragas/llms/haystack_wrapper.py ## Repo Inspection Evidence - repo_clone_verified: true - repo_inspection_verified: true - repo_commit: `298b68274234c060deacab3cf5fb52aa3a20e885` - inspected_files: `README.md`, `pyproject.toml`, `docs/_static/annotated_data.json`, `docs/_static/edited_chain_runs.json`, `docs/_static/js/commonroom.js`, `docs/_static/js/header_border.js`, `docs/_static/js/mathjax.js`, `docs/_static/js/mendable_chat_bubble.js`, `docs/_static/js/toggle.js`, `docs/_static/sample_annotated_summary.json`, `docs/alfred.py`, `docs/community/index.md`, `docs/community/pdf_export.md`, `docs/concepts/components/eval_dataset.md`, `docs/concepts/components/eval_sample.md`, `docs/concepts/components/index.md`, `docs/concepts/components/prompt.md`, `docs/concepts/datasets.md`, `docs/concepts/experimentation.md`, `docs/concepts/feedback/index.md` Host AI hard rules: - Without repo_clone_verified=true, do not claim that the source code has been read. - Without repo_inspection_verified=true, do not write README, docs, or package-file conclusions as facts. - Without quick_start_verified=true, do not claim that the Quick Start path has run successfully. ## Doramagic Pitfall Constraints These rules come from Doramagic discovery, validation, or compilation findings. The host AI must treat them as operating constraints, not background notes. ### Constraint 1: Installation risk requires verification - Trigger: Project evidence flags a installation risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2760 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 2: Installation risk requires verification - Trigger: Project evidence flags a installation risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2748 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 3: Configuration risk requires verification - Trigger: Project evidence flags a configuration risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2774 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 4: Security or permission risk requires verification - Trigger: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2067 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 5: Security or permission risk requires verification - Trigger: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2732 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 6: Security or permission risk requires verification - Trigger: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2622 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 7: Security or permission risk requires verification - Trigger: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2649 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 8: Security or permission risk requires verification - Trigger: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2692 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 9: Security or permission risk requires verification - Trigger: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: community_evidence:github | https://github.com/vibrantlabsai/ragas/issues/2779 - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it. ### Constraint 10: Capability evidence risk requires verification - Trigger: README/documentation is current enough for a first validation pass. - Host AI rule: Reproduce the official install and quickstart path in an isolated environment. - Why it matters: May increase setup, validation, or first-run risk for the user. - Evidence: capability.assumptions | https://github.com/vibrantlabsai/ragas - Hard boundary: Do not present this pitfall as solved, verified, or ignorable unless later evidence explicitly closes it.