# https://github.com/gabrielchua/RAGxplorer Project Manual

Generated at: 2026-07-13 03:49:49 UTC

## Table of Contents

- [Overview and Installation](#page-1)
- [Core Architecture and Module Layout](#page-2)
- [Query Expansion and Retrieval Methods](#page-3)
- [Visualization Pipeline and Customization](#page-4)

<a id='page-1'></a>

## Overview and Installation

### Related Pages

Related topics: [Core Architecture and Module Layout](#page-2)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [README.md](https://github.com/gabrielchua/RAGxplorer/blob/main/README.md)
- [setup.py](https://github.com/gabrielchua/RAGxplorer/blob/main/setup.py)
- [requirements.txt](https://github.com/gabrielchua/RAGxplorer/blob/main/requirements.txt)
- [ragxplorer/__init__.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/__init__.py)
- [app.py](https://github.com/gabrielchua/RAGxplorer/blob/main/app.py)
</details>

# Overview and Installation

RAGxplorer is an open-source Python package that helps developers and practitioners visualize and explore the internal behaviour of a Retrieval-Augmented Generation (RAG) pipeline. Rather than treating the retriever as a black box, RAGxplorer projects the embedded document chunks and the user query into a two-dimensional space using UMAP, so users can visually inspect which chunks the retriever considers most relevant. The project is distributed on PyPI under the name `ragxplorer` and is published alongside a Streamlit-based hosted web application for users who prefer not to install the library locally. Source: [README.md:1-40]()

## Purpose and Scope

The core purpose of RAGxplorer is to make retrieval quality observable. Given a PDF document and a query, the library loads and chunks the document, embeds each chunk through a user-selected embedding model, projects the resulting vectors to 2D, and highlights the chunks that would be returned by the configured retrieval strategy. This makes it easy to compare retrieval techniques — such as naive similarity, HYDE, and multi-query — on the same document and query. The package is intended as a diagnostic and educational tool for engineers tuning RAG systems, not as a production retrieval framework. Source: [README.md:20-60]()

The scope is intentionally narrow: RAGxplorer focuses on visualization and inspection. It does not provide its own vector store, does not manage long-running document indexes, and does not replace frameworks such as LangChain or LlamaIndex. Instead, it composes with them and delegates embedding, chunking, and answer generation to those libraries. Pinecone support is tracked as an open enhancement request, indicating that the maintainers view RAGxplorer as a thin visualizer over multiple backends. Source: [README.md:60-90]() Community references: [issues/38](https://github.com/gabrielchua/RAGxplorer/issues/38)

## Installation

RAGxplorer is installed from PyPI using the standard `pip` workflow:

```bash
pip install ragxplorer
```

The package metadata is defined in `setup.py`, which declares the project name, version, and the runtime dependencies required for embedding, retrieval, and projection. The most recent published release is `v0.1.2`, which packages the library for general PyPI distribution. Source: [setup.py:1-40]() Source: [README.md:10-20]()

Because the package is a thin wrapper around LangChain, OpenAI, UMAP, and Plotly, all of these are pulled in transitively by the install command. A supplementary `requirements.txt` is provided for users who prefer an explicit, pinned dependency list — recommended for reproducible environments. Source: [requirements.txt:1-20]()

> ⚠️ **Known installation pitfall (Issue #53):** The `langchain` dependency declared in `setup.py` is not pinned. Recent versions of LangChain removed the `langchain.text_splitter` module, so a fresh `pip install ragxplorer` can resolve to an incompatible LangChain release and fail at import time with `ModuleNotFoundError: No module named 'langchain.text_splitter'`. Until the dependency is pinned upstream, the recommended workaround is to pin LangChain manually before installing RAGxplorer:
>
> ```bash
> pip install "langchain<0.2" ragxplorer
> ```
>
> Source: [setup.py:10-30]() Community reference: [issues/53](https://github.com/gabrielchua/RAGxplorer/issues/53)

## Dependencies and Runtime Requirements

RAGxplorer relies on a small set of external services and libraries at runtime:

| Component | Role | Configuration surface |
|---|---|---|
| LangChain | Document loading, chunking, retrieval chains | Selected via `retrieval_method` |
| OpenAI / Ollama | LLM calls (HYDE, multi-query) | API key or local endpoint |
| Embedding model (e.g. `nomic-embed-text`) | Vectorizes chunks and queries | Passed to `RAGxplorer(embedding_model=...)` |
| UMAP | Projects high-dim vectors to 2D | Internal, no user config |
| Plotly | Renders the interactive scatter | Internal, no user config |

Source: [ragxplorer/__init__.py:1-40]() Source: [requirements.txt:1-20]()

For full functionality — including HYDE and multi-query retrieval — an OpenAI API key (or a compatible alternative such as Azure OpenAI or a local Ollama server) must be available in the environment. Source: [README.md:30-60]()

## Quick Start

The minimal end-to-end workflow is:

```python
from ragxplorer import RAGxplorer

client = RAGxplorer(embedding_model="nomic-embed-text")
client.load_pdf(document_path="./tutorials/nais2023.pdf", verbose=True)
client.visualize_query("How will AI innovation be balanced with ethics?")
```

`load_pdf` ingests and chunks the document, while `visualize_query` embeds the query, projects both the chunks and the query into 2D, and renders an interactive Plotly figure with the most relevant chunks highlighted. Source: [ragxplorer/__init__.py:20-80]() Source: [README.md:40-80]()

For users who prefer not to install the package, a hosted Streamlit application (`app.py`) exposes the same capabilities through a browser UI. Uploads to that webapp are processed transiently and are not persisted beyond the session. Community reference: [issues/45](https://github.com/gabrielchua/RAGxplorer/issues/45), [issues/44](https://github.com/gabrielchua/RAGxplorer/issues/44)

## Common First-Run Issues

Two recurring issues are worth highlighting before going further:

1. **3-D array shape error (`Found array with dim 3. None expected <= 2.`)** — typically caused by an embedding model whose output shape is incompatible with UMAP's 2-D projection, or by a LangChain version mismatch. Verify the embedding model and LangChain version before opening an issue. Community reference: [issues/46](https://github.com/gabrielchua/RAGxplorer/issues/46)
2. **`Expected document to be a str`** when using `retrieval_method="multi_qns"` — the multi-query retriever expects string documents; ensure that chunking returns `Document` objects whose `page_content` is a string, especially when substituting Azure OpenAI for OpenAI. Community reference: [issues/47](https://github.com/gabrielchua/RAGxplorer/issues/47)

For development or contribution, the repository does not yet ship a test suite; adding one is tracked as an open enhancement. Community reference: [issues/30](https://github.com/gabrielchua/RAGxplorer/issues/30)

---

<a id='page-2'></a>

## Core Architecture and Module Layout

### Related Pages

Related topics: [Overview and Installation](#page-1), [Query Expansion and Retrieval Methods](#page-3), [Visualization Pipeline and Customization](#page-4)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [ragxplorer/ragxplorer.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/ragxplorer.py)
- [ragxplorer/rag.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/rag.py)
- [ragxplorer/query_expansion.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/query_expansion.py)
- [ragxplorer/projections.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/projections.py)
- [ragxplorer/constants.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/constants.py)
</details>

# Core Architecture and Module Layout

RAGxplorer is a lightweight Python library that helps developers and practitioners **visually inspect how a Retrieval-Augmented Generation (RAG) pipeline retrieves context**. Its core architecture follows a small, layered design where a single orchestrator class coordinates dedicated modules for retrieval, query expansion, and projection-based visualization.

## High-Level Module Layout

The package is centered on a single user-facing class, `RAGxplorer`, exported from the top-level module. Internally it delegates work to three specialized submodules and a constants module:

| Module | Primary Responsibility |
|---|---|
| `ragxplorer.py` | Public API: `RAGxplorer` orchestrator (load, query, visualize) |
| `rag.py` | Retrieval logic, chunk handling, and LLM prompt assembly |
| `query_expansion.py` | Query rewriting strategies (`naive`, `hyde`, `multi_qns`) |
| `projections.py` | Dimensionality reduction (UMAP) for 2D/3D embedding visualization |
| `constants.py` | Default prompts, model identifiers, and shared string constants |

Source: [ragxplorer/ragxplorer.py:1-30]()
Source: [ragxplorer/constants.py:1-40]()

This layout keeps each concern isolated: the orchestrator owns the user-facing workflow, while `rag`, `query_expansion`, and `projections` are independently testable units. The `constants` module avoids hard-coded prompts or model names from being scattered across files.

## The `RAGxplorer` Orchestrator

`RAGxplorer` is the single entry point exposed to users via `from ragxplorer import RAGxplorer`. The class follows a typical three-step pipeline: **load**, **retrieve**, **visualize**.

```python
from ragxplorer import RAGxplorer

client = RAGxplorer(embedding_model="nomic-embed-text")
client.load_pdf(document_path="./doc.pdf", verbose=True)
client.visualize_query("How will AI innovation be balanced with ethics?")
```

The constructor stores configuration (embedding model identifier, retrieval method, chunk size) and prepares an Ollama-compatible embedding endpoint. Source: [ragxplorer/ragxplorer.py:1-60]()

Key methods:

- **`load_pdf`** — Uses LangChain's `PyPDFLoader` to ingest the document, splits the text into chunks, embeds each chunk through the configured model, and stores both raw chunks and embeddings on the instance.
- **`chunk_and_embed`** — Bridges the loader output and the internal vector store; this is where the chunks are turned into NumPy arrays suitable for projection.
- **`visualize_query`** — Accepts a natural-language question, embeds it, retrieves relevant chunks via the `RAG` module, projects both chunks and query onto a 2D plane, and renders an interactive plotly chart.

Because the orchestrator exposes only a few methods, integration code stays compact. Source: [ragxplorer/ragxplorer.py:60-160]()

## The `RAG` Module

`rag.py` encapsulates retrieval logic. It defines a `RAG` class that accepts a list of chunks, their embeddings, and a query, then returns the most relevant chunks. The class delegates query rewriting to `query_expansion` and prompt construction to constants, so its own surface area stays narrow. Source: [ragxplorer/rag.py:1-80]()

Retrieval flow inside `RAG`:

1. Receive the user's query string and the configured retrieval method.
2. Call the appropriate expansion strategy from `query_expansion.py`.
3. Embed the (possibly expanded) query using the same embedding model.
4. Compute similarity (cosine or dot product) between the query embedding and all chunk embeddings.
5. Return the top-*k* chunks together with their similarity scores.

By isolating this logic, the orchestrator remains decoupled from retrieval mechanics, which is also helpful for future extensions such as Pinecone-backed vector stores (tracked as an open feature request in issue #38). Source: [ragxplorer/rag.py:80-140]()

## Query Expansion Strategies

`query_expansion.py` provides a small registry of strategies that rewrite a user's query before retrieval. The orchestrator selects the strategy based on the `retrieval_method` argument. The supported strategies, defined as constants in `constants.py`, are:

- **`naive`** — Pass the query through unchanged. Useful as a baseline.
- **`hyde`** — "Hypothetical Document Embeddings": asks the LLM to generate a synthetic answer, then embeds that answer instead of the raw question. Improves recall when the question is short or abstract.
- **`multi_qns`** — Generates several reformulations of the query and retrieves against the union of results, improving recall at the cost of extra LLM calls.

Source: [ragxplorer/query_expansion.py:1-60]()
Source: [ragxplorer/constants.py:20-80]()

The community has reported that the `multi_qns` path can raise `Expected document to be a str` when the backing LLM returns non-string outputs (issue #47). Because the expansion module is the sole producer of these LLM-derived strings, fixing the issue should be localized here — for example, by coercing LLM outputs to `str` before downstream use. Source: community context referencing issue #47.

## Projection and Visualization

`projections.py` is responsible for turning high-dimensional embeddings into coordinates that can be plotted. It wraps UMAP and exposes a function (commonly `project_embeddings`) that returns 2D arrays. The orchestrator calls this function once for the chunk embeddings and again for the query embedding, then assembles a plotly figure distinguishing retrieved chunks from non-retrieved ones. Source: [ragxplorer/projections.py:1-60]()

A common source of confusion reported by users (issue #46) is the error `Found array with dim 3. None expected <= 2.`. This originates here: if chunk embeddings have an extra dimension (for example, when the embedding model returns batched outputs that were not flattened), UMAP fails. The fix lives in the orchestration layer — chunks must be reshaped to 2D before being passed to the projection module.

## End-to-End Data Flow

```mermaid
flowchart LR
    PDF[PDF Document] --> Loader[LangChain PyPDFLoader]
    Loader --> Chunker[Text Splitter]
    Chunker --> EmbedChunks[Embedding Model]
    EmbedChunks --> Store[(Chunk Embeddings)]
    Query[User Query] --> Expand[query_expansion.py]
    Expand --> EmbedQuery[Embedding Model]
    Store --> Rag[rag.py: similarity + top-k]
    EmbedQuery --> Rag
    Rag --> TopK[Top-k Chunks]
    Store --> Proj[projections.py: UMAP]
    TopK --> Proj
    Proj --> Plot[plotly Visualization]
```

Source: [ragxplorer/ragxplorer.py:60-200]()
Source: [ragxplorer/rag.py:1-140]()
Source: [ragxplorer/projections.py:1-60]()

## Known Architectural Constraints

A few constraints are worth noting for contributors:

- **Pinned LangChain dependency**: An unpinned `langchain` requirement has caused `ModuleNotFoundError: No module named 'langchain.text_splitter'` after a major LangChain release (issue #53). The orchestrator's reliance on LangChain's `text_splitter` import path means the `setup.py` constraint must stay aligned with the supported LangChain major version.
- **Single dimensionality reduction technique**: UMAP is currently the only projection method (issue #36). Adding t-SNE or PCA would require a new module analogous to `projections.py` rather than edits inside it, preserving the single-responsibility layout.
- **No automated test suite**: As of issue #30, the project lacks formal tests; the modular layout is, however, friendly to unit testing of `rag.py`, `query_expansion.py`, and `projections.py` in isolation.

---

<a id='page-3'></a>

## Query Expansion and Retrieval Methods

### Related Pages

Related topics: [Core Architecture and Module Layout](#page-2), [Visualization Pipeline and Customization](#page-4)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [ragxplorer/query_expansion.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/query_expansion.py)
- [ragxplorer/rag.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/rag.py)
- [ragxplorer/constants.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/constants.py)
- [ragxplorer/embedding.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/embedding.py)
- [ragxplorer/visualize.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/visualize.py)
- [ragxplorer/loaders.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/loaders.py)
- [ragxplorer/__init__.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/__init__.py)
- [setup.py](https://github.com/gabrielchua/RAGxplorer/blob/main/setup.py)
</details>

# Query Expansion and Retrieval Methods

## Overview and Purpose

Query expansion is the mechanism by which RAGxplorer augments a user's raw question with additional context before embedding and retrieval. The library exposes three retrieval strategies — `naive`, `HYDE`, and `multi_qns` — selectable through the `retrieval_method` parameter on the `RAGxplorer` client. These strategies are implemented in `ragxplorer/query_expansion.py` and dispatched from `ragxplorer/rag.py`, while method names and prompt templates are centralized as constants in `ragxplorer/constants.py`.

The motivation for offering multiple retrieval methods is to help users compare how different expansion strategies affect which chunks surface as the most relevant neighbors in the 2D UMAP projection. Each method produces a different set of vectors to query against the chunk store, so the resulting visualization highlights different parts of the embedding space.

Source: [ragxplorer/query_expansion.py:1-40]()
Source: [ragxplorer/rag.py:1-60]()
Source: [ragxplorer/constants.py:1-20]()

## Supported Retrieval Methods

The three retrieval methods currently available are:

| Method key | Strategy | Prompt/Expansion Behavior |
|---|---|---|
| `naive` | Pass the user query directly without modification | Embeds the original question and retrieves nearest chunks by cosine similarity |
| `HYDE` | Hypothetical Document Embeddings — instructs the LLM to generate a hypothetical answer that is embedded instead of the raw question | Bypasses the asymmetry between short queries and longer passage embeddings |
| `multi_qns` | Multi-Question expansion — instructs the LLM to generate several related sub-questions, each embedded separately and averaged or iterated | Aims to broaden recall by capturing multiple phrasings of intent |

Each method is selected at the `RAGxplorer` client level and routed through a dispatcher in `rag.py`. The available keys are kept as string constants in `constants.py` so that downstream code and tutorials reference the same identifiers.

Source: [ragxplorer/query_expansion.py:10-80]()
Source: [ragxplorer/constants.py:5-25]()
Source: [ragxplorer/rag.py:30-90]()

## Implementation Flow

The end-to-end flow for a query visualization is:

1. `load_pdf` (defined in `ragxplorer/loaders.py`) reads the PDF, splits it into chunks using a LangChain text splitter, embeds each chunk via the configured `embedding_model` (Ollama by default, e.g. `nomic-embed-text`), and stores the chunk matrix in memory on the client.
2. `visualize_query` (in `rag.py`) accepts the raw question, looks up the chosen `retrieval_method`, and calls the corresponding function from `query_expansion.py`.
3. The expansion function invokes the LLM through the embedding/chat client, parses the response, and returns either a single string (for `naive` and `HYDE`) or a list of strings (for `multi_qns`).
4. The expanded text(s) are embedded, compared against the chunk embeddings using cosine similarity, and the top-k results are projected alongside the query vector with UMAP for plotting.

Source: [ragxplorer/rag.py:40-130]()
Source: [ragxplorer/query_expansion.py:20-100]()
Source: [ragxplorer/embedding.py:1-60]()
Source: [ragxplorer/loaders.py:1-50]()

## Known Limitations and Community-Reported Issues

Several community-reported issues trace back to the query expansion pipeline:

- **3D array errors in Colab/Hugging Face Spaces**: Users hit `Found array with dim 3. None expected <= 2.` because the `multi_qns` path returns a list of expanded queries, each producing its own 2D embedding, and the downstream UMAP reducer or scatter plot receives the stacked array without the expected `reshape`/`np.mean` step (issue #46).
- **"Expected document to be a str" with `multi_qns`**: When users swap the default OpenAI client for Azure OpenAI, the LLM response is sometimes returned as a list of message objects rather than a string. The expansion function then passes a non-string into the embedding step, which raises this error (issue #47). A workaround is to coerce the response to a plain `str` before embedding.
- **Unpinned LangChain dependency**: `setup.py` does not pin `langchain`, so `pip install ragxplorer` can pull a version that has removed `langchain.text_splitter`, which causes `load_pdf` to fail before the retrieval layer is even reached (issue #53).
- **Single dimensionality reducer**: Only UMAP is wired in. Requests to add t-SNE or PCA (issue #36) would require touching `visualize.py` rather than `query_expansion.py`, but the expansion methods are orthogonal to the choice of reducer.

Source: [ragxplorer/query_expansion.py:50-120]()
Source: [ragxplorer/rag.py:80-140]()
Source: [setup.py:1-40]()

## Extending With a New Method

To add a new retrieval method, three files typically need to change:

1. Append a new key and prompt template to `ragxplorer/constants.py`.
2. Implement the expansion function in `ragxplorer/query_expansion.py` and register it in the dispatcher used by `rag.py`.
3. Ensure the function returns either a `str` (single-query) or `List[str]` (multi-query), and that any list output is flattened/averaged before being fed to the embedding model, so that downstream UMAP and scatter code in `ragxplorer/visualize.py` receives a 2D array.

The `__init__.py` file re-exports the `RAGxplorer` class, so no changes there are required when adding internal methods.

Source: [ragxplorer/constants.py:1-40]()
Source: [ragxplorer/query_expansion.py:1-30]()
Source: [ragxplorer/rag.py:1-50]()
Source: [ragxplorer/__init__.py:1-20]()
Source: [ragxplorer/visualize.py:1-80]()

---

<a id='page-4'></a>

## Visualization Pipeline and Customization

### Related Pages

Related topics: [Core Architecture and Module Layout](#page-2), [Query Expansion and Retrieval Methods](#page-3)

<details>
<summary>Related Source Files</summary>

The following source files were used to generate this page:

- [ragxplorer/ragxplorer.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/ragxplorer.py)
- [ragxplorer/projections.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/projections.py)
- [ragxplorer/__init__.py](https://github.com/gabrielchua/RAGxplorer/blob/main/ragxplorer/__init__.py)
- [app.py](https://github.com/gabrielchua/RAGxplorer/blob/main/app.py)
- [setup.py](https://github.com/gabrielchua/RAGxplorer/blob/main/setup.py)
- [requirements.txt](https://github.com/gabrielchua/RAGxplorer/blob/main/requirements.txt)
- [README.md](https://github.com/gabrielchua/RAGxplorer/blob/main/README.md)
</details>

# Visualization Pipeline and Customization

## Purpose and Scope

RAGxplorer's visualization pipeline is a thin orchestration layer that turns a source document into an interactive 2-D scatter plot of embedded chunks, with optional overlays that mark which chunks a retrieval strategy would surface for a given natural-language query. The pipeline is exposed both as a Python API (via the `RAGxplorer` class in `ragxplorer/ragxplorer.py`) and as a Streamlit front-end (via `app.py`), so the same stages execute in both surfaces.

The pipeline is intentionally minimal: there are exactly four user-visible stages — `load_pdf`, `chunk`, `embed`, `project` — followed by the query-time `visualize_query` step. Source: [ragxplorer/ragxplorer.py:1-40](). This bounded design is what makes the pipeline easy to customize: each stage exposes a small set of knobs rather than a deep class hierarchy.

## Pipeline Stages

| Stage | Method | Default Behavior | Customization Knob |
|-------|--------|------------------|--------------------|
| Load | `load_pdf(document_path, verbose=False)` | PyPDF loader via LangChain | `verbose` flag |
| Chunk | `chunk(chunk_size, chunk_overlap)` | `RecursiveCharacterTextSplitter` | chunk size / overlap |
| Embed | `embed(embedding_model)` | Ollama-compatible embeddings endpoint | `embedding_model` name |
| Project | `project()` | UMAP to 2-D | hard-coded UMAP defaults |
| Visualize | `visualize_query(query, retrieval_method, ...)` | `naive` retrieval, color highlight | method, color, alpha, hover |

`chunk` delegates to LangChain's `RecursiveCharacterTextSplitter` under the hood, which is why an unpinned LangChain dependency has caused installation failures in the wild — `langchain.text_splitter` was removed in newer releases. Source: [setup.py:1-30](), [issue #53](https://github.com/gabrielchua/RAGxplorer/issues/53).

`project` always returns a 2-D NumPy array of shape `(n_chunks, 2)`. This contract is the source of the well-known `Found array with dim 3. None expected <= 2` error reported in issue #46: any embedding model that returns vectors with an extra dimension (for example, some HF pipelines that wrap outputs in a batch axis) will be rejected by UMAP. Source: [ragxplorer/projections.py:1-40](), [issue #46](https://github.com/gabrielchua/RAGxplorer/issues/46).

```mermaid
flowchart LR
  A[load_pdf] --> B[chunk]
  B --> C[embed]
  C --> D[project / UMAP]
  D --> E[visualize_query]
  Q[User Query] --> E
  E --> R[Plotly scatter]
```

## Customization Points

**Embedding model selection.** `RAGxplorer(embedding_model=...)` accepts any name that the local Ollama-compatible endpoint can resolve. There is no built-in fallback for OpenAI, Cohere, or Hugging Face endpoints, which is why users have forked the client to swap in `AzureOpenAI` (issue #47). When swapping providers, callers must guarantee that the returned vector is strictly `(n, d)` — any wrapper batch dimension will trip UMAP. Source: [ragxplorer/ragxplorer.py:1-80](), [issue #47](https://github.com/gabrielchua/RAGxplorer/issues/47).

**Retrieval method.** `visualize_query` accepts `retrieval_method` with supported values `naive`, `hyde`, and `multi_qns`. Each strategy produces a different ranking over the embedded chunks, and the visualization colors the top-k hits to make the strategy's bias visible. `multi_qns` expands the user query into several sub-questions via the same LLM endpoint; users have reported that a non-string intermediate (e.g., a dict) returned by a custom LLM wrapper surfaces as `Expected document to be a str` (issue #47). Source: [ragxplorer/ragxplorer.py:80-160](), [issue #47](https://github.com/gabrielchua/RAGxplorer/issues/47).

**Visual styling.** `visualize_query` exposes `color`, `point_size`, and `point_alpha` parameters that feed directly into the underlying `plotly.express.scatter` call. The Plotly figure is returned to the caller rather than rendered inline, which lets both the Streamlit app and standalone notebooks render it through `st.plotly_chart` or `fig.show()` respectively. Source: [app.py:1-80](), [ragxplorer/ragxplorer.py:80-160]().

**Dimensionality reduction.** UMAP is the only projection backend shipped today. Community request #36 explicitly asks for t-SNE and PCA back-ends, and the existing `projections.py` module is the natural extension point — it currently exposes a single function that wraps `umap.UMAP(n_components=2)` and is called unconditionally from `project()`. Source: [ragxplorer/projections.py:1-40](), [issue #36](https://github.com/gabrielchua/RAGxplorer/issues/36).

## Known Limitations and Community-Reported Friction

Three recurring classes of failure show up in the issue tracker and are worth knowing before customizing the pipeline:

1. **Embedding dimensionality.** UMAP refuses any vector with rank ≠ 2; if your embedding endpoint returns `(batch, n, d)` or `(n, 1, d)`, you must squeeze before calling `embed()`. Source: [issue #46](https://github.com/gabrielchua/RAGxplorer/issues/46).
2. **LangChain API drift.** The package imports `langchain.text_splitter`, which was relocated in LangChain 1.x. Pinning or migrating the import is required for fresh installs. Source: [issue #53](https://github.com/gabrielchua/RAGxplorer/issues/53).
3. **Single projection algorithm.** Adding `method="tsne"` or `method="pca"` to `project()` is the most requested extension and would not require touching the rest of the pipeline, since every downstream consumer expects only `(n, 2)` output. Source: [issue #36](https://github.com/gabrielchua/RAGxplorer/issues/36).

The Streamlit surface in `app.py` does not introduce any additional pipeline stage — it simply calls the same Python API and renders the returned Plotly figure through `st.plotly_chart`. Privacy questions about uploaded PDFs (issues #44, #45) are scoped to the hosted Streamlit deployment and do not affect the local Python pipeline, which never persists the document beyond the in-memory `chunks` list on the client instance. Source: [app.py:1-120](), [issue #45](https://github.com/gabrielchua/RAGxplorer/issues/45).

---

<!-- evidence_pipeline_checked: true -->
<!-- evidence_injected: true -->

---

## Pitfall Log

Project: gabrielchua/RAGxplorer

Summary: Found 11 structured pitfall item(s), including 1 high/blocking item(s). Top priority: Installation risk - Installation risk requires verification.

## 1. Installation risk - Installation risk requires verification

- Severity: high
- Evidence strength: source_linked
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/gabrielchua/RAGxplorer/issues/3

## 2. Installation risk - Installation risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a installation risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/gabrielchua/RAGxplorer/issues/41

## 3. Configuration risk - Configuration risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a configuration risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/gabrielchua/RAGxplorer/issues/47

## 4. Capability evidence risk - Capability evidence risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: README/documentation is current enough for a first validation pass.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: capability.assumptions | https://github.com/gabrielchua/RAGxplorer

## 5. Maintenance risk - Maintenance risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a maintenance risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/gabrielchua/RAGxplorer

## 6. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: downstream_validation.risk_items | https://github.com/gabrielchua/RAGxplorer

## 7. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: no_demo
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: risks.scoring_risks | https://github.com/gabrielchua/RAGxplorer

## 8. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/gabrielchua/RAGxplorer/issues/29

## 9. Security or permission risk - Security or permission risk requires verification

- Severity: medium
- Evidence strength: source_linked
- Finding: Project evidence flags a security or permission risk. Review the linked source before relying on this workflow.
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: community_evidence:github | https://github.com/gabrielchua/RAGxplorer/issues/53

## 10. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: issue_or_pr_quality=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/gabrielchua/RAGxplorer

## 11. Maintenance risk - Maintenance risk requires verification

- Severity: low
- Evidence strength: source_linked
- Finding: release_recency=unknown。
- User impact: May increase setup, validation, or first-run risk for the user.
- Evidence: evidence.maintainer_signals | https://github.com/gabrielchua/RAGxplorer

<!-- canonical_name: gabrielchua/RAGxplorer; human_manual_source: deepwiki_human_wiki -->