AI use cases RAG, knowledge & memory Chat with your documents (RAG)
Chat with your documents (RAG)
Let an AI answer questions from your own files with retrieval-augmented generation. Example prompts, a workflow and the ranked engines to build it.
- Tools compared
- 4
- Open source
- 2of 4
- Reviewed
- 16 Oct
- 1RAGFlowAI tool69Caution
- 2Open WebUIAI tool68Caution
- 3DifyAI tool56Caution
- 4LlamaIndexAgent framework43Caution
What the score and shapes mean
Score = public maintenance, adoption and docs signals, as a percentile of 52 tools. Nobody here tested these hands-on. Shapes are the separate safety label: No issues found, No dependencies scanned, Caution.
What it does for you
Teams with many documents and developers building search or Q&A over them.
- Ask questions across PDFs, notes and wikis
- Get answers with citations to the passage used
- Search by meaning, not just keywords
- Summarise a large document set
- Build a private assistant over internal content
More about this job
Retrieval-augmented generation finds the relevant passages in your documents and gives them to the model, so answers come from your material and can point to a source. It is the standard way to build document Q&A.
Workflow in 5 steps
- Gather documents and convert them to clean text.
- Split into chunks that keep a heading or section together.
- Index with an embedding model and test retrieval before generation.
- Write a question set with known answers and measure how often the right passage is retrieved.
- Add citations and a "not found" behaviour, then iterate on the failures.
Full steps
- Gather documents and convert them to clean text.
- Split into chunks that keep a heading or section together.
- Index with an embedding model and test retrieval before generation.
- Write a question set with known answers and measure how often the right passage is retrieved.
- Add citations and a "not found" behaviour, then iterate on the failures.
Prompts to copy
Starting points we wrote, not tested results.
Answer using only the passages provided. Quote the sentence you relied on and name the document. If the passages do not answer the question, say "not found".
2 more prompts
Which documents mention the 2024 data retention policy, and do they agree with each other? List differences.
Rewrite this question as three alternative search queries that might match how the documents are worded.
Watch out for
- AI output can be fluent and wrong.
- Prompts sent to a hosted model leave your machine.
- Treat AI changes like a pull request from a new colleague: read the diff, run the tests and keep a human sign-off before anything ships.
- Most RAG failures are retrieval failures.
- Licence to check before commercial use: Open WebUI (Open WebUI License), Dify (Dify Open Source License (modified Apache-2.0)).
- CautionRAGFlow3 critical-severity known vulnerabilities in runtime dependencies (OSV)
- CautionOpen WebUI13 critical-severity known vulnerabilities in runtime dependencies (OSV)
- CautionDify1 critical-severity known vulnerability in runtime dependencies (OSV)
- CautionLlamaIndex39 critical-severity known vulnerabilities in runtime dependencies (OSV)
Why these warnings?
- AI output can be fluent and wrong. Check facts, code and citations yourself before you rely on them or publish them.
- Prompts sent to a hosted model leave your machine. For confidential material use a local model or check the provider's data terms first.
- Treat AI changes like a pull request from a new colleague: read the diff, run the tests and keep a human sign-off before anything ships.
- Most RAG failures are retrieval failures. If the right passage is not retrieved, a better model will not help.
- RAGFlow: 3 critical-severity known vulnerabilities in runtime dependencies (OSV)
- Open WebUI: 13 critical-severity known vulnerabilities in runtime dependencies (OSV)
- Dify: 1 critical-severity known vulnerability in runtime dependencies (OSV)
- LlamaIndex: 39 critical-severity known vulnerabilities in runtime dependencies (OSV)
Recommended tools
From our ranked data, highest score first. A high score does not mean a clean scan.
- 69RAGFlowAI tool · Apache-2.0 · #6 overall
- 68Open WebUIAI tool · Open WebUI License · #8 overall
- 56DifyAI tool · modified Apache-2.0 · #20 overall
- 43LlamaIndexAgent framework · MIT · #35 overall
Related jobs
Sources for tool facts
infiniflow/ragflow · open-webui/open-webui · langgenius/dify · run-llama/llama_index. Data fetched 2 Nov 2026. Scores and labels are automated, relative and not a hands-on test.
Get the board every Monday
One email a week: the new top 10, the biggest score changes and every safety label that changed.
Free. Double opt-in: you confirm by email first, unsubscribe in one click.