Agentic RAG

You can now enable Agentic RAG in your deployments to improve the relevance of retrieved context and the overall output quality.

Once toggled on, simply select a model to act as the agent. The agent will automatically check if the retrieved knowledge chunks are relevant to the user query. If they aren’t, it rewrites the query — preserving the original intent — to improve retrieval results.

This iterative refinement loop increases the chance of surfacing useful context from your Knowledge Base, giving the language model better grounding to generate high-quality, reliable responses. The setup includes two key components:

• Document Grading Agent – determines if relevant chunks were retrieved.
• Query Refinement Agent – rewrites the query if needed.

See the screenshot below on how the input query gets refined. Input query: 'is my suitcase too big?' is reformulated to > 'luggage size requirements and restrictions for carry-on and checked baggage'.

How to enable this? Just toggle the Agentic RAG feature on and select your model.

This feature is part of our ongoing effort to help you ship more robust AI features, let us know if you have any feedback!