Skip to main content
Use Cases
  • Reordering results from an initial search so the most relevant documents appear first.
  • Improving RAG answer quality by scoring each document directly against the query.
  • Screening user input or generated output for harmful content before it is shown.

Rerank

Overview

The AI Gateway exposes POST /rerank on the OpenAI-compatible API base URL (https://api.orq.ai/v3/router). Send a query and a list of documents, and receive the documents ordered by relevance to the query. Reranking re-orders the results of an initial retrieval (for example a vector or keyword search) with a cross-encoder model, a model that reads the query and each document together. This is slower but more accurate than comparing embeddings alone. The request requires three fields: query, documents, and model. Send no more than 1,000 documents in a single request.

Quick Start

See Supported Models for the rerank models available through the gateway.

Response

Each entry in the results array contains: Results are ordered by relevance_score descending, so the most relevant document is first. Pass top_n to limit the number of results returned; it defaults to the full length of documents. Pass return_documents: true to include the original document text in each result. Knowledge Base search already supports reranking: the search-knowledge-base endpoint accepts a rerank_config object and returns a rerank_score alongside the search_score for every retrieved chunk. rerank_config fields:
  • model: The rerank model to use
  • top_k: Number of top results to return after reranking; defaults to the Knowledge Base top_k
  • threshold: Only return documents with a relevance score above this value; defaults to 0
Where reranking runs:
  • Inside a Knowledge Base: pass rerank_config when retrieval stays inside the gateway.
  • Directly via POST /rerank: call the endpoint when the initial candidates come from an external store, for example a third-party vector database, and only the final ordering should happen in the gateway.

Moderations

Overview

The AI Gateway exposes POST /moderations on the OpenAI-compatible API base URL (https://api.orq.ai/v3/router). Send text and receive safety classifications: for each input, the response reports whether it is flagged and, per category, a boolean categories verdict and a category_scores confidence score. The request requires input; model defaults to openai/omni-moderation-latest. Other available models include openai/text-moderation-latest, openai/text-moderation-stable, and mistral/mistral-moderation-2603.

Quick Start

Response

Each entry in the results array corresponds to one input and contains: Categories include hate, harassment, illicit, self-harm, sexual, violence, and their subcategories such as hate/threatening and self-harm/intent. Use flagged or the individual category scores to decide what the application does, for example blocking output above a violence score threshold.
Category names and scores vary by model. OpenAI-compatible models return the 13-category set above. Mistral models (mistral/mistral-moderation-*) return their own set: sexual, hate_and_discrimination, violence_and_threats, dangerous_and_criminal_content, selfharm, health, financial, law, pii.

Moderations vs Guardrails

Moderations is a scoring endpoint: it classifies text and returns scores. It does not block anything, and the caller decides what to do with the result. Guardrails are the enforcement layer: LLM-as-a-Judge or Python evaluators attached to Guardrail Rules that block non-compliant requests and responses automatically, before they reach the caller. Use moderations when the application inspects or acts on the scores itself, and guardrails when the gateway should enforce the policy without application code.