Perplexity DeepSeek V4 Flash: A Routing and Retrieval Test Protocol
A preregistered test for DeepSeek V4 Flash through Perplexity model-gateway and agent routes, with identical inputs and separate retrieval, citation, latency, and quality fields.
Protocol status: Perplexity’s changelog lists perplexity/deepseek-v4-flash-0731 for the Agent API and Model Gateway. The changelog describes a one-million-token context window. SearchEngineAnswer has not independently verified that limit, benchmarked the model, or compared the two routes.
The method therefore separates three layers: the underlying model service, the gateway contract, and the agent’s retrieval and orchestration behavior. A good or bad final answer cannot identify which layer caused the outcome without preserved traces.
Define the two routes
| Route | Primary question | Required evidence |
|---|---|---|
| Model Gateway | How does the selected model respond under the gateway contract? | Request, model, usage, latency, raw output |
| Agent API | How does the orchestrated workflow retrieve and answer? | Tools, sources, citations, steps, final output |
| Cross-route | What changes when orchestration is added? | Identical task, bounded settings, separate rubrics |
Do not call the routes interchangeable. The agent may search, select sources, transform context, retry, or use other tools that a direct model request does not.
Build a paired task set
Use tasks that cover closed-context reasoning, current web research, source verification, long-document retrieval, comparison, and questions that should return uncertainty. For each task, define whether outside retrieval is permitted and what a correct response must include.
- Freeze the exact input, attachments, locale, date, and success rubric.
- Save the requested and returned model identifiers.
- Record route-specific tools, sources, citations, intermediate errors, and retries.
- Capture latency, token or usage fields, and cost information exposed by the API.
- Blind reviewers to the route for final-answer quality scoring.
- Retain refusals, timeouts, empty citations, and zero-result retrievals.
Long-context fixtures should contain answer keys and distractors at known positions. An accepted request is not proof that the model used the entire context correctly.
Score retrieval and generation separately
For the agent route, score source coverage, source ownership, freshness, passage support, duplication, and retrieval errors before evaluating the final prose. Then score factual correctness, completeness, boundary preservation, citation placement, uncertainty, and task completion.
For the gateway route, distinguish knowledge inside the supplied context from unsupported external claims. If web retrieval is not part of the request, a fluent current answer may be unverifiable rather than impressive.
Do not create one “quality” number that hides whether the route was fast but uncited, well sourced but incomplete, or accurate only after expensive retries.
Report with provider boundaries
Publish the model string, endpoint, date, region, account configuration, request parameters, task set, route, raw artifacts, rubric, reviewer process, missing data, and stopping rule. Attribute the context-window statement to Perplexity unless independently verified under a declared method.
A result obtained through Perplexity does not automatically describe the same model served by another provider. Routing, quantization, context handling, tool access, safety layers, and infrastructure can differ.
Publish a failure taxonomy beside the averages. Separate authentication and quota failures, unsupported parameters, provider routing errors, tool failures, empty retrieval, malformed citations, timeouts, truncation, safety refusals, and reviewer rejection. A retry can convert a visible error into a polished answer while increasing latency and cost; preserve both attempts. Decide in advance whether a retried success counts as reliable completion, degraded completion, or failure.
Use the Perplexity Gateway versus Agent API guide for architectural boundaries and the search API reproducibility guide for run manifests.
Ask a question or join the discussion