Perplexity’s visual retrievers can split indexing and queries across models

Perplexity’s new visual retrieval models can split indexing and live queries across different model sizes. The pplx-embed-v2-late family shares an embedding space, allowing a larger model to encode documents offline while a smaller model handles incoming questions.

Perplexity listed the release in its October 19 changelog. The accompanying technical report was submitted on October 21. These are separate dates for the release notice and the paper, rather than two independent launches.

Why the shared space changes the deployment choice

The 0.6B model card and 9B model card describe compatible representations. A team can index with 9B and query with 0.6B instead of requiring the same model size on both sides. This creates a configuration to evaluate; it does not guarantee a particular latency or retrieval-quality tradeoff on a publisher’s collection.

The models retain multiple 128-dimensional token vectors rather than compressing each item into one vector. Late interaction compares those vectors at retrieval time, using MaxSim scoring to match query evidence against document evidence. The report describes the training and evaluations behind the family.

Page images need a different audit from extracted text

Visual retrieval matters when a chart, layout, or scanned page carries information that text extraction loses. Finding the right page is still distinct from reading a figure correctly and citing the passage or region that supports an answer.

Consider a fictional annual report with a chart whose legend disappears during text extraction. A text-only index might retrieve a nearby paragraph without the chart. An image-based index might recover the page, but an answer system could still confuse two series. Preserve those as different failure states when evaluating the pipeline.

• Compare extracted text and page images on the same bounded document set.
• Record missed figures, wrong pages and unsupported answers separately.
• Measure index size and query latency for each indexing/query model pairing.
• Keep the source page and expected supporting region beside each judged result.

Implementation and evidence limits

The cards require separate text-only and image-only encoding batches; mixed text-and-image inputs in one batch are unsupported. They use native Sentence Transformers modules without custom Python code. The model weights carry an MIT license, while the paper has its own publication license.

Benchmark results in the report are Perplexity’s evaluations. SearchEngineAnswer did not reproduce them or measure these models on a production collection. The proposed comparison above is an evaluation plan, not an executed benchmark.

For the surrounding retrieval pipeline, the Photon explainer separates search infrastructure from answer generation, while the Q2D-Web brief examines query-to-document evaluation. Neither retrieval scores nor model availability establish visibility gains for a website.