Cloudflare AI Search Metadata: 10 KiB vs 64 Bytes

What the August 25 release note says

Cloudflare’s AI Search release notes say custom metadata values can be larger within a shared 10 KiB metadata envelope per vector. The envelope includes system metadata and JSON overhead. The same note says the first 64 UTF-8 bytes of each indexed string remain filterable.

A value can fit inside the 10 KiB envelope while its distinguishing suffix lies outside the filterable window. Conversely, short filter keys can remain safe while many fields and JSON overhead push the complete metadata record toward the shared envelope.

UTF-8 bytes are not characters

These byte counts are SearchEngineAnswer calculations from the literal UTF-8 strings in the downloadable fixture. They are not Cloudflare API observations. Unicode normalization can also change byte sequences for visually similar text, so production pipelines should normalize consistently before hashing or comparing keys.

Download the Cloudflare AI Search metadata byte fixtures (CSV). The eight rows are marked EXAMPLE-REMOVE and include ASCII, accented text, CJK, emoji, mixed text, a long category, and serialized JSON. Replace them with your own production keys and record live envelope behavior separately.

Use short machine keys and separate display labels

Do not filter on a long prose label when a stable machine key will do. Store a concise value such as tech-seo, fr-BE, v3, or an internal content-type code in the filter field. Keep the full label, explanation, or breadcrumb in a separate metadata value used for display or downstream context.

This pattern also reduces accidental collisions where two long labels share the same first 64 bytes and differ only later.

Measure the serialized envelope, not a spreadsheet estimate

The shared envelope includes more than visible custom values. Serialize the exact JSON produced by the application, measure its UTF-8 byte length, and leave headroom for system metadata the service adds. Record the SDK, API version, field set, serializer, normalization rule, and observed response.

1. Create one minimal vector with the smallest intended metadata record.
2. Add fields one at a time and save the exact serialized request.
3. Measure bytes after serialization, not before escaping and field names are added.
4. Test values just below, at, and above the expected internal budget.
5. Run filters against prefixes that differ before and after byte 64.
6. Record accepted, rejected, truncated, filter-matched, filter-missed, or inconclusive states.

Do not publish account IDs, service tokens, private document content, or internal access labels in a public fixture.

Version the metadata contract before values grow

Write a field contract for every filterable value: type, normalization, allowed character set, maximum byte length, null behavior, case rule, owner, and migration version. Reject or shorten a value before indexing rather than discovering after retrieval that the distinguishing suffix fell beyond the documented filterable prefix.

Keep a collision test in continuous integration. The fixture should contain two values that differ early, two that differ only after byte 64, and multilingual examples that reach the boundary with fewer than 64 visible characters. Calculate the byte length after the same normalization and serialization used in production.

Test collisions near the filterable prefix

Create two controlled strings with an identical first 64-byte prefix and different suffixes. Index them as separate fixture records, then run filters that try to distinguish the suffix. The release note says only the first 64 bytes remain filterable, so suffix-based distinctions should not be assumed. Record actual behavior without inventing a truncation mechanism the documentation does not state.

Repeat with a difference that occurs safely before byte 64. If the early difference can be filtered and the late difference cannot, the result supports the documented boundary in that environment. If behavior differs, save the full request and open a version-specific support investigation.

What this article measured—and did not measure

We measured the UTF-8 byte length of eight literal fixture values locally and validated consistent CSV columns. We did not upload them to Cloudflare AI Search, observe envelope rejection, or run live metadata filters. The article therefore reports the documented product limits plus transparent project calculations, not an API benchmark.

The retrieval-augmented search protocol explains how to separate fixtures from findings, and the technical SEO pre-publish checklist provides a wider deterministic QA model.

Source, method, and recheck trigger

Primary source: Cloudflare’s AI Search release notes, August 25 entry, checked September 1, 2026.

Method: We translated the documented 64 UTF-8 byte boundary into literal multilingual fixtures and calculated character and byte lengths locally. The 10 KiB envelope remains a documented service limit rather than an observed SearchEngineAnswer account result.

Recheck trigger: Revisit the fixture if Cloudflare changes the envelope, filterable prefix, metadata field count, serializer behavior, or documentation for metadata attributes.