How to Choose an AI SEO Agency Without Paying for Vanity Metrics

Choose an AI SEO agency by the quality of its evidence, exports, measurement definitions and exit plan—not by a proprietary visibility score. A useful partner can explain what is documented, what it observed, how it validates citations, and what you will own if the relationship ends.

Define the business decision before the pitch

Write the target audience, markets, priority products and qualified outcomes. Decide whether the first problem is crawl access, content usefulness, platform observation, analytics or editorial operations. Ask agencies to respond to that scope. A broad promise to “dominate AI search” is not a diagnosis.

Give every bidder the same small page and prompt sample so comparisons reflect method rather than presentation.

Ask for measurement definitions

Request definitions for mention, citation, cited page, visibility, sentiment, referral and conversion. Ask what counts in the denominator, how failed prompts are handled, whether prompts are fixed, and how country, account state and personalization are recorded.

Compare the answer with our AI visibility measurement crosswalk. If the agency cannot export the rows behind a score, the score is not auditable.

Inspect the evidence workflow

Ask how sources are selected, how claim support is checked, and how product changes trigger updates. Google states in its AI optimization guidance that standard SEO remains relevant and there are no special AI-only technical requirements. A proposal centered on secret schema or guaranteed citations conflicts with that baseline.

Request one anonymized deliverable showing raw observations, source links, uncertainty and a decision—not just a slide with percentages.

Separate platform access from outcomes

An agency may correctly configure OAI-SearchBot, PerplexityBot or a CDN policy. That proves a control was applied; it does not prove citation, traffic or revenue. The plan should show how crawl evidence connects to index eligibility, observed citations, referrals and qualified actions while keeping each layer distinct.

Our AI SEO audit checklist provides a neutral sequence for evaluating the proposed work.

Put ownership and exit terms in the contract

• You receive raw prompt observations and source URLs.
• You own dashboards, analytics configurations, content and images.
• Definitions and calculation methods are documented.
• Credentials use least privilege and are revoked at exit.
• Automated publication requires approval and rollback.
• The handoff includes current tests, limitations and pending risks.

Avoid contracts where historical data disappears with the subscription or the agency owns the only account containing the work.

Run a paid pilot with pass/fail gates

Use four to six weeks and a bounded page set. Capture the baseline, fix one known constraint, improve a few pages, run a fixed prompt sample and deliver raw exports. Judge the pilot on analytical clarity, implementation quality, communication and useful outcomes—not whether every metric rises in a short period.

Good signal	Red flag
Shows unknowns and limits	Promises guaranteed inclusion
Provides raw rows	Provides score screenshots only
Links claims to primary sources	Cites agency blogs for platform rules
Documents rollback	Publishes at scale before review

Make the final comparison reproducible

Score each agency on evidence quality, measurement transparency, technical competence, editorial usefulness, security, ownership, communication and cost. Record the proof for every score. Revisit the matrix after references and the pilot rather than letting charisma override missing controls.

Download the AI SEO agency selection scorecard. Replace the examples with evidence from each proposal and keep the “raw export available” and “exit ownership” columns visible during negotiation.

Use the AI SEO guide and task hub as a neutral scope checklist when an agency bundles crawling, content, citation sampling, analytics, and attribution into one promise.