Does ChatGPT Rely on Capterra? A SaaS Source-Frequency Test Protocol

A preregistered test separating mentions, citations, recommendations, and referrals across bounded SaaS recommendation tasks. Results pending.

By Abdessalam AlaouiPublished: Reading time3 min read
Sonar the Answer Whale counts Mention, Citation, and Recommendation cards while blocking an arrow labeled No cause and marking results Pending.

Protocol published August 13, 2026; testing pending. This study will measure how often ChatGPT surfaces Capterra and other review platforms in a declared set of SaaS recommendation tasks. It does not currently claim that Capterra reviews, ratings, or listings cause a product to be recommended.

A practitioner discussion observed Capterra in answers and asked whether reviews influence AI visibility. That is a testable hypothesis, but source frequency, recommendation frequency, and causal influence are different claims.

Research questions

  1. How often is each domain cited or linked in the declared sample?
  2. How often is a platform mentioned without a citation?
  3. Which products are recommended, and for which user jobs?
  4. Does the source mix change when the task constraints change without naming a review platform?

Predeclare the sample

Select five SaaS categories and four realistic user scenarios per category. For each scenario, write a neutral recommendation prompt with budget, team size, required integration, region, and one exclusion. Do not mention Capterra or another source in the base prompts.

Record the ChatGPT surface, plan, account state, location, language, displayed model where available, browsing/search state, and UTC timestamp. Run each prompt a fixed number of times in a new conversation and preserve answers with no citations or no recommendation.

Code four distinct events

Do not collapse source behavior into one visibility score
Event Definition Does not prove
Mention A domain or platform is named in answer text. That it supplied evidence
Citation The answer links or attributes a supporting source. That the source caused product selection
Recommendation A product is proposed for the user job. That a review rating drove it
Referral The user can click to a recorded destination. That the click or conversion occurred

Source-role coding

For every citation, classify the source as official product documentation, vendor page, review platform, publisher analysis, community discussion, marketplace, or other. Record which claim the citation supports. A review platform may supply category context while the product recommendation relies on official pricing or integration documentation.

Sensitivity check

Repeat a subset with one user-job constraint changed, such as moving from a two-person team to a regulated enterprise. Do not add a platform name. If source frequency changes, report the task difference rather than claiming the engine “prefers” a domain universally.

Analysis and reporting

  • Publish prompt text, run count, date, context, and coding rubric.
  • Report domain citations per run and per category with raw counts.
  • Separate recommendation frequency from citation frequency.
  • Show answers with no review-site source.
  • Report run-to-run variance and disagreements between coders.
  • Do not correlate public review count or rating with recommendations unless those fields are separately collected and the design supports it.

Limits

The sample will not describe every ChatGPT user, model, location, or future date. Search indexes, product data, and model behavior can change during collection. A domain appearing frequently may reflect broad coverage, task fit, retrieval availability, or other unobserved factors. Frequency is not causation.

OpenAI’s ChatGPT Search help describes the search surface for users; it does not publish a formula assigning causal weight to Capterra. The related SEA GA4 referral guide explains why recorded clicks are yet another layer.

Release condition

The results section will remain marked pending until the complete run set, screenshots, citations, coding sheet, and verification sample are saved. Null results will be published. No vendor outreach or listing change will occur during the run window.

Primary documentation

Community discussion

Discuss: Does ChatGPT Rely on Capterra? A SaaS Source-Frequency Test Protocol

Have a question, a useful example, or a different perspective? Join the discussion, share evidence, and help other readers reach a better answer.

0 replies Moderated
No replies yet.

Be the first to ask a focused question, share a practical example, or add useful evidence.

Ask a question or join the discussion

Share evidence, a useful example, or a clear question. Be specific, stay on topic, and challenge ideas without attacking people. First-time replies may be held for moderation.