← Tool directory
TOOL PROFILE

Promptfoo

Test whether a search or content assistant includes expected source URLs in its answers.

Open source

Community support

Upvotes come from SearchEngineAnswer members. GitHub stars and reader reviews are counted separately.

Claim this tool if you maintain it.

Best for
SEO agents
Setup required
Use a supported Node.js release and npm. The example uses the local echo provider and requires no model API key.
Costs to check
This local example uses open-source software and requires no paid service.
License in catalog
MIT

Project screenshot

Promptfoo report showing one passed source URL check and one expected failure
Verified command output from a SearchEngineAnswer example ↗

GitHub repository

GitHub stars
27,134
Forks
2,727
Last push
2026-11-13
Primary language
TypeScript

GitHub snapshot from . Stars count GitHub bookmarks. Reader ratings appear separately below.

How to install Promptfoo

  1. Create a Node.js project and install Promptfoo as a development dependency.
    npm install --save-dev promptfoo
  2. Run npx promptfoo --version, then save the example configuration as checks.yaml.

First tutorial for Promptfoo

  1. Run npx promptfoo eval -c checks.yaml --no-cache. The local echo provider returns each sample answer as written.
    description: Check source URLs in sample SEO assistant answers
    prompts:
      - '{{answer}}'
    providers:
      - echo
    tests:
      - vars:
          answer: 'Check https://example.test/migration/ for the documented procedure.'
        assert:
          - type: contains
            value: 'https://example.test/migration/'
      - vars:
          answer: 'The procedure is documented in the migration guide.'
        assert:
          - type: contains
            value: 'https://example.test/migration/'
  2. Confirm that the report shows one pass and one expected failure. Open the failing row to see that the answer omits the required URL.
  3. Before testing a live assistant, choose source URLs that genuinely support the expected answer and add a separate check for claim support.

SearchEngineAnswer verified one pass and one expected failure using the local echo provider. Live model or hosted service usage may carry separate costs.

Official installation and usage documentation ↗

Where Promptfoo fits

Use Promptfoo when an SEO assistant or internal search workflow needs repeatable checks for missing source URLs.

When to choose another tool
Use human review to assess whether a cited page actually supports a claim. A URL presence check only verifies that the expected string occurs.
First evaluation
Run two fixed answers through the echo provider. Include the expected migration URL in one answer and omit it from the other.
How to interpret the result
Confirm that the first answer passes and the second fails. Keep this fixture as a check on the test configuration before connecting a live model.

Reviews and discussion

Reader reviews

Be the first to share your experience with this tool.

Discussion

Ask a question or share a setup tip.

Write a review or join the discussion

Reviews should describe your own experience. All submissions are moderated.

Flag an outdated detail

Include the field that needs updating and a source link. Copy your draft, then use the corrections policy to contact the editorial team.