Promptfoo
Test whether a search or content assistant includes expected source URLs in its answers.
Community support
Upvotes come from SearchEngineAnswer members. GitHub stars and reader reviews are counted separately.
Claim this tool if you maintain it.
- Best for
- SEO agents
- Setup required
- Use a supported Node.js release and npm. The example uses the local echo provider and requires no model API key.
- Costs to check
- This local example uses open-source software and requires no paid service.
- License in catalog
- MIT
Project screenshot
GitHub repository
- GitHub stars
- 27,134
- Forks
- 2,727
- Last push
- 2026-11-13
- Primary language
- TypeScript
GitHub snapshot from . Stars count GitHub bookmarks. Reader ratings appear separately below.
How to install Promptfoo
-
Create a Node.js project and install Promptfoo as a development dependency.
npm install --save-dev promptfoo - Run npx promptfoo --version, then save the example configuration as checks.yaml.
First tutorial for Promptfoo
-
Run npx promptfoo eval -c checks.yaml --no-cache. The local echo provider returns each sample answer as written.
description: Check source URLs in sample SEO assistant answers prompts: - '{{answer}}' providers: - echo tests: - vars: answer: 'Check https://example.test/migration/ for the documented procedure.' assert: - type: contains value: 'https://example.test/migration/' - vars: answer: 'The procedure is documented in the migration guide.' assert: - type: contains value: 'https://example.test/migration/' - Confirm that the report shows one pass and one expected failure. Open the failing row to see that the answer omits the required URL.
- Before testing a live assistant, choose source URLs that genuinely support the expected answer and add a separate check for claim support.
SearchEngineAnswer verified one pass and one expected failure using the local echo provider. Live model or hosted service usage may carry separate costs.
Where Promptfoo fits
Use Promptfoo when an SEO assistant or internal search workflow needs repeatable checks for missing source URLs.
- When to choose another tool
- Use human review to assess whether a cited page actually supports a claim. A URL presence check only verifies that the expected string occurs.
- First evaluation
- Run two fixed answers through the echo provider. Include the expected migration URL in one answer and omit it from the other.
- How to interpret the result
- Confirm that the first answer passes and the second fails. Keep this fixture as a check on the test configuration before connecting a live model.
Reviews and discussion
Reader reviews
Be the first to share your experience with this tool.
Discussion
Ask a question or share a setup tip.
Flag an outdated detail
Include the field that needs updating and a source link. Copy your draft, then use the corrections policy to contact the editorial team.
Write a review or join the discussion