Promptfoo
AI Dev Platforms · MLOps & Evaluation Open source free under MIT license

Promptfoo

Open-source CLI and platform for LLM red teaming, prompt evaluation, and application security

User Satisfaction 100%
42 Community Reviews

Free Tier & Quota Rules

Core evaluation CLI and red teaming library are 100% free and open source for local and CI workflows

Overview & Core Features

Promptfoo is an open-source evaluation framework and red-teaming tool designed for LLM applications. It systematically tests prompts, agent outputs, and RAG pipelines against jailbreaks, hallucinations, and security vulnerabilities. Seamlessly integrating into CI/CD pipelines, Promptfoo helps developers benchmark models, assess regressions, and ship hardened AI software with confidence.

Interface Preview & Architecture

🔒 https://www.promptfoo.dev
Promptfoo interface screenshot

Pros & Limitations

Strengths

  • ✓ 100% free open-source core with turnkey integration into GitHub Actions and CI/CD pipelines
  • ✓ Extensive built-in red teaming suites that automatically probe for jailbreaks and prompt injections
  • ✓ Intuitive visual web dashboard for reviewing side-by-side model outputs and latency metrics

Considerations

  • ✕ Running exhaustive red-teaming test matrices against paid LLMs consumes provider API quotas
  • ✕ Crafting specialized custom assertions requires writing lightweight JavaScript or Python hooks

Ready to try Promptfoo?

Explore features directly on their official platform.

Open Promptfoo ↗

Similar Alternatives

💬 Discussion

GitHub Powered