Open-Source Tools: all 10 tracked, with live status (2026)
Open-Source Tools are the public GitHub projects for testing and defending LLMs. llmthreat tracks 10 of them, each with live license, language, maintainer, and repository status from the GitHub API. 6 are Active as of ; the rest are flagged Archived, Dormant, or Ownership changed.
Open-source tools
| Tool | License | Language | Stars | Last push | Status | Primary use |
|---|---|---|---|---|---|---|
| Agentic Security | Apache-2.0 | Python | 1,925 | 2026-06-23 | Active | Agentic LLM vulnerability scanner / AI red-teaming kit for testing autonomous agent pipelines against security threats |
| garak | Apache-2.0 | Python | 8,401 | 2026-07-10 | Active | LLM vulnerability scanner — automated probing for jailbreaks, prompt injection, data leakage, hallucination, and other failure modes |
| Giskard | Apache-2.0 | Python | 5,506 | 2026-07-10 | Active | Open-source evaluation and testing library for LLM agents/RAG — detects hallucination, bias, prompt injection, and other quality/security issues |
| Guardrails | Apache-2.0 | Python | 7,130 | 2026-07-10 | Active | Framework for adding structural, type, and quality guardrails to LLM outputs, including input/output validators for security risks like prompt injection and PII leakage |
| LLM Guard | MIT | Python | 3,164 | 2026-07-08 | Archived | Security toolkit for LLM interactions — input/output scanners for prompt injection, PII, toxicity, jailbreaks, and data leakage |
| NeMo Guardrails | Apache-2.0 | Python | 6,669 | 2026-07-10 | Active | Open-source toolkit for adding programmable guardrails (topical, safety, security rails) to LLM-based conversational systems |
| promptfoo | MIT | TypeScript | 23,165 | 2026-07-12 | Ownership changed | Prompt/agent/RAG testing and LLM red-teaming — vulnerability scanning, eval, and CI/CD security testing for AI applications |
| PyRIT | MIT | Python | 4,086 | 2026-07-12 | Active | Python Risk Identification Tool — open-source framework for red-teaming/risk-identification of generative AI systems |
| Rebuff | Apache-2.0 | TypeScript | 1,511 | 2024-08-07 | Archived | LLM prompt injection detector using heuristics, an LLM-based detector, vector-store similarity, and canary tokens |
| Vigil | Apache-2.0 | Python | 490 | 2024-01-31 | Dormant | Detects prompt injections, jailbreaks, and other risky LLM inputs via a pluggable scanner pipeline |
How to choose an open-source tool here
Open-source LLM security splits into two jobs buyers conflate: pre-deploy testing tools that find vulnerabilities (garak, PyRIT, promptfoo) and runtime guardrails that block them in production (NeMo Guardrails, Guardrails AI, llm-guard). Get the split wrong and you either ship untested or ship unprotected.
- 01 Test-time scanner vs. runtime guardrail
- garak, PyRIT, and promptfoo run offline or in CI — they generate attacks and score the model, adding zero production latency. NeMo Guardrails, Guardrails AI, and llm-guard sit inline in the request path and add latency on every call. These are two different jobs; picking one covers pre-deploy testing, the other covers runtime blocking, and most stacks need both.
- 02 Probe breadth vs. attack sophistication vs. CI fit
- garak ships 120+ static probes for broad known-failure-mode coverage but weak agentic/RAG support. PyRIT specializes in multi-turn adversarial orchestration (Crescendo, TAP) but needs custom Python per attack chain. promptfoo covers 50+ vulnerability types with a built-in OWASP LLM Top 10 preset and CI/CD hooks, trading attack depth for pipeline integration and regression gating.
- 03 Latency budget per guardrail layer
- llm-guard's latency swings hard by scanner: a token-limit check is sub-millisecond, but a secrets or anonymize scan runs tens to hundreds of milliseconds on CPU. Guardrails AI and NeMo Guardrails each add their own per-validator cost on top. Stack more than one layer and you're budgeting per tool, not per pipeline — sub-100ms p50 on input and sub-150ms p50 on output is the ceiling before the guardrail becomes the app's latency story.
- 04 Repo maintenance status, not star count
- llm-guard has 3,164 stars and was archived read-only on 2026-07-09, days after we last checked. Rebuff (1,511 stars) was archived May 2025. Vigil (490 stars) isn't formally archived but hasn't had a commit since January 2024 — 2.5 years dormant. Check the archive banner and last-push date yourself before adopting.
- 05 License and ownership churn
- MIT (PyRIT, promptfoo, llm-guard) carries fewer redistribution obligations than Apache-2.0 (garak, Giskard, Guardrails AI, NeMo Guardrails), which adds an explicit patent grant — relevant if you fork and ship. Ownership changes separately: OpenAI acquired promptfoo in March 2026 (kept it MIT) while Protect AI's tools went dormant after its 2025 Palo Alto Networks acquisition. A new owner doesn't guarantee continued maintenance.
Stars measure hype, not health. llm-guard sat at 3,164 stars the same week it went read-only (archived 2026-07-09); Vigil sits at 490 stars, isn't archived, and hasn't shipped a commit in 2.5 years. Check the archive flag and last-push date yourself — don't infer maintenance from popularity.
Frequently asked questions
What is Open-Source Tools?
Public GitHub projects for testing and defending LLMs, tracked with live stars, license, and repo status so a dead project can't sneak into your stack.
What open-source LLM-security tools does llmthreat track?
10: Agentic Security, garak, Giskard, Guardrails, LLM Guard, NeMo Guardrails, and more. Each is listed with license, language, maintainer, and current repository status.
Which open-source LLM-security tools are still maintained?
6 of the 10 tracked repositories are Active, as of 2026-07-13. The rest are flagged Archived, Dormant, or Ownership changed in the table above.