Skip to main content

TOOL-003: DeepTeam

CategoryTools & Frameworks
FrameworksOpen Source · github.com/confident-ai/deepteam

LLM penetration testing framework aligned with OWASP Top 10 for LLMs and NIST AI RMF. Built-in attack modules mapped to industry standards.

Technique​

# pip install deepteam

from deepteam import red_team

# Scan for OWASP LLM Top 10 vulnerabilities
results = red_team(
    model=your_model,
    attacks=["prompt_injection",
             "jailbreak",
             "pii_leakage"],
)

# Framework mappings:
# OWASP LLM Top 10, NIST AI RMF

Key Concepts​

  • DeepTeam maps attack modules directly to compliance frameworks. Each built-in attack is tagged with its corresponding OWASP LLM Top 10 and NIST AI RMF category, making it straightforward to demonstrate compliance coverage or identify gaps in security testing.
  • The API is designed for simplicity. A single red_team() function call with a model reference and attack list produces structured results, lowering the barrier to entry for teams that need LLM security testing but lack dedicated red team expertise.
  • Attack modules cover the core LLM vulnerability classes. Prompt injection, jailbreak, and PII leakage modules address the most commonly exploited LLM vulnerabilities, with additional modules for harmful content, bias, and hallucination detection.
  • Results are structured for programmatic consumption. The output format supports integration with dashboards, CI/CD pipelines, and security information and event management (SIEM) systems.

Use Cases​

  • Rapid security baseline for LLM applications. Run the full attack suite against a new LLM deployment in minutes to get a quick understanding of its vulnerability profile before investing in deeper manual testing.
  • OWASP LLM Top 10 compliance testing. Systematically test against each category in the OWASP LLM Top 10, generating evidence of testing coverage for audit and compliance purposes.
  • Developer-accessible security testing. The simple API makes it practical for application developers (not just security teams) to run basic LLM security checks during development, shifting security left in the development lifecycle.
  • Continuous monitoring integration. Schedule periodic red team scans to detect security regressions when models are updated, fine-tuned, or when system prompts change.

Getting Started​

Install with pip install deepteam and import the red_team function. Pass your model (supports OpenAI-compatible APIs, HuggingFace models, and custom endpoints) along with a list of attack types to run. Start with the three core attacks (prompt_injection, jailbreak, pii_leakage) to establish a baseline, then expand to the full attack catalog. Review the results object for per-attack pass/fail status and detailed response logs. DeepTeam integrates with DeepEval for broader LLM evaluation workflows, so teams already using DeepEval for quality testing can add security testing with minimal additional setup.