Skip to main content

TOOL-002: PyRIT (Microsoft)

CategoryTools & Frameworks
FrameworksOpen Source · github.com/microsoft/pyrit

Python Risk Identification Tool for generative AI. Enterprise-focused automated red teaming with orchestrated attack strategies and scoring.

Technique​

# pip install pyrit

from pyrit.orchestrator import (
    PromptSendingOrchestrator
)
from pyrit.prompt_target import (
    AzureOpenAITextChatTarget
)

# PyRIT automates:
# - Multi-turn attack strategies
# - Prompt variation generation
# - Response scoring/classification
# - Attack tree exploration

Key Concepts​

  • PyRIT uses orchestrators to manage complex attack strategies. Rather than sending individual prompts, orchestrators coordinate multi-turn conversations, branching attack paths, and adaptive strategies that respond to model behavior in real time.
  • The scoring system classifies responses automatically. PyRIT evaluates whether an attack succeeded by using configurable scorers (LLM-based, regex-based, or custom) to classify model responses, eliminating manual review for large-scale assessments.
  • Attack tree exploration enables systematic coverage. PyRIT can explore branching attack paths where each node represents a different prompt variation or escalation strategy, ensuring comprehensive coverage of the attack surface.
  • Enterprise integration is a first-class concern. PyRIT supports Azure OpenAI, local models, and custom endpoints, with built-in logging and reporting designed for enterprise security workflows and compliance documentation.

Use Cases​

  • Automated multi-turn attack campaigns. Use orchestrators to run sophisticated attack strategies that adapt based on model responses, simulating realistic adversarial behavior over extended conversations.
  • Large-scale prompt variation testing. Generate thousands of prompt variations from attack templates and systematically test each one, identifying which phrasings and strategies bypass specific guardrails.
  • Enterprise AI risk assessment. Conduct structured red team engagements against production Azure OpenAI deployments with full audit logging and scored results suitable for risk reporting.
  • Research and benchmarking. Compare the robustness of different models, system prompts, or guardrail configurations against the same standardized attack suite.

Getting Started​

Install PyRIT with pip install pyrit and configure your target model credentials in a .env file or through environment variables. PyRIT supports Azure OpenAI, OpenAI, and local model endpoints. Start with the PromptSendingOrchestrator for basic prompt injection testing, then progress to RedTeamingOrchestrator for multi-turn attacks and CrescendoOrchestrator for gradual escalation strategies. The Microsoft documentation includes Jupyter notebooks with worked examples for each orchestrator type. PyRIT's memory system tracks all interactions for analysis and reporting.