Cybersecurity · intermediate

LLM Red-Team Lab with Garak, PyRIT, and Promptfoo

A safe, local lab where you run automated adversarial testing tools against a model you host yourself — jailbreaks, prompt injection, data leakage, toxic output — and produce a findings report with severity ratings and mitigations, the way an AI security engineer would.

Project overview

Almost two thirds of cybersecurity postings now ask for AI skills, and AI red teamer is one of the fastest-growing new titles. Testing a model you host is legal, free, and produces the exact artifact hiring managers ask for: a real findings report.

Real-world use case

A company is about to ship a customer-facing chatbot. Before launch, the security team runs a structured red-team against it and hands engineering a prioritized list of what to fix.

Tools and technologies

• Ollama (local model)
• Garak
• PyRIT
• Promptfoo
• Python
• Markdown Report

Skills you can demonstrate

AI Red Teaming, Adversarial Prompting, Vulnerability Assessment, Security Reporting, LLM Safety Controls

Build roadmap

1. Host a target model
   Install Ollama and pull a small open-weight chat model. Confirm you can query it over its local API. Everything you attack from here on is yours.
2. Run Garak probes
   Point Garak at the Ollama endpoint and run its prompt injection, jailbreak, and leakage probe sets. Save the raw output.
3. Run PyRIT attack strategies
   Set up PyRIT with a target and a scorer, and run multi-turn attack strategies. Note which succeed and after how many turns.
4. Build a Promptfoo eval suite
   Write your own red-team test cases in Promptfoo for the app’s specific use case — the scenarios a generic tool would miss.
5. Add guardrails and re-test
   Write a system prompt with safety instructions, add an input filter, and re-run everything. Record the change in success rates per category.
6. Write the findings report
   Executive summary, methodology, findings with severity and OWASP mapping, mitigations, and before/after metrics. Export to PDF.

Ways to extend the project

Add a system prompt with guardrails, re-run the full suite, and quantify how much the attack success rate dropped — then try to bypass your own guardrails.

How to showcase your work

• GitHub: The lab setup scripts, tool configs, raw results, and the polished findings report as a PDF. Redact nothing — it is your own model.
• LinkedIn: Post the summary table: probe categories, attack success rate before and after guardrails. That before/after is the story.
• Resume: Use the industry vocabulary: "Conducted structured LLM red-team assessments using Garak, PyRIT, and Promptfoo, mapped findings to OWASP LLM Top 10."