Computer Science · advanced

Autonomous AI Agent for Software Testing

An agent that reads a Python codebase, generates pytest tests, runs them, analyzes failures, and proposes patches — all from a CI pipeline.

Project overview

Hiring is shifting from "can you write code" to "can you orchestrate AI to ship code safely." This project proves you can build the autonomous QA layer modern teams are racing to deploy.

Real-world use case

A startup engineering team that wants >80% test coverage on a fast-moving codebase without dedicating a full QA hire.

Tools and technologies

Skills you can demonstrate

Agentic Workflows, Code Analysis, Test Generation, CI/CD

Build roadmap

  1. Pick the target codebase
    Use a small open-source Python repo (or one of your own) with under 70% test coverage as your testbed.
  2. Build the AST walker
    Use Python ast to extract function signatures, docstrings, and existing test references. Output a structured JSON map of the codebase.
  3. Write the test-generation prompt
    Send each function + its context to the LLM with a strict format: arrange-act-assert pytest tests, including edge cases (None, empty, large input).
  4. Run and triage
    Execute pytest in a sandbox. Parse the failures. Send each failure back to the LLM with the source + traceback to propose a fix.
  5. Wrap it in GitHub Actions
    Trigger on PR. The agent writes tests, runs them, posts results as a PR comment, and (optionally) commits passing tests.
  6. Measure and demo
    Track coverage delta and bugs found per run. Open a real PR on a real repo with the agent — that PR is your portfolio piece.

Ways to extend the project

Add a self-healing mode where the agent auto-commits passing patches to a feature branch and opens a PR with a written explanation.

How to showcase your work

Open this project in BuildersBenchBrowse Computer Science projects