Skip to content

Add test plan agent#6237

Open
jpangas wants to merge 6 commits into
mozilla:masterfrom
jpangas:add_test_plans_agent
Open

Add test plan agent#6237
jpangas wants to merge 6 commits into
mozilla:masterfrom
jpangas:add_test_plans_agent

Conversation

@jpangas

@jpangas jpangas commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

This pull request introduces a new "test-plan-generator" agent that automatically generates and executes Firefox QA test plans based on feature details.

@jpangas jpangas marked this pull request as ready for review June 26, 2026 20:09
Copilot AI review requested due to automatic review settings June 26, 2026 20:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new Hackbot agent, test-plan-generator, wired into the Hackbot API registry and local docker-compose workflow, to generate a Firefox QA test plan from feature details, execute it via the Firefox DevTools MCP server, and return a structured pass/fail/unsuitable result.

Changes:

  • Registers a new test-plan-generator agent in hackbot-api with a dedicated input schema and associated tests.
  • Introduces a new agent package under agents/test-plan-generator/ including prompts, result schema/collector MCP server, Firefox install helper, DevTools MCP wiring, Dockerfile, and compose service.
  • Updates workspace/dependency lock and root compose include to build/run the new agent.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
uv.lock Adds the new agent package as a workspace member with its dependencies.
services/hackbot-api/tests/test_agents.py Adds tests for the new input schema, env serialization, and registry entry.
services/hackbot-api/app/schemas.py Adds TestPlanGeneratorInputs for the new agent’s API contract.
services/hackbot-api/app/agents.py Registers the new agent in AGENT_REGISTRY and exposes it via the API.
docker-compose.yml Includes the agent’s compose file for local runs.
agents/test-plan-generator/pyproject.toml Declares the agent distribution and its runtime dependencies.
agents/test-plan-generator/hackbot.toml Documents that this agent downloads Firefox at startup (no platform prep needed).
agents/test-plan-generator/hackbot_agents/test_plan_generator/result.py Defines structured result models and an in-process MCP submit_result tool.
agents/test-plan-generator/hackbot_agents/test_plan_generator/prompts/system.md Adds the system prompt governing generation/execution/reporting rules.
agents/test-plan-generator/hackbot_agents/test_plan_generator/firefox_install.py Implements Nightly download/install at agent startup.
agents/test-plan-generator/hackbot_agents/test_plan_generator/devtools_mcp.py Builds the stdio MCP server config for the Firefox DevTools MCP server.
agents/test-plan-generator/hackbot_agents/test_plan_generator/config.py Defines the allowed DevTools MCP tool allowlist for execution.
agents/test-plan-generator/hackbot_agents/test_plan_generator/agent.py Orchestrates MCP servers, prompts, and result collection for the run.
agents/test-plan-generator/hackbot_agents/test_plan_generator/__main__.py Reads env inputs and runs the agent entrypoint, installing Firefox first.
agents/test-plan-generator/hackbot_agents/test_plan_generator/__init__.py Declares the agent subpackage.
agents/test-plan-generator/Dockerfile Builds the agent image with Python deps, Node/npm, and required Firefox libs.
agents/test-plan-generator/compose.yml Adds a local compose service for running the agent with env wiring.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread services/hackbot-api/app/agents.py
Comment thread services/hackbot-api/tests/test_agents.py
@jpangas

jpangas commented Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

The lint check is failing for a file that is untouched in this PR. When I committed on my side, everything passed. Should I commit the fix it's recommending?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants