Testing
Plugin
agent-qa
The self-improving QA agent for software teams. A test harness with memory. Write tests in natural language for web and mobile. agent-qa learns from every run, adapts to UI changes, and catches regressions before you ship.
Plugin Signal73/100Solid
maintained97
capable41
documented100
adopted48
Install
$ npx -y skills add vostride/agent-qa --agent claude-code
Run the curl in your terminal, the rest in Claude Code.
Stats
175
Stars
11
Forks
Active
Maintenance
TypeScript
Language
6d ago
Last commit
3mo ago
Created
Repo: vostride/agent-qa
What's inside
Skills
6/agent-qa-authoringUse when creating, editing, validating, or running agent-qa tests, suites, or hooks. Prefer agent-qa MCP tools, enforce canonical agent-qa IDs, and use the bundled schema reference to avoid hallucinated config keys or YAML fields./agent-qa-debug-fixUse after an agent-qa run has failed and you need to debug, patch, and verify the issue using MCP evidence, logs, artifacts, and local code changes instead of generated fix suggestions./agent-qa-result-triageUse when investigating failed agent-qa runs, inspecting artifacts, classifying failures, or comparing recent runs. Prefer agent-qa MCP run/artifact tools and produce evidence-backed triage./agent-qa-authoringUse when creating, editing, validating, or running agent-qa tests, suites, or hooks. Prefer agent-qa MCP tools, enforce canonical agent-qa IDs, and use the bundled schema reference to avoid hallucinated config keys or YAML fields./agent-qa-debug-fixUse after an agent-qa run has failed and you need to debug, patch, and verify the issue using MCP evidence, logs, artifacts, and local code changes instead of generated fix suggestions./agent-qa-result-triageUse when investigating failed agent-qa runs, inspecting artifacts, classifying failures, or comparing recent runs. Prefer agent-qa MCP run/artifact tools and produce evidence-backed triage.
Badges
4
README
1
agent-qa
The self-improving Agentic QA harness with Memory
Write tests in natural language for web and mobile. agent-qa learns from past runs, adapts to UI changes, and catches regressions before you ship.
Features
- Write tests in natural language for web and mobile: Define actions and assertions in human language while agents work from visible roles, labels, and screen state.
- Self-healing test execution: When any sub-action, such as click, fill, or select, fails, agent-qa re-observes the UI and tries a different path in the same run. Tests recover from UI drift and flaky interactions instead of failing on the first broken action.
- Self-improves with Memory: With every test run, agent-qa builds execution memory from product, suite, and test observations, then adds that context to future runs. agent-qa also curates memory from steps that were healed during execution, helping future runs avoid the same mistake.
- Built for humans and machines: A polished dashboard and CLI for developers, plus MCP and skills for coding agents.
- Accelerate runs with smart Cache: The action cache reuses validated plans across similar subsequent test runs, reducing planner work, token usage, and runtime overhead.
- Run sandboxed hooks during tests: Run Node, Bun, Python, or Bash hooks in isolated Docker containers to set up environments, call APIs, seed fixtures, tear down state, or pass structured outputs back into the active test run.
- Open source, reviewable QA: The harness is open source, and tests, configs, hooks, memory, and suite logic all live as version-controlled code, so every change can be diffed, reviewed, reused, and shared across teams.
- Bring your own LLM: Run tests with the model of your choice via OpenAI- and Anthropic-compatible endpoints, Gemini, local or open-source models, and subscriptions like Codex and Claude Code.
Quickstart
Install the package:
npm install -D agent-qa
For Codex or Claude Code subscription auth, also install:
npm install -D @vostride/agent-qa-subscription-auth
Install Docker before using hooks. agent-qa runs hooks in a sandboxed runtime, and Docker is required for the Node, Bun, Python, and Bash hook containers.
Initialize agent-qa and install the runtime support you need:
npx agent-qa init
npx agent-qa install-browsers --chromium
# Mobile projects:
npx agent-qa install-mobile-drivers --all
Start the dashboard, complete auth, and run tests from the UI:
npx agent-qa dashboard --open
For the full setup flow, use the quickstart.
CLI
Run tests from the CLI:
npx agent-qa run tests/hacker-news-top-story.yaml
Docs
File tree
912vostride/agent-qaView on GitHub
.dockerignore
.github/
workflows/
docker-release.yml
release.yml
.gitignore
.nvmrc
.prettierrc
AGENTS.md
demo-project/
agent-qa.config.yaml
agent-qa.release.config.yaml
agent-rules.md
hooks.yaml
release-action-pack/
upload-fixture.txt
scripts/
release-action-preflight.mjs
tests/
fixtures/
sample-upload.txt
web/
38-hooks-demo.yaml
40-hook-runjs-compare.yaml
41-env-runjs-compare.yaml
43-inline-hook-demo.yaml
45-hook-and-runjs-prestep.yaml
bad-a11y.yaml
docker/
Dockerfile.android
Dockerfile.hooks-bash
Dockerfile.hooks-bun
Dockerfile.hooks-node
Dockerfile.hooks-python
Dockerfile.web
docs/
assets/
readme-banner.png
glama.json
knip.json
LICENSE.md
NOTICE.md
package.json
packages/
android/
AGENTS.md
LICENSE.md
NOTICE.md
package.json
src/
__tests__/
adapter.test.ts
adapter.ts
index.ts
session.ts
types.ts
tsconfig.json
tsup.config.ts
vitest.config.ts
cli/
AGENTS.md
LICENSE.md
NOTICE.md
package.json
README.md
scripts/
copy-skills.mjs
skills/
agent-qa-authoring/
agents/
openai.yaml
references/
agent-qa-contracts.json
SKILL.md
agent-qa-debug-fix/
agents/
openai.yaml
SKILL.md
agent-qa-result-triage/
agents/
openai.yaml
references/
triage-categories.md
SKILL.md
src/
__tests__/
auth-state-capture.test.ts
auth-test.test.ts
cache-purge.test.ts
clean-memory.test.ts
config.test.ts
create-suite.test.ts
create-test.test.ts
dashboard.test.ts
devices.test.ts
doctor.test.ts
ids.test.ts
init-config-schema.test.ts
init.test.ts
install-browsers.test.ts
install-mobile-drivers.test.ts
llm-utils.test.ts
mcp.test.ts
removed-cli-surface.test.ts
run-mobile.test.ts
run.hook-ids.test.ts
run.test.ts
skills.test.ts
targets.test.ts
validate.test.ts
version-notice.test.ts
version.test.ts
cli.ts
commands/
__tests__/
doctor.test.ts
auth-state.ts
auth.ts
cache.ts
clean-memory.ts
config.ts
create-suite.ts
create-test.ts
dashboard.ts
devices.ts
doctor.ts
ids.ts
index.ts
init.ts
install-browsers.ts
install-mobile-drivers.ts
mcp.ts
queue.ts
run.ts
skills.ts
validate.ts
config.ts
devices.ts
index.ts
llm-utils.ts
model-defaults.ts
removed-cli-surface.ts
targets.ts
version-notice.ts
tsconfig.json
tsup.config.ts
vitest.config.ts
core/
AGENTS.md
LICENSE.md
NOTICE.md
package.json
scripts/
ensure-posthog-project.cjs
src/
__tests__/
auth-store.test.ts
config-schema.test.ts
coord-contract.test.ts
fetch-wrapper.test.ts
junit-reporter.test.ts
llm-auth-resolver.test.ts
log-manager.test.ts
log-schemas.test.ts
pkce.test.ts
plugin-loader.test.ts
run-attributes.test.ts
runtime-paths.test.ts
sqlite.test.ts
suite-schema.test.ts
agent/
__tests__/
action-docs.test.ts
assert-visual.test.ts
asserter.test.ts
failure-summary.test.ts
hack-regression.test.ts
hook-inline.test.ts
integration.test.ts
loop.test.ts
observation-hash.test.ts
planner.test.ts
prompts-action-complete.test.ts
prompts-context.test.ts
prompts-dom.test.ts
prompts-memory.test.ts
prompts-qa-mindset.test.ts
prompts-viewport-contract.test.ts
provider.test.ts
runjs-inline.test.ts
runner-runjs.test.ts
runner-variables.test.ts
runner.test.ts
token-budget.test.ts
trace.test.ts
variables.test.ts
verifier.test.ts
vision-fallback.test.ts
appium-inline.ts
asserter.ts
failure-summary.ts
hook-inline.ts
index.ts
loop.ts
observation.ts
planner.ts
prompts.ts
provider.ts
runjs-inline.ts
runner.ts
secrets.ts
token-budget.ts
types.ts
variables.ts
verifier.ts
analytics/
__tests__/
events.test.ts
identity.test.ts
posthog-transport.test.ts
run-reporter.test.ts
service.test.ts
version.test.ts
events.ts
identity.ts
index.ts
posthog-transport.ts
run-reporter.ts
service.ts
transport.ts
types.ts
artifacts/
index.ts
run-artifact.ts
auth/
auth-state/
__tests__/
auth-state.test.ts
redaction.test.ts
hook-env.ts
index.ts
redaction.ts
resolver.ts
runtime.ts
schema.ts
store.ts
fetch-wrapper.ts
index.ts
pkce.ts
plugin-loader.ts
plugin-registry.ts
resolver.ts
store.ts
types.ts
cache/
__tests__/
file-cache.test.ts
null-cache.test.ts
file-cache.ts
hasher.ts
index.ts
null-cache.ts
types.ts
farm/
__tests__/
browserstack.test.ts
browserstack.ts
capability-mapper.ts
credentials.ts
index.ts
registry.ts
types.ts
hooks/
__tests__/
migrate-hook-refs.test.ts
orchestrator.test.ts
parser.test.ts
runner-hooks.test.ts
sandbox-runner.test.ts
schema.test.ts
index.ts
migrate-hook-refs.ts
orchestrator.ts
parser.ts
sandbox-runner.ts
schema.ts
types.ts
ids/
index.ts
run-id.ts
index.ts
logging/
cli-formatter.ts
index.ts
log-manager.ts
types.ts
memory/
__tests__/
ablation.test.ts
circuit-breaker.test.ts
curator.test.ts
local-provider.test.ts
memory-config.test.ts
memory-index.test.ts
observation-id.test.ts
observation-io.test.ts
observation-schema.test.ts
security-scanner.test.ts
similarity.test.ts
xml-escape.test.ts
ablation.ts
circuit-breaker.ts
config.ts
curator.ts
factory.ts
index.ts
local-provider.ts
lock-manager.ts
memory-index.ts
observation-id.ts
observation-io.ts
provider.ts
schema.ts
security-scanner.ts
similarity.ts
types.ts
xml-escape.ts
mobile/
__tests__/
appium-executable.test.ts
element-resolver.test.ts
gestures.test.ts
launch-resolver.test.ts
xml-parser.test.ts
appium-executable.ts
element-resolver.ts
gestures.ts
index.ts
launch-resolver.ts
role-map.ts
types.ts
xml-parser.ts
parser/
__tests__/
file-resolver.test.ts
fixtures/
malformed-schema.yaml
malformed-syntax.yaml
multi-test.yaml
structured-steps.yaml
valid-test.yaml
suite-parser.test.ts
yaml-parser.test.ts
errors.ts
file-resolver.ts
index.ts
suite-parser.ts
yaml-parser.ts
provider/
__tests__/
middleware.test.ts
index.ts
middleware.ts
reporter/
__tests__/
console-reporter.test.ts
multi-reporter.test.ts
reporter-types.test.ts
screenshot.test.ts
stdout-live-reporter.test.ts
console-reporter.ts
index.ts
junit-reporter.ts
screenshot.ts
stdout-live-reporter.ts
types.ts
run-attributes.ts
runtime-paths.ts
schema/
__tests__/
device-schema.test.ts
suite-schema.test.ts
test-schema.test.ts
action-schema.ts
analytics-schema.ts
config-schema.ts
index.ts
primitives.ts
registry-schema.ts
services-schema.ts
suite-schema.ts
test-schema.ts
use-schema.ts
workspace-schema.ts
screenshot/
__tests__/
compress.test.ts
align.ts
compress.ts
image-dimensions.ts
... 658 moreFAQ
agent-qa is a Claude Code plugin with 6 hand-picked skills for testing work, indexed on Flowy. Install it with the command on its page. It includes agent-qa-authoring, agent-qa-debug-fix, agent-qa-result-triage. Its skills do not fire on their own yet. Request auto-invocation to have Flowy route them as you prompt. Free and open source.
- What is agent-qa?
- The self-improving QA agent for software teams. A test harness with memory. Write tests in natural language for web and mobile. agent-qa learns from every run, adapts to UI changes, and catches regressions before you ship.
- How do I install agent-qa?
- Run these in Claude Code: npx -y skills add vostride/agent-qa --agent claude-code. Then prompt normally.
- Does agent-qa auto-invoke its skills?
- Not yet. It is indexed on Flowy as a plain plugin. Request auto-invocation on its page and Flowy will route its skills for you as you prompt.
- Is agent-qa free?
- Yes. Flowy is free and open source, with nothing gated. You can read every skill in full before you install.