Does rigorpilot-skills auto-invoke its skills?

Not yet. It is indexed on Flowy as a plain plugin. Request auto-invocation on its page and Flowy will route its skills for you as you prompt.

Is rigorpilot-skills free?

Yes. Flowy is free and open source, with nothing gated. You can read every skill in full before you install.

Research

Plugin

rigorpilot-skills

Research-first Agent Skills for Deep Learning Experiments. Main idea: RigorPilot keeps AI-assisted deep learning research grounded in comparability, reproducible evidence, and auditable changes while an agent reproduces, improves, or explores a research

FLOW.md

lllllllama@lllllllama

GitHub

Plugin Signal78/100Solid

maintained92

capable58

documented100

adopted58

Install

$ npx -y skills add lllllllama/ai-paper-reproduction-skill --agent claude-code

Stats

Python Bash PowerShell Linux GitHub

+4 tags

Mermaid Anthropic OpenAI Gemini

471

Stars

Forks

Active

Maintenance

Python

Language

MIT

License

22d ago

Last commit

4mo ago

Created

Repo: lllllllama/ai-paper-reproduction-skill

In this plugin

11 skills 4 commands

See everything inside

What's inside

RigorPilot Skills

Research-first Agent Skills for Deep Learning Experiments.

Main idea: RigorPilot keeps AI-assisted deep learning research grounded in comparability, reproducible evidence, and auditable changes while an agent reproduces, improves, or explores a research repository.

Not just higher scores. Meaningful deep learning research progress.

⚡ At a Glance

Focus	Summary
🧭 Purpose	Research-first workflow skills for deep learning experiments, not a generic coding agent or score-chasing framework.
🔒 Default rule	`trusted by default`: ambiguous requests route to reproduction, setup, run, train, analysis, or safe debugging.
🧪 Exploration boundary	Explore work starts only when the researcher explicitly authorizes candidate-only exploration.
📄 Flagship output	Every reproduction ends with an annotated README: your README replayed verbatim with color-coded, evidence-linked per-section results.
🧠 Thinking loop	Exploration follows a greedy, evidence-anchored research cycle: observe → ground → hypothesize → design → run → fair compare → keep or roll back.
🌱 Continuous learning	An immutable rigor core plus a user-owned lessons overlay that personalizes safely with use.
📦 Evidence outputs	Artifacts are written to `repro_outputs/`, `analysis_outputs/`, `train_outputs/`, `debug_outputs/`, `explore_outputs/`, and related directories.
🌐 Works across agents	Skills follow the Agent Skills open standard (Claude Code, Codex, Cursor, VS Code, Gemini CLI, …); a root `AGENTS.md` routes any AGENTS.md-aware agent.

🚀 Start Fast

Most users only need one of these commands:

Goal	Command
Install the full RigorPilot skill set	`npx skills add lllllllama/rigorpilot-skills --all`
Install the trusted reproduction entrypoint	`npx skills add lllllllama/rigorpilot-skills --skill ai-research-reproduction`
Install the explicit exploration entrypoint	`npx skills add lllllllama/rigorpilot-skills --skill ai-research-explore`

Claude Code project commands:

/ai-research-reproduction
/ai-research-explore
/analyze-project
/safe-debug

The project brand is RigorPilot Skills; the recommended GitHub repository slug is rigorpilot-skills.

Preferred install source: lllllllama/rigorpilot-skills
Compatibility fallback: lllllllama/ai-paper-reproduction-skills
ai-paper-reproduction migrated to ai-research-reproduction
research-explore migrated to ai-research-explore
Existing compatible skill slugs remain supported; rigor-* names are display modes today, not install aliases.

📄 Annotated README

After a Rigor Reproduce run, repro_outputs/ANNOTATED_README.md replays the target repository's README verbatim — split into heading blocks, each followed by a color-coded annotation of what the agent actually did there. One glance shows what ran, what was skipped, and where your attention is needed; every annotation links back to the evidence bundle (SUMMARY.md · COMMANDS.md · LOG.md · status.json) for details.

Color legend: 🟢 executed successfully · 🔵 informational / not executed · ⚪ read only · 🟡 partial · 🔴 blocked · 🟣 decision required.

The header carries a rubric-style section-coverage scoreboard (e.g. 🟢 1 · 🟡 2 · 🟣 1 · 🔵 1 · ⚪ 8 (13 sections)), also recorded machine-readably as readme_section_coverage in status.json — inspired by how replication benchmarks such as PaperBench grade reproduction attempts with hierarchical rubrics rather than a single pass/fail.

What it looks like — the original section stays untouched, the colored, evidence-linked verdicts sit right below it:

Full real example, generated by actually running ai-research-reproduction on a realistic segmentation-repo README (badges, model zoo, installation, data prep, evaluation, training, FAQ, citation) — two scenarios, told by the same command before and after asset preparation:

Scenario	What you see
First attempt →	🟡 evaluation fails (checkpoint missing, real error excerpt) · 🟡 dataset not ready · 🟣 training awaits authorization
After assets are ready →	🟢 evaluation succeeds with observed `mIoU` / `aAcc`

The evidence links inside both files (SUMMARY, COMMANDS, LOG, status.json) resolve to the real generated bundle committed next to them.

🎯 Choose an Entry Point

What you want to do	RigorPilot display name	Current skill slug
Reproduce a deep learning repository from README commands	Rigor Reproduce	`ai-research-reproduction`
Analyze repository structure, entrypoints, and risks without editing	Rigor Analyze / Audit	`analyze-project`
Prepare environment, datasets, weights, and cache assumptions	Rigor Setup	`env-and-assets-bootstrap`
Run documented inference or evaluation conservatively	Rigor Run	`minimal-run-and-audit`
Start or verify training conservatively	Rigor Train	`run-train`
Debug a failure safely, diagnose before patching	Rigor Debug / Audit	`safe-debug`
Explore candidates on top of `current_research`	Rigor Explore	`ai-research-explore`
Implement candidate changes on an isolated branch	Rigor Improve	`explore-code`
Run small probes or short-cycle experiments	Rigor Explore / Improve	`explore-run`

Bundled helper skills are usually called by orchestrators:

repo-intake-and-plan
paper-context-resolver

🛣️ Lane Model

🔒 Trusted Lane

Use this lane for reproduction, setup, read-only analysis, conservative execution, training verification, and safe debugging.

Primary entrypoint: ai-research-reproduction
Output directories: repro_outputs/, train_outputs/, analysis_outputs/, debug_outputs/
Core requirement: preserve scientific meaning, minimize semantic changes, and record assumptions, blockers, and evidence.

🧪 Explore Lane

Use this lane only when the researcher explicitly authorizes candidate-only exploration.

Primary entrypoint: ai-research-explore
Leaf skills: explore-code, explore-run
Output directory: explore_outputs/
Key anchor: current_research

current_research should be a durable research state such as a branch, commit, checkpoint, run record, or already-trained local model state. Explore outputs are always candidate results. They must not claim trusted reproduction success, complete benchmark results, or verified novelty.

🔬 Core Research Principles

Do not chase scores blindly: score gains must have explanatory value.
Do not claim novelty lightly: novelty needs literature, code, or experimental evidence.
Do not break comparability silently: if evaluation conditions change, say that results are not directly comparable.
Do not disguise engineering fixes as research contributions.
Do not leave collaborators out of control: important changes must be auditable, reversible, and explainable.

See references/research-rigor-principles.md and references/agent-operating-principles.md.

🔁 Lifecycle View

The repository follows a shallow lifecycle-oriented routing model:

flowchart LR
    A[Understand] --> B[Reproduce]
    B --> C[Set up]
    C --> D[Run or train]
    D --> E[Debug]
    E --> F[Report]
    B -. explicit only .-> G[Explore]
    G --> H[Rank candidates]
    H --> F

The lifecycle helps the agent choose the right lane and evidence target. It does not force every repository into a fixed implementation sequence.

🧠 Research Thinking Loop

Agents implement well but often think in engineering steps. Once the researcher freezes the evaluation contract and explicitly authorizes exploration, ai-research-explore runs a codified greedy research cycle — from observation to a fair keep-or-rollback decision (full contract):

flowchart LR
    A[Observe run evidence] --> B[Ground: paper · code · prior runs · intuition]
    B --> C[Falsifiable hypothesis]
    C --> D[Single-variable design]
    D --> E[Bounded run]
    E --> F{Fair comparison vs current best}
    F -- better --> G[Keep as new candidate best]
    F -- worse or unfair --> H[Roll back, record why]
    G --> I[Ledger entry]
    H --> I
    I --> A

Every hypothesis carries a labeled evidence anchor — paper, code, prior-run, or intuition; unanchored ideas queue in the idea bank and never execute.
Greedy applies to selection, not honesty: a keep requires comparable evidence under the frozen contract; ties favor the simpler, cheaper change.
Underneath: hard-gated idea ranking, atomic idea decomposition, three-layer implementation fidelity (planned / heuristic / observed), and executor-emitted file-level evidence.
Lineage: adapts the greedy solution-space search of AIDE and the managed agentic tree search of AI-Scientist-v2, constrained by RigorPilot's comparability-first gates.

🌱 Continuous Learning

The shipped skills are an immutable universal rigor core; personalization lives in a user-owned overlay (policy):

Failed runs — and their later fixes — are auto-recorded as one-line lessons in ~/.rigorpilot/lessons.jsonl (opt out with RIGORPILOT_LESSONS=0).
python shared/scripts/lessons_store.py summarize distills them into ~/.rigorpilot/PERSONAL_RIGOR.md, which skills consult at run start as the researcher's standing preferences and known pitfalls.
Hard rules: lessons are advisory only — they never relax rigor gates, never store secrets, never edit skill files. Delete the folder and the skills return to the universal base.

🧾 Suggested Research Evidence

Artifact	Purpose
`SCIENTIFIC_CHANGELOG.md`	Records what changed, why it changed, whether it affects scientific meaning, and whether it remains comparable.
`COMPARABILITY_REPORT.md`	Explains whether results can still be compared to the README, paper, baseline, or SOTA reference.
`REPRODUCIBILITY_NOTES.md`	Records commands, configs, seeds, checkpoints, datasets, environment assumptions, and known gaps.
`NOVELTY_CLAIM.md`	States possible novelty as a hypothesis, with supporting evidence, missing evidence, limitations, and required ablations.
`ABLATION_PLAN.md`	Describes which variables must be isolated to validate a candidate change.
`EXPERIMENT_LEDGER.md`	Records runs, metrics, commands, artifacts, changed files, and evidence status.

SCIENTIFIC_CHANGELOG.md, COMPARABILITY_REPORT.md, and EXPERIMENT_LEDGER.md are already generated by standard trusted / explore writers. The remaining names (REPRODUCIBILITY_NOTES.md, NOVELTY_CLAIM.md, ABLATION_PLAN.md) are future-compatible evidence concepts.

📁 Output Directories

Directory	Contents
`repro_outputs/`	Trusted reproduction bundle, including `ANNOTATED_README.md`
`train_outputs/`	Trusted training bundle
`analysis_outputs/`	Read-only analysis, research map, change map, eval contract, idea seeds, atomic idea map, implementation fidelity, and related outputs
`debug_outputs/`	Safe debug diagnosis and patch plan
`sources/`	Free-first research lookup records, repo-local extraction, and auditable index
`explore_outputs/`	Changeset, idea gate, experiment plan, manifest, ledger, candidate ranking, and related outputs

🧩 Campaign Inputs

ai-research-explore still accepts variant_spec.json, but research_campaign.json or research_campaign.yaml is preferred for Rigor Explore campaigns.

Durable core fields:

current_research
task_family
dataset
benchmark
evaluation_source
sota_reference
compute_budget

Optional fields:

candidate_ideas
variant_spec
research_lookup
idea_policy
idea_generation
source_constraints
feasibility_policy

See skills/ai-research-explore/references/research-campaign-spec.md.

🌐 Multi-Agent, Multi-Model

RigorPilot is model-agnostic by construction:

Agent Skills standard — every skill is a spec-compliant SKILL.md (agentskills.io), the format adopted by Claude Code, OpenAI Codex, Cursor, VS Code, Gemini CLI, and 30+ other tools. npx skills add lllllllama/rigorpilot-skills works for any of them.
AGENTS.md routing — the root AGENTS.md gives AGENTS.md-aware agents (Codex, Cursor, Copilot, Gemini CLI, Aider, Zed, …) the lane model, entrypoint table, and hard rules without any install step.
Same contract, any model — SKILL.md instructions carry no model-specific tool syntax; the evidence bundles (status.json, ANNOTATED_README.md, …) are identical whichever model executes the run, so results stay comparable across GPT-, Claude-, and Gemini-based agents.
Per-skill client mirrors — skills/*/agents/openai.yaml and .claude/commands/* keep Codex- and Claude-specific entry points in sync with the canonical contract.

🛠️ Local Install

Use the Python installer only when developing locally, needing a project-scoped install, or manually targeting client directories.

python scripts/install_skills.py --client agents --target "$HOME/.agents/skills" --force
python scripts/install_skills.py --client codex --target "$HOME/.codex/skills" --force
python scripts/install_skills.py --client claude --target "$HOME/.claude/skills" --force

Project-scoped examples:

python scripts/install_skills.py --client agents --target ./.agents/skills --force
python scripts/install_skills.py --client claude --target ./.claude/skills --force

These commands are written to work in both Windows PowerShell and Linux shells.

💬 Example Prompts

Trusted reproduction

Use ai-research-reproduction on this deep learning research repo. Stay README-first, prefer documented inference or evaluation, avoid unnecessary repo changes, and write outputs to repro_outputs/.

Read-only analysis

Use analyze-project on this repo. Read the code, map the model and training entrypoints, and flag suspicious patterns without editing files.

Safe debug

Use safe-debug on this traceback. Diagnose the failure first, propose the smallest safe fix, and do not patch until I approve.

Candidate exploration

Use ai-research-explore with research_campaign.json. Treat the task family, dataset, evaluation source, and SOTA table as frozen inputs. Rank candidate ideas and write evidence outputs to analysis_outputs/ and explore_outputs/.

✅ Local Validation

Run everything (CI entrypoint):

python scripts/run_all_tests.py

Basic checks:

python scripts/validate_repo.py
python scripts/test_skill_registry.py
python scripts/test_trigger_boundaries.py
python scripts/test_operating_principles_structure.py
python scripts/test_claude_command_wrappers.py
python scripts/test_readme_selection.py

Core output and explore regressions:

python scripts/test_output_rendering.py
python scripts/test_readme_annotation.py
python scripts/test_train_output_rendering.py
python scripts/test_analysis_output_rendering.py
python scripts/test_safe_debug_output_rendering.py
python scripts/test_research_explore_dry_run.py
python scripts/test_research_explore_campaign_flow.py
python scripts/test_research_explore_artifact_consistency.py
python scripts/test_research_explore_variant_execution.py
python scripts/test_research_explore_nontraining_execution.py
python scripts/test_atomic_idea_decomposition.py
python scripts/test_idea_seed_generation.py
python scripts/test_implementation_fidelity.py

Install-related regressions:

python scripts/test_bootstrap_env.py
python scripts/test_install_targets.py
python scripts/test_setup_planning.py

🧭 Current Repo Snapshot

11 skills total: 9 public skills and 2 helper skills.
6 trusted-lane public skills and 3 explore-lane public skills.
4 project-scoped Claude Code wrappers under .claude/commands/.
48 Python scripts, including 45 test scripts.
Documentation and command examples are kept usable from both Windows PowerShell and Linux shells.

⚠️ Current Limits

run-train is a bounded training monitor, not a long-running training scheduler.
Trusted reproduction avoids silent semantic changes.
Helper skills stay narrow and are not public catch-all entrypoints.
Exploratory work must stay isolated from trusted baselines.
ai-research-explore is the governed Rigor Explore compatible slug, not an open-ended autonomous research agent.

📚 References

🧱 Scope

RigorPilot Skills is a research-first skill repository for deep learning experiments. It focuses on scientific meaning, comparability, reproducibility, collaborator control, and auditable workflow boundaries. It helps agents move research forward more reliably, but it does not replace researcher judgment.

File tree

lllllllama/ai-paper-reproduction-skillView on GitHub

.claude/
  commands/
    ai-research-explore.md
    ai-research-reproduction.md
    analyze-project.md
    safe-debug.md
.editorconfig
.github/
  workflows/
    validate.yml
.gitignore
AGENTS.md
assets/
  annotated-readme-preview.png
  annotated-readme-preview.zh-CN.png
CHANGELOG.md
CONTRIBUTING.md
examples/
  annotated-readme-demo/
  annotated-readme-demo-zh/
    after-setup/
      ANNOTATED_README.md
      COMMANDS.md
      COMPARABILITY_REPORT.md
      LOG.md
      SCIENTIFIC_CHANGELOG.md
      status.json
      SUMMARY.md
    first-run/
      ANNOTATED_README.md
      COMMANDS.md
      COMPARABILITY_REPORT.md
      LOG.md
      SCIENTIFIC_CHANGELOG.md
      status.json
      SUMMARY.md
    after-setup/
      ANNOTATED_README.md
      COMMANDS.md
      COMPARABILITY_REPORT.md
      LOG.md
      SCIENTIFIC_CHANGELOG.md
      status.json
      SUMMARY.md
    first-run/
      ANNOTATED_README.md
      COMMANDS.md
      COMPARABILITY_REPORT.md
      LOG.md
      SCIENTIFIC_CHANGELOG.md
      status.json
      SUMMARY.md
  example_prompt_main.md
  example_prompt_subskills.md
  real_repo_trials.md
LICENSE
README.md
README.zh-CN.md
references/
  agent-operating-principles.md
  branch-and-commit-policy.md
  client-compatibility-policy.md
  continuous-learning-policy.md
  deep-learning-experiment-principles.md
  explore-module-roadmap.md
  explore-orchestrator-design.md
  explore-variant-spec.md
  output-contract.md
  research-pitfall-checklist.md
  research-rigor-principles.md
  research-thinking-loop.md
  routing-policy.md
  skill-registry.json
  trigger-boundary-policy.md
scripts/
  install_skills.py
  run_all_tests.py
  test_analysis_output_rendering.py
  test_analyze_project_structural_hints.py
  test_atomic_idea_decomposition.py
  test_bootstrap_env.py
  test_claude_command_wrappers.py
  test_explore_output_rendering.py
  test_explore_variant_matrix.py
  test_idea_seed_generation.py
  test_implementation_fidelity.py
  test_install_targets.py
  test_lessons_store.py
  test_operating_principles_structure.py
  test_orchestrator_dry_run.py
  test_output_rendering.py
  test_readme_annotation.py
  test_readme_selection.py
  test_research_explore_artifact_consistency.py
  test_research_explore_blocked_manifest.py
  test_research_explore_campaign_abandon.py
  test_research_explore_campaign_checkpoint.py
  test_research_explore_campaign_flow.py
  test_research_explore_contracts.py
  test_research_explore_dry_run.py
  test_research_explore_execution_feasibility.py
  test_research_explore_improvement_bank.py
  test_research_explore_legacy_no_analysis_contract.py
  test_research_explore_legacy_variant_contract.py
  test_research_explore_nontraining_execution.py
  test_research_explore_smoke_report.py
  test_research_explore_source_mapping.py
  test_research_explore_variant_execution.py
  test_research_lookup_arxiv_provider.py
  test_research_lookup_cache.py
  test_research_lookup_doi_provider.py
  test_research_lookup_evidence_layers.py
  test_research_lookup_github_provider.py
  test_research_lookup_inventory_rendering.py
  test_research_lookup_repo_extractor.py
  test_research_lookup_url_provider.py
  test_safe_debug_output_rendering.py
  test_setup_planning.py
  test_skill_registry.py
  test_train_output_rendering.py
  test_training_lane_routing.py
  test_trigger_boundaries.py
  validate_repo.py
shared/
  scripts/
    lessons_store.py
    write_explore_bundle.py
    write_run_bundle.py
skills/
  ai-research-explore/
    agents/
      openai.yaml
    references/
      ai-research-explore-policy.md
      idea-evaluation-framework.md
      research-campaign-spec.md
      smoke-validation-policy.md
      source-mapping-policy.md
      sources-naming-policy.md
    scripts/
      lookup/
        __init__.py
        cache_store.py
        inventory_writer.py
        normalizers.py
        providers/
          __init__.py
          arxiv_provider.py
          base.py
          doi_provider.py
          github_provider.py
          optional_provider.py
          url_provider.py
        record_schema.py
        repo_extractors.py
        source_support.py
      orchestrate_explore.py
      passes/
        __init__.py
        atomic_idea_decomposition.py
        candidate_idea_generation.py
        execution_feasibility.py
        idea_cards.py
        idea_ranking.py
        implementation_fidelity.py
        improvement_bank.py
        lookup_sources.py
        source_mapping.py
      write_outputs.py
    SKILL.md
  ai-research-reproduction/
    agents/
      openai.yaml
    assets/
      COMMANDS.template.md
      LOG.template.md
      PATCHES.template.md
      status.template.json
      SUMMARY.template.md
    references/
      architecture.md
      language-policy.md
      output-spec.md
      patch-policy.md
      research-safety-principles.md
    scripts/
      annotate_readme.py
      orchestrate_repro.py
    SKILL.md
  analyze-project/
    agents/
      openai.yaml
    references/
      analysis-policy.md
    scripts/
      analyze_project.py
    SKILL.md
  env-and-assets-bootstrap/
    agents/
      openai.yaml
    references/
      assets-policy.md
      env-policy.md
    scripts/
      bootstrap_env.py
      bootstrap_env.sh
      plan_setup.py
      prepare_assets.py
    SKILL.md
  explore-code/
    agents/
      openai.yaml
    references/
      explore-policy.md
    scripts/
      plan_code_changes.py
      write_outputs.py
    SKILL.md
  explore-run/
    agents/
      openai.yaml
    references/
      execution-policy.md
    scripts/
      plan_variants.py
      write_outputs.py
    SKILL.md
  minimal-run-and-audit/
    agents/
      openai.yaml
    references/
      reporting-policy.md
    scripts/
      run_command.py
      write_outputs.py
    SKILL.md
  paper-context-resolver/
    agents/
      openai.yaml
    references/
      paper-assisted-reproduction.md
    SKILL.md
  repo-intake-and-plan/
    agents/
      openai.yaml
    references/
      repo-scan-rules.md
    scripts/
      extract_commands.py
      scan_repo.py
    SKILL.md
  run-train/
    agents/
      openai.yaml
    references/
      training-policy.md
    scripts/
      run_training.py
      write_outputs.py
    SKILL.md
  safe-debug/
    agents/
      openai.yaml
    references/
      debug-policy.md
    scripts/
      safe_debug.py
    SKILL.md
tests/
  readme_selection_cases.json
  trigger_cases.json

More Flows by @lllllllama.

VerifiedPlugin

Development