/inno-experiment-analysis

id: inno-experiment-analysis
name: inno-experiment-analysis
version: 0.1.0
description: |-
  This skill should be used when the user asks to "analyze experimental results", "generate results section", "statistical analysis of experiments", "compare model performance", "create results visualization", or mentions connecting experime...
stages: ["experiment"]
tools: ["read_file", "search_project", "write_file"]
summary: |-
  This skill should be used when the user asks to "analyze experimental results", "generate results section", "statistical analysis of experiments", "compare model performance", "create results visualization", or mentions connecting experime...
primaryIntent: evaluation
intents: ["evaluation", "experiment"]
capabilities: ["evaluation-benchmarking"]
domains: ["general"]
keywords: ["research", "analysis", "statistics", "visualization", "paper writing", "inno-experiment-analysis", "evaluation-benchmarking", "inno", "experiment", "this", "should", "be"]
source: builtin
status: verified
upstream:
  repo: dr-claw
  path: skills/inno-experiment-analysis
  revision: 8322dc4ef575affaa374aa7922c0a0971c6db7d7
resourceFlags:
  hasReferences: true
  hasScripts: false
  hasTemplates: false
  hasAssets: false
  referenceCount: 4
  scriptCount: 0
  templateCount: 0
  assetCount: 0
  optionalScripts: false

inno-experiment-analysis

Canonical Summary

This skill should be used when the user asks to "analyze experimental results", "generate results section", "statistical analysis of experiments", "compare model performance", "create results visualization", or mentions connecting experime...

Trigger Rules

Use this skill when the user request matches its research workflow scope. Prefer the bundled resources instead of recreating templates or reference material. Keep outputs traceable to project files, citations, scripts, or upstream evidence.

Resource Use Rules

• Read from `references/` only when the current task needs the extra detail.

Execution Contract

• Resolve every relative path from this skill directory first.
• Prefer inspection before mutation when invoking bundled scripts.
• If a required runtime, CLI, credential, or API is unavailable, explain the blocker and continue with the best manual fallback instead of silently skipping the step.
• Do not write generated artifacts back into the skill directory; save them inside the active project workspace.

Upstream Instructions

Results Analysis for ML/AI Research

A systematic experimental results analysis workflow connecting experimental data to paper writing.

Core Features

This skill provides three core capabilities:

1. **Experimental Data Analysis** - Read and analyze experimental data in various formats 2. **Statistical Validation** - Perform statistical significance tests and performance comparisons 3. **Paper Content Generation** - Generate text and visualizations for the Results section

When to Use

Use this skill when you need to:

• Analyze experimental results (CSV, JSON, TensorBoard logs)
• Generate the Results section of a paper
• Compare performance across multiple models
• Perform statistical significance tests
• Create publication-quality visualizations
• Validate the reliability of experimental results

Workflow

Standard Analysis Pipeline

Data Loading → Data Validation → Statistical Analysis → Visualization → Writing → Quality Check

Step 1: Data Loading and Validation

**Supported Data Formats:**

• CSV files - Tabular data
• JSON files - Structured results
• TensorBoard logs - Training curves
• Python pickle - Complex objects

**Data Validation Checks:**

• Completeness check - Missing values, outliers
• Consistency check - Data format, units
• Reproducibility check - Random seeds, version info

Select appropriate tools for data loading and preliminary validation based on data format.

Step 2: Statistical Analysis

**Basic Statistics:**

• Mean
• Standard Deviation
• Standard Error
• Confidence Interval

**Significance Tests:**

• t-test - Two-group comparison
• ANOVA - Multi-group comparison
• Wilcoxon test - Non-parametric test
• Bonferroni correction - Multiple comparison correction

Select appropriate statistical tests based on data characteristics.

**Key Principles:**

• Report complete statistical information (mean ± std/SE)
• Specify the test method and significance level used
• Report p-values and effect sizes
• Consider multiple comparison issues

See `references/statistical-methods.md` for the complete statistical methods guide.

Step 3: Model Performance Comparison

**Comparison Dimensions:**

• Accuracy/Performance metrics
• Training time/Inference speed
• Model complexity/Parameter count
• Robustness/Generalization ability

**Comparison Methods:**

• Baseline comparison - Compare with existing methods
• Ablation study - Validate component contributions
• Cross-dataset validation - Test generalization

Systematically compare performance across different methods, ensuring fair comparison.

Step 4: Visualization

**Publication-Quality Visualization Requirements:**

• Vector format (PDF/EPS)
• Colorblind-friendly palette
• Clear labels and legends
• Appropriate error bars
• Readable in black-and-white print

**Common Chart Types:**

• Line chart - Training curves, trend analysis
• Bar chart - Performance comparison
• Box plot - Distribution display
• Heatmap - Correlation analysis
• Scatter plot - Relationship display

Use appropriate visualization tools to generate publication-quality figures.

See `references/visualization-best-practices.md` for the visualization guide.

Step 5: Writing the Results Section

**Results Section Structure:**

## Results

### Overview of Main Findings
[1-2 paragraphs summarizing core results]

### Experimental Setup
[Brief description of experimental configuration; details in appendix]

### Performance Comparison
[Comparison with baseline methods, including tables and figures]

### Ablation Study
[Va