/results-analysis

name: results-analysis
description: Use when analyzing experimental results, validating comparisons, generating paper-ready results text, or turning model-evaluation outputs into figures, tables, and defensible claims.

Results Analysis for ML/AI Research

A systematic experimental results analysis workflow connecting experimental data to paper writing.

Core Features

This skill provides three core capabilities:

1. **Experimental Data Analysis** - Read and analyze experimental data in various formats 2. **Statistical Validation** - Perform statistical significance tests and performance comparisons 3. **Paper Content Generation** - Generate text and visualizations for the Results section

When to Use

Use this skill when you need to:

• Analyze experimental results (CSV, JSON, TensorBoard logs)
• Generate the Results section of a paper
• Compare performance across multiple models
• Perform statistical significance tests
• Create publication-quality visualizations
• Validate the reliability of experimental results

Workflow

Standard Analysis Pipeline

Data Loading → Data Validation → Statistical Analysis → Visualization → Writing → Quality Check

Step 1: Data Loading and Validation

**Supported Data Formats:**

• CSV files - Tabular data
• JSON files - Structured results
• TensorBoard logs - Training curves
• Python pickle - Complex objects

**Data Validation Checks:**

• Completeness check - Missing values, outliers
• Consistency check - Data format, units
• Reproducibility check - Random seeds, version info

Select appropriate tools for data loading and preliminary validation based on data format.

Step 2: Statistical Analysis

**Basic Statistics:**

• Mean
• Standard Deviation
• Standard Error
• Confidence Interval

**Significance Tests:**

• t-test - Two-group comparison
• ANOVA - Multi-group comparison
• Wilcoxon test - Non-parametric test
• Bonferroni correction - Multiple comparison correction

Select appropriate statistical tests based on data characteristics.

**Key Principles:**

• Report complete statistical information (mean ± std/SE)
• Specify the test method and significance level used
• Report p-values and effect sizes
• Consider multiple comparison issues
• Recompute examples and report values from the same unrounded inputs; mean, sample SD, n,

test statistic, P and effect size must describe the same comparison and analysis unit.

• For a mean across datasets, first form each run's mean, then estimate variation across runs;

averaging dataset SDs does not recover this uncertainty. Report only diagnostics actually run.

See `references/statistical-methods.md` for the complete statistical methods guide.

Step 3: Model Performance Comparison

**Comparison Dimensions:**

• Accuracy/Performance metrics
• Training time/Inference speed
• Model complexity/Parameter count
• Robustness/Generalization ability

**Comparison Methods:**

• Baseline comparison - Compare with existing methods
• Ablation study - Validate component contributions
• Cross-dataset validation - Test generalization

Systematically compare performance across different methods, ensuring fair comparison.

Step 4: Visualization

**Publication-Quality Visualization Requirements:**

• Vector format (PDF/EPS)
• Colorblind-friendly palette
• Clear labels and legends
• Appropriate error bars
• Readable in black-and-white print

**Common Chart Types:**

• Line chart - Training curves, trend analysis
• Bar chart - Performance comparison
• Box plot - Distribution display
• Heatmap - Correlation analysis
• Scatter plot - Relationship display

Use appropriate visualization tools to generate publication-quality figures.

See `references/visualization-best-practices.md` for the visualization guide.

Step 5: Writing the Results Section

**Results Section Structure:**

## Results

### Overview of Main Findings
[1-2 paragraphs summarizing core results]

### Experimental Setup
[Brief description of experimental configuration; details in appendix]

### Performance Comparison
[Comparison with baseline methods, including tables and figures]

### Ablation Study
[Validate contributions of each component]

### Statistical Reporting
[Effect sizes with intervals and n; inferential tests only where the design supports them, with the policy declared once in Methods]

### Qualitative Analysis
[Case studies, visualization examples]

**Writing Principles:**

• Clearly state the hypothesis each experiment validates
• Guide readers to observe key phenomena: "Figure X shows..."
• Report complete statistical information
• Report null and negative results plainly, as findings; state the scope of the work once, in the Discussion

See `references/results-writing-guide.md` for the complete writing guide.

Step 6: Quality Check

**Checklist:**

• [ ] All values include error bars/confidence intervals
• [ ] Statistical test methods are specified
• [ ] Figures are clear and readable (including black-and-white print)
• [ ] Hyperparameter search ranges are reported
• [ ] Computational resources are specified (GPU type, time)
• [ ] Random seed settings are specified
• [ ] Results are reproducible (code/data available)

Common Mistakes and Pitfalls

Statistical Errors

❌ **Wrong approach:**

• Reporting only the best results (cherry-picking)
• Confusing standard deviation and standard error
• Claiming a difference without its effect size, interval and n
• Reporting several inferential tests without a multiplicity policy

✅ **Correct approach:**

• Report all experimental results
• Clearly specify whether standard deviation or standard error is used
• Perform statistical tests where the design supports them, with the unit of replication stated
• State one multiplicity policy (for example Holm or Benjamini-Hochberg) when several tests are reported

Visualization Errors

❌ **Wrong approach:**

• Using non-colorblind-friendly palettes
• Y-axis not starting from 0 (exaggerating differences)
• M