/dnanexus-integration

name: dnanexus-integration
description: "DNAnexus cloud genomics platform. Build apps/applets, manage data (upload/download), dxpy Python SDK, run workflows, FASTQ/BAM/VCF, for genomics pipeline development and execution."

DNAnexus Integration

Overview

DNAnexus is a cloud platform for biomedical data analysis and genomics. Build and deploy apps/applets, manage data objects, run workflows, and use the dxpy Python SDK for genomics pipeline development and execution.

When to Use This Skill

This skill should be used when:

• Creating, building, or modifying DNAnexus apps/applets
• Uploading, downloading, searching, or organizing files and records
• Running analyses, monitoring jobs, creating workflows
• Writing scripts using dxpy to interact with the platform
• Setting up dxapp.json, managing dependencies, using Docker
• Processing FASTQ, BAM, VCF, or other bioinformatics files
• Managing projects, permissions, or platform resources

Core Capabilities

The skill is organized into five main areas, each with detailed reference documentation:

1. App Development

**Purpose**: Create executable programs (apps/applets) that run on the DNAnexus platform.

**Key Operations**:

• Generate app skeleton with `dx-app-wizard`
• Write Python or Bash apps with proper entry points
• Handle input/output data objects
• Deploy with `dx build` or `dx build --app`
• Test apps on the platform

**Common Use Cases**:

• Bioinformatics pipelines (alignment, variant calling)
• Data processing workflows
• Quality control and filtering
• Format conversion tools

**Reference**: See `references/app-development.md` for:

• Complete app structure and patterns
• Python entry point decorators
• Input/output handling with dxpy
• Development best practices
• Common issues and solutions

2. Data Operations

**Purpose**: Manage files, records, and other data objects on the platform.

**Key Operations**:

• Upload/download files with `dxpy.upload_local_file()` and `dxpy.download_dxfile()`
• Create and manage records with metadata
• Search for data objects by name, properties, or type
• Clone data between projects
• Manage project folders and permissions

**Common Use Cases**:

• Uploading sequencing data (FASTQ files)
• Organizing analysis results
• Searching for specific samples or experiments
• Backing up data across projects
• Managing reference genomes and annotations

**Reference**: See `references/data-operations.md` for:

• Complete file and record operations
• Data object lifecycle (open/closed states)
• Search and discovery patterns
• Project management
• Batch operations

3. Job Execution

**Purpose**: Run analyses, monitor execution, and orchestrate workflows.

**Key Operations**:

• Launch jobs with `applet.run()` or `app.run()`
• Monitor job status and logs
• Create subjobs for parallel processing
• Build and run multi-step workflows
• Chain jobs with output references

**Common Use Cases**:

• Running genomics analyses on sequencing data
• Parallel processing of multiple samples
• Multi-step analysis pipelines
• Monitoring long-running computations
• Debugging failed jobs

**Reference**: See `references/job-execution.md` for:

• Complete job lifecycle and states
• Workflow creation and orchestration
• Parallel execution patterns
• Job monitoring and debugging
• Resource management

4. Python SDK (dxpy)

**Purpose**: Programmatic access to DNAnexus platform through Python.

**Key Operations**:

• Work with data object handlers (DXFile, DXRecord, DXApplet, etc.)
• Use high-level functions for common tasks
• Make direct API calls for advanced operations
• Create links and references between objects
• Search and discover platform resources

**Common Use Cases**:

• Automation scripts for data management
• Custom analysis pipelines
• Batch processing workflows
• Integration with external tools
• Data migration and organization

**Reference**: See `references/python-sdk.md` for:

• Complete dxpy class reference
• High-level utility functions
• API method documentation
• Error handling patterns
• Common code patterns

5. Configuration and Dependencies

**Purpose**: Configure app metadata and manage dependencies.

**Key Operations**:

• Write dxapp.json with inputs, outputs, and run specs
• Install system packages (execDepends)
• Bundle custom tools and resources
• Use assets for shared dependencies
• Integrate Docker containers
• Configure instance types and timeouts

**Common Use Cases**:

• Defining app input/output specifications
• Installing bioinformatics tools (samtools, bwa, etc.)
• Managing Python package dependencies
• Using Docker images for complex environments
• Selecting computational resources

**Reference**: See `references/configuration.md` for:

• Complete dxapp.json specification
• Dependency management strategies
• Docker integration patterns
• Regional and resource configuration
• Example configurations

Quick Start Examples

Upload and Analyze Data

import dxpy

# Upload input file
input_file = dxpy.upload_local_file("sample.fastq", project="project-xxxx")

# Run analysis
job = dxpy.DXApplet("applet-xxxx").run({
    "reads": dxpy.dxlink(input_file.get_id())
})

# Wait for completion
job.wait_on_done()

# Download results
output_id = job.describe()["output"]["aligned_reads"]["$dnanexus_link"]
dxpy.download_dxfile(output_id, "aligned.bam")

Search and Download Files

import dxpy

# Find BAM files from a specific experiment
files = dxpy.find_data_objects(
    classname="file",
    name="*.bam",
    properties={"experiment": "exp001"},
    project="project-xxxx"
)

# Download each file
for file_result in files:
    file_obj = dxpy.DXFile(file_result["id"])
    filename = file_obj.describe()["name"]
    dxpy.download_dxfile(file_result["id"], filename)

Create Simple App

# src/my-app.py
import dxpy
import subprocess

@dxpy.entry_point('main')
def main(input_file, quality_threshold=30):
    # Download input
    dxpy.download_d