/context-load-offline-data-platform

allowed-tools: mcp__context7__resolve-library-id, mcp__context7__get-library-docs, WebFetch, Task, Read, Write, Bash(gdate:*), Bash(fd:*), Bash(rg:*), Bash(kubectl:*), Bash(jq:*), Bash(wc:*)
name: "Context Load Offline Data Platform"
description: "Load comprehensive lakehouse architecture context with parallel documentation loading and project-specific optimization"
author: "wcygan"
tags: ["context","web"]
version: "1.0.0"
created_at: "2025-07-14T00:00:00Z"
updated_at: "2025-07-14T00:00:00Z"

/context-load-offline-data-platform

Context

• Session ID: !`gdate +%s%N 2>/dev/null || date +%s%N 2>/dev/null || echo "$(date +%s)$(jot -r 1 100000 999999 2>/dev/null || shuf -i 100000-999999 -n 1 2>/dev/null || echo $RANDOM$RANDOM)"`
• Current directory: !`pwd`
• Project structure: !`fd . -t d -d 3 | head -10 || echo "No directories found"`
• Kubernetes manifests: !`fd "\.(yaml|yml)$" . | rg -l "(kind:|apiVersion:)" | wc -l | tr -d ' ' || echo "0"`
• Data platform indicators: !`fd "(trino|iceberg|nessie|spark|airflow|ceph)" . -i | head -5 || echo "No data platform files detected"`
• Helm charts: !`fd "Chart.yaml" . | wc -l | tr -d ' ' || echo "0"`
• Docker compose files: !`fd "(docker-compose|compose)\.(yml|yaml)$" . | wc -l | tr -d ' ' || echo "0"`
• Configuration files: !`fd "\.(toml|json|conf|properties)$" . | head -5 || echo "No config files"`
• Git repository: !`git rev-parse --is-inside-work-tree 2>/dev/null && echo "Yes" || echo "No"`

Your Task

STEP 1: Initialize comprehensive lakehouse documentation loading session

• VALIDATE required tools availability (fd, rg, git, jq)
• CREATE session state file: `/tmp/context-lakehouse-$SESSION_ID.json`
• SET initial state:

  {
    "sessionId": "$SESSION_ID",
    "phase": "initialization",
    "project_context": "auto-detect",
    "documentation_sources": {
      "trino": { "status": "pending", "method": "unknown" },
      "iceberg": { "status": "pending", "method": "unknown" },
      "nessie": { "status": "pending", "method": "unknown" },
      "spark": { "status": "pending", "method": "unknown" },
      "airflow": { "status": "pending", "method": "unknown" },
      "rook_ceph": { "status": "pending", "method": "unknown" }
    },
    "context_loaded": {},
    "completion_status": "in_progress"
  }

• ANALYZE project context from detected files and structure

STEP 2: Project-specific context analysis and optimization strategy

Think deeply about the optimal documentation loading strategy based on the detected project characteristics and infrastructure.

• IF Kubernetes manifests > 0:
• PRIORITIZE Kubernetes-native deployment patterns
• FOCUS on operator-based installations and configurations
• EMPHASIZE cloud-native storage and networking considerations
• IF Docker compose files > 0:
• PRIORITIZE container-based local development setups
• FOCUS on service orchestration and networking
• EMPHASIZE development workflow optimizations
• ELSE:
• PROVIDE general lakehouse architecture guidance
• FOCUS on deployment flexibility and technology selection

STEP 3: Context7 availability detection and method selection

TRY:

• TEST Context7 MCP server availability by attempting to resolve a library ID
• UPDATE session state with detection results

IF Context7 available:

• SET documentation_method = "context7_mcp"
• EXECUTE parallel Context7 documentation loading:

1. **Trino Agent**: Load Trino distributed SQL documentation 2. **Iceberg Agent**: Load Apache Iceberg table format documentation 3. **Nessie Agent**: Load Nessie catalog version control documentation 4. **Spark Agent**: Load Apache Spark on Kubernetes documentation 5. **Airflow Agent**: Load Apache Airflow orchestration documentation 6. **Ceph Agent**: Load Rook Ceph storage documentation

ELSE:

• SET documentation_method = "webfetch_fallback"
• EXECUTE parallel WebFetch documentation loading with Task tool:

1. **Trino Documentation Agent**: `https://trino.io/docs/current/`

• FOCUS: cluster configuration, query optimization, connector setup, performance tuning

2. **Apache Iceberg Agent**: `https://iceberg.apache.org/docs/latest/`

• FOCUS: table format, partitioning, schema evolution, catalog management

3. **Nessie Agent**: `https://projectnessie.org/docs/`

• FOCUS: version control concepts, branching strategies, catalog integration

4. **Apache Spark K8s Agent**: `https://spark.apache.org/docs/latest/running-on-kubernetes.html`

• FOCUS: operator deployment, job submission, resource management

5. **Apache Airflow Agent**: `https://airflow.apache.org/docs/apache-airflow/stable/`

• FOCUS: KubernetesExecutor, DAG design, operator usage, monitoring

6. **Rook Ceph Agent**: `https://rook.io/docs/rook/latest-release/Storage-Configuration/Object-Storage-RGW/object-storage/`

• FOCUS: S3-compatible object storage, bucket policies, performance tuning, multi-site replication

CATCH (context7_unavailable OR webfetch_network_error OR tool_availability_error):

• LOG error details to session state
• PROVIDE comprehensive internal knowledge fallback
• SAVE offline documentation recommendations to `/tmp/context-lakehouse-$SESSION_ID-offline.md`
• CONTINUE with available information and graceful degradation

STEP 4: Parallel documentation synthesis and organization

• COORDINATE findings from all documentation loading agents
• ORGANIZE context by architectural layers:
• **Compute Layer**: Trino distributed SQL and Spark processing
• **Storage Layer**: Apache Iceberg table format and Rook Ceph object storage
• **Catalog Layer**: Nessie version control and metadata management
• **Orchestration Layer**: Apache Airflow workflow management
• **Infrastructure Layer**: Kubernetes deployment and networking

STEP 5: Project-specific context synthesis and recommendations

Think harder about lakehouse implementation strategies based on detected project context and infrastructure patterns.

• SYNTHESIZE project-specific r