/api-integration

name: memstack-automation-api-integration
description: "Use this skill when the user says 'API integration', 'connect APIs', 'sync data', 'data mapping', 'rate limiting', or needs system-to-system connectors with authentication, rate limit handling, and error recovery. Generates API integration code with authentication (OAuth, API key, JWT), request/response mapping, rate limit handling, error recovery with circuit breakers, and sync monitoring. Do NOT use for visual n8n workflows or webhook receiving."
version: 1.0.0
license: "Proprietary, MemStack™ Pro by CW Affiliate Investments LLC. See LICENSE.txt"

API Integration: Building system connector...

*Develops system-to-system connectors with REST/GraphQL patterns, authentication flows, rate limit handling, data mapping, error recovery, and SDK wrapper generation.*

Activation

When this skill activates, output:

`API Integration: Building system connector...`

Then execute the protocol below.

Context Guard

| Context | Status | |---------|--------| | User says "API integration", "connect APIs", "sync data" | ACTIVE | | User says "data mapping" or "rate limiting" | ACTIVE | | User needs to build a connector between two systems | ACTIVE | | User wants a visual n8n workflow | DORMANT: use n8n Workflow Builder | | User wants to receive webhooks | DORMANT: use Webhook Designer |

Common Mistakes

| Mistake | Why It's Wrong | |---------|---------------| | "Ignore rate limits" | Getting blocked by the API wastes hours of debugging. Implement rate limiting from day one. | | "No retry logic" | APIs have transient failures. Without retry + backoff, your sync silently drops data. | | "Store tokens in code" | Use environment variables or a secrets manager. Tokens in code end up in git history. | | "Map fields manually every time" | Build a reusable mapping layer. Manual field-by-field transforms are fragile and hard to update. | | "No pagination handling" | Most APIs return paginated results. If you only read page 1, you're missing data. |

Protocol

Step 1: Gather Integration Requirements

If the user hasn't provided details, ask:

> 1. **Source system**: where does the data come from? (API name, docs URL) > 2. **Destination**: where does it go? (your DB, another API, file) > 3. **Data**: what entities are synced? (users, orders, products, events) > 4. **Direction**: one-way, two-way, or event-driven? > 5. **Auth**: how does the API authenticate? (API key, OAuth 2.0, JWT, Basic) > 6. **Volume**: how much data? How often? (1K records/day vs 1M)

Step 2: Implement Authentication

| Auth Type | Implementation | Token Lifecycle | |-----------|---------------|----------------| | **API Key** | Header: `X-API-Key: {key}` or query param | Static: rotate manually | | **Bearer Token** | Header: `Authorization: Bearer {token}` | Expires: refresh needed | | **OAuth 2.0** | Auth code flow → access + refresh tokens | Auto-refresh on 401 | | **JWT** | Sign claims → `Authorization: Bearer {jwt}` | Short-lived: re-sign | | **Basic Auth** | Header: `Authorization: Basic {base64}` | Static | | **HMAC** | Sign request body → custom header | Per-request signing |

**OAuth 2.0 token refresh pattern:**

class ApiClient {
  private accessToken: string;
  private refreshToken: string;
  private expiresAt: number;

  async request(method: string, path: string, body?: any): Promise<any> {
    if (Date.now() >= this.expiresAt - 60_000) { // Refresh 60s early
      await this.refreshAccessToken();
    }

    const response = await fetch(`${this.baseUrl}${path}`, {
      method,
      headers: {
        'Authorization': `Bearer ${this.accessToken}`,
        'Content-Type': 'application/json',
      },
      body: body ? JSON.stringify(body) : undefined,
    });

    if (response.status === 401) {
      await this.refreshAccessToken();
      return this.request(method, path, body); // Retry once
    }

    return response.json();
  }
}

Step 3: Handle Rate Limiting

**Rate limit detection and backoff:**

async function requestWithRateLimit(
  fn: () => Promise<Response>,
  maxRetries = 3
): Promise<Response> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const response = await fn();

    if (response.status === 429) {
      const retryAfter = parseInt(response.headers.get('Retry-After') || '0');
      const waitMs = retryAfter > 0
        ? retryAfter * 1000
        : Math.min(1000 * Math.pow(2, attempt), 30_000); // Exponential backoff, max 30s

      console.log(`Rate limited. Waiting ${waitMs}ms (attempt ${attempt + 1})`);
      await sleep(waitMs);
      continue;
    }

    return response;
  }
  throw new Error('Rate limit exceeded after max retries');
}

**Proactive rate limiting (token bucket):**

class RateLimiter {
  private tokens: number;
  private lastRefill: number;

  constructor(
    private maxTokens: number,    // e.g., 100
    private refillRate: number,   // tokens per second, e.g., 10
  ) {
    this.tokens = maxTokens;
    this.lastRefill = Date.now();
  }

  async acquire(): Promise<void> {
    this.refill();
    if (this.tokens <= 0) {
      const waitMs = (1 / this.refillRate) * 1000;
      await sleep(waitMs);
      this.refill();
    }
    this.tokens--;
  }

  private refill(): void {
    const now = Date.now();
    const elapsed = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.maxTokens, this.tokens + elapsed * this.refillRate);
    this.lastRefill = now;
  }
}

Step 4: Build Data Mapper

**Mapping layer pattern:**

// Define field mappings declaratively
interface FieldMapping {
  source: string;        // Source field path (dot notation)
  target: string;        // Target field path
  transform?: (val: any) => any;  // Optional transformation
  required?: boolean;
}

const orderMappings: FieldMapping[] = [
  { source: 'id',              target: 'externalId',  required: true },
  { source: 'customer.email',  ta