/using-document-databases

name: using-document-databases
description: Document database implementation for flexible schema applications. Use when building content management, user profiles, catalogs, or event logging. Covers MongoDB (primary), DynamoDB, Firestore, schema design patterns, indexing strategies, and aggregation pipelines.

Document Database Implementation

Guide NoSQL document database selection and implementation for flexible schema applications across Python, TypeScript, Rust, and Go.

When to Use This Skill

Use document databases when applications need:

• **Flexible schemas** - Data models evolve rapidly without migrations
• **Nested structures** - JSON-like hierarchical data
• **Horizontal scaling** - Built-in sharding and replication
• **Developer velocity** - Object-to-database mapping without ORM complexity

Database Selection

Quick Decision Framework

DEPLOYMENT ENVIRONMENT?
├── AWS-Native Application → DynamoDB
│   ✓ Serverless, auto-scaling, single-digit ms latency
│   ✗ Limited query flexibility
│
├── Firebase/GCP Ecosystem → Firestore
│   ✓ Real-time sync, offline support, mobile-first
│   ✗ More expensive for heavy reads
│
└── General-Purpose/Complex Queries → MongoDB
    ✓ Rich aggregation, full-text search, vector search
    ✓ ACID transactions, self-hosted or managed

Database Comparison

| Database | Best For | Latency | Max Item | Query Language | |----------|----------|---------|----------|----------------| | **MongoDB** | General-purpose, complex queries | 1-5ms | 16MB | MQL (rich) | | **DynamoDB** | AWS serverless, predictable performance | <10ms | 400KB | PartiQL (limited) | | **Firestore** | Real-time apps, mobile-first | 50-200ms | 1MB | Firebase queries |

See `references/mongodb.md` for MongoDB details See `references/dynamodb.md` for DynamoDB single-table design See `references/firestore.md` for Firestore real-time patterns

Schema Design Patterns

Embedding vs Referencing

**Use the decision matrix in `references/schema-design-patterns.md`**

Quick guide:

| Relationship | Pattern | Example | |--------------|---------|---------| | One-to-Few | Embed | User addresses (2-3 max) | | One-to-Many | Hybrid | Blog posts → comments | | One-to-Millions | Reference | User → events (logging) | | Many-to-Many | Reference | Products ↔ Categories |

Embedding Example (MongoDB)

// User with embedded addresses
{
  _id: ObjectId("..."),
  email: "user@example.com",
  name: "Jane Doe",
  addresses: [
    {
      type: "home",
      street: "123 Main St",
      city: "Boston",
      default: true
    }
  ],
  preferences: {
    theme: "dark",
    notifications: { email: true, sms: false }
  }
}

Referencing Example (E-commerce)

// Orders reference products
{
  _id: ObjectId("..."),
  userId: ObjectId("..."),
  items: [
    {
      productId: ObjectId("..."),      // Reference
      priceAtPurchase: 49.99,          // Denormalize (historical)
      quantity: 2
    }
  ],
  totalAmount: 99.98
}

**When to denormalize:**

• Frequently read together
• Historical snapshots (prices, names)
• Read-heavy workloads

Indexing Strategies

MongoDB Index Types

// 1. Single field (unique email)
db.users.createIndex({ email: 1 }, { unique: true })

// 2. Compound index (ORDER MATTERS!)
db.orders.createIndex({ status: 1, createdAt: -1 })

// 3. Partial index (index subset)
db.orders.createIndex(
  { userId: 1 },
  { partialFilterExpression: { status: { $eq: "pending" }}}
)

// 4. TTL index (auto-delete after 30 days)
db.sessions.createIndex(
  { createdAt: 1 },
  { expireAfterSeconds: 2592000 }
)

// 5. Text index (full-text search)
db.articles.createIndex({
  title: "text",
  content: "text"
})

**Index Best Practices:**

• Add indexes for all query filters
• Compound index order: Equality → Range → Sort
• Use covering indexes (query + projection in index)
• Use `explain()` to verify index usage
• Monitor with Performance Advisor (Atlas)

**Validate indexes with the script:**

python scripts/validate_indexes.py

See `references/indexing-strategies.md` for complete guide.

MongoDB Aggregation Pipelines

**Key Operators:** `$match` (filter), `$group` (aggregate), `$lookup` (join), `$unwind` (arrays), `$project` (reshape)

**For complete pipeline patterns and examples, see:** `references/aggregation-patterns.md`

DynamoDB Single-Table Design

Design for access patterns using PK/SK patterns. Store multiple entity types in one table with composite keys.

**For complete single-table design patterns and GSI strategies, see:** `references/dynamodb.md`

Firestore Real-Time Patterns

Use `onSnapshot()` for real-time listeners and Firestore security rules for access control.

**For complete real-time patterns and security rules, see:** `references/firestore.md`

Multi-Language Examples

**Complete implementations available in `examples/` directory:**

• `examples/mongodb-fastapi/` - Python FastAPI + MongoDB
• `examples/mongodb-nextjs/` - TypeScript Next.js + MongoDB
• `examples/dynamodb-serverless/` - Python Lambda + DynamoDB
• `examples/firestore-react/` - React + Firestore real-time

Frontend Skill Integration

• **Media Skill** - Use MongoDB GridFS for large file storage with metadata
• **AI Chat Skill** - MongoDB Atlas Vector Search for semantic conversation retrieval
• **Feedback Skill** - DynamoDB for high-throughput event logging with TTL

**For integration examples, see:** `references/skill-integrations.md`

Performance Optimization

**Key practices:**

• Always use indexes for query filters (verify with `.explain()`)
• Use connection pooling (reuse clients across requests)
• Avoid collection scans in production

**For complete optimization guide, see:** `references/performance.md`

Common Patterns

**Pagination:** Use cursor-based pagination for large datasets (recommended over offset) **Soft Deletes:** Mark as deleted with timestamp instead of removing **Audit Logs:**