/rag-chunking-strategy

name: rag-chunking-strategy
description: Document chunking with multiple strategies including semantic, recursive, and fixed-size chunking
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash
  - Glob
  - Grep
graph:
  domains: [domain:software-engineering]
  specializations: [specialization:ai-agents-conversational]
  skillAreas: [skill-area:retrieval-augmented-generation, skill-area:natural-language-processing]
  roles: [role:ml-engineer, role:backend-engineer]
  workflows: [workflow:ml-model-lifecycle, workflow:feature-development]

RAG Chunking Strategy Skill

Capabilities

• Implement multiple document chunking strategies
• Configure semantic chunking based on content boundaries
• Set up recursive character text splitting
• Design fixed-size chunking with overlap
• Implement document-aware chunking (markdown, code, etc.)
• Optimize chunk sizes for retrieval quality

Target Processes

• rag-pipeline-implementation
• chunking-strategy-design

Implementation Details

Chunking Strategies

1. **RecursiveCharacterTextSplitter**: Hierarchical splitting with separators 2. **SemanticChunker**: Embedding-based semantic boundaries 3. **TokenTextSplitter**: Token-aware splitting 4. **MarkdownHeaderTextSplitter**: Structure-aware markdown splitting 5. **CodeSplitter**: Language-aware code chunking

Configuration Options

• Chunk size (characters or tokens)
• Chunk overlap percentage
• Separator hierarchy
• Embedding model for semantic chunking
• Document type detection

Best Practices

• Match chunk size to embedding model limits
• Use appropriate overlap for context preservation
• Test retrieval quality with different strategies
• Consider document structure in strategy selection

Dependencies

• langchain-text-splitters
• sentence-transformers (for semantic chunking)