/senior-computer-vision

name: "senior-computer-vision"
description: "Computer vision models: object detection, segmentation, training a custom model, and optimizing or deploying inference (YOLO, DETR, SAM, ONNX, TensorRT). Use whenever the user wants to detect objects in images or video, train on their own labeled images, or speed up a slow vision model."

Senior Computer Vision Engineer

Production computer vision engineering skill for object detection, image segmentation, and visual AI system deployment.

Table of Contents

• [Quick Start](#quick-start)
• [Core Expertise](#core-expertise)
• [Tech Stack](#tech-stack)
• [Workflow 1: Object Detection Pipeline](#workflow-1-object-detection-pipeline)
• [Workflow 2: Model Optimization and Deployment](#workflow-2-model-optimization-and-deployment)
• [Workflow 3: Custom Dataset Preparation](#workflow-3-custom-dataset-preparation)
• [Architecture Selection Guide](#architecture-selection-guide)
• [Reference Documentation](#reference-documentation)

Quick Start

# Generate training configuration for YOLO or Faster R-CNN
python scripts/vision_model_trainer.py models/ --task detection --arch yolov8

# Analyze model for optimization opportunities (quantization, pruning)
python scripts/inference_optimizer.py model.pt --target onnx --benchmark

# Build dataset pipeline with augmentations
python scripts/dataset_pipeline_builder.py images/ --format coco --augment

Core Expertise

This skill provides guidance on:

• **Object Detection**: YOLO family (v5-v11), Faster R-CNN, DETR, RT-DETR
• **Instance Segmentation**: Mask R-CNN, YOLACT, SOLOv2
• **Semantic Segmentation**: DeepLabV3+, SegFormer, SAM (Segment Anything)
• **Image Classification**: ResNet, EfficientNet, Vision Transformers (ViT, DeiT)
• **Video Analysis**: Object tracking (ByteTrack, SORT), action recognition
• **3D Vision**: Depth estimation, point cloud processing, NeRF
• **Production Deployment**: ONNX, TensorRT, OpenVINO, CoreML

Tech Stack

| Category | Technologies | |----------|--------------| | Frameworks | PyTorch, torchvision, timm | | Detection | Ultralytics (YOLO), Detectron2, MMDetection | | Segmentation | segment-anything, mmsegmentation | | Optimization | ONNX, TensorRT, OpenVINO, torch.compile | | Image Processing | OpenCV, Pillow, albumentations | | Annotation | CVAT, Label Studio, Roboflow | | Experiment Tracking | MLflow, Weights & Biases | | Serving | Triton Inference Server, TorchServe |

Workflow 1: Object Detection Pipeline

Use this workflow when building an object detection system from scratch.

Step 1: Define Detection Requirements

Analyze the detection task requirements:

Detection Requirements Analysis:
- Target objects: [list specific classes to detect]
- Real-time requirement: [yes/no, target FPS]
- Accuracy priority: [speed vs accuracy trade-off]
- Deployment target: [cloud GPU, edge device, mobile]
- Dataset size: [number of images, annotations per class]

Step 2: Select Detection Architecture

Choose architecture based on requirements:

| Requirement | Recommended Architecture | Why | |-------------|-------------------------|-----| | Real-time (>30 FPS) | YOLOv8/v11, RT-DETR | Single-stage, optimized for speed | | High accuracy | Faster R-CNN, DINO | Two-stage, better localization | | Small objects | YOLO + SAHI, Faster R-CNN + FPN | Multi-scale detection | | Edge deployment | YOLOv8n, MobileNetV3-SSD | Lightweight architectures | | Transformer-based | DETR, DINO, RT-DETR | End-to-end, no NMS required |

Step 3: Prepare Dataset

Convert annotations to required format:

# COCO format (recommended)
python scripts/dataset_pipeline_builder.py data/images/ \
    --annotations data/labels/ \
    --format coco \
    --split 0.8 0.1 0.1 \
    --output data/coco/

# Verify dataset
python -c "from pycocotools.coco import COCO; coco = COCO('data/coco/train.json'); print(f'Images: {len(coco.imgs)}, Categories: {len(coco.cats)}')"

Step 4: Configure Training

Generate training configuration:

# For Ultralytics YOLO
python scripts/vision_model_trainer.py data/coco/ \
    --task detection \
    --arch yolov8m \
    --epochs 100 \
    --batch 16 \
    --imgsz 640 \
    --output configs/

# For Detectron2
python scripts/vision_model_trainer.py data/coco/ \
    --task detection \
    --arch faster_rcnn_R_50_FPN \
    --framework detectron2 \
    --output configs/

Step 5: Train and Validate

# Ultralytics training
yolo detect train data=data.yaml model=yolov8m.pt epochs=100 imgsz=640

# Detectron2 training
python train_net.py --config-file configs/faster_rcnn.yaml --num-gpus 1

# Validate on test set
yolo detect val model=runs/detect/train/weights/best.pt data=data.yaml

Step 6: Evaluate Results

Key metrics to analyze:

| Metric | Target | Description | |--------|--------|-------------| | mAP@50 | >0.7 | Mean Average Precision at IoU 0.5 | | mAP@50:95 | >0.5 | COCO primary metric | | Precision | >0.8 | Low false positives | | Recall | >0.8 | Low missed detections | | Inference time | <33ms | For 30 FPS real-time |

Workflow 2: Model Optimization and Deployment

Use this workflow when preparing a trained model for production deployment.

Step 1: Benchmark Baseline Performance

# Measure current model performance
python scripts/inference_optimizer.py model.pt \
    --benchmark \
    --input-size 640 640 \
    --batch-sizes 1 4 8 16 \
    --warmup 10 \
    --iterations 100

Expected output:

Baseline Performance (PyTorch FP32):
- Batch 1: 45.2ms (22.1 FPS)
- Batch 4: 89.4ms (44.7 FPS)
- Batch 8: 165.3ms (48.4 FPS)
- Memory: 2.1 GB
- Parameters: 25.9M

Step 2: Select Optimization Strategy

| Deployment Target | Optimization Path | |-------------------|-------------------| | NVIDIA GPU (cloud) | PyTorch → ONNX → TensorRT FP16 | | NVIDIA GPU (edge) | PyTorch → TensorRT INT8 | | Intel CPU | PyTorch → ONNX → OpenVINO | | Apple Silicon | PyTorch → CoreML | | Generic CPU | PyT