/fluentbit-generator

name: fluentbit-generator
description: Generate/create Fluent Bit configs — INPUT, FILTER, OUTPUT, parsers, log pipeline.

Fluent Bit Config Generator

Trigger Guidance

Use this skill when the user asks for any of the following:

• Create or update a Fluent Bit config (`fluent-bit.conf`, `parsers.conf`)
• Build a log pipeline (INPUT -> FILTER -> OUTPUT)
• Configure Kubernetes logging with metadata enrichment
• Send logs/metrics to Elasticsearch, Loki, CloudWatch, S3, Kafka, OTLP, or Prometheus remote write
• Implement parser, multiline, lua, or stream processing behavior

Do not use this skill for pure validation-only requests; use `fluentbit-validator` in that case.

Execution Flow

Follow this sequence exactly. Do not skip stages.

Stage 1: Set Working Context and Preflight

Use one of these deterministic command patterns.

# Option A (recommended): run from this skill directory
cd /Users/akinozer/GolandProjects/cc-devops-skills/devops-skills-plugin/skills/fluentbit-generator
python3 scripts/generate_config.py --help

# Option B: run from any cwd with absolute paths
python3 /Users/akinozer/GolandProjects/cc-devops-skills/devops-skills-plugin/skills/fluentbit-generator/scripts/generate_config.py --help

Preflight checks:

• Confirm `python3` is available.
• Confirm script help loads without errors.
• If using relative output paths, run from the intended target cwd.

Fallback:

• If `python3` or script execution is unavailable, switch to manual generation flow (Stage 4) using `examples/` templates.

Stage 2: Clarification Questionnaire (Explicit Template)

Collect these fields before generation.

Required questions: 1. Primary use case: `kubernetes`, `application logs`, `syslog`, `http ingest`, `metrics`, or `custom`? 2. Inputs: which sources and paths/ports (for example `tail /var/log/containers/*.log`)? 3. Outputs: destination plugin(s) and endpoint(s) (host, port, uri/topic/index/bucket)? 4. Reliability/security requirements: TLS on/off, retry expectations, buffering limits? 5. Environment context: cluster name, environment name, cloud region (if applicable)?

Optional but important: 1. Expected log format: json, regex, cri, docker, multiline stack traces? 2. Throughput profile: low/medium/high and acceptable latency? 3. Constraints: offline environment, missing binaries, read-only filesystem?

If any required answer is missing, ask focused follow-up questions before generating.

Stage 3: Decide Script vs Manual Generation

Use this decision table.

| Condition | Path | | --- | --- | | Request matches a built-in use case and only needs supported flags | Script path (Stage 5) | | Request needs uncommon plugin options not represented by script flags | Manual path (Stage 4) | | Complex multi-filter chain, custom parser/lua logic, or specialized plugin tuning | Manual path (Stage 4) | | Script cannot run in environment (missing Python/permissions) | Manual path (Stage 4) | | User explicitly requests hand-crafted config | Manual path (Stage 4) |

Supported script use cases:

• `kubernetes-elasticsearch`
• `kubernetes-loki`
• `kubernetes-cloudwatch`
• `kubernetes-opentelemetry`
• `application-multiline`
• `syslog-forward`
• `file-tail-s3`
• `http-kafka`
• `multi-destination`
• `prometheus-metrics`
• `lua-filtering`
• `stream-processor`
• `custom`

Always state the decision explicitly, including why the other path was not used.

Stage 4: Manual Generation (When Script Is Not the Best Fit)

1. Read the closest template in `examples/` first. 2. Read `examples/parsers.conf` before defining new parsers. 3. Assemble config in this order:

• `[SERVICE]`
• `[INPUT]`
• `[FILTER]`
• `[OUTPUT]`
• parser definitions (if needed)

4. Reuse known-good sections from examples and only customize required parameters. 5. Keep tags and parser references consistent across sections.

Required local template selection:

• Kubernetes + Elasticsearch: `examples/kubernetes-elasticsearch.conf`
• Kubernetes + Loki: `examples/kubernetes-loki.conf`
• Kubernetes + CloudWatch: `examples/cloudwatch.conf`
• Kubernetes + OpenTelemetry: `examples/kubernetes-opentelemetry.conf`
• App multiline: `examples/application-multiline.conf`
• Syslog forward: `examples/syslog-forward.conf`
• File to S3: `examples/file-tail-s3.conf`
• HTTP to Kafka: `examples/http-input-kafka.conf`
• Multi destination: `examples/multi-destination.conf`
• Metrics: `examples/prometheus-metrics.conf`
• Lua filtering: `examples/lua-filtering.conf`
• Stream processor: `examples/stream-processor.conf`
• Parsers: `examples/parsers.conf`

Fallback:

• If no matching example exists, start from `scripts/generate_config.py --use-case custom` output shape and extend manually.

Stage 5: Script Generation Commands (Deterministic Examples)

Run from skill directory unless output path is absolute.

cd /Users/akinozer/GolandProjects/cc-devops-skills/devops-skills-plugin/skills/fluentbit-generator

# Kubernetes -> Elasticsearch
python3 scripts/generate_config.py \
  --use-case kubernetes-elasticsearch \
  --cluster-name prod-cluster \
  --environment production \
  --es-host elasticsearch.logging.svc \
  --es-port 9200 \
  --output output/fluent-bit.conf

# Kubernetes -> OpenTelemetry
python3 scripts/generate_config.py \
  --use-case kubernetes-opentelemetry \
  --cluster-name prod-cluster \
  --environment production \
  --otlp-endpoint otel-collector.observability.svc:4318 \
  --output output/fluent-bit-otlp.conf

# File tail -> S3
python3 scripts/generate_config.py \
  --use-case file-tail-s3 \
  --log-path /var/log/app/*.log \
  --s3-bucket my-logs-bucket \
  --s3-region us-east-1 \
  --output output/fluent-bit-s3.conf

Fallback:

• If a required option is unsupported by the script, document that gap and switch to Stage 4.

Stage 6: Plugin Documentation Lookup Fallback Chain

Use this strict order when plugin behavior or parameters are uncertain.

1. Context7 (first choice)

• Resolve library id for Flu