/composition

name: fishjam-composition
description: "Fishjam compositions: server-side, real-time mixing of several live streams (room peers, WHIP, RTMP, MP4) into new output streams with layouts, overlays, captions, and audio mixes, built on Smelter. Covers lifecycle and billing, static scenes, React templates, composing a room, and the raw Composition API. Use when building a grid or picture-in-picture of a call, restreaming a room to a livestream or to YouTube / Twitch over RTMP, or adding overlays or captions driven from a backend. Trigger on: 'Fishjam composition', 'compose a room', 'CompositionClient', 'createComposition', 'create_composition', 'registerWhipOutput', 'register_whip_output', 'registerTemplateOutput', 'register_template_output', 'updateOutput', 'update_output', 'forwardRoomTracks', 'forward_room_tracks', 'track_forwardings', '@fishjam-cloud/composition', '@fishjam-cloud/composition-cli', 'rtc.fishjam.io', 'Smelter'."
license: MIT

Fishjam Compositions

A composition takes several live streams, lays them out in a scene, and pushes the result out as new streams in real time. Rendering happens on Fishjam's servers, built on Software Mansion's Smelter.

Because the layout is rendered once on the server, every viewer receives one ready-made stream: it looks the same on every device, and a low-end phone only has to play a single video instead of decoding every participant.

> **Read `../platform/SKILL.md` first.** It defines rooms, peers, tracks, management tokens, livestream rooms, and notifications that this skill builds on.

What you can build

• **A video call turned into a broadcast.** The room's participants in a grid, speaker, or picture-in-picture layout, with names, speaking highlights, and camera-off placeholders, streamed to a livestream audience in your app.
• **Restreaming.** A call, an OBS feed, or any other input sent to YouTube, Twitch, or another RTMP platform.
• **Overlays and broadcast graphics driven by your backend.** A LIVE badge, captions, lower thirds, sponsor banners, or a score bug, toggled as things happen in your product.
• **A standby or placeholder channel.** Graphics only, or a looping video, switched to the live layout when the show starts. An output shows one scene at a time; events or updates switch it.

Mental model

inputs                       composition                 outputs
room tracks (forwarded) ──┐                            ┌─▶ WHIP → Fishjam livestream → viewers
WHIP publisher ───────────┼─▶ scene per output ────────┤
RTMP encoder ─────────────┤   video tree + audio mix   └─▶ RTMP → YouTube / Twitch / other
MP4 file URL ─────────────┘

| Concept | What it is | |---|---| | **Composition** | A rendering workspace on the Composition API (`https://rtc.fishjam.io`), authenticated with the management token (`../platform/auth-model.md`). Holds inputs, outputs, and assets; billing is per input and output, not per composition. | | **Input** | A stream coming in, addressed by an `input_id` you choose. Registered by you (WHIP, WHEP, RTMP, MP4) or created for you when a Fishjam room is forwarded. | | **Output** | A stream going out, addressed by an `output_id` you choose. Pushes over WHIP or RTMP. Each output renders its own scene. | | **Template** | A bundled React component, uploaded while registering a template output, that controls the output's scene and re-renders when its state changes. Can react to room events or events sent by your backend. | | **Static scene** | The alternative to a template: a JSON component tree (`view`, `tiles`, `rescaler`, `input_stream`, `text`, `image`) plus an audio mix, sent by your backend. | | **Assets** | Images (registered by URL) and fonts (uploaded), usable from templates and scenes. |

**A composition produces video; it does not serve it.** There is no URL a player can open. Every output must push to a destination that viewers can reach.

Designing a composition

Answer three questions: where the streams come from, where the result goes, and what draws the layout.

Where do the streams come from?

| Source | How it gets in | Read | |---|---|---| | Peers in a Fishjam room (`../platform/room-types.md`) | Forward the whole room; inputs appear per track | `room-composition.md` | | Anything else: OBS or another WHIP publisher, an RTMP encoder, a remote WHEP stream, an MP4 file | Register an input of the matching type | `inputs-and-outputs.md` |

Where does the result go?

| Destination | Output | Read | |---|---|---| | Your own web or mobile app | WHIP output into a Fishjam livestream room. Your app's users join that room as livestream viewers with the client SDK (`../react-client/livestream.md`, same hooks in React Native). | `inputs-and-outputs.md` | | YouTube, Twitch, or another streaming platform | RTMP output to the platform's ingest URL and stream key | `inputs-and-outputs.md` |

What draws the layout?

**Use a template unless the layout is fixed.** Most real compositions react to something: peers joining a call, someone speaking, a caption or "live" badge toggled from the backend. With static scenes your backend has to track all of that state and rebuild and resend the scene on every change; a template does it inside the composition.

| | Template (recommended) | Static scene | |---|---|---| | Layout logic | React component running next to the composition | JSON tree built by your backend | | Room peers join or leave | Re-renders by itself (`usePeers`) | Backend rebuilds and resends the whole scene | | Backend-driven change | `sendEvent` with a small payload | `updateOutput` with the whole scene | | Extra setup | Build step with `@fishjam-cloud/composition-cli` | None | | Pick it for | Room composition, overlays, captions, anything dynamic | A few known inputs in a fixed layout, quick prototypes |

Templates are written in TypeScript and React whatever the backend language; a Python backend uploads the built bundle the same way. Read `templates.md` for templates and `scen