# OctoHub 0.8.0: The Proxy Grew a Unified Media API

> OctoHub 0.8.0 puts image, video, speech and transcription behind the same self-hosted front door as your chat completions — one JSON envelope for four tasks, five providers, queued jobs that survive a restart, and a per-request cost that says "unpriced" instead of guessing zero. Built on octolib 0.36.1. Open source, Rust, Apache-2.0.

# OctoHub 0.8.0: The Proxy Grew a Unified Media API

A month ago the question that made us build [OctoHub](/blog/introducing-octohub-llm-proxy-for-observability) was: your agent made forty model calls — what did it send, what did it cost, and which upstream answered?

Here is the same question with the money moved. Your agent generated eleven images while iterating on a design, then a nine-second video, then read the summary aloud in a synthetic voice. Four providers were involved. Three of them bill in units that aren't tokens — GPU-seconds, video-seconds, characters. One of them didn't tell you what it cost at all. Now: which customer do you charge?

For chat completions we'd answered that. For everything else, the answer was still a spreadsheet.

**OctoHub 0.8.0 closes the gap.** Image generation, video generation, speech synthesis and transcription now run through the same proxy, the same client keys, the same allow-lists, the same request log and the same cost column as `/v1/completions`. Five providers — fal, ElevenLabs, Replicate, Runway and OpenRouter — behind one JSON envelope.

---

## The division of labor that made it small

This landed on top of [octolib](/blog/octolib-the-engine-behind-our-ai-stack) **0.36.1**, tagged the same afternoon, which is where the media stack actually lives: typed request structs for the four tasks, the provider adapters, the job lifecycle, the capability descriptors, and a reference rate table for pricing what providers won't price themselves. We covered that release in [the early-September round-up](/blog/release-round-early-september-2026).

The OctoHub side follows from one rule: **anything about models belongs in octolib, anything about tenants belongs here.** Routing grammar, adapters, job handles and pricing are octolib's. Keys, quotas, persistence, metrics and the wire API are OctoHub's.

That rule is why the config gained no new syntax. A media alias is a model alias:

```toml
[media_models]
"flux" = ["fal:fal-ai/flux/dev", "replicate:black-forest-labs/flux-1.1-pro"]
"veo"  = ["openrouter:google/veo-3.1"]
"tts"  = ["elevenlabs:eleven_flash_v2_5"]

[providers.fal]        # concurrency and rate windows, unchanged
concurrency = 8
requests_per_minute = 60
```

Same `provider:model` grammar, same list-means-load-balance semantics, same per-provider limiter. Media never _feeds_ `tokens_per_minute` — media providers report no tokens — but the windows are keyed by provider name and shared with completions, so on a provider you use for both, a token budget your chat traffic already spent will turn a media request away.

Bad configuration fails at boot rather than on the first paid request: an unknown provider, a malformed `provider:model`, an empty mirror list, or an alias that collides with `[models]` all refuse to start.

---

## Four tasks, one envelope

```
POST /v1/images/generations     generate | edit | inpaint | variation
POST /v1/videos                 text_to_video | image_to_video | reference_to_video | extend | edit
POST /v1/audio/speech
POST /v1/audio/transcriptions
GET  /v1/media/{id}             fetch or advance a job
POST /v1/media/{id}/cancel
GET  /v1/media/models           capabilities, parameters, reference price
```

They're client endpoints, authenticated exactly like completions — same bearer key, same per-key model allow-list, same `X-Request-Id` correlation:

```bash
curl -sX POST http://127.0.0.1:8080/v1/images/generations \
  -H "Authorization: Bearer <client-key>" \
  -d '{"model":"flux","prompt":"a red panda astronaut","count":2,"size":"1024x1024"}'
```

Every response — image, video, voice, transcript, finished or still running — is the same object:

```jsonc
{
  "id": "med_9f3c1e0b…", "object": "media", "task": "text_to_image",
  "status": "succeeded", "model": "fal-ai/flux/dev", "provider": "fal",
  "progress": 1.0,
  "artifacts": [ { "kind": "image", "media_type": "image/png",
                   "source": { "type": "url", "value": "https://…" },
                   "size_bytes": 812345, "expires_at": 1767225600 } ],
  "usage": { "cost": 0.08, "cost_source": "provider", "currency": "USD", … },
  "warnings": [], "safety": { "status": "passed", … }, "error": null
}
```

One shape means one code path for persistence, one row format for the log, and one thing for your client to parse. Transcription is the only task whose payload isn't an artifact, so it adds a `result` object with `text`, `language`, `segments` and `words`.

Two deliberate deviations from OpenAI's API, stated up front rather than discovered: **everything is JSON, never `multipart/form-data`** — binary inputs are `{"type":"url"…}` or `{"type":"base64"…}` objects — and image edit, inpaint and variation are a `mode` field rather than separate paths. `/v1/images/generations` borrows OpenAI's path and its `model` / `prompt` / `size` spellings, but not its wire format — image count is `count`, not `n`, and the reply is the envelope above rather than OpenAI's `{"created", "data"}`. An OpenAI SDK won't parse it; call it over plain HTTP or a thin wrapper.

---

## A job that outlives the request that started it

This is the part that genuinely differs from proxying a chat completion, and it's where the design decisions are.

A completion is one call that returns or fails. **A media job commits money upstream the moment the provider accepts it, and then may run for minutes.** A video is not a slow request; it's a purchase followed by a wait. Everything else follows from taking that seriously:

**The row is written before the wait, not after.** As soon as a queue-based provider accepts — fal, Replicate, Runway, OpenRouter video — OctoHub persists the record and the credential-free `JobHandle`, and only then starts waiting. A restart, a timeout, a client that hangs up: none of them can orphan a job you already paid for. The handle is in your database and the job is resumable from it. (ElevenLabs and OpenRouter's synchronous endpoints have no queue to hand back a handle for — they do the whole job inside the submit call, so there is no window to be interrupted in and nothing to resume.)

**`202` is not a failure.** Send `wait: false` and you get the id as soon as the provider accepts. Send `wait: true` and exceed `server.upstream_timeout_secs` and you get the same `202` with `status: "queued"` or `"running"`. The remote work continues; the id is live; nothing was lost. Poll `GET /v1/media/{id}` when you're ready.

**Polling is what advances a job — there is no background worker.** A deliberate non-goal: a worker would mean a scheduler, leases, and a second failure mode for jobs nobody is waiting on. The consequence is stated in the docs rather than hidden: a job you never poll stays `queued` and its cost is never recorded. Reading a job that already finished is free and never re-bills — a terminal row is served from the database with no upstream call at all.

**The provider permit covers submit only.** OctoHub's per-provider concurrency limiter guards the submit call, then releases. A four-minute video does not pin one of your eight fal slots for four minutes. The per-tenant slot is held for the whole request, though, so a customer's media work drains the same budget as their completions.

**Failover happens at submit, where it's safe.** Turn on `server.failover_on_error` — off by default, same as for completions — and a provider fault at submit drops that candidate and hands the request to the next mirror in the alias. The fault also counts toward that provider's failure streak: set `server.provider_error_cooldown_secs` (`0`, off, by default) and three consecutive provider-side failures put it on cooldown, which sorts it behind healthy candidates rather than blocking it. Left at the defaults, the fault goes straight back to the caller. Either way, once a job is _accepted_ there's nothing to fail over — it's been paid for.

Records are scoped to the key that created them. Another tenant's id returns `404`, not `403` — you don't get to learn that an id exists.

---

## The parameter problem, and the honest answer to it

Every media provider has a different idea of what a request looks like. fal wants `num_inference_steps` and `guidance_scale`; some endpoints call the prompt `text`; Runway sells credits and thinks in its own model names. A unified API has to decide what to do about that, and there are two bad answers: expose only the intersection (useless), or invent a translation layer that pretends everything is the same (lies, expensively).

OctoHub's answer is three parts:

**A portable core with one spelling everywhere** — `prompt`, `count`, `seed`, `size`, `duration_secs`, `negative_prompt`, `output_format`. `size` takes `"1024x1024"` or `"16:9"`; anything else is a `400`. Portable means one name, not universal support: Runway has no equivalent for `count`, `negative_prompt` or `output_format`, and neither does OpenRouter's video endpoint, so under the default strict policy those come back as a `400` on those providers rather than being quietly ignored — which is the third part below.

**An escape hatch that passes anything through verbatim**, namespaced by provider:

```jsonc
"provider_options": {
  "fal": { "input": { "num_inference_steps": 28, "guidance_scale": 3.5 },
           "field_map": { "prompt": "text" } }
}
```

`field_map` remaps a portable name onto whatever the endpoint actually calls it — so portable `prompt` keeps working against an endpoint whose field is `text`. Send every namespace at once when an alias spans providers; only the winning candidate's namespace is forwarded and the rest are dropped, which is what makes a multi-provider alias usable at all.

**A policy for what happens when a parameter can't be honored.** `unsupported_parameters: "error"` (the default) fails _before money is spent_ — correct for production. `"warn_and_drop"` drops it and returns a warning — useful when one alias fans out across providers with uneven support. You choose per request, because only you know which one you meant.

And `GET /v1/media/models` tells you which is which before you spend anything: each configured candidate's execution and parameter capability flags, limits, its adapter's own `provider_options` JSON Schema, and the reference price. Two rough edges in 0.8.0, since you'd find them anyway: discovery probes every candidate through the image adapter, so the `tasks` field reads `["text_to_image"]` even for a video alias, and an `elevenlabs` candidate has no image adapter at all — it comes back with a price and a `null` descriptor. Many capability fields honestly read `unknown` — the adapters cannot know every endpoint's schema, and saying so is better than a confident wrong answer. That's precisely why the escape hatch exists.

---

## Cost that refuses to guess

This is the feature the release is really about, and the one place we were most stubborn.

`usage.cost` is the number billed. `usage.cost_source` says where it came from:

| `cost_source` | Meaning                                                                 |
| ------------- | ----------------------------------------------------------------------- |
| `provider`    | The upstream returned actual dollars. OpenRouter and Replicate do this. |
| `estimate`    | Computed locally from octolib's reference rate table.                   |
| `unavailable` | Nothing could price it — `cost` is `null`.                              |

**`null` is not zero.** A request nothing could price is recorded as unpriced, never as free. It shows up in `octohub_media_cost_unknown_total` and carries a `cost_unavailable` warning, instead of silently dragging your spend total down and making a dashboard look better than reality.

The estimates come from octolib's reference table, which is unit-aware because the providers are: ElevenLabs bills characters, Runway sells seconds of video converted from credits, fal falls back to GPU wall-clock because that's the only quantity its queue metrics report. Where a rate would be a guess, there is no rate — Replicate community models bill GPU-seconds against an unknown GPU class, so they resolve to nothing and stay unpriced rather than get stamped with a plausible number.

The known gap, said out loud: **ElevenLabs transcription is unpriced.** Scribe bills input-audio duration, which isn't reported back, and no reference rate covers it. Transcription elsewhere does get a number — fal falls through to its per-GPU-second catch-all, and Replicate and OpenRouter price from whatever dollar amount the upstream reports. When we can price Scribe, we will; until then it's `unavailable`, not `0.00`.

On the aggregate side, `GET /v1/admin/usage` gains `media_count` and `total_cost` — which now sums completions, embeddings _and_ media into the one number you'd actually put on an invoice. `GET /v1/admin/media` lists individual records with the same filters as the other two, newest first, in-flight jobs included with a non-terminal status and a `null` `completed_at`. A job _is_ its record at an earlier status; there's no separate queue to inspect.

Prometheus, per task, model and provider — plus an `api_key_id` label on the request counter when `metrics.per_key` is on. The cost counters stay unlabelled by key; per-tenant spend comes from `GET /v1/admin/usage`, where it's exact rather than sampled:

```
octohub_media_requests_total{task,model,provider,status}
octohub_media_duration_seconds{task,model,provider}
octohub_media_cost_microusd_total{task,model,provider,source}
octohub_media_cost_unknown_total{task,model,provider}
```

Costs are counted in micro-USD because a counter of dollars for images that cost $0.003 is a rounding-error generator. There's deliberately no outstanding-jobs gauge: an accurate one would have to count non-terminal rows in the database, and an in-process counter would be wrong the moment a job is polled by a different replica or survives a restart.

---

## What we said no to

The features that aren't here are as load-bearing as the ones that are:

**OctoHub does not become a blob store.** There's no object store, no CDN, no artifact lifecycle to run, and it never fetches an artifact on your behalf. A provider that answers with a URL is stored as a URL — the row holds a link and metadata, nothing more, and that's most of the traffic. A provider that answers with the payload itself is the exception you should size for: ElevenLabs speech always does, and fal, Replicate or OpenRouter sometimes do. Those bytes are base64'd into the response, and the same base64 is persisted in the record's `result` column — because that row is exactly what a later `GET /v1/media/{id}` replays without touching the upstream. An MP3 in the `media` table costs about 4/3 of its own size, so a heavy text-to-speech deployment should plan for a table that is part ledger, part media library.

**Client-supplied file paths are rejected.** octolib's `MediaSource` supports `file`, `provider_file` and `object_storage`; all three return `400` here. A path in a request to a _server_ is a request to read the server's filesystem — an SSRF/LFI hole, not a feature. Inline base64 is size-checked against `media.max_source_bytes` (20 MiB default) before anything reaches a provider, and the payload never lands in the database: the stored request keeps the shape and the byte count, not the bytes.

**Upstream credentials stay on the server.** Provider keys come from the server's environment (`FAL_API_KEY`, `ELEVENLABS_API_KEY`, and so on) and are never accepted from a client. `provider_options.<provider>.cost_estimate` is rejected outright — pricing is resolved server-side, and a client doesn't get to tell you what it owes you.

Also not in 0.8.0: streaming TTS, and `multipart/form-data`. Both are additive when someone actually needs them — though streaming will need its own answer for cost first, because octolib's streaming speech path reports no usage at all.

---

## Upgrading

The `media` table is created on startup alongside the others, on SQLite, MySQL and PostgreSQL — no migration step. Existing config keeps working: if you add no `[media_models]`, the new endpoints simply have nothing to route and the rest of the proxy behaves exactly as it did in 0.7.

```bash
# Linux x86_64, static musl build
curl -fsSL https://github.com/Muvon/octohub/releases/download/0.8.0/octohub-0.8.0-x86_64-unknown-linux-musl.tar.gz | tar xz
./octohub
```

Binaries ship for six targets (Linux musl, macOS, Windows — x86_64 and ARM64), or `cargo build --release` from source.

The release is marked breaking for one reason: the `Storage` trait grew media methods. That matters only if you maintain your own storage backend against OctoHub's internals; if you run the binary, there is nothing to change. One smaller thing worth knowing: since 0.7.12, OctoHub forwards attribution headers end to end, and as of this cycle it identifies itself upstream as `Octohub/<version>` rather than the generic octolib default — so provider-side dashboards name the proxy that made the call.

---

## The point

The reason to put a proxy in front of your models was never the routing. It was that one place every request passes through is the only place the whole truth exists. That argument doesn't get weaker when the requests start returning pixels and audio instead of tokens — it gets stronger, because media is where the per-request cost stops being a rounding error and starts being the bill.

As of 0.8.0 the answer to "which customer generated this video, on which provider, and what did it cost" is one row in your own database, next to the chat completions, in the same currency, with a column that admits it when nobody knows.

— Don

_OctoHub is open source under Apache-2.0, developed by [Muvon Un Limited](https://muvon.io). Get it on [GitHub](https://github.com/Muvon/octohub) — issues and pull requests welcome. Full media documentation: [doc/11-media.md](https://github.com/Muvon/octohub/blob/master/doc/11-media.md)._