kenzuya/unshackle
1
0
Fork 2
mirror of https://github.com/unshackle-dl/unshackle.git synced 2026-11-12 12:05:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: apply simplified technical english across docs and docstrings.

Browse Source
This commit is contained in:
imSp4rky
2026-08-19 16:31:06 -06:00
parent 4177cb2267
commit 66971230c3
135 changed files with 2651 additions and 2570 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/dev/rest-api/authentication.md
+43 -43
View File
@@ -1,16 +1,16 @@
# REST API Authentication
The unshackle REST API protects every endpoint (with one exception) behind a single API key. Clients present that key in a request header, and the server checks it against the keys defined in your configuration. This page explains how that check works, how to configure keys, how clients send them, and the extra controls that build on top of the key: per-key service allowlists, per-key CDM device access, and session IP binding.
The unshackle REST API protects every endpoint (with one exception) behind a single API key. Clients present that API key in a request header, and the server examines it against the API keys in your configuration. This page explains how that check works, how to configure API keys, and how clients send them. It also explains the extra controls on top of the API key: per-key service allowlists, per-key CDM device access, and remote session IP binding.
The API is started with the [`unshackle serve`](../../guide/cli-reference.md) command. Authentication is on by default: if you start the server without an API key configured, it refuses to launch.
You start the API with the [`unshackle serve`](../../guide/cli-reference.md) command. Authentication is on by default: if you start the server without an API key configured, it refuses to launch.
!!! note "Which server am I running?"
`unshackle serve` can run in two shapes, and each uses a slightly different piece of code to check your key:
`unshackle serve` can operate in two shapes, and each uses a slightly different piece of code to examine your API key:
- The **integrated server** (the default) exposes the REST API *and* the pywidevine / pyplayready CDM HTTP endpoints.
- The **API-only server** (`--api-only`, also implied by `--remote-only`) exposes only the REST API.
The header, the config, and the behaviour you see as a client are the same in both. The differences are internal and are called out in the developer notes below.
The header, the config, and the behaviour you see as a client are the same in both. The differences are internal, and the developer notes below give them.
## The short version
@@ -36,7 +36,7 @@ serve:
api_secret: "a-long-random-secret-string"
```
This value becomes the one and only key that the server accepts. Requests that present it in the `X-Secret-Key` header are allowed; everything else is rejected.
This value becomes the one and only API key that the server accepts. The server allows requests that present it in the `X-Secret-Key` header. The server rejects everything else.
!!! warning "The server will not start without a key"
If you start `unshackle serve` with no `api_secret` configured, and you have *not* passed `--no-key`, the command fails immediately with:
@@ -45,11 +45,11 @@ This value becomes the one and only key that the server accepts. Requests that p
API secret key is not configured. Please add 'api_secret' to the 'serve' section in your config.
```
Choose a long, unpredictable value. The key is the only thing standing between the internet and your CDM and download endpoints.
Choose a long, unpredictable value. The API key is the only thing standing between the internet and your CDM and download endpoints.
### Multiple keys with `users`
For finer control you can define named users, each with their own key. A user entry can also grant CDM device access and restrict which services that key may touch:
For finer control you can define named users, each with their own API key. A user entry can also grant CDM device access and restrict which services that API key may touch:
```yaml title="unshackle.yaml"
serve:
@@ -65,16 +65,16 @@ serve:
devices: ["my_widevine_device"]
```
The key of each entry under `users` (for example `a-secret-key-for-alice`) *is* the secret that client sends in `X-Secret-Key`. The `username` is a human-readable label used in logs and in job/history metadata.
The config key of each entry under `users` (for example `a-secret-key-for-alice`) *is* the secret that client sends in `X-Secret-Key`. The `username` is a human-readable label used in logs and in job/history metadata.
!!! info "How `api_secret` and `users` combine (integrated server)"
On the integrated server, the configured `api_secret` is automatically added as an extra accepted key alongside everyone in `users`. It is granted access to all loaded Widevine and PlayReady devices and is labelled `api_user`. So `api_secret` behaves as an admin-level key in addition to any named users you define.
On the integrated server, the server automatically adds the configured `api_secret` as an extra accepted API key alongside everyone in `users`. It gives that API key access to all loaded Widevine and PlayReady devices, and labels it `api_user`. So `api_secret` behaves as an admin-level API key in addition to any named users you define.
On the API-only server, only `api_secret` is accepted: the single-key model. (See the developer notes for the exact difference.)
On the API-only server, the server accepts only `api_secret`: the single-key model. (See the developer notes for the exact difference.)
### CDM device access per key
### CDM device access per API key
Under the integrated server, a key can only run DRM licensing through a CDM device that its user entry lists:
Under the integrated server, an API key can do DRM licensing only through a CDM device that its user entry lists:
| Config key | Grants access to |
|---|---|
@@ -82,21 +82,21 @@ Under the integrated server, a key can only run DRM licensing through a CDM devi
| `playready_devices` | PlayReady `.prd` devices, by name |
!!! warning "PlayReady access is opt-in per user"
If a user entry omits `playready_devices`, that key is given **no** PlayReady access, and the server logs a warning at startup. You must list PlayReady devices explicitly for any key that needs them. Widevine and PlayReady device files themselves are auto-loaded from your WVDs and PRDs directories.
If a user entry omits `playready_devices`, the server gives that API key **no** PlayReady access, and logs a warning at startup. You must name the PlayReady devices explicitly for any API key that needs them. Widevine and PlayReady device files themselves are auto-loaded from your WVDs and PRDs directories.
!!! note "Device lists *are* the server-side-decryption switch"
There is no separate tier, capability flag, or permission toggle for whether the server will hand back keys (`KID:KEY`) for a session download. The presence of devices on the calling key decides it. With **empty** `devices` and `playready_devices`, the server can only proxy CDM challenges, so the client must run its own CDM. Once those lists are **populated**, the client may request `mode:server_cdm` and receive keys back. Don't go looking for a tier setting; configuring devices is what enables server-side decryption.
There is no separate tier, capability flag, or permission toggle for whether the server will hand back content keys (`KID:KEY`) for a remote session download. The presence of devices on the calling API key decides it. With **empty** `devices` and `playready_devices`, the server can only proxy CDM challenges, so the client must operate its own CDM. Once those lists are **populated**, the client may request `mode:server_cdm` and receive content keys back. Do not look for a tier setting. The devices you configure are what enables server-side decryption.
### Disabling authentication
The `--no-key` flag turns authentication off entirely. Every request is allowed, `X-Secret-Key` is ignored, and no key needs to be configured.
The `--no-key` flag turns authentication off entirely. The server allows every request, ignores `X-Secret-Key`, and needs no API key in your configuration.
```bash
unshackle serve --no-key
```
!!! danger "Only use `--no-key` on a trusted, isolated network"
With `--no-key` there is nothing stopping any client that can reach the port from listing your services, starting downloads, or using your CDM devices. Never expose a `--no-key` server to an untrusted network.
With `--no-key`, any client that can reach the port can show your services, start downloads, and use your CDM devices. Never put a `--no-key` server on an untrusted network.
## How clients present credentials
@@ -106,18 +106,18 @@ Clients authenticate by sending the secret in a single HTTP request header on ev
X-Secret-Key: <your-secret>
```
The value must exactly match one of the configured keys (`api_secret`, or one of the keys under `users`). There is no login step, no session cookie, and no token exchange for API access. The same static key is sent on each request.
The value must match one of the configured API keys exactly (`api_secret`, or one of the keys under `users`). There is no login step, no HTTP session cookie, and no token exchange for API access. The client sends the same static API key on each request.
!!! info "One route also accepts the key as a query parameter"
The job event stream, `GET /api/download/jobs/{job_id}/events`, also accepts the key in a `secret_key` query parameter. This is for browsers: the `EventSource` API cannot set request headers. The exception applies to that one route only, and it works in both server modes.
The job event stream, `GET /api/download/jobs/{job_id}/events`, also accepts the API key in a `secret_key` query parameter. This is for browsers: the `EventSource` API cannot set request headers. The exception applies to that one route only, and it works in both server modes.
```javascript
new EventSource(`http://127.0.0.1:8786/api/download/jobs/${jobId}/events?secret_key=${key}`);
```
The header always wins. If the request has an `X-Secret-Key` header, the server checks that header and ignores the query parameter, so a wrong header is rejected even when the query parameter holds a valid key.
The header always wins. If the request has an `X-Secret-Key` header, the server examines that header and ignores the query parameter. The server rejects a wrong header even when the query parameter holds a valid API key.
A key in a URL is easier to leak than a key in a header, because proxies and browser history can record it. Use the header wherever your client can set one.
An API key in a URL is easier to leak than an API key in a header, because proxies and browser history can record it. Use the header wherever your client can set one.
=== "curl"
@@ -150,11 +150,11 @@ The value must exactly match one of the configured keys (`api_secret`, or one of
```
!!! tip "The reference client sets it for you"
unshackle's own [remote download](../../guide/downloading.md) client (`RemoteService`) sends `X-Secret-Key` automatically, reading the key from your `remote_services` config. It also sends a `User-Agent` of `unshackle/<version>`. You only deal with the header directly when writing your own client.
unshackle's own [remote download](../../guide/downloading.md) client (`RemoteService`) sends `X-Secret-Key` automatically, reading the API key from your `remote_services` config. It also sends a `User-Agent` of `unshackle/<version>`. You only deal with the header directly when writing your own client.
### The health check is always open
One endpoint is exempt from authentication: `GET /api/health`. It never requires a key, so you can use it for liveness and version probes without embedding a secret in your monitoring:
One endpoint is exempt from authentication: `GET /api/health`. It never requires an API key, so you can use it for liveness and version probes without embedding a secret in your monitoring:
```bash title="No key needed"
curl http://127.0.0.1:8786/api/health
@@ -172,11 +172,11 @@ curl http://127.0.0.1:8786/api/health
}
```
Every other endpoint (including the Swagger UI at `/api/docs/` on the API-only server) requires a valid key.
Every other endpoint (including the Swagger UI at `/api/docs/` on the API-only server) requires a valid API key.
## Authentication failures
When a request is rejected for authentication reasons, the response has an HTTP `401 Unauthorized` status and a small JSON body.
When the server rejects a request for authentication reasons, the response has an HTTP `401 Unauthorized` status and a small JSON body.
=== "Missing header"
@@ -191,7 +191,7 @@ When a request is rejected for authentication reasons, the response has an HTTP
=== "Wrong key"
Sending a key that is not in your config:
Sending an API key that is not in your config:
```json
{
@@ -203,17 +203,17 @@ When a request is rejected for authentication reasons, the response has an HTTP
!!! warning "Auth errors use a different JSON shape than other errors (developer note)"
The rest of the API reports errors with a structured body containing `error_code`, `message`, `timestamp`, and other fields (see [Errors](errors.md)). The authentication middleware is different: it returns `{"status": 401, "message": "..."}`, where `status` is an **integer**, and there is no `error_code` or `timestamp`.
A robust client should treat any `401` as an authentication problem and be prepared to parse *both* shapes rather than assuming the standard error body is always present.
In your client, treat any `401` as an authentication problem and parse *both* shapes, because the standard error body is not always present.
## Building on the key: access controls
## Access controls on top of the API key
Once a request is authenticated, several additional controls decide what that specific key is allowed to do.
After the server authenticates a request, several additional controls decide what that specific API key can do.
### Per-key service allowlists
You can restrict which services a key may use. Two layers combine:
You can restrict which services an API key may use. Two layers combine:
- **Global allowlist**: `serve.services`, applied to every key.
- **Global allowlist**: `serve.services`, applied to every API key.
- **Per-key allowlist**: `services` inside that key's `users` entry.
The effective allowlist for a request is:
@@ -235,11 +235,11 @@ serve:
services: ["EXAMPLE1"] # this key ends up allowed only EXAMPLE1
```
Endpoints that name a service (listing services, searching, listing titles/tracks, downloading, profiles, history, and remote sessions) all filter against this allowlist. A request for a service outside the key's allowlist is rejected as an invalid service. Service tags are matched after normalization, so casing and aliases resolve consistently.
Endpoints that name a service (listing services, searching, listing titles/tracks, downloading, profiles, history, and remote sessions) all filter against this allowlist. The server rejects a request for a service outside the API key's allowlist as an invalid service. The server matches service tags after normalisation, so casing and aliases give the same result.
### CDM overrides and per-job credentials
Two download-time capabilities are gated behind server config and, unless enabled, are refused with `403 Forbidden` even for an authenticated key:
The server config gates two download-time capabilities. Unless you enable them, the server refuses them with `403 Forbidden`, even for an authenticated API key:
| Config key | Default | Controls |
|---|---|---|
@@ -254,26 +254,26 @@ serve:
```
!!! tip "When to enable these gates"
The two gates are fully independent and can be combined. The intended posture is that a locked-down or public-facing deployment leaves **both** unset (they default off and reject with `403`) so client requests can never steer the server's CDM or feed it their own logins. Enabling them only makes sense for a trusted single-client deployment, where the caller and the operator are effectively the same party.
The two gates are fully independent, and you can combine them. For a locked-down or public-facing deployment, leave **both** unset. They then default off and reject with `403`, so client requests can never steer the server's CDM or feed it their own logins. Enable them only for a trusted single-client deployment, where the caller and the operator are effectively the same party.
!!! note "Client-supplied credentials get isolated token caches"
When `allow_job_credentials` is enabled and a download job supplies its own `credential` / `credentials`, each distinct credential is given its **own** isolated token cache on the server. Client-supplied logins are never shared or merged with the server's configured-credential caches, so they cannot cross-contaminate or read the server's own token state, and they don't clobber each other.
When you enable `allow_job_credentials` and a download job supplies its own `credential` / `credentials`, the server gives each distinct credential its **own** isolated token cache. The server never shares or merges client-supplied logins with its configured-credential caches. They cannot cross-contaminate or read the server's own token state, and they do not clobber each other.
See [Downloads](endpoints.md) for how these fields are used on the download endpoint.
See [Downloads](endpoints.md) for how the download endpoint uses these fields.
### Session IP binding (remote sessions)
The remote-download session endpoints (`/api/session/*`) add an extra check on top of the key. When a session is created, the server records the client's IP address. Any later request for that session must come from the **same** IP, or it is rejected with `403 Forbidden`. This means a session is bound both to the key that created it and to the address it was created from.
The remote-download session endpoints (`/api/session/*`) add an extra check on top of the API key. When the server creates a remote session, it records the client's IP address. Any later request for that remote session must come from the **same** IP, or the server rejects it with `403 Forbidden`. Thus the server binds a remote session both to the API key that created it and to the address it came from.
## Transport security (HTTPS)
The API server speaks plain HTTP. Because your key is sent on every request, you should not expose it over an unencrypted connection on an untrusted network. To serve over HTTPS, run a reverse proxy in front of unshackle. The `serve` command can launch Caddy for you:
The API server speaks plain HTTP. Because the client sends your API key on every request, do not use it over an unencrypted connection on an untrusted network. To serve over HTTPS, operate a reverse proxy in front of unshackle. The `serve` command can launch Caddy for you:
```bash
unshackle serve --caddy
```
With `--caddy`, unshackle also starts `caddy run` using the `Caddyfile` located next to your unshackle config, letting you terminate TLS and reverse-proxy to the API. Configuring Caddy itself (certificates, domains, upstream) is done in that `Caddyfile`.
With `--caddy`, unshackle also starts `caddy run` using the `Caddyfile` located next to your unshackle config, letting Caddy end the TLS connection and reverse-proxy to the API. Configuring Caddy itself (certificates, domains, upstream) is done in that `Caddyfile`.
## Quick reference
@@ -291,7 +291,7 @@ With `--caddy`, unshackle also starts `caddy run` using the `Caddyfile` located
---
## Developer notes: how the check is implemented
## Developer notes: how the server does the check
!!! note "This section is for people writing or debugging the server, not for API consumers."
@@ -299,13 +299,13 @@ With `--caddy`, unshackle also starts `caddy run` using the `Caddyfile` located
The header and config are identical between modes, but the code that enforces them differs:
- **API-only server** (`--api-only`, or `--remote-only` which forces API-only). An `api_key_authentication` middleware runs on every request. It lets `GET /api/health` through unconditionally, then reads `X-Secret-Key`. An empty header returns the *"Secret Key is Empty."* body; a header that is not a configured user returns *"Secret Key is Invalid."*. In this mode the accepted-keys map holds your `api_secret` (labelled `api_user`, with no devices) plus every key under `serve.users`, so a per-user key authenticates here exactly as it does on the integrated server, and that user's `services` allowlist applies as usual. With `--no-key`, this middleware is simply not installed and the users map is empty.
- **API-only server** (`--api-only`, or `--remote-only` which forces API-only). An `api_key_authentication` middleware runs on every request. It lets `GET /api/health` through unconditionally, then reads `X-Secret-Key`. An empty header returns the *"Secret Key is Empty."* body. A header that is not a configured user returns *"Secret Key is Invalid."*. In this mode the accepted-keys map holds your `api_secret` (labelled `api_user`, with no devices) plus every key under `serve.users`. A per-user key therefore authenticates here exactly as it does on the integrated server, and that user's `services` allowlist applies as usual. With `--no-key`, this middleware is not installed and the users map is empty.
- **Integrated server** (default). A `serve_authentication` middleware wraps pywidevine's authentication (and pyplayready's for `/playready*` paths), and the REST routes live in the same application, guarded by the same check. Here the accepted-keys map is your full `serve` config: `api_secret` (auto-added with access to all loaded devices) plus every entry under `users`, each with its declared `devices` / `playready_devices`.
Because the API-only middleware guards *all* non-health paths, the Swagger UI at `/api/docs/` is also behind `X-Secret-Key` there. In `--remote-only` mode the Swagger UI is not mounted at all.
Both middlewares read the caller's key through the shared `request_secret_key()` helper in `unshackle/core/api/handlers.py`, which returns the `X-Secret-Key` header and falls back to the `secret_key` query parameter only when the matched route is the job event stream. The integrated server needs one extra branch for this: pywidevine's `authentication` is header-only, so `serve_authentication` compares the query-parameter key itself when the header is absent, and delegates to pywidevine in every other case. That is why a present-but-wrong header is rejected in both modes.
Both middlewares read the caller's API key through the shared `request_secret_key()` helper in `unshackle/core/api/handlers.py`. That helper returns the `X-Secret-Key` header, and falls back to the `secret_key` query parameter only when the matched route is the job event stream. The integrated server needs one extra branch for this. pywidevine's `authentication` is header-only. So `serve_authentication` compares the API key from the query parameter itself when the header is absent, and delegates to pywidevine in every other case. That is why both modes reject a present-but-wrong header.
### CORS and the `Authorization` header
@@ -318,8 +318,8 @@ Access-Control-Allow-Headers: Content-Type, X-Secret-Key, Authorization
Access-Control-Max-Age: 3600
```
The REST API authenticates with `X-Secret-Key`; `Authorization` is advertised because the co-hosted pywidevine / pyplayready CDM endpoints use it. `PUT` appears in the allowed methods list but no REST route uses it; only `GET`, `POST`, and `DELETE` are wired up.
The REST API authenticates with `X-Secret-Key`. The middleware advertises `Authorization` because the co-hosted pywidevine / pyplayready CDM endpoints use it. `PUT` appears in the allowed-methods list, but no REST route uses it. Only `GET`, `POST`, and `DELETE` have routes.
### Where device access is read
The map of accepted keys lives at `request.app["config"]["users"]`. For DRM operations, a key's permitted Widevine devices come from its `devices` list and PlayReady devices from `playready_devices`; the first configured device is used when a request does not (or may not) override it. A server-wide `config.cdm[<service>]` mapping, if present, takes precedence over the key's device list when resolving which device to use.
The map of accepted API keys lives at `request.app["config"]["users"]`. For DRM operations, an API key's permitted Widevine devices come from its `devices` list, and PlayReady devices from `playready_devices`. The server uses the first configured device when a request does not (or may not) override it. A server-wide `config.cdm[<service>]` mapping, if present, takes precedence over the API key's device list when the server chooses which device to use.
docs/dev/rest-api/endpoints.md
+61 -61
View File
@@ -1,25 +1,25 @@
# REST API Endpoints
This is the full endpoint reference for the unshackle REST API. It covers every route: its method and path, the request body it accepts, the response shape it returns, and the status codes it can produce. Routes are grouped into general, download jobs, server management, and remote-dl sessions.
This is the full endpoint reference for the unshackle REST API. It covers every route: its method and path, the request body it accepts, the response shape it returns, and the status codes it can give. This page groups the routes into general, download jobs, server management, and remote-dl sessions.
Start the server with [`unshackle serve`](../../guide/cli-reference.md). By default it binds to `127.0.0.1:8786`, and every route lives under the `/api/` prefix. Authentication, server modes, and the `X-Secret-Key` header are covered in detail on the [Authentication](authentication.md) page. Read that first if you have not already. For an end-to-end walkthrough, see the [Quickstart](quickstart.md).
Start the server with [`unshackle serve`](../../guide/cli-reference.md). By default it binds to `127.0.0.1:8786`, and every route lives under the `/api/` prefix. The [Authentication](authentication.md) page gives authentication, server modes, and the `X-Secret-Key` header in detail. Read that first if you have not already. For an end-to-end walkthrough, see the [Quickstart](quickstart.md).
!!! note "Base URL"
All examples below assume a base URL of `http://127.0.0.1:8786`. Replace the host and port to match your own `unshackle serve -h <host> -p <port>` invocation.
All examples below assume a base URL of `http://127.0.0.1:8786`. Replace the host and port so that they match your own `unshackle serve -h <host> -p <port>` invocation.
!!! tip "Interactive docs"
A live Swagger UI is mounted at **`/api/docs/`** whenever the server is not running in `--remote-only` mode. Its title is "Unshackle REST API" and its version tracks the installed package version. In API-only mode with a key configured, the Swagger UI itself sits behind `X-Secret-Key`; only `/api/health` is exempt.
The server mounts a live Swagger UI at **`/api/docs/`** whenever it does not operate in `--remote-only` mode. Its title is "Unshackle REST API" and its version tracks the installed package version. In API-only mode with an API key configured, the Swagger UI itself sits behind `X-Secret-Key`. Only `/api/health` is exempt.
## Conventions
- **Authentication.** Every request except `GET /api/health` requires the `X-Secret-Key` header when a key is configured. Starting the server with `--no-key` disables the check entirely. See [Authentication](authentication.md).
- **Content type.** Request bodies are JSON (`Content-Type: application/json`); all success responses are JSON unless a `204 No Content` is documented.
- **CORS.** Every response carries permissive CORS headers, and `OPTIONS` preflight requests are answered automatically.
- **Authentication.** Every request except `GET /api/health` requires the `X-Secret-Key` header when you configure an API key. `--no-key` disables the check entirely. See [Authentication](authentication.md).
- **Content type.** Request bodies are JSON (`Content-Type: application/json`). All success responses are JSON unless this page documents a `204 No Content`.
- **CORS.** Every response carries permissive CORS headers, and the server answers `OPTIONS` preflight requests automatically.
- **Compression.** JSON responses of 256 bytes or more are gzip-compressed when the client sends `Accept-Encoding: gzip`.
- **Service allowlist.** Service-facing endpoints are filtered by the effective allowlist for your API key (the intersection of the global `serve.services` list and your per-key list). Services you are not allowed to use are treated as unknown.
- **Service allowlist.** The effective allowlist for your API key filters the service-facing endpoints (the intersection of the global `serve.services` list and your per-key list). The server treats services you cannot use as unknown.
!!! warning "Error responses have two shapes"
Structured application errors (raised inside a handler) return `{"status": "error", "error_code": "...", "message": "...", "timestamp": "..."}`. The authentication middleware, however, returns a **different** shape: `{"status": 401, "message": "..."}` with an integer `status` and no `error_code`. Clients must handle both. The full error model is documented in the [Error responses](#error-responses) section at the bottom of this page.
Structured application errors (raised inside a handler) return `{"status": "error", "error_code": "...", "message": "...", "timestamp": "..."}`. The authentication middleware, however, returns a **different** shape: `{"status": 401, "message": "..."}` with an integer `status` and no `error_code`. Clients must accept both shapes. The [Error responses](#error-responses) section at the bottom of this page gives the full error model.
---
@@ -52,7 +52,7 @@ Health check. **This is the only route exempt from authentication**: you can cal
}
```
`code_hash` fingerprints the framework source the server is running, so it identifies the code itself rather than the release. It is the same value for a git checkout and for an installed package built from that checkout, and it changes if anything under `unshackle/core`, `unshackle/commands`, `unshackle/utils`, or `unshackle/vaults` is edited. Service modules are not included. The value is `null` only when the source files cannot be read.
`code_hash` fingerprints the framework source the server operates, so it names the code itself rather than the release. It is the same value for a git checkout and for an installed package built from that checkout. It changes when you edit anything under `unshackle/core`, `unshackle/commands`, `unshackle/utils`, or `unshackle/vaults`. Service modules are not included. unshackle sets the value to `null` only when it cannot read the source files.
If the update check fails (for example, no network), `update_available` and `latest_version` are `null` while `current_version` still reports the installed version.
@@ -62,7 +62,7 @@ If the update check fails (for example, no network), `update_available` and `lat
### `GET /api/services`
List the streaming services available on this server, filtered by your allowlist. Each entry describes the service's tag, matching rules, capability flags, and its CLI parameters (useful for building a UI that accepts service-specific options).
Show the streaming services available on this server, filtered by your allowlist. Each entry gives the service's tag, matching rules, capability flags, and its CLI parameters (useful when you make a UI that accepts service-specific options).
=== "Request"
@@ -127,7 +127,7 @@ Field notes:
### `POST /api/search`
Search a service for titles by query string. The service must implement `search()` (see `has_search` above).
Find titles in a service by query string. The service must have a `search()` method (see `has_search` above).
**Request body**
@@ -175,7 +175,7 @@ Search a service for titles by query string. The service must implement `search(
### `POST /api/list-titles`
Resolve a title ID into its list of titles (episodes, movie, etc.) without downloading anything.
Get the list of titles behind a title ID, for example episodes or a movie, without downloading anything.
**Request body**
@@ -188,7 +188,7 @@ Resolve a title ID into its list of titles (episodes, movie, etc.) without downl
| `no_proxy` | boolean | no | Force-disable proxy. |
| `cdm_type` | string | no | Preferred CDM type. |
Service-specific CLI options may also be passed as additional keys.
You can also pass service-specific CLI options as additional keys.
=== "Request"
@@ -221,13 +221,13 @@ Service-specific CLI options may also be passed as additional keys.
}
```
Each serialized title carries `type` (`"episode"`, `"movie"`, or `"other"`), `name`, `id`, `language`, `description`, `date`, and `cover_url`. Episodes and movies add `year`; episodes additionally add `series_title`, `season`, and `number`.
Each serialized title carries `type` (`"episode"`, `"movie"`, or `"other"`), `name`, `id`, `language`, `description`, `date`, and `cover_url`. Episodes and movies add `year`. Episodes also add `series_title`, `season`, and `number`.
!!! warning "A song serializes as `"other"`"
A music track has no `type` of its own. It is sent as `"other"`, and the response
A song has no `type` of its own. The server sends it as `"other"`, and the response
carries none of its music fields: no artist, no album, no track or disc number, and no
`year`. `name` holds the track name. To read the disc and track numbers of a release,
run the CLI with `--list-titles`. Selecting music tracks over the API still works,
`year`. `name` holds the song name. To read the disc and track numbers of a release,
operate the CLI with `--list-titles`. You can still select songs over the API,
through the `wanted` parameter.
These keys are sent only when the title carries them, so a title without them serializes exactly as before:
@@ -243,8 +243,8 @@ These keys are sent only when the title carries them, so a title without them se
!!! info "`part` on a split episode"
A service that splits one episode into several separately playable videos reports each
one as its own episode. Those titles share a `season` and `number` and add a `part`
integer counting from 1. No other title carries the key, so a client that ignores it
is unaffected. See [Split episodes](../creating-a-service.md#split-episodes).
integer counting from 1. No other title carries the `part` field, so a client can
ignore it safely. See [Split episodes](../creating-a-service.md#split-episodes).
| Status | Error code | Meaning |
| --- | --- | --- |
@@ -257,10 +257,10 @@ These keys are sent only when the title carries them, so a title without them se
### `POST /api/list-tracks`
List the video, audio, and subtitle tracks for a title. For series, you can scope to specific episodes.
Show the video, audio, and subtitle tracks for a title. For series, you can scope to specific episodes.
!!! tip "Spotting the original-language audio"
Each audio track carries `is_original`, which marks the track the default `lang` of `orig` selects. Unshackle resolves it with the same language matcher the downloader uses, so it stays correct for regional variants a client cannot separate from tag text alone (`pt` marks `pt-BR` rather than `pt-PT`). Services need not report a title's language, so every track can legitimately come back `false`.
Each audio track carries `is_original`, which marks the track the default `lang` of `orig` selects. unshackle finds it with the same language matcher the downloader uses. Thus it stays correct for regional variants a client cannot separate from tag text alone (`pt` marks `pt-BR` rather than `pt-PT`). Services need not report a title's language, so every track can legitimately come back `false`.
**Request body**
@@ -274,7 +274,7 @@ List the video, audio, and subtitle tracks for a title. For series, you can scop
| `part` | int/string | no | Part index of a split episode (combined with `season` and `episode`). |
| `profile`, `proxy`, `no_proxy`, `cdm_type` | - | no | As on `list-titles`. |
When both `season` and `episode` are given, they are combined into a `"{season}x{episode}"` selector. Adding `part` narrows that to `"{season}x{episode}.{part}"`, one part of a [split episode](../creating-a-service.md#split-episodes); without it you get every part of the episode. `part` is read only when `season` and `episode` are both given, and `wanted` takes precedence over all three. To scope a part by range, put it in `wanted` instead (`"S01E01.2"`).
When you give both `season` and `episode`, the server combines them into a `"{season}x{episode}"` selector. `part` narrows that to `"{season}x{episode}.{part}"`, one part of a [split episode](../creating-a-service.md#split-episodes). Without `part` you get every part of the episode. The server reads `part` only when you give both `season` and `episode`, and `wanted` takes precedence over all three. To scope a part by range, put it in `wanted` instead (`"S01E01.2"`).
=== "Single title / movie: Response `200`"
@@ -321,7 +321,7 @@ When both `season` and `episode` are given, they are combined into a `"{season}x
}
```
Tracks are sorted by bitrate, descending. Episodes that fail to resolve are collected into `unavailable_episodes` rather than failing the whole request. Track objects here do **not** include a download `url`; segment URLs are only exposed through the remote-dl session endpoints.
The server sorts tracks by bitrate, descending. Episodes the server cannot get go into `unavailable_episodes`, rather than failing the whole request. Track objects here do **not** include a download `url`. Only the remote session endpoints give segment URLs.
The `drm` array on a track is a list of `{type, pssh?, kids?, content_keys?, license_url?}` objects.
@@ -337,14 +337,14 @@ The `drm` array on a track is a list of `{type, pssh?, kids?, content_keys?, lic
## Download jobs
Downloads are asynchronous. `POST /api/download` enqueues a job and returns immediately with a `job_id`; the server runs each job in a background worker subprocess. You then poll the job endpoints for progress and results.
Downloads are asynchronous. `POST /api/download` enqueues a job and returns immediately with a `job_id`. The server runs each job in a background job worker subprocess. You then poll the job endpoints for progress and results.
!!! info "Job states"
A job moves through `queued` → `downloading` → one of the terminal states `completed`, `failed`, or `cancelled`. Only queued jobs can be prioritized; only terminal jobs can be retried or removed.
A job moves through `queued` → `downloading` → one of the terminal states `completed`, `failed`, or `cancelled`. You can prioritise only queued jobs. You can retry or remove only terminal jobs.
### `POST /api/download`
Create a download job. Requires `service` and `title_id`; every other field is an optional download parameter with a server-side default. On success this returns **`202 Accepted`**, not `200`.
Make a download job. It requires `service` and `title_id`. Every other field is an optional download parameter with a server-side default. On success this returns **`202 Accepted`**, not `200`.
=== "Request"
@@ -418,22 +418,22 @@ Create a download job. Requires `service` and `title_id`; every other field is a
| `output_dir` | string | `null` | Override output directory. |
| `no_cache` / `reset_cache` | boolean | `false` | Title cache controls. |
**Validation.** Invalid values return `400 INVALID_PARAMETERS`. Notably: `vcodec` must be one of H264/H265/H.264/H.265/AVC/HEVC/VC1/VC-1/VP8/VP9/AV1; `acodec` one of AAC/AC3/EC3/EAC3/DD/DD+/AC4/OPUS/FLAC/ALAC/VORBIS/OGG/DTS; `range` one of SDR/HDR10/HDR10P/DV/HLG/HYBRID (`HDR10+` is accepted); bitrate/worker/download counts must be positive integers; at most one of the `*_only` flags may be set; `no_subs` cannot be combined with `subs_only`, `no_audio` with `audio_only`, or `s_lang` with `require_subs`.
**Validation.** Invalid values return `400 INVALID_PARAMETERS`. `vcodec` must be one of H264/H265/H.264/H.265/AVC/HEVC/VC1/VC-1/VP8/VP9/AV1. `acodec` must be one of AAC/AC3/EC3/EAC3/DD/DD+/AC4/OPUS/FLAC/ALAC/VORBIS/OGG/DTS. `range` must be one of SDR/HDR10/HDR10P/DV/HLG/HYBRID, and `HDR10+` is also valid. The bitrate, download worker, and download counts must be positive integers. You may set at most one of the `*_only` flags. You cannot combine `no_subs` with `subs_only`, `no_audio` with `audio_only`, or `s_lang` with `require_subs`.
!!! danger "Rejected parameters"
`postscript`, `post_script` and `post_scripts` are never accepted. A body containing any
of them is rejected with `400 INVALID_PARAMETERS`, because a command string arriving from
an HTTP caller would be remote code execution. Post-download hooks are configured only in
`unshackle.yaml`, where they still run for API jobs. See
of them gets `400 INVALID_PARAMETERS`, because a command string that arrives from
an HTTP caller would be remote code execution. You configure post-download hooks only in
`unshackle.yaml`, where they still operate for API jobs. See
[Post-download scripts](../../reference/configuration/post-scripts.md).
!!! warning "Gated parameters (developer)"
A few keys are accepted but only permitted when the server config opts in, otherwise the request is rejected with `403 FORBIDDEN`:
The server accepts a few keys, but permits them only when the server config opts in. Otherwise the server rejects the request with `403 FORBIDDEN`:
- `cdm` (per-request CDM device name): allowed only if `serve.cdm_overrides` is `true` or a list containing that device name.
- `credential` (`"user:pass"`) and `credentials` (a map): allowed only if `serve.allow_job_credentials` is truthy (off by default).
Server-side, effective parameters are merged as defaults → `serve:` config overrides → service-specific defaults → your request body.
On the server, unshackle merges the effective parameters in this order: defaults → `serve:` config overrides → service-specific defaults → your request body.
| Status | Error code | Meaning |
| --- | --- | --- |
@@ -445,7 +445,7 @@ Create a download job. Requires `service` and `title_id`; every other field is a
### `GET /api/download/jobs`
List download jobs, with optional filtering and sorting.
Show download jobs, with optional filtering and sorting.
**Query parameters**
@@ -533,10 +533,10 @@ The `data` field of every event is the same job object that `GET /api/download/j
The server sends a `: keep-alive` comment each 15 seconds while the job is quiet, to keep proxies from closing an idle connection. Ignore these lines.
If the job is already in a terminal state, the server sends the `snapshot` event, then the terminal event, and closes the stream immediately.
If the job is already in a terminal state, the server sends the `snapshot` event, then the terminal event, and closes the event stream immediately.
!!! tip "Authentication from a browser"
`EventSource` cannot set headers. For this endpoint only, the server also accepts the key in the `secret_key` query parameter. This works in both server modes, the default integrated server and `--api-only`:
`EventSource` cannot set headers. For this endpoint only, the server also accepts the API key in the `secret_key` query parameter. This works in both server modes, the default integrated server and `--api-only`:
```javascript
const events = new EventSource(
@@ -546,7 +546,7 @@ If the job is already in a terminal state, the server sends the `snapshot` event
events.addEventListener("completed", () => events.close());
```
The header always wins. Send the key in the query parameter *or* in the header, not both: if an `X-Secret-Key` header is present, the server checks that header and ignores the query parameter. See [Authentication](authentication.md#how-clients-present-credentials).
The header always wins. Send the API key in the query parameter *or* in the header, not both. If an `X-Secret-Key` header is present, the server examines that header and ignores the query parameter. See [Authentication](authentication.md#how-clients-present-credentials).
=== "Request"
@@ -583,7 +583,7 @@ Cancel or remove a job. The behaviour depends on the job's current state:
- **Queued or downloading job**: cancelled, returning `200` `{"status": "success", "message": "Job cancelled"}`.
!!! warning
A `204` response has no body. Do not attempt to parse JSON from it.
A `204` response has no body. Do not try to parse JSON from it.
| Status | Error code | Meaning |
| --- | --- | --- |
@@ -607,7 +607,7 @@ Remove every terminal (completed/failed/cancelled) job from the manager. No requ
Enqueue a **new** job that reuses a terminal job's service, title, and parameters. The service allowlist and parameter gates are re-checked. Returns `202` with the new job's ID.
!!! warning "Retry reuses the original, unredacted parameters"
Retry replays the finished job's stored parameters exactly as they were submitted, including any secrets such as `credential`/`credentials`, not the redacted `parameters` view that `GET /api/download/jobs` and `GET /api/history` expose. This is deliberate: it lets a retried job reproduce a credentialed download that the redacted job record alone never could. It also means anyone who can retry a job can re-run it with its original credentials, even though those credentials are masked everywhere else in the API.
A retry replays the finished job's stored parameters exactly as the original client submitted them, including any secrets such as `credential`/`credentials`. It does not use the redacted `parameters` view that `GET /api/download/jobs` and `GET /api/history` give. This is deliberate: it lets a retried job reproduce a credentialed download that the redacted job record alone never could. The API masks those credentials everywhere else. But anyone who can retry a job can operate it again with the original credentials.
=== "Response `202`"
@@ -651,7 +651,7 @@ Endpoints for profiles, effective configuration, download history, environment c
### `GET /api/profiles`
List the named credential profiles configured per service (usable as the `profile` parameter elsewhere). Only services whose credentials are a mapping of profile name to credential are listed; services with a single unnamed credential are omitted. Filtered by your allowlist.
Show the named credential profiles configured per service (usable as the `profile` parameter elsewhere). The response holds only services whose credentials are a mapping of profile name to credential. It leaves out services with a single unnamed credential. Your allowlist filters the response.
=== "Response `200`"
@@ -666,7 +666,7 @@ List the named credential profiles configured per service (usable as the `profil
### `GET /api/config`
Return a read-only, redacted view of the effective server configuration, suitable for a settings page in a UI. Secrets are never emitted: keys matching `secret`, `password`, `token`, `api_key`, or `credential`, and any userinfo in proxy URLs, are masked.
Return a read-only, redacted view of the effective server configuration, suitable for a settings page in a UI. The server never sends secrets: it masks every config key whose name contains `secret`, `password`, `token`, `api_key`, or `credential`, and any userinfo in proxy URLs.
=== "Response `200`"
@@ -695,7 +695,7 @@ Return a read-only, redacted view of the effective server configuration, suitabl
### `GET /api/history`
Read the persisted history of jobs that reached a terminal state, newest first. History is stored in `api_history.jsonl` in the cache directory; corrupt lines are skipped and a missing file yields an empty list.
Read the persisted history of jobs that reached a terminal state, newest first. unshackle stores the history in `api_history.jsonl` in the cache directory. It skips corrupt lines, and a missing file yields an empty list.
**Query parameters**
@@ -742,12 +742,12 @@ Remove a single persisted history entry. Returns **`204 No Content`** (empty bod
### `POST /api/maintenance/clear-cache` and `POST /api/maintenance/clear-temp`
Delete and recreate the cache or temp directory. Neither takes a request body. Both are blocked with `409 CONFLICT` if any job is currently `downloading` (the `details` include the offending `active_jobs`).
Delete and recreate the cache or temp directory. Neither takes a request body. The server blocks both with `409 CONFLICT` if any job is currently `downloading` (the `details` include the offending `active_jobs`).
The `409` guard is about active file I/O, not an arbitrary lock: an in-flight job may still be *reading* cached title data under the cache directory, and active download workers are *writing* segment files under temp. Removing either directory mid-download would corrupt those running jobs, so the endpoints refuse until nothing is downloading.
The `409` guard is about active file I/O, not an arbitrary lock. An in-flight job may still be *reading* cached title data under the cache directory, and active download workers are *writing* segment files under temp. The removal of either directory mid-download would corrupt those jobs, so the endpoints refuse while a download is in progress.
!!! warning "clear-cache also wipes download history"
The persistent history read by [`GET /api/history`](#get-apihistory) lives in `api_history.jsonl` **inside the cache directory**. Because clear-cache deletes and recreates that whole directory, calling it also erases your entire download history, not just cached title data. If you want to keep history, do not clear the cache; remove individual entries with [`DELETE /api/history/{job_id}`](#delete-apihistoryjob_id) instead.
The persistent history read by [`GET /api/history`](#get-apihistory) lives in `api_history.jsonl` **inside the cache directory**. Because clear-cache deletes and recreates that whole directory, calling it also erases your entire download history, not only cached title data. If you want to keep history, do not clear the cache. Remove individual entries with [`DELETE /api/history/{job_id}`](#delete-apihistoryjob_id) instead.
=== "Response `200`"
@@ -762,7 +762,7 @@ The `409` guard is about active file I/O, not an arbitrary lock: an in-flight jo
### `POST /api/maintenance/refresh-services`
git-pull every service repository configured under `directories.services`. `refreshed` is `true` when all repos updated successfully (or none are configured).
git-pull every service repository configured under `directories.services`. `refreshed` becomes `true` when all repos updated successfully, or when you configure no repository.
=== "Response `200`"
@@ -795,11 +795,11 @@ Report the install status of the external binaries unshackle depends on (the sam
## Remote-dl sessions
!!! note "Developer feature"
The `/api/session/*` endpoints power unshackle's **remote download** mode, where a thin local client authenticates once against a remote server, then fetches titles, tracks, and segments and proxies DRM licensing back through it. These endpoints are exposed in `--remote-only` mode, and they are what the built-in `RemoteService` client drives. Most end users never call them directly.
The `/api/session/*` endpoints power unshackle's **remote download** mode. A thin local client authenticates once against a remote server, then fetches titles, tracks, and segments, and proxies DRM licensing back through it. The server mounts these endpoints in `--remote-only` mode, and they are what the built-in `RemoteService` client drives. Most end users never call them directly.
Sessions are held in memory. They expire after `serve.session_ttl` (default 300s), except while awaiting interactive authentication input, which uses a 600s window. A session is bound to the IP that created it, so a request from a different IP is rejected with `403 FORBIDDEN`.
The server holds remote sessions in memory. They expire after `serve.session_ttl` (default 300s), except while awaiting interactive authentication input, which uses a 600s window. A remote session binds to the IP that made it, so the server rejects a request from a different IP with `403 FORBIDDEN`.
### Session lifecycle overview
### Remote session lifecycle overview
```text
POST /api/session/create → returns session_id, status "authenticating"
@@ -813,7 +813,7 @@ DELETE /api/session/{id} → tear down, harvest updated cache
### `POST /api/session/create`
Create a session for a service and title. Authentication runs asynchronously in the background. This call returns immediately with `status: "authenticating"`, and you then poll the prompt endpoint. The body accepts `service` and `title_id` (both required) plus a broad set of optional keys (`additionalProperties` is allowed): `credentials` (`{username, password, extra?}`), `cookies` (base64 of zlib-compressed Netscape cookie file), `proxy`, `no_proxy`, `profile`, `cache` (map of forwarded cache files), `client_region`, `cdm_type`, `range_`, `vcodec`, `quality`, `best_available`, and any service CLI options.
Make a remote session for a service and title. Authentication runs asynchronously in the background. This call returns immediately with `status: "authenticating"`, and you then poll the prompt endpoint. The body accepts `service` and `title_id` (both required). It also accepts a broad set of optional keys, because the body allows `additionalProperties`. These are `credentials` (`{username, password, extra?}`), `cookies` (base64 of zlib-compressed Netscape cookie file), `proxy`, `no_proxy`, `profile`, `cache` (map of forwarded cache files), `client_region`, `cdm_type`, `range_`, `vcodec`, `quality`, `best_available`, and any service CLI options.
=== "Response `200`"
@@ -883,7 +883,7 @@ Submit a response to a pending prompt. Body: `{ "response": "..." }` (required).
### `GET /api/session/{session_id}/titles`
Fetch the titles for an authenticated session.
Fetch the titles for an authenticated remote session.
=== "Response `200`"
@@ -894,11 +894,11 @@ Fetch the titles for an authenticated session.
}
```
Requires an authenticated session; `404 SESSION_NOT_FOUND` otherwise.
Requires an authenticated remote session. Otherwise it returns `404 SESSION_NOT_FOUND`.
### `POST /api/session/{session_id}/tracks`
Get tracks and chapters for a specific title within the session. Body: `{ "title_id": "..." }` (required). Unlike `list-tracks`, track objects here **include** download URLs, along with session headers/cookies and manifest data the client needs to download directly.
Get tracks and chapters for a specific title within the remote session. Body: `{ "title_id": "..." }` (required). Unlike `list-tracks`, track objects here **include** download URLs, along with session headers/cookies and manifest data the client needs to download directly.
=== "Response `200`"
@@ -925,7 +925,7 @@ Get tracks and chapters for a specific title within the session. Body: `{ "title
### `POST /api/session/{session_id}/segments`
Resolve the download URL, DRM info, headers, cookies, and raw track data for selected tracks. Body: `{ "track_ids": ["v-1", "a-1"] }` (required).
Get the download URL, DRM info, headers, cookies, and raw track data for the selected tracks. Body: `{ "track_ids": ["v-1", "a-1"] }` (required).
=== "Response `200`"
@@ -945,7 +945,7 @@ Resolve the download URL, DRM info, headers, cookies, and raw track data for sel
### `POST /api/session/{session_id}/license`
Acquire DRM keys. Two modes are supported via the `mode` field.
Get the content keys for the DRM. The `mode` field selects one of two modes.
**Proxy mode** (`mode: "proxy"`, the default) forwards the client's CDM `challenge` to the service's license endpoint and returns the raw license bytes:
@@ -994,7 +994,7 @@ Acquire DRM keys. Two modes are supported via the `mode` field.
### `GET /api/session/{session_id}`
Return session metadata and validity.
Return remote session metadata and validity.
=== "Response `200`"
@@ -1009,11 +1009,11 @@ Return session metadata and validity.
}
```
`404 SESSION_NOT_FOUND` if the session does not exist or has expired.
`404 SESSION_NOT_FOUND` if the remote session does not exist or has expired.
### `DELETE /api/session/{session_id}`
Tear down a session. Its input bridge is cancelled, any updated cache files are harvested, and the session plus its cache directory are deleted.
Tear down a remote session. The server cancels its input bridge, harvests any updated cache files, and deletes the remote session and its cache directory.
=== "Response `200`"
@@ -1024,13 +1024,13 @@ Tear down a session. Its input bridge is cancelled, any updated cache files are
}
```
The `cache` field is present only if the session produced cache files. `404 SESSION_NOT_FOUND` if the session does not exist.
The `cache` field is present only if the remote session produced cache files. `404 SESSION_NOT_FOUND` if the remote session does not exist.
---
## Error responses
Every structured error raised inside a handler is serialized to the same JSON envelope:
The server serialises every structured error raised inside a handler to the same JSON envelope:
```json
{
@@ -1046,7 +1046,7 @@ Every structured error raised inside a handler is serialized to the same JSON en
- `details` appears only when non-empty.
- `retryable` appears only when the error is retryable (e.g. network errors, rate limiting).
- `debug_info` appears only when the server was started with `--debug-api`.
- `debug_info` appears only when you start the server with `--debug-api`.
!!! warning "Authentication errors differ"
Errors raised by the API-key middleware use a distinct shape (`{"status": 401, "message": "Secret Key is Empty."}` or `{"status": 401, "message": "Secret Key is Invalid."}`) with an integer `status` and no `error_code`, `timestamp`, or `details`. A malformed JSON body on any POST, by contrast, returns the standard envelope with `error_code: "INVALID_INPUT"`.
docs/dev/rest-api/errors.md
+28 -28
View File
@@ -1,20 +1,20 @@
# Error Responses
Every endpoint of the unshackle REST API reports failures in a single, predictable
shape. Whether a request was rejected because a field was missing, a service refused
to authenticate, or the download worker crashed, the server answers with a JSON body
shape. The cause can be a missing field, a service that refuses to authenticate, or a
download worker that crashed. In each case the server answers with a JSON body
that carries a machine-readable **error code**, a human-readable **message**, and a
matching **HTTP status**. This page documents that format and catalogues every error
code the server can emit.
code the server can send.
If you are building a client against the API, read this page alongside the endpoint
reference. Each endpoint lists the specific codes it raises, and the meanings of
those codes are defined here.
If you write a client against the API, read this page alongside the endpoint
reference. Each endpoint lists the specific codes it raises, and this page gives the
meanings of those codes.
!!! note "Where this comes from"
The error format is defined in `unshackle/core/api/errors.py`, and the wrapper
`unshackle/core/api/errors.py` defines the error format, and the wrapper
that turns raised errors into responses lives in `unshackle/core/api/routes.py`.
Nothing on this page is invented; it reflects the actual server behaviour.
Nothing on this page is invented. It reflects the actual server behaviour.
## The standard error shape
@@ -59,7 +59,7 @@ included when it is non-empty, so a client must treat a missing `details` as "no
extra information". Some concrete examples you may encounter:
- A validation failure may include the offending field or the list of valid values.
- A "title not found" error on a remote-dl session includes `available_titles`.
- A "title not found" error on a remote session includes `available_titles`.
- A maintenance conflict includes `active_jobs`, the jobs blocking the operation.
- Categorised exceptions add a `reason` tag, such as `"network_connectivity"` or
`"geofence_restriction"`.
@@ -68,8 +68,8 @@ extra information". Some concrete examples you may encounter:
The `retryable` field appears **only when it is `true`**. It is set for conditions
the server believes are transient: network errors, rate limiting, and temporary
service unavailability. When you see it, backing off and retrying is reasonable; when
it is absent, assume the request will keep failing until you change something.
service unavailability. When you see it, you can back off and try the request again.
When it is absent, assume the request will keep failing until you change something.
!!! example "A retryable network error"
```json
@@ -125,7 +125,7 @@ but the defaults hold for the overwhelming majority of responses.
If a code is ever emitted without a mapped status, the server falls back to
`500`. In practice every code above has an explicit mapping.
## How errors are produced
## How the server produces errors
Understanding where an error comes from helps interpret it.
@@ -171,8 +171,8 @@ the server returns `INVALID_INPUT` (400) with the parse error in
## The authentication error exception
There is one place where the response does **not** follow the standard shape: the API
key authentication middleware. When you send a request to a protected endpoint
There is one place where the response does **not** follow the standard shape: the
API key authentication middleware. When you send a request to a protected endpoint
without a valid `X-Secret-Key` header, the middleware short-circuits with a legacy
body that uses an integer `status` and no `error_code` or `timestamp`.
@@ -189,7 +189,7 @@ body that uses an integer `status` and no `error_code` or `timestamp`.
```
!!! warning "Handle both shapes"
A robust client must expect two error formats: the standard structured error
A client must expect two error formats: the standard structured error
(string `status` of `"error"`, with `error_code`), and this authentication error
(integer `status`, no `error_code`). The `GET /api/health` endpoint is exempt
from authentication, so it never returns the key-error shape.
@@ -201,13 +201,13 @@ Not every failure or completion has a JSON body. Some endpoints answer with
deleting a history entry.
!!! warning "Do not parse `204` bodies"
A `204` response has no body. Clients must check the status code before attempting
to decode JSON, or the decode will fail on an empty string.
A `204` response has no body. Clients must examine the status code before they
decode JSON, or the decode will fail on an empty string.
## Debug information
By default, error responses never include tracebacks or internal diagnostics. If the
server was started with the `--debug-api` flag, every structured error gains a
By default, error responses never include tracebacks or internal diagnostics. If you
start the server with the `--debug-api` flag, every structured error gains a
`debug_info` object:
```json title="Error with --debug-api enabled"
@@ -225,7 +225,7 @@ server was started with the `--debug-api` flag, every structured error gains a
The `debug_info` object always carries `exception_type` and, for exception-derived
errors, a formatted `traceback`. Some endpoints add extra diagnostics such as
captured worker stderr.
captured job worker stderr.
!!! warning "Developer use only"
`--debug-api` exposes internal tracebacks in responses. Enable it only for local
@@ -235,16 +235,16 @@ captured worker stderr.
For anyone writing a consumer of the API, a durable strategy is:
1. **Read the HTTP status first.** It tells you the broad class of outcome and lets
you handle the `204`/no-body and authentication-error cases before touching the
1. **Read the HTTP status first.** It tells you the broad class of outcome, and it
lets you find the `204`/no-body and authentication-error cases before you read the
body.
2. **Branch on `error_code`.** For structured errors, this is the stable identifier
to switch on. Do not match on `message`.
3. **Respect `retryable`.** When it is `true`, back off and retry; the reference
client retries `429`, `500`, `502`, `503`, and `504` with exponential backoff. When
absent, fix the request rather than repeating it.
4. **Fall back gracefully.** Unrecognised codes should be surfaced using the
`message` plus the HTTP status.
to switch on. Do not use `message` as the identifier.
3. **Respect `retryable`.** When it is `true`, back off and try the request again. The
reference client retries `429`, `500`, `502`, `503`, and `504` with exponential
backoff. When absent, fix the request rather than repeating it.
4. **Fall back gracefully.** For an unrecognised code, show the `message` and the
HTTP status.
!!! note "Reference consumer"
unshackle's own remote-download client treats any status of `400` or above as
docs/dev/rest-api/index.md
+22 -22
View File
@@ -1,7 +1,7 @@
# REST API Overview
unshackle ships with a built-in HTTP server. Alongside the command-line tool, you
can run `unshackle serve` and drive the same download engine over a REST API:
can operate `unshackle serve` and drive the same download engine over a REST API:
searching services, listing titles and tracks, queueing downloads, watching live
progress, and browsing history, all as JSON over HTTP.
@@ -9,7 +9,7 @@ This page explains what the API is for, how to start the server, where it listen
and how the endpoints are organised. Later pages cover each area in depth.
!!! info "Who this section is for"
The REST API is aimed at people building automation, dashboards, or remote
The REST API is for people who make automation, dashboards, or remote
clients on top of unshackle. If you only download the occasional title, the
[command-line tool](../../guide/cli-reference.md) does everything the API does
and is simpler to use. Reach for the API when you want a long-running service, a
@@ -20,20 +20,20 @@ and how the endpoints are organised. Later pages cover each area in depth.
The server exposes the same capabilities as the `dl` command, but as discrete HTTP
calls you can script against:
- **Discovery**: list the services available on the server, run a service's search,
- **Discovery**: show the services available on the server, use a service's search,
and enumerate the titles and tracks behind a title ID.
- **Downloading**: submit a download as a background *job*, then follow its status and
live progress (by polling, or by subscribing to its Server-Sent Events stream),
cancel it, retry it, or bump it to the front of the queue.
- **History and housekeeping**: read the log of finished jobs, inspect the server's
effective (redacted) configuration, check external tool versions, and clear caches
effective (redacted) configuration, examine external tool versions, and clear caches
or temp files.
- **Remote-download sessions**: a thin local client can authenticate against a
service *on the server*, then pull titles, tracks, and segment info back, and proxy
DRM licensing. This is how one machine with the credentials and CDMs can serve
downloads to another. See [Remote Sessions](remote-sessions.md).
Because every download runs as a queued job in a separate worker process, the API is
Because every download runs as a queued job in a separate job worker process, the API is
well suited to running unattended: submit work, come back later, and read the result.
## Running the server
@@ -77,27 +77,27 @@ unshackle serve --api-only --host 0.0.0.0 --port 8786
!!! danger "Binding to a public interface"
`--no-key` disables authentication completely. Anyone who can reach the port can
search, download, and read your configuration. Only combine it with a host like
find titles, download them, and read your configuration. Only combine it with a host like
`127.0.0.1`, or keep the default key-based auth when binding to `0.0.0.0`. Put a
TLS-terminating reverse proxy (Caddy, nginx) in front for anything internet-facing.
### Server modes
The set of routes that is actually mounted depends on how you start the server:
The routes the server mounts depend on how you start the server:
=== "Default (CDM + REST)"
Both the pywidevine/pyplayready CDM endpoints and the full REST API are mounted.
This is the mode used when you run `unshackle serve` with no mode flags.
The server mounts both the pywidevine/pyplayready CDM endpoints and the full REST API.
This is the mode you get when you operate `unshackle serve` with no mode flags.
=== "`--api-only`"
Only the REST API is mounted; the CDM HTTP endpoints are omitted. Authentication
uses the single configured API key (or none, with `--no-key`).
The server mounts only the REST API. It does not mount the CDM HTTP endpoints.
Authentication uses the single configured API key (or none, with `--no-key`).
=== "`--remote-only`"
Only the remote-session subset of routes is exposed (health, services, search,
The server mounts only the remote-session subset of routes (health, services, search,
and everything under `/api/session/*`). This mode implies `--api-only`, and the
interactive Swagger UI is **not** mounted.
@@ -115,12 +115,12 @@ health check is `GET http://127.0.0.1:8786/api/health`.
!!! tip "Interactive API docs"
Unless you started the server with `--remote-only`, an interactive Swagger UI is
available at [`/api/docs/`](http://127.0.0.1:8786/api/docs/). It is generated from
the same route table the server actually serves, so it stays in sync with the
running build. Note that in `--api-only` mode the docs page is itself behind the
API key; only `/api/health` is exempt from authentication.
available at [`/api/docs/`](http://127.0.0.1:8786/api/docs/). The server generates
it from the same route table it serves, so it always matches the version you
operate. Note that in `--api-only` mode the docs page is itself behind the
API key. Only `/api/health` is exempt from authentication.
A quick smoke test that needs no key:
A quick smoke test that needs no API key:
```shell title="Confirm the server is up"
curl http://127.0.0.1:8786/api/health
@@ -141,7 +141,7 @@ curl http://127.0.0.1:8786/api/health
## Endpoint map
Every REST path uses JSON request and response bodies and lives under `/api`. The
table below is a map of the surface; the [Endpoints](endpoints.md) page documents each
table below is a map of the surface. The [Endpoints](endpoints.md) page gives each
request and response in full.
### Discovery
@@ -216,15 +216,15 @@ curl -X POST http://127.0.0.1:8786/api/search \
-d '{"service": "EXAMPLE", "query": "example"}'
```
A missing or unknown key returns `401`. The full model, including per-key service
allowlists, is covered in [Authentication](authentication.md).
A missing or unknown API key returns `401`. [Authentication](authentication.md) gives
the full model, including per-key service allowlists.
## Where to go next
<div class="grid cards" markdown>
- **[Quick Start](quickstart.md)**: start the server, set a key, and run your first
end-to-end download over the API.
- **[Quick Start](quickstart.md)**: start the server, set an API key, and operate your
first end-to-end download over the API.
- **[Authentication](authentication.md)**: the `X-Secret-Key` header, `--no-key`,
per-key service allowlists, and CORS.
- **[Endpoints](endpoints.md)**: every request body, query parameter, and response
docs/dev/rest-api/job-lifecycle.md
+82 -79
View File
@@ -1,23 +1,23 @@
# Download Job Lifecycle
When you submit a download over the REST API, the server does not block the request
until the file is written. Instead it creates a **job**, hands it back to you with an
until it writes the file. Instead it creates a **job**, hands it back to you with an
ID, and works through it in the background. Everything you do afterwards (watching
progress, cancelling, retrying, reading the result) is done by referring to that job
ID.
This page explains the states a job moves through, how live progress is reported, how
to cancel or retry, and, in clearly marked developer sections, how the worker and
manager behind the queue actually run the download.
This page explains the states a job moves through, how the server reports live
progress, and how to cancel or retry. Clearly marked developer sections tell you how
the job worker and the manager behind the queue operate the download.
!!! info "Who this page is for"
The first half is for anyone driving the API: the states, the fields you can read,
and the calls that move a job around. The **Under the hood** sections near the end
describe the internal worker/manager model and are aimed at contributors and people
debugging the server. You do not need them to use the API.
give the internal job worker and manager model, for contributors and for people
who debug the server. You do not need them to use the API.
For the full request and response shape of each endpoint, see
[Endpoints](endpoints.md). For how errors are reported, see
[Endpoints](endpoints.md). For how the server reports errors, see
[Error Responses](errors.md).
## The five states
@@ -26,15 +26,15 @@ A job is always in exactly one of five states, exposed as the `status` field:
| Status | Meaning |
|---|---|
| `queued` | Created and waiting for a free worker. No download has started yet. |
| `downloading` | A worker has picked the job up and the download is in progress. |
| `queued` | Created and waiting for a free job worker. No download has started yet. |
| `downloading` | A job worker has picked the job up and the download is in progress. |
| `completed` | The download finished successfully; `output_files` lists the results. |
| `failed` | The download stopped with an error; `error_message` and `error_code` explain why. |
| `cancelled` | The job was cancelled before it could finish, from either the queued or downloading state. |
The last three (`completed`, `failed`, and `cancelled`) are **terminal**: once a
job reaches one of them it never changes state again. Terminal jobs can be removed,
retried, or read from history, but they will not resume.
job reaches one of them it never changes state again. You can remove, retry, or read
terminal jobs from history, but they will not resume.
## How a job moves between states
@@ -59,7 +59,7 @@ The transitions in prose:
- **`queued` → `downloading`** happens automatically. As soon as one of the server's
workers is free, it takes the next job off the queue and starts it. You do not
trigger this; you only observe it.
trigger this. You only observe it.
- **`downloading` → `completed`** is the happy path: the download engine wrote all the
requested files and reported success.
- **`downloading` → `failed`** is any error during the download: a service failure, a
@@ -72,12 +72,12 @@ The transitions in prose:
!!! note "Priority does not change status"
Moving a job to the front of the queue with the priority endpoint reorders *when*
it will start, but it stays `queued` until a worker actually picks it up. It is not
it will start, but it stays `queued` until a job worker picks it up. It is not
a state change.
## Creating a job
A download is queued with `POST /api/download`. The request needs at least a `service`
You queue a download with `POST /api/download`. The request needs at least a `service`
and a `title_id`, plus any download options you want to set. On success the server
replies with **`202 Accepted`** and the new job's identity:
@@ -101,14 +101,15 @@ curl -X POST http://127.0.0.1:8786/api/download \
}
```
The job starts in `queued`. Workers auto-start on the first job submitted, so there is
nothing else to switch on. Keep the `job_id`; every later call is addressed to it.
The job starts in `queued`. The job workers auto-start on the first job you submit, so
you switch nothing else on. Keep the `job_id`. Every later call uses it.
!!! tip "The full parameter set"
`POST /api/download` accepts the same download options as the `dl` command
(quality, codecs, language selection, `wanted` episode ranges, and many more), with
server-side defaults filling in anything you omit. The complete list, their
defaults, and validation rules are documented on the [Endpoints](endpoints.md) page.
server-side defaults filling in anything you do not write. The
[Endpoints](endpoints.md) page gives the complete list, their defaults, and the
validation rules.
## Reading a job's status and progress
@@ -136,7 +137,7 @@ There are two ways to look at jobs.
-H "X-Secret-Key: your-api-secret"
```
The list can be filtered by `status` and `service`, and ordered with `sort_by`
You can filter the list by `status` and `service`, and order it with `sort_by`
(`created_time`, `started_time`, `completed_time`, `progress`, `status`, or
`service`) and `sort_order` (`asc` / `desc`). See [Endpoints](endpoints.md) for the
query-parameter reference.
@@ -179,7 +180,7 @@ The full detail adds `started_time`, `completed_time`, `output_files`, the redac
!!! tip "Or subscribe instead of polling"
`GET /api/download/jobs/{job_id}/events` pushes the same job object as Server-Sent
Events, and closes the stream when the job ends. See
Events, and closes the event stream when the job ends. See
[Endpoints](endpoints.md#get-apidownloadjobsjob_idevents).
!!! note "What the percentage means"
@@ -201,15 +202,15 @@ The full detail adds `started_time`, `completed_time`, `output_files`, the redac
### Completed
A `completed` job has its `output_files` populated with the paths of the muxed
results, and `progress` is `100`. This is the state you are waiting for.
results, and `progress` is `100`. This is the state you want.
### Failed
A `failed` job carries the error. `error_message` is the short human message,
`error_code` is one of the API's structured error codes (for example `NETWORK_ERROR`,
`DRM_ERROR`, `AUTH_FAILED`, or `DOWNLOAD_ERROR`), and `error_details` gives more
context. When the server was started with `--debug-api`, `error_traceback` and
`worker_stderr` are also filled in for diagnosis.
context. When you start the server with `--debug-api`, `error_traceback` and
`worker_stderr` are also filled in, to help you find the cause.
!!! tip "Secrets are scrubbed from error text"
A service can echo a credential or a proxy URL into its output. The server masks
@@ -221,8 +222,8 @@ The error-code catalogue and how each maps to an HTTP status is on the
### Cancelled
A `cancelled` job was stopped on request before finishing. It may have partial output,
but it is not considered complete. See below.
The server stopped a `cancelled` job on request, before it finished. It may have
partial output, but it is not considered complete. See below.
## Cancelling a job
@@ -241,7 +242,7 @@ both stops in-flight work and tidies up afterwards.
!!! warning "Do not parse the body of a 204"
Removing a terminal job returns `204 No Content` with an empty body. A client must
not try to decode JSON from it; check the status code first.
not try to decode JSON from it. Examine the status code first.
```shell title="Cancel a running job"
curl -X DELETE http://127.0.0.1:8786/api/download/jobs/b1e6c7de-6d0a-4f2f-9d1e-3a2b4c5d6e7f \
@@ -254,14 +255,15 @@ which returns `{"removed": <count>}`.
## Retrying a job
`POST /api/download/jobs/{job_id}/retry` re-runs a finished job by creating a **new**
job with the same service, title, and parameters. The original job is untouched and
stays in its terminal state; the response gives you the new job's ID with `202`.
job with the same service, title, and parameters. The retry does not change the
original job. It stays in its terminal state, and the response gives you the new job's
ID with `202`.
- Only terminal jobs can be retried. Retrying a `queued` or `downloading` job returns
- You can retry only terminal jobs. A retry of a `queued` or `downloading` job returns
`409` with error code `CONFLICT`.
- A missing job returns `404` (`JOB_NOT_FOUND`).
- The service allowlist and any download gates are re-checked for the new job, exactly
as they were for the original submission, so a retry can be rejected with
- The server examines the service allowlist and any download gates again for the new
job, exactly as for the original submission. Thus the server can reject a retry with
`INVALID_SERVICE` or `FORBIDDEN` if permissions changed.
```json title="202 Accepted, a fresh job"
@@ -275,38 +277,39 @@ stays in its terminal state; the response gives you the new job's ID with `202`.
## Prioritising a job
`POST /api/download/jobs/{job_id}/priority` moves a **queued** job to the front of the
line, so it will be the next one a free worker picks up.
line, so it will be the next one a free job worker picks up.
- The job must be `queued`. Prioritising a job in any other state returns `409`
(`CONFLICT`); a missing job returns `404`.
(`CONFLICT`). A missing job returns `404`.
- On success the response is `{"job_id": "<id>", "position": "front"}`.
This only reorders the queue; the job's status stays `queued` until a worker starts
it.
This only reorders the queue. The job's status stays `queued` until a job worker
starts it.
## History and retention
Every job that reaches a terminal state is also written once to a persistent history
log, so it survives even after the job is removed from the live list.
The server also writes every job that reaches a terminal state once to a persistent
history log. The job survives there even after the server removes it from the live
list.
- Read it with `GET /api/history` (newest first; supports `limit` and `service`
- Read it with `GET /api/history` (newest first, with `limit` and `service`
filters).
- Delete a single entry with `DELETE /api/history/{job_id}` (returns `204`).
The live in-memory jobs are separate from this on-disk history. A background cleanup
task periodically drops old terminal jobs from the live list based on a retention
period; the history file keeps them for later reference. See [Endpoints](endpoints.md)
period. The history file keeps them for later reference. See [Endpoints](endpoints.md)
for the history response shape.
!!! note "Why the history file lives under the cache directory"
`api_history.jsonl` is placed under the cache directory on purpose, so it stays out
of the config/data tree. It is disposable runtime state, not user configuration or
unshackle puts `api_history.jsonl` under the cache directory on purpose, so it
stays out of the config/data tree. It is disposable runtime state, not user configuration or
curated data, which is exactly why clearing the cache is a safe operation against
it.
---
## Under the hood: the worker/manager model
## Under the hood: the job worker and manager model
!!! info "Developer section"
Everything below describes server internals. It is useful for contributors and for
@@ -316,18 +319,18 @@ for the history response shape.
A single process-wide **download queue manager** owns all jobs. It holds them in
memory keyed by ID, backs the wait list with a FIFO queue, and keeps a separate
priority deque for jobs that have been bumped to the front. On the first job, it starts
a fixed pool of worker tasks plus one cleanup task; starting is idempotent, so later
jobs reuse the same pool.
priority deque for the jobs it bumped to the front. On the first job, it starts
a fixed pool of job worker tasks plus one cleanup task. The start is idempotent, so
later jobs reuse the same pool.
- **Concurrency** is capped by `max_concurrent_downloads` (default **2**). That many
jobs can be in `downloading` at once; the rest wait in `queued`.
- **Retention** is governed by `job_retention_hours` (default **24**). An hourly
- **Concurrency**: `max_concurrent_downloads` (default **2**) caps it. That many
jobs can be in `downloading` at once. The rest wait in `queued`.
- **Retention**: `job_retention_hours` (default **24**) controls it. An hourly
cleanup task removes terminal jobs whose completion time (or, if never started,
creation time) is older than the retention window.
!!! note "Two concurrency knobs that are easy to conflate"
`max_concurrent_downloads` caps how many separate download **jobs** run at once.
`max_concurrent_downloads` caps how many separate download **jobs** operate at once.
The separate `serve.downloads` setting controls how many **tracks** download in
parallel *within* a single job. They are independent, and raising one does not affect
the other.
@@ -336,64 +339,64 @@ jobs reuse the same pool.
The manager reads these as top-level config attributes
(`config.max_concurrent_downloads` and `config.download_job_retention_hours`).
Those keys are not part of the standard config schema, so in practice the built-in
defaults of **2** and **24** apply. The live values are surfaced by
[`GET /api/config`](endpoints.md). The related `serve.history_limit` (default
defaults of **2** and **24** apply. [`GET /api/config`](endpoints.md) gives the
live values. The related `serve.history_limit` (default
**100**) instead controls how many entries the on-disk history file keeps.
Each worker loop drains the priority deque first, then the FIFO queue. A job that was
promoted has its stale FIFO copy skipped so it does not run twice. When a worker takes
a job it flips it to `downloading`, stamps `started_time`, runs it, and on finish
stamps `completed_time` and records history.
Each job worker loop drains the priority deque first, then the FIFO queue. The loop
skips the stale FIFO copy of a promoted job, so it does not do the job twice. When
a job worker takes a job it flips it to `downloading`, stamps `started_time`, does the
download, and on finish stamps `completed_time` and records history.
### Each job runs in its own subprocess
The manager does **not** run the download in the server's event loop or even in the
The manager does **not** operate the download in the server's event loop or even in the
server process. For each job it spawns a separate Python subprocess:
```text
python -m unshackle.core.api.download_worker <payload.json> <result.json> <progress.json>
```
Three temporary files bridge the parent and the worker:
Three temporary files bridge the parent and the job worker:
- **`payload.json`**: written by the manager before launch; contains the job ID,
- **`payload.json`**: the manager writes it before launch. It contains the job ID,
service, title ID, and merged parameters.
- **`progress.json`**: written continuously by the worker as the download proceeds.
- **`result.json`**: written by the worker at the end.
- **`progress.json`**: the job worker writes it continuously as the download proceeds.
- **`result.json`**: the job worker writes it at the end.
Running each download out-of-process isolates the server from crashes, memory growth,
and blocking work inside the download engine, and makes hard cancellation (killing the
process) reliable.
The worker's exit codes are `0` for success, `1` for a download error, and `2` for bad
The job worker's exit codes are `0` for success, `1` for a download error, and `2` for bad
arguments. Its `result.json` is either `{"status": "success", "output_files": [...]}`
or `{"status": "error", "message", "error_details", "error_code", "traceback"}`. If the
process exits non-zero, or the result is missing or not a success, the parent marks the
job `failed` and adopts the worker's `error_code` (or derives one by categorising the
job `failed` and adopts the job worker's `error_code` (or derives one by categorising the
exception).
### How progress reaches the API
While the worker runs, the manager polls `progress.json` every **0.5 s** and copies the
While the job worker operates, the manager polls `progress.json` every **0.5 s** and copies the
fields it finds onto the live `DownloadJob`. That is what the `GET` endpoints return.
Inside the worker, progress is aggregated across all selected tracks. Each track's
completion fraction is weighted by its bitrate, so video and audio dominate the bar and
subtitles barely move it (subtitles get a small fixed weight). Track fractions are kept
**monotonic**: the percentage only ever climbs. The download portion is scaled to fill
`0` to `90`; the remaining `90` to `100` is driven by the download engine as it repackages
and muxes. A track counts as done when it reaches a terminal state such as
The job worker aggregates progress across all selected tracks. It weights each track's
completion fraction by its bitrate, so video and audio dominate the bar and subtitles
barely move it (subtitles get a small fixed weight). Track fractions are kept
**monotonic**: the percentage only ever climbs. The job worker scales the download
portion to fill `0` to `90`. The download engine drives the remaining `90` to `100` as
it repackages and muxes. A track counts as done when it reaches a terminal state such as
`Downloaded`, `Decrypted`, or a skipped subtitle.
### How cancellation actually works
### How cancellation works
Each job carries a cancellation event. When you `DELETE` a live job:
- If it is **`queued`**, it flips straight to `cancelled`, its history is recorded
immediately (it never reaches a worker), and the worker loop skips it if it later
pops it off the queue.
- If it is **`queued`**, it flips straight to `cancelled`, the manager records its
history immediately (it never reaches a job worker), and the job worker loop skips it
if it later pops it off the queue.
- If it is **`downloading`**, the manager sets the cancellation event and flips the job
to `cancelled`. The polling loop notices the event, `terminate()`s the worker
to `cancelled`. The polling loop notices the event, `terminate()`s the job worker
subprocess (escalating to `kill()` if it does not exit within 5 seconds), and raises
a cancellation so the task unwinds. The subprocess model is what makes this a clean,
immediate stop rather than a cooperative one.
@@ -403,11 +406,11 @@ Each job carries a cancellation event. When you `DELETE` a live job:
Job parameters may carry secrets: a raw `user:pass` credential, or a proxy URL with
embedded userinfo. Whenever the manager serializes a job (for an API response or the
history file), it masks the sensitive keys (`credential`, `credentials`, `password`,
`token`, `api_key`) and strips userinfo from proxy URLs. Free-text fields that a
service might have leaked a secret into (`error_message`, `error_details`,
`error_traceback`, `worker_stderr`, and worker stdout/stderr in the logs) are scrubbed
of the job's known secret values, longest-match first, before they are ever returned or
logged.
`token`, `api_key`) and strips userinfo from proxy URLs. The manager also scrubs
free-text fields that a service might have leaked a secret into (`error_message`,
`error_details`, `error_traceback`, `worker_stderr`, and job worker stdout/stderr in the
logs). It removes the job's known secret values, longest-match first, before it returns
or logs them.
## See also
docs/dev/rest-api/quickstart.md
+27 -27
View File
@@ -1,11 +1,11 @@
# REST API Quick Start
Unshackle ships with a built-in REST API that exposes the same download engine you drive from the command line. It is served by the `unshackle serve` command and lets you queue downloads, poll their progress, browse services, and manage history over HTTP instead of the terminal.
Unshackle ships with a built-in REST API that exposes the same download engine you drive from the command line. The `unshackle serve` command serves it and lets you queue downloads, poll their progress, browse services, and manage history over HTTP instead of the terminal.
The full loop runs end to end: start the server, authenticate, submit a download job, and poll it to completion, using real `curl` commands built directly from the API routes.
The full loop runs end to end: start the server, authenticate, submit a download job, and poll it to completion. The `curl` commands here come directly from the API routes.
!!! note "Who this page is for"
This is a **developer** guide. It assumes you are comfortable with HTTP, JSON, and a tool like `curl`. If you just want to download something, the [command line](../../index.md) is simpler. The REST API is meant for building tools, UIs, and automation on top of unshackle.
This is a **developer** guide. It assumes you are comfortable with HTTP, JSON, and a tool like `curl`. If you only want to download something, the [command line](../../index.md) is simpler. The REST API is meant for building tools, UIs, and automation on top of unshackle.
---
@@ -30,7 +30,7 @@ You need a working unshackle install with services and CDM devices already confi
## Step 1: Configure an API secret
Every request (except the health check) is authenticated with a secret key sent in the `X-Secret-Key` header. The server reads this key from the `serve` section of your `unshackle.yaml` config.
The server authenticates every request (except the health check) with an API key that you send in the `X-Secret-Key` header. The server reads this API key from the `serve` section of your `unshackle.yaml` config.
At minimum you must set `api_secret`:
@@ -39,7 +39,7 @@ serve:
api_secret: "choose-a-long-random-secret"
```
If you want the API to be able to license and download DRM-protected content, you also need to grant CDM devices to the key. Configure that under `serve.users`, keyed by the secret:
If you want the API to license and download DRM-protected titles, you must also grant CDM devices to the API key. Configure that under `serve.users`, keyed by the secret:
```yaml title="unshackle.yaml"
serve:
@@ -55,7 +55,7 @@ serve:
If `api_secret` is not set and you do **not** pass `--no-key`, `unshackle serve` refuses to start and raises an error. Treat the secret like a password. Anyone who has it can queue downloads on your machine.
!!! tip "Disabling auth for local testing"
Passing `--no-key` to `serve` turns authentication off entirely: every request is allowed and the `X-Secret-Key` header is ignored. This is convenient for a throwaway local test, but never expose a `--no-key` server to a network you do not fully trust.
`--no-key` turns authentication off entirely: the server allows every request and ignores the `X-Secret-Key` header. This is convenient for a throwaway local test, but never put a `--no-key` server on a network you do not fully trust.
---
@@ -67,7 +67,7 @@ Start the REST API with the `serve` command:
unshackle serve --api-only
```
By default the server binds to `127.0.0.1:8786`. The `--api-only` flag serves just the REST API and omits the Widevine/PlayReady CDM HTTP endpoints; drop it if you also want to expose those.
By default the server binds to `127.0.0.1:8786`. The `--api-only` flag serves only the REST API and omits the Widevine/PlayReady CDM HTTP endpoints. Drop it when you also want those endpoints.
Common options:
@@ -90,9 +90,9 @@ Swagger UI available at http://127.0.0.1:8786/api/docs/
```
!!! note "Swagger UI is also behind auth"
In `--api-only` mode (without `--no-key`), the Swagger UI at `/api/docs/` requires the `X-Secret-Key` header just like every other route. Only `/api/health` is exempt from authentication.
In `--api-only` mode (without `--no-key`), the Swagger UI at `/api/docs/` requires the `X-Secret-Key` header, the same as every other route. Only `/api/health` is exempt from authentication.
For the rest of this page we will assume the server is reachable at `http://127.0.0.1:8786` and store the secret in a shell variable:
For the rest of this page we will assume the server is reachable at `http://127.0.0.1:8786` and store the secret in an environment variable:
```shell
export SECRET="choose-a-long-random-secret"
@@ -101,9 +101,9 @@ export BASE="http://127.0.0.1:8786"
---
## Step 3: Confirm the server is alive
## Step 3: Make sure that the server is alive
The health endpoint is the one route that never requires a key. Use it to confirm the server is running and to read its version:
The health endpoint is the one route that never requires an API key. Use it to make sure that the server operates, and to read its version:
```shell
curl "$BASE/api/health"
@@ -127,7 +127,7 @@ The `update_check` block is best effort. If unshackle cannot reach the update so
## Step 4: Authenticate a real request
Every other endpoint needs the `X-Secret-Key` header. A quick way to prove your key works is to list the services the server can see:
Every other endpoint needs the `X-Secret-Key` header. A quick way to prove your API key works is to show the services the server can see:
```shell
curl "$BASE/api/services" \
@@ -154,14 +154,14 @@ curl "$BASE/api/services" \
}
```
If the key is missing or wrong, you get a `401` with a plain body:
If the API key is missing or wrong, you get a `401` with a plain body:
```json title="401 Unauthorized"
{ "status": 401, "message": "Secret Key is Invalid." }
```
!!! warning "Two error shapes exist"
The authentication middleware returns the short `{"status": 401, "message": "..."}` shape shown above. Every **other** error the API produces uses the richer structured shape (`error_code`, `timestamp`, `details`, ...) described in [Error handling](#error-handling). A robust client should handle both.
The authentication middleware returns the short `{"status": 401, "message": "..."}` shape shown above. Every **other** error the API produces uses the richer structured shape (`error_code`, `timestamp`, `details`, ...) described in [Error handling](#error-handling). Parse both shapes in your client.
The `tag` value for the service you want is the identifier you pass as `service` in the calls below.
@@ -241,7 +241,7 @@ The request body accepts the full download parameter set. A few of the most usef
Validation is strict: invalid codecs, subtitle formats, dynamic-range names, non-positive counts, or mutually exclusive flags (for example both `video_only` and `audio_only`) come back as `400` with error code `INVALID_PARAMETERS`. See the full reference on the [download reference page](endpoints.md) for every accepted field.
!!! note "Gated fields"
Passing a per-request CDM (`cdm`) or inline credentials (`credential` / `credentials`) is refused with `403 FORBIDDEN` unless the server operator has explicitly opted in via `serve.cdm_overrides` and `serve.allow_job_credentials`. Both are off by default.
The server refuses a per-request CDM (`cdm`) or inline credentials (`credential` / `credentials`) with `403 FORBIDDEN`, unless the server operator has explicitly opted in with `serve.cdm_overrides` and `serve.allow_job_credentials`. Both are off by default.
---
@@ -260,7 +260,7 @@ curl "$BASE/api/download/jobs/$JOB" \
-H "X-Secret-Key: $SECRET"
```
While the download is running you will see `status: "downloading"` and a `progress` value climbing from `0` toward `100`:
While the download runs you will see `status: "downloading"` and a `progress` value climbing from `0` toward `100`:
```json title="200 OK (in progress)"
{
@@ -313,14 +313,14 @@ done
A few seconds between polls is plenty. The server updates a job's progress roughly twice a second internally, and each poll reads the current snapshot.
!!! tip "Or let the server push the updates"
`GET /api/download/jobs/{job_id}/events` streams the same job record as Server-Sent Events, and closes the stream when the job ends. It removes the polling loop above. See [Endpoints](endpoints.md#get-apidownloadjobsjob_idevents).
`GET /api/download/jobs/{job_id}/events` streams the same job record as Server-Sent Events, and closes the event stream when the job ends. It removes the polling loop above. See [Endpoints](endpoints.md#get-apidownloadjobsjob_idevents).
```bash
curl -N "$BASE/api/download/jobs/$JOB/events" \
-H "X-Secret-Key: $SECRET"
```
If a job ends in `failed`, the full record includes `error_message`, `error_code`, and (when the server was started with `--debug-api`) `error_traceback` and `worker_stderr` to help you diagnose it. Sensitive values in these fields are scrubbed before they are returned.
If a job ends in `failed`, the full record includes `error_message`, `error_code`, and (when you start the server with `--debug-api`) `error_traceback` and `worker_stderr` to help you find the cause. The server scrubs sensitive values in these fields before it returns them.
---
@@ -333,11 +333,11 @@ curl -X DELETE "$BASE/api/download/jobs/$JOB" \
-H "X-Secret-Key: $SECRET"
```
- If the job is **terminal** (completed/failed/cancelled) it is removed from the manager and you get **`204 No Content`** with an empty body.
- If the job is still **queued or downloading** it is cancelled instead, and you get `200` with `{"status": "success", "message": "Job cancelled"}`.
- If the job is **terminal** (completed/failed/cancelled), the manager removes it. You get **`204 No Content`** with an empty body.
- If the job is still **queued or downloading**, the manager cancels it instead. You get `200` with `{"status": "success", "message": "Job cancelled"}`.
!!! warning "Do not parse JSON on 204"
A `204` response has no body. This applies to terminal-job removal here and to deleting history entries; check the status code before attempting to decode JSON.
A `204` response has no body. This applies to terminal-job removal here and to deleting history entries. Examine the status code before you decode JSON.
To clear every finished job at once instead of one at a time:
@@ -365,7 +365,7 @@ A handful of endpoints round out job control. All require the `X-Secret-Key` hea
| `POST` | `/api/download/jobs/{job_id}/priority` | Move a queued job to the front of the queue |
| `POST` | `/api/download/jobs/clear-finished` | Remove all terminal jobs |
Listing supports query parameters for filtering by `status` or `service`, sorting via `sort_by` (`created_time`, `started_time`, `completed_time`, `progress`, `status`, `service`) and `sort_order` (`asc` / `desc`), and requesting full per-job detail with `full=true`:
Listing supports query parameters for filtering by `status` or `service`, sorting with `sort_by` (`created_time`, `started_time`, `completed_time`, `progress`, `status`, `service`) and `sort_order` (`asc` / `desc`), and requesting full per-job detail with `full=true`:
```shell
curl "$BASE/api/download/jobs?status=downloading&sort_by=progress&sort_order=desc" \
@@ -390,7 +390,7 @@ curl "$BASE/api/download/jobs?status=downloading&sort_by=progress&sort_order=des
}
```
Retrying reuses the original job's service, title, and parameters to enqueue a brand new job (returning `202` with a fresh `job_id`); it only works on terminal jobs and returns `409 CONFLICT` otherwise. Prioritizing only works on jobs still in the `queued` state.
A retry reuses the original job's service, title, and parameters to enqueue a brand new job, and returns `202` with a fresh `job_id`. A retry works only on terminal jobs, and returns `409 CONFLICT` otherwise. A priority change works only on jobs still in the `queued` state.
---
@@ -398,10 +398,10 @@ Retrying reuses the original job's service, title, and parameters to enqueue a b
The download loop above is the core of the API, but the same server exposes more:
- **Discovery**: `POST /api/search` finds titles by query, and `POST /api/list-titles` / `POST /api/list-tracks` inspect what a title offers before you commit to downloading it. See [list titles and tracks](endpoints.md).
- **Discovery**: `POST /api/search` finds titles by query, and `POST /api/list-titles` / `POST /api/list-tracks` inspect what a title offers before you commit to downloading it. See [titles and tracks](endpoints.md).
- **History**: `GET /api/history` reads a persisted log of jobs that reached a terminal state (newest first), and `DELETE /api/history/{job_id}` removes a single entry. See [history](endpoints.md).
- **Introspection**: `GET /api/profiles` lists configured credential profiles per service, `GET /api/config` returns a redacted view of the effective server config, and `GET /api/env/check` reports which external binaries are installed.
- **Maintenance**: `POST /api/maintenance/clear-cache`, `.../clear-temp`, and `.../refresh-services` perform housekeeping (the cache/temp clears are blocked with `409` while a download is active).
- **Introspection**: `GET /api/profiles` lists configured credential profiles per service, `GET /api/config` returns a redacted view of the effective server config, and `GET /api/env/check` reports which external binaries the machine has.
- **Maintenance**: `POST /api/maintenance/clear-cache`, `.../clear-temp`, and `.../refresh-services` do housekeeping (the server blocks the cache and temp clears with `409` while a download is active).
- **Remote sessions**: the `/api/session/*` endpoints drive the thin-client "remote download" flow, where the server authenticates and licenses on behalf of a local client. See [remote sessions](remote-sessions.md).
The interactive **Swagger UI at `/api/docs/`** documents every route and lets you try them in the browser (send your `X-Secret-Key` there too).
@@ -422,7 +422,7 @@ Apart from the auth middleware's short form, every error the API returns uses a
}
```
`details` appears only when there is something to add, `retryable: true` appears only for transient failures, and a `debug_info` block (exception type and traceback) is included only when the server runs with `--debug-api`.
`details` appears only when the server has more to add. `retryable: true` appears only for transient failures. The server includes a `debug_info` block (exception type and traceback) only when it runs with `--debug-api`.
The `error_code` maps to an HTTP status. The ones you will meet most often:
docs/dev/rest-api/remote-sessions.md
+85 -83
View File
@@ -1,6 +1,6 @@
# Remote Service Sessions
A **remote service session** lets one machine do the hard, service-specific work
A **remote session** lets one machine do the hard, service-specific work
(authenticating to a streaming service, listing titles, resolving tracks and
manifests, and, optionally, DRM licensing) while another machine runs the actual
download, decryption, and muxing locally.
@@ -9,14 +9,14 @@ This is the mechanism behind unshackle's client/server "remote download" workflo
A thin local client (`RemoteService`) talks to an unshackle `serve` instance over
HTTP. The server keeps an authenticated service instance alive between requests so
the client can authenticate once and then make many follow-up calls against the
same session.
same remote session.
!!! note "Why you'd use this"
- The server has service accounts, cookies, or a CDM you don't want to copy to
- The server has service accounts, cookies, or a CDM you do not want to copy to
every machine.
- You want to run downloads on a fast/local machine but keep credentials and
licensing centralized.
- A service is region-locked to where the server lives, and the client isn't.
- You want to do downloads on a fast/local machine but keep credentials and
licensing centralised.
- A service is region-locked to where the server lives, and the client is not.
---
@@ -43,7 +43,7 @@ selection (downloading segments, decrypting, and muxing) runs locally.
The `RemoteService` adapter lives inside the ordinary `dl` command and replaces
the old standalone `remote_dl` command. If you have prior scripts or docs that
invoke `remote_dl`, use `dl` with a configured remote server instead. The
functionality now rides on `dl` via `RemoteService`.
functionality now rides on `dl` through `RemoteService`.
For the auth-facing endpoints, header format, and error shapes referenced below,
see [Authentication](authentication.md) and the endpoint reference in
@@ -79,18 +79,18 @@ remote_services:
!!! warning "Port default"
The `unshackle serve` default port is **`8786`** (not `8080`). Use the port
your server actually binds. Some older example snippets show `8080`, which is
your server binds. Some older example snippets show `8080`, which is
not the default.
### Selecting a server
If exactly one server is configured, unshackle uses it implicitly:
If you configure exactly one server, unshackle uses it implicitly:
```bash
unshackle dl EXAMPLE 0ABCDEF
```
If **more than one** server is configured, you must pick one with `--server`:
If you configure **more than one** server, you must pick one with `--server`:
```bash
unshackle dl --server my_server EXAMPLE 0ABCDEF
@@ -104,17 +104,17 @@ you to add the block shown above.
## What happens during a download
From your point of view the command looks like an ordinary download. Under the
hood the client walks a session through its lifecycle.
hood the client walks a remote session through its lifecycle.
=== "1. Create + authenticate"
The client calls `POST /api/session/create`, forwarding whatever it can so the
server doesn't have to prompt:
server does not have to prompt:
- Local **credentials** for the service/profile (`{username, password, extra?}`)
- Local **cookies**, compressed and base64-encoded
- A resolved **proxy**, or, if you didn't set one, your detected
**client region** so the server can auto-proxy to match
- A resolved **proxy**, or, if you did not set one, your detected
**client region** so the server can auto-proxy to match it
- Track-selection hints (`range_`, `vcodec`, `quality`, `best_available`) so
the server fetches the right manifests
- Your local per-service **cache files** (e.g. refreshed tokens)
@@ -125,7 +125,7 @@ hood the client walks a session through its lifecycle.
=== "2. Interactive prompts (if needed)"
Some services need a one-time code, PIN, or device confirmation. When the
server's auth flow asks for input, the session enters `pending_input`. The
server's auth flow asks for input, the remote session enters `pending_input`. The
client polls, displays the prompt to you locally, collects your answer, and
sends it back. The server's auth thread resumes with your response.
@@ -136,29 +136,29 @@ hood the client walks a session through its lifecycle.
=== "3. Titles + tracks"
Once authenticated, the client fetches the title list, then the tracks for the
chosen title. The tracks come back with playback URLs, and DASH/ISM manifests
are shipped as compressed XML so the client can re-parse them locally for
chosen title. The tracks come back with playback URLs, and the server ships
DASH/ISM manifests as compressed XML so the client can re-parse them locally for
downloading. Any `session_headers` / `session_cookies` the server used are
merged into the client's local HTTP session.
=== "4. Licensing"
For DRM content the client either **proxies** its CDM challenge through the
server, or, if `server_cdm: true`, asks the server to run the full CDM flow
For a DRM-protected title the client either **proxies** its CDM challenge through
the server, or, if `server_cdm: true`, asks the server to do the full CDM flow
and return the content keys directly. See
[Server-side vs. proxied CDM](#server-side-vs-proxied-cdm) below.
=== "5. Download + close"
The client downloads, decrypts, and muxes locally. On completion it deletes
the session. If the server has updated cache files (e.g. a refreshed token),
they're returned on delete and saved locally so the **next** session can skip
interactive auth.
the remote session. If the server has updated cache files (for example a refreshed
token), the delete returns them and the client saves them locally, so the **next**
remote session can skip interactive auth.
!!! tip "Renaming remote titles locally"
You can rename titles for a remote service you don't have installed locally by
You can rename titles for a remote service you do not have installed locally by
adding a `title_map` under that service in your client `remote_services.<name>.services`
config. The server sends raw titles; your local map wins.
config. The server sends raw titles. Your local map wins.
This is deliberate: the server does **no** `title_map` remapping of its own and
sends titles exactly as the service returns them. All remapping happens on the
@@ -178,26 +178,26 @@ There are two ways DRM keys get resolved, chosen by the client's `server_cdm` fl
to `POST /api/session/{id}/license`, the server forwards it to the service, and
the raw license bytes come back for your local CDM to parse.
- Keeps your CDM local; the server just relays the license request.
- Keeps your CDM local. The server only relays the license request.
- Used when `server_cdm` is `false` (the default).
=== "Server-side CDM (server_cdm: true)"
The **server's CDM** does everything. The client sends track IDs (or a PSSH),
the server checks its key vaults, loads the device configured for your API key,
runs the CDM flow, and returns `KID:KEY` pairs directly. No CDM is needed on the
client.
runs the CDM flow, and returns `KID:KEY` pairs directly. The client needs no
CDM.
- Enable with `server_cdm: true` in the server entry.
- The server tells the client which DRM type it actually used
- The server tells the client which DRM type it used
(`widevine` or `playready`).
---
## Session lifecycle and expiry
## Remote session lifecycle and expiry
Sessions live in an **in-memory store on the server** and expire on a timer. You
generally never touch this directly, but it explains behavior you might observe.
Remote sessions live in an **in-memory store on the server** and expire on a timer. You
generally never touch this directly, but it explains behaviour you might observe.
| Behavior | Value | Notes |
|---|---|---|
@@ -207,9 +207,10 @@ generally never touch this directly, but it explains behavior you might observe.
| Cleanup sweep | every 60s | Expired sessions are removed and their input prompts cancelled |
!!! note "Auth is never rushed"
A session that is still `authenticating` or waiting on `pending_input` is **not**
subject to the short 300s TTL; it gets the full 600s auth window. This gives you
time to enter an interactive code without the session vanishing mid-prompt.
A remote session that is still `authenticating` or waiting on `pending_input` is
**not** subject to the short 300s TTL. It gets the full 600s auth window. This gives
you time to enter an interactive code without the remote session vanishing
mid-prompt.
The TTL and max-session limits are read from server config:
@@ -223,17 +224,18 @@ serve:
## Security properties worth knowing
- **API key required.** Every session request carries `X-Secret-Key`; requests
without a valid key are rejected (health check aside). See
- **API key required.** Every remote session request carries `X-Secret-Key`. The
server rejects requests without a valid API key (health check aside). See
[Authentication](authentication.md).
- **IP binding.** A session records the creator's IP at create time. If a later
request for the same session comes from a different IP, the server returns
`403 FORBIDDEN`. Sessions are not portable between hosts.
- **Namespaced, isolated cache.** Each session gets its own cache directory,
namespaced by a hash of the API key and the session ID, so sessions can't read
each other's cached tokens. The directory is deleted when the session ends.
- **Secrets are scrubbed from logs.** Session IDs, service tags, and other user
values are sanitized before logging.
- **IP binding.** A remote session records the creator's IP when the server makes it.
If a later request for the same remote session comes from a different IP, the server
returns `403 FORBIDDEN`. Remote sessions are not portable between hosts.
- **Namespaced, isolated cache.** Each remote session gets its own cache directory,
namespaced by a hash of the API key and the remote session ID, so remote sessions
cannot read each other's cached tokens. The server deletes the directory when the
remote session ends.
- **The server redacts secrets in logs.** It redacts remote session IDs, service tags,
and other user values before it writes them to the log.
---
@@ -243,18 +245,18 @@ serve:
The material below documents the server-side implementation and the HTTP
contract. End users configuring a client can stop at the sections above.
### Session endpoints
### Remote session endpoints
All routes are exposed even in `--remote-only` server mode. Paths use the
`session_id` returned by create.
The server mounts all these routes, even in `--remote-only` mode. Paths use the
`session_id` that `create` returns.
!!! note "Why `--remote-only` exists"
`--remote-only` narrows the server to just the health, services, search, and
session subset (and emits CORS headers) precisely so it is safe to sit behind
`--remote-only` narrows the server to only the health, services, search, and
remote session subset (and emits CORS headers), precisely so it is safe to sit behind
Cloudflare or serve cross-origin browser clients. That trimmed surface, rather
than the full `--api-only` mode, is what makes a CORS/Cloudflare-fronted
deployment practical: reach for it when the server is public-facing or accessed
from a browser origin, not when you simply want the local HTTP API.
from a browser origin, not when you want only the local HTTP API.
| Method | Path | Purpose |
|---|---|---|
@@ -284,8 +286,8 @@ The response returns **before** authentication finishes:
```
Authentication runs on a background thread (`asyncio.to_thread(authenticate, ...)`).
The session starts in `AUTHENTICATING`; the client must poll the prompt endpoint
until it reaches `authenticated` (or `failed`).
The remote session starts in `AUTHENTICATING`. The client must poll the prompt
endpoint until it reaches `authenticated` (or `failed`).
### Interactive auth: the `InputBridge`
@@ -296,7 +298,7 @@ layer.
- `AuthStatus` values: `authenticating`, `pending_input`, `authenticated`, `failed`.
- `InputBridge.request_input(prompt, timeout=600)` blocks the sync auth thread on a
`threading.Event` until `submit_response()` or `cancel()` fires.
- A timeout raises `TimeoutError` and marks the session `FAILED`.
- A timeout raises `TimeoutError` and marks the remote session `FAILED`.
- `AUTH_INPUT_TIMEOUT = 600.0` seconds. This is also the TTL granted to
`AUTHENTICATING` / `PENDING_INPUT` sessions in the store.
@@ -309,30 +311,30 @@ layer.
{ "status": "failed", "error": "...message..." }
```
A missing session returns `404 SESSION_NOT_FOUND`; an IP mismatch returns
A missing remote session returns `404 SESSION_NOT_FOUND`. An IP mismatch returns
`403 FORBIDDEN`. `POST /api/session/{id}/prompt` takes `{ "response": "..." }` and
returns `{ "status": "accepted" }`; posting with no pending prompt is an
returns `{ "status": "accepted" }`. A post with no pending prompt is an
`INVALID_INPUT` error.
### `SessionStore` internals
Source: `unshackle/core/api/session_store.py`. A singleton obtained via
Source: `unshackle/core/api/session_store.py`. A singleton you get from
`get_session_store()`.
- **Config-driven limits.** `serve.session_ttl` (default `300`) and
`serve.max_sessions` (default `100`) are read as properties, so config changes
take effect without recreating the store.
- **`create()`** evicts the least-recently-accessed session when at capacity,
then stores a new `SessionEntry` (defaulting `auth_status` to `AUTHENTICATED`;
the create handler overrides it to `AUTHENTICATING`).
- **`get()`** refreshes `last_accessed` via `touch()`. It returns `None` (and
deletes the entry) if an authenticated session has been idle longer than the TTL.
Sessions in `AUTHENTICATING` / `PENDING_INPUT` are exempt from TTL expiry.
- **`cleanup_expired()`** runs every 60s: authenticated sessions expire at `ttl`,
in-flight-auth sessions expire at `AUTH_INPUT_TIMEOUT`. Removing a session
cancels its `InputBridge` and deletes its cache directory (pruning empty parent
dirs up to, but not including, the cache root).
- **`cancel_all_bridges()`** is called on server shutdown to unblock any waiting
- **`create()`** evicts the least-recently-accessed remote session when at capacity,
then stores a new `SessionEntry`. It defaults `auth_status` to `AUTHENTICATED`, and
the `create` handler overrides it to `AUTHENTICATING`.
- **`get()`** refreshes `last_accessed` through `touch()`. It returns `None` (and
deletes the entry) if an authenticated remote session has been idle longer than the
TTL. Remote sessions in `AUTHENTICATING` / `PENDING_INPUT` are exempt from TTL expiry.
- **`cleanup_expired()`** runs every 60s: authenticated remote sessions expire at
`ttl`, in-flight-auth remote sessions expire at `AUTH_INPUT_TIMEOUT`. The removal of
a remote session cancels its `InputBridge` and deletes its cache directory (pruning
empty parent dirs up to, but not including, the cache root).
- The server calls **`cancel_all_bridges()`** on shutdown, to unblock any waiting
auth threads.
`SessionEntry` fields:
@@ -352,22 +354,22 @@ Source: `unshackle/core/api/session_store.py`. A singleton obtained via
### Per-session cache namespacing
The create handler builds a `Cacher` namespaced as:
The `create` handler builds a `Cacher` namespaced as:
```text
_sessions/<pbkdf2_hmac(sha256, X-Secret-Key, "unshackle-session-ns", 100000)[:12]>/<session_id>/<service>
```
Forwarded `cache` files are written into that directory before authentication.
The handler writes forwarded `cache` files into that directory before authentication.
On `DELETE`, the handler harvests updated cache files (compressing each with zlib
and base64-encoding, **excluding** `titles_*` files) and returns them under a
`cache` key so the client can persist refreshed tokens:
`cache` field, so the client can keep refreshed tokens:
```json
{ "status": "ok", "cache": { "tokens": "...base64(zlib(bytes))..." } }
```
### Session info response
### Remote session info response
`GET /api/session/{id}`:
@@ -384,12 +386,12 @@ and base64-encoding, **excluding** `titles_*` files) and returns them under a
!!! warning "`expires_in` is the configured TTL, not time remaining"
The `expires_in` value reports the store's configured `session_ttl`, not the
seconds left before this specific session expires.
seconds left before this specific remote session expires.
### Reference client: `RemoteService`
Source: `unshackle/core/remote_service.py`. This is the canonical consumer of the
session API and a good template for any client.
remote session API and a good template for any client.
- **`RemoteClient.request`** sets `X-Secret-Key` and `User-Agent: unshackle/<version>`,
uses a 120s timeout for `POST` and 30s for `GET`/`DELETE`, and treats any
@@ -398,10 +400,10 @@ session API and a good template for any client.
- **Retries.** The download-side HTTP session mounts an adapter with
`Retry(total=5, backoff_factor=0.2, status_forcelist=[429, 500, 502, 503, 504])`.
- **Flow.** `authenticate()` → `create` (+ poll `prompt` every 2s up to a 600s
deadline, answering `pending_input` prompts; with stdin closed the client logs an
deadline, answering `pending_input` prompts. With stdin closed the client logs an
error and raises `SystemExit(1)` instead of posting an empty answer) → `get_titles()` → `get_tracks()`
(merging returned `session_headers`/`session_cookies`, re-parsing `manifests`) →
license via proxy or `server_cdm` → `close()` (`DELETE`, saving any returned
get the licence in proxy mode or `server_cdm` mode → `close()` (`DELETE`, saving any returned
`cache`).
- **Server resolution.** `resolve_server()` reads `config.remote_services.<name>`
into `{url, api_key, services, server_cdm}`, injecting the `server_cdm` flag into
@@ -412,20 +414,20 @@ session API and a good template for any client.
## Troubleshooting
!!! example "\"Could not connect to remote server ... Is it running?\""
The client couldn't reach the URL. Confirm `unshackle serve` is running on the
server, the `url`/port in `remote_services` are correct (default port `8786`),
The client could not reach the URL. Make sure that `unshackle serve` operates on
the server, the `url`/port in `remote_services` are correct (default port `8786`),
and any firewall or reverse proxy allows the connection.
!!! example "`403 FORBIDDEN` mid-download"
The request came from a different IP than the one that created the session.
Sessions are IP-bound. Don't move between networks (or NAT egress IPs) during a
The request came from a different IP than the one that made the remote session.
Remote sessions are IP-bound. Do not move between networks (or NAT egress IPs) during a
remote download.
!!! example "Auth times out or the prompt never resolves"
Interactive auth allows up to 600s. If you miss that window the session is
marked `failed` and cleaned up; just re-run the download. If the server never
prompts you, check that credentials/cookies forwarded from the client are valid
for the service.
Interactive auth allows up to 600s. If you miss that window, the server marks the
remote session `failed` and cleans it up. Operate the download again. If the server
never prompts you, make sure that the credentials and cookies the client forwarded
are valid for the service.
!!! example "\"Multiple remote services configured. Use --server ...\""
You have more than one entry under `remote_services`. Pass `--server <name>`