mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-11-14 13:05:09 +00:00
docs: restructure documentation into MkDocs Material site
This commit is contained in:
imSp4rky
@@ -0,0 +1,634 @@
|
||||
# Creating a Service
|
||||
|
||||
A **service** is a plugin that teaches unshackle how to talk to one streaming
|
||||
provider: how to log in, how to look up a title, what video/audio/subtitle
|
||||
tracks exist, and how to license any DRM. Everything else (track selection,
|
||||
downloading, decryption, muxing, naming) is handled by the unshackle core.
|
||||
|
||||
!!! info "Who this page is for"
|
||||
This is a **developer guide**. If you only want to *use* services, see
|
||||
[downloading](../guide/downloading.md) and
|
||||
[configuration](../reference/configuration.md). Nothing here is required to run
|
||||
the CLI. Service code is deliberately kept separate from the core so it can
|
||||
live in its own private repository.
|
||||
|
||||
Unshackle ships one reference service, **EXAMPLE**, at
|
||||
`unshackle/services/EXAMPLE/`. It is a deliberately exhaustive, non-runnable
|
||||
showcase of every framework feature in a single file. Read it alongside this
|
||||
page; most snippets below are drawn from it.
|
||||
|
||||
---
|
||||
|
||||
## How a service is discovered
|
||||
|
||||
At startup unshackle scans every path in the `directories.services` config key
|
||||
(a **list**). Each entry is either a local directory of services or a remote
|
||||
repo spec (a git URL or `owner/repo` shorthand, which is cloned automatically).
|
||||
The default is the bundled `unshackle/services` directory.
|
||||
|
||||
Within a services directory, **every subfolder that contains an `__init__.py`
|
||||
is a service**, and the folder name is the service **tag**:
|
||||
|
||||
```text
|
||||
services/
|
||||
└── EXAMPLE/
|
||||
├── __init__.py # required — defines the service class
|
||||
└── config.yaml # optional — per-service configuration
|
||||
```
|
||||
|
||||
Two rules the loader enforces:
|
||||
|
||||
1. **The class name must exactly match the folder/tag name.** Folder `AMZN` must
|
||||
define `class AMZN`. A mismatch raises a `RuntimeError`.
|
||||
2. **List order is priority.** The first services path to define a given tag
|
||||
wins; later duplicates are shadowed. Put your own local overrides *last* to
|
||||
use them as fallbacks, or *first* to override a repo.
|
||||
|
||||
Once installed, a service is invoked as a subcommand of `dl`:
|
||||
|
||||
```bash
|
||||
unshackle dl EXAMPLE 20914
|
||||
unshackle dl EX 20914 # via an ALIAS
|
||||
```
|
||||
|
||||
!!! tip "Config lives next to the code"
|
||||
A service's own settings go in `config.yaml` **inside the service folder**
|
||||
(`directories.services/<TAG>/config.yaml`). This is separate from the global
|
||||
`unshackle.yaml`. Access it in code as `self.config[...]`. Never hardcode
|
||||
URLs, user agents, or certificates; put them here.
|
||||
|
||||
!!! warning "`config.yaml` is shared: keep secrets out of it"
|
||||
Treat `config.yaml` as a checked-in file of **defaults**: it belongs in the
|
||||
service repo and is shared by everyone who runs the service. Per-user secrets
|
||||
(API keys, device IDs, account tokens) must **not** live here. Those go in the
|
||||
user's own `unshackle.yaml` under `services.<TAG>`, which is **merged into
|
||||
`self.config` at runtime**. That checked-in-defaults / per-user-overrides
|
||||
split is exactly why every URL lives in `config.yaml` with `{}`/`{name}`
|
||||
placeholders while the secrets that fill them stay out of it. Both halves
|
||||
read back through the same `self.config[...]`.
|
||||
|
||||
---
|
||||
|
||||
## The Service base class
|
||||
|
||||
Every service subclasses `unshackle.core.service.Service`. The minimum is a
|
||||
Click `cli` entry point plus three abstract methods.
|
||||
|
||||
### Class variables
|
||||
|
||||
Declared at class level to configure framework behaviour:
|
||||
|
||||
| Variable | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `ALIASES` | `tuple[str, ...]` | Extra tags that resolve to this service (e.g. `("EX", "DOMAIN")`). Default `()`. |
|
||||
| `GEOFENCE` | `tuple[str, ...]` | IP region codes the service requires. Empty = no geofence. The **first** entry is treated as the main region for auto-proxy. |
|
||||
| `VAULT_TAG` | `Optional[str]` | Overrides the key-vault namespace so sibling services can share one vault. Default `None` (use the service's own tag). |
|
||||
| `AUTH_METHODS` | `Optional[tuple[str, ...]]` | Auth methods accepted (`"cookies"` / `"credentials"`). When `None`, the REST `/services` endpoint infers them from `authenticate()`. |
|
||||
| `NO_SUBTITLES` | `bool` | Set `True` on a service with no subtitle tracks to skip subtitle handling entirely. |
|
||||
|
||||
!!! note "`NO_SUBTITLES` is a convention, not a base-class attribute"
|
||||
`NO_SUBTITLES` is checked by `dl.py` via `hasattr` and is deliberately **not**
|
||||
declared on the `Service` base class, so it only exists if your service
|
||||
defines it. Setting `NO_SUBTITLES = True` is what makes the pipeline skip
|
||||
subtitle work for services that never carry subtitles; leave it off otherwise.
|
||||
|
||||
`GEOFENCE` drives an automatic proxy: on startup, if you did not pass an
|
||||
explicit `--proxy`, the base class does a live IP check and, if your region is
|
||||
blocked, fetches a proxy to `GEOFENCE[0]` from your configured proxy providers.
|
||||
|
||||
```python
|
||||
class EXAMPLE(Service):
|
||||
ALIASES = ("EX", "DOMAIN")
|
||||
GEOFENCE = ("US", "UK")
|
||||
VAULT_TAG = "DIFFERENT_NAME"
|
||||
```
|
||||
|
||||
### What `Service.__init__` wires up
|
||||
|
||||
When you call `super().__init__(ctx)`, the base class sets up the attributes you
|
||||
will use throughout the service:
|
||||
|
||||
| Attribute | What it is |
|
||||
|---|---|
|
||||
| `self.config` | This service's `config.yaml` contents (a dict), or `None` if absent. |
|
||||
| `self.log` | A `logging.Logger` named after your class. |
|
||||
| `self.session` | A prepared `requests.Session` with config headers and a retry adapter (5 retries, backoff 0.2, retries on 429/500/502/503/504). |
|
||||
| `self.cache` | A `Cacher` for arbitrary key/value data (tokens, etc.). |
|
||||
| `self.title_cache` | A region/account-aware `TitleCacher` used by `get_titles_cached`. |
|
||||
| `self.cache_dir` | `config.directories.cache / <ClassName>`. |
|
||||
| `self.track_request` | A `TrackRequest` built from the CLI `--vcodec` / `--range` / `--best-available` flags. |
|
||||
| `self.credential` | `None` until `authenticate()` runs. |
|
||||
| `self.current_region` | The two-letter region resolved from proxy/IP. |
|
||||
|
||||
`self.track_request` is a small dataclass you can read *and rewrite* before
|
||||
fetching tracks:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class TrackRequest:
|
||||
codecs: list[Video.Codec] = [] # empty == accept any codec
|
||||
ranges: list[Video.Range] = [Video.Range.SDR]
|
||||
best_available: bool = False
|
||||
```
|
||||
|
||||
!!! note "Read the request, don't second-guess the user"
|
||||
Services may narrow `track_request` for hard technical constraints (e.g.
|
||||
*"HDR on this service is only delivered as HEVC"*), but must **not** filter
|
||||
the tracks they return by resolution, bitrate, or language. All selection is
|
||||
done by the core after `get_tracks` returns.
|
||||
|
||||
!!! tip "Override `get_session()` to defeat TLS-fingerprint bot detection"
|
||||
The base `get_session()` returns a plain `requests.Session` (config headers +
|
||||
retry adapter) and **deliberately does not** do TLS impersonation. When a
|
||||
provider blocks you on TLS fingerprint alone, override it to return
|
||||
`session("Chrome131")` (any `rnet.Impersonate` preset). `RnetSession` is a
|
||||
drop-in `requests.Session` replacement (same `.get`/`.post`/cookie-jar
|
||||
interface), so nothing else in the service changes. If instead you need
|
||||
specific SSL *cipher* behaviour, mount
|
||||
`unshackle.core.utils.sslciphers.SSLCiphers` onto a plain `requests.Session`.
|
||||
|
||||
---
|
||||
|
||||
## The methods you implement
|
||||
|
||||
Methods run in the order shown. Only the three abstract ones are mandatory.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[cli] --> B[__init__] --> C[authenticate] --> D[get_titles] --> E[get_tracks] --> F[get_chapters]
|
||||
```
|
||||
|
||||
### `cli`: the Click entry point
|
||||
|
||||
A static method decorated as a Click command. It defines the CLI arguments and
|
||||
returns an **instance** of your service. Keep the command `name=` matching the
|
||||
tag.
|
||||
|
||||
!!! note "`name=` is cosmetic: the directory name is what resolves the command"
|
||||
`unshackle dl EXAMPLE` is dispatched by the **folder/class name**, not by the
|
||||
`name=` in `@click.command`. A mismatched `name=` still loads and runs; it
|
||||
just produces confusing `--help`/usage output that names the wrong command.
|
||||
That confusing output is the reason to keep it in sync, not any dispatch
|
||||
requirement.
|
||||
|
||||
```python
|
||||
@staticmethod
|
||||
@click.command(name="EXAMPLE", short_help="https://domain.com", help=__doc__)
|
||||
@click.argument("title", type=str)
|
||||
@click.option("-m", "--movie", is_flag=True, default=False, help="Treat the title as a movie.")
|
||||
@click.option("-d", "--device", type=click.Choice(["android_tv", "web", "ios"]), default="android_tv")
|
||||
@click.pass_context
|
||||
def cli(ctx: click.Context, **kwargs: Any) -> EXAMPLE:
|
||||
return EXAMPLE(ctx, **kwargs)
|
||||
|
||||
def __init__(self, ctx: click.Context, title: str, movie: bool, device: str):
|
||||
self.title = title
|
||||
self.movie = movie
|
||||
self.device = device
|
||||
super().__init__(ctx) # wires up config, log, session, caches, track_request...
|
||||
self.cdm = ctx.obj.cdm
|
||||
```
|
||||
|
||||
Your docstring becomes the CLI `--help` text (`help=__doc__`), so document the
|
||||
accepted URL/ID formats and options there.
|
||||
|
||||
!!! tip "CDM-aware behaviour"
|
||||
The resolved CDM is on `ctx.obj.cdm` (may be `None` for DRM-free runs). Use
|
||||
`is_widevine_cdm()` / `is_playready_cdm()` from `unshackle.core.cdm.detect`
|
||||
to classify it. These correctly handle local *and* remote CDMs, so never
|
||||
hand-roll an `isinstance` check. Services often pick a device profile or
|
||||
manifest endpoint based on which DRM the CDM speaks.
|
||||
|
||||
### `authenticate`: optional login
|
||||
|
||||
Override to log in with cookies and/or credentials. **Call `super().authenticate()`
|
||||
first**: the base implementation loads the cookie jar into `self.session.cookies`
|
||||
and stores `self.credential`. Do all token fetching here; it runs before
|
||||
`get_titles`.
|
||||
|
||||
```python
|
||||
def authenticate(self, cookies=None, credential=None) -> None:
|
||||
super().authenticate(cookies, credential)
|
||||
self.session.headers.update({"user-agent": self.config["client"][self.device]["user_agent"]})
|
||||
|
||||
cache = self.cache.get(f"tokens_{self.device}_{self.profile}")
|
||||
if cache and cache.data.get("expires_in", 0) > int(datetime.now().timestamp()):
|
||||
self.log.info(" + Using cached tokens")
|
||||
else:
|
||||
token = self.session.post(self.config["endpoints"]["login"], data=body).json()
|
||||
cache.set(data=token, expiration=token.get("expires_in"))
|
||||
self.token = cache.data["token"]
|
||||
```
|
||||
|
||||
Cookies come from files under `directories.cookies`; credentials from the
|
||||
`credentials` map in `unshackle.yaml`. For interactive prompts (OTP, captcha)
|
||||
use `self.request_input(prompt)`, never a bare `input()`. Under `serve` mode
|
||||
there is no local terminal, so a bare `input()` would hang the server waiting on
|
||||
stdin that never arrives; `request_input` instead relays the prompt to the
|
||||
remote client through the attached `InputBridge`. Locally it routes through the
|
||||
shared Rich console (`console.input`) so the prompt renders correctly alongside
|
||||
progress and log output.
|
||||
|
||||
!!! warning "Key token caches by whatever varies per session"
|
||||
The cache key above is `tokens_{device}_{profile}` on purpose. A token cache
|
||||
must include every dimension that changes the token (device, profile, or
|
||||
`credential.sha1`), or two profiles/devices will share one cache entry and
|
||||
stomp each other's tokens. This isn't obvious from the caching API, which
|
||||
happily lets you use a single flat key.
|
||||
|
||||
!!! warning "Handle the credential carefully"
|
||||
Use the `Credential` only inside `authenticate()` to obtain tokens; don't
|
||||
stash it or the raw cookie jar elsewhere. The base class already caches
|
||||
identity for you.
|
||||
|
||||
### `get_titles`: required
|
||||
|
||||
Return a titles collection for the given ID. The return type depends on content:
|
||||
|
||||
| Return | Contains | Use for |
|
||||
|---|---|---|
|
||||
| `Movies` | `Movie` objects | Films |
|
||||
| `Series` | `Episode` objects | Shows (flatten seasons into episodes) |
|
||||
| `Album` | `Song` objects | Music |
|
||||
|
||||
Every title carries a `language` (its **original recorded language**) and an
|
||||
arbitrary `data` dict you can use to stash metadata for later methods. At least
|
||||
one title must be returned, or the ID is treated as invalid.
|
||||
|
||||
```python
|
||||
def get_titles(self) -> Titles_T:
|
||||
match = re.match(self.TITLE_RE, self.title)
|
||||
if not match:
|
||||
raise ValueError("Could not parse a title ID - is the URL/ID correct?")
|
||||
metadata = self.session.get(
|
||||
self.config["endpoints"]["metadata"].format(title_id=match.group("title_id")),
|
||||
params={"token": self.token},
|
||||
).json()
|
||||
original_lang = Language.find(metadata["languages"][0])
|
||||
|
||||
return Movies([
|
||||
Movie(
|
||||
id_=metadata["id"],
|
||||
service=self.__class__,
|
||||
name=metadata["title"],
|
||||
year=metadata.get("releaseYear") or None,
|
||||
language=original_lang, # ORIGINAL audio language, not the -l preference
|
||||
data=metadata, # read later as title.data
|
||||
)
|
||||
])
|
||||
```
|
||||
|
||||
Common constructor fields:
|
||||
|
||||
=== "Movie"
|
||||
|
||||
```python
|
||||
Movie(id_, service, name, year=None, language=None, data=None, description=None)
|
||||
```
|
||||
|
||||
=== "Episode"
|
||||
|
||||
```python
|
||||
Episode(id_, service, title, season, number, name=None, year=None,
|
||||
language=None, data=None, description=None, air_date=None)
|
||||
```
|
||||
|
||||
Pass `air_date` for daily/sports content to name by date instead of
|
||||
`SxxExx`. Return a `Series([...])` of episodes.
|
||||
|
||||
=== "Song"
|
||||
|
||||
```python
|
||||
Song(id_, service, name, artist, album, track, disc, year,
|
||||
language=None, data=None)
|
||||
```
|
||||
|
||||
Return an `Album([...])` of songs.
|
||||
|
||||
!!! note "The ID must be unique and stable"
|
||||
`id_` must be truthy and, if it has a length, at least 4 characters, to
|
||||
avoid clashes. Use the service's own stable IDs.
|
||||
|
||||
!!! tip "Title caching"
|
||||
You can call `self.get_titles_cached()` instead of `get_titles()` from the
|
||||
framework path to reuse cached results and gracefully fall back to stale data
|
||||
when the API errors. It also applies the per-service `title_map` config.
|
||||
|
||||
### `get_tracks`: required
|
||||
|
||||
Given one title, return a `Tracks` object holding `Video`, `Audio`, and
|
||||
`Subtitle` tracks. In almost all cases you build these by parsing a manifest,
|
||||
not by hand.
|
||||
|
||||
The manifest parsers live in `unshackle.core.manifests`:
|
||||
|
||||
```python
|
||||
from unshackle.core.manifests import DASH, HLS, ISM
|
||||
```
|
||||
|
||||
Each exposes `from_url(url, session=...)` (or `from_text(text, url)`) and then
|
||||
`.to_tracks(language=...)`:
|
||||
|
||||
=== "DASH (.mpd)"
|
||||
|
||||
```python
|
||||
def get_tracks(self, title: Title_T) -> Tracks:
|
||||
return DASH.from_url(manifest_url, session=self.session) \
|
||||
.to_tracks(language=title.language)
|
||||
```
|
||||
|
||||
=== "HLS (.m3u8)"
|
||||
|
||||
```python
|
||||
def get_tracks(self, title: Title_T) -> Tracks:
|
||||
return HLS.from_url(manifest_url, session=self.session) \
|
||||
.to_tracks(language=title.language)
|
||||
```
|
||||
|
||||
The URL must be a **variant (master) playlist**, not a media playlist.
|
||||
|
||||
=== "ISM (Smooth Streaming)"
|
||||
|
||||
```python
|
||||
def get_tracks(self, title: Title_T) -> Tracks:
|
||||
return ISM.from_url(manifest_url, session=self.session) \
|
||||
.to_tracks(language=title.language)
|
||||
```
|
||||
|
||||
!!! warning "Always pass `language=`"
|
||||
Pass the title's original language to `to_tracks(language=...)`. For DASH and
|
||||
HLS this is **required as a fallback**. If a track's language can't be
|
||||
derived and no valid fallback was given, `to_tracks` raises `ValueError`.
|
||||
It's also what lets the parser flag `is_original_lang` on each track (via
|
||||
`is_close_match`), driving `-l best/all` selection and the filename language
|
||||
token.
|
||||
|
||||
Some services deliver a **separate manifest per codec/range**. The base class
|
||||
provides `_get_tracks_for_variants(title, fetch_fn)` to fan out over every
|
||||
codec×range in the `TrackRequest`, including `HYBRID` (fetch HDR10 + DV and
|
||||
merge) and `--best-available` skip-on-error handling:
|
||||
|
||||
```python
|
||||
def get_tracks(self, title: Title_T) -> Tracks:
|
||||
def _fetch_variant(title, codec, range_) -> Tracks:
|
||||
vcodec = "H265" if codec == Video.Codec.HEVC else "H264"
|
||||
return self._fetch_dash_manifest(title, vcodec=vcodec, range_=range_)
|
||||
return self._get_tracks_for_variants(title, _fetch_variant)
|
||||
```
|
||||
|
||||
After parsing, you may **correct** track metadata the manifest gets wrong:
|
||||
stamp the real `video.range`, fix odd audio channel counts, mark descriptive
|
||||
audio, add hand-built subtitles or an `Attachment` (e.g. cover art). Do *not*
|
||||
filter for resolution/bitrate. See `_fetch_dash_manifest` in the EXAMPLE service
|
||||
for a thorough demonstration.
|
||||
|
||||
!!! note "Flip HDR10 → HDR10+ by hand when you know the platform embeds it"
|
||||
HDR10+ is a **bitstream (SEI)** feature. The HLS parser intentionally does
|
||||
not sniff it from the stream, so a manifest that embeds HDR10+ SEI but labels
|
||||
the variant plain HDR10 will parse as HDR10. The label is the only signal
|
||||
the parser has, and it's wrong. Only the service author knows the platform
|
||||
embeds HDR10+, so it's on you to correct `video.range` to HDR10+ here for
|
||||
those services.
|
||||
|
||||
!!! warning "`get_tracks` is your only chance to capture license inputs"
|
||||
The license callbacks (`get_widevine_license`, etc.) fire **much later**, in
|
||||
the download/decrypt step, long after `get_tracks` has returned, and they
|
||||
receive **only** `challenge`, `title`, and `track`. Nothing else from the
|
||||
manifest response reaches them. So anything a license call needs (the license
|
||||
URL, `dt-custom-data`, a session token) has to be stashed **now**, during
|
||||
`get_tracks`, onto `self.license_data` or `title.data`. This is a
|
||||
lifecycle-timing consequence you can't infer from the callback signatures.
|
||||
|
||||
!!! tip "Provide the KID if you cheaply can"
|
||||
If you can obtain a track's Key ID (32-char hex) without downloading stream
|
||||
data, set `track.kid`; it speeds up the decryption/key-lookup path. And be
|
||||
sure encrypted tracks carry the correct `drm` so the core licenses them.
|
||||
|
||||
### `get_chapters`: required
|
||||
|
||||
Return a `Chapters` object (0 or more `Chapter`s). You don't need to number or
|
||||
sort them; `Chapters` does that automatically and inserts a `00:00:00.000`
|
||||
marker if missing. A `Chapter` timestamp accepts `"HH:MM:SS[.mmm]"`, an int in
|
||||
milliseconds, or a float in seconds.
|
||||
|
||||
```python
|
||||
def get_chapters(self, title: Title_T) -> Chapters:
|
||||
chapters = Chapters()
|
||||
seen: set[int] = set()
|
||||
for ch in title.data.get("chapters", []):
|
||||
chapter = Chapter(timestamp=ch["start"], name=ch.get("name"))
|
||||
if chapter.timestamp == 0 or chapter.timestamp in seen:
|
||||
continue # see the warning below
|
||||
seen.add(chapter.timestamp)
|
||||
chapters.add(chapter)
|
||||
return chapters
|
||||
```
|
||||
|
||||
!!! warning "Skip markers at 0 and de-duplicate timestamps before `add()`"
|
||||
`Chapters.add()` auto-injects a `Chapter(0)` at `00:00:00.000` the first time
|
||||
you add *any* chapter, **and** it raises `ValueError` if a chapter already
|
||||
exists at an exact timestamp. So an API marker reported at `0` (an intro or
|
||||
recap that starts at the very beginning) collides with that auto-inserted
|
||||
opening chapter, and two markers sharing a timestamp collide with each other.
|
||||
Guard both cases: drop any marker at timestamp `0` and de-duplicate timestamps
|
||||
before adding, as above.
|
||||
|
||||
!!! note "An unnamed chapter is the idiom for closing a named range"
|
||||
A `Chapter` with no name is perfectly valid. Inserting an unnamed marker is
|
||||
the standard way to *close out* a named range: e.g. add `Chapter(name="Intro")`
|
||||
at the start and an unnamed `Chapter` where the intro ends, so the "Intro"
|
||||
label applies only to that span.
|
||||
|
||||
!!! warning "Don't invent chapter names"
|
||||
Never name chapters `"Chapter 1"` yourself. Leave unnamed markers unnamed;
|
||||
users who want generic names set `chapter_fallback_name` (e.g.
|
||||
`"Chapter {i:02}"`) in their config.
|
||||
|
||||
---
|
||||
|
||||
## Handling DRM
|
||||
|
||||
The core drives licensing. Your job is to answer the license challenges the CDM
|
||||
produces. Mark encrypted tracks with the right DRM in `get_tracks` (the manifest
|
||||
parsers do this automatically for PSSH/`EXT-X-KEY` found in the manifest), then
|
||||
implement the license callbacks you need. Each receives the challenge plus the
|
||||
current `title` and `track`:
|
||||
|
||||
| Method | For | Returns |
|
||||
|---|---|---|
|
||||
| `get_widevine_service_certificate` | Widevine privacy mode | The service certificate (bytes or base64 str), or `None`. |
|
||||
| `get_widevine_license` | Widevine | The raw license response (bytes or base64 str), **unmodified**. |
|
||||
| `get_playready_license` | PlayReady | The license response. Defaults to delegating to `get_widevine_license`. |
|
||||
| `get_clearkey_license` | DASH `org.w3.clearkey` | The JWK Set (dict/JSON/bytes), or `None` to let the framework POST the challenge to the manifest's Laurl. |
|
||||
|
||||
```python
|
||||
def get_widevine_license(self, *, challenge: bytes, title, track):
|
||||
response = self.session.post(
|
||||
self.license_data["url"],
|
||||
data=challenge, # POST the challenge as-is
|
||||
headers={"dt-custom-data": self.license_data["data"]},
|
||||
)
|
||||
response.raise_for_status()
|
||||
try:
|
||||
return response.json()["license"] # some services wrap it in JSON
|
||||
except (ValueError, KeyError):
|
||||
return response.content # others return raw bytes
|
||||
```
|
||||
|
||||
!!! warning "Return the license untouched"
|
||||
Do **not** base64-encode or decode the challenge or response. Pass them
|
||||
through verbatim. A malformed license request can get your CDM/device
|
||||
flagged, banned, or downgraded.
|
||||
|
||||
!!! tip "`ClearKeyCENC` (`org.w3.clearkey`) has three escalating integration levels"
|
||||
Pick the lowest level that works for the platform:
|
||||
|
||||
1. **The manifest carries a `<Laurl>`**: implement nothing. The framework
|
||||
POSTs the challenge to that URL itself.
|
||||
2. **A custom endpoint or extra headers are needed**: override
|
||||
`get_clearkey_license` to do the POST yourself.
|
||||
3. **Keys arrive obfuscated or through a bespoke channel**: fetch and unwrap
|
||||
the key in `get_tracks` and pre-populate
|
||||
`drm.content_keys[kid] = key_hex` on the track's `ClearKeyCENC`. The
|
||||
non-obvious payoff: when every KID is already keyed, the framework skips
|
||||
the license round-trip entirely.
|
||||
|
||||
Things you don't handle:
|
||||
|
||||
- **HLS AES-128 (`ClearKey`)** and **DRM-free** content have no license
|
||||
callback. The key comes from the manifest and is applied directly.
|
||||
- **Key vaults** cache `KID:KEY` pairs across runs, so repeat downloads skip the
|
||||
license round-trip entirely.
|
||||
- **Decryption tooling** (shaka-packager or mp4decrypt) and the choice of local
|
||||
vs remote CDM are user configuration, not service concerns.
|
||||
|
||||
See the [DRM & CDM reference](../guide/drm-and-cdm.md) for the full picture.
|
||||
|
||||
---
|
||||
|
||||
## Optional event hooks
|
||||
|
||||
Override any of these to react to pipeline stages (all no-ops by default):
|
||||
`on_segment_downloaded`, `on_track_downloaded`, `on_track_decrypted`,
|
||||
`on_track_repacked`, `on_track_multiplex`. Also override `search()` to yield
|
||||
`SearchResult` objects for `unshackle search`.
|
||||
|
||||
```python
|
||||
def search(self) -> Generator[SearchResult, None, None]:
|
||||
results = self.session.get(self.config["endpoints"]["search"],
|
||||
params={"q": self.title}).json()
|
||||
for r in results["entries"]:
|
||||
yield SearchResult(id_=r["id"], title=r["title"],
|
||||
description=r.get("description"), url=r.get("url"))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Minimal service skeleton
|
||||
|
||||
A complete, minimal single-manifest service. Save as
|
||||
`services/MYSVC/__init__.py` (class name `MYSVC`), with an optional
|
||||
`services/MYSVC/config.yaml` beside it.
|
||||
|
||||
```python title="services/MYSVC/__init__.py"
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any, Optional
|
||||
|
||||
import click
|
||||
|
||||
from unshackle.core.manifests import DASH
|
||||
from unshackle.core.service import Service
|
||||
from unshackle.core.titles import Movie, Movies, Title_T, Titles_T
|
||||
from unshackle.core.tracks import Chapters, Tracks
|
||||
|
||||
|
||||
class MYSVC(Service):
|
||||
"""
|
||||
MyService — https://myservice.com
|
||||
|
||||
\b
|
||||
Usage: unshackle dl MYSVC <title-id-or-url>
|
||||
"""
|
||||
|
||||
ALIASES = ("MY",)
|
||||
GEOFENCE = ("US",)
|
||||
TITLE_RE = r"^(?:https?://myservice\.com/watch/)?(?P<id>[\w-]+)"
|
||||
|
||||
@staticmethod
|
||||
@click.command(name="MYSVC", short_help="https://myservice.com", help=__doc__)
|
||||
@click.argument("title", type=str)
|
||||
@click.pass_context
|
||||
def cli(ctx: click.Context, **kwargs: Any) -> "MYSVC":
|
||||
return MYSVC(ctx, **kwargs)
|
||||
|
||||
def __init__(self, ctx: click.Context, title: str):
|
||||
self.title = title
|
||||
super().__init__(ctx)
|
||||
|
||||
def authenticate(self, cookies=None, credential=None) -> None:
|
||||
super().authenticate(cookies, credential)
|
||||
# obtain any tokens here, e.g. self.token = ...
|
||||
|
||||
def get_titles(self) -> Titles_T:
|
||||
match = re.match(self.TITLE_RE, self.title)
|
||||
if not match:
|
||||
raise ValueError("Could not parse a title ID.")
|
||||
meta = self.session.get(
|
||||
self.config["endpoints"]["metadata"].format(id=match.group("id"))
|
||||
).json()
|
||||
return Movies([
|
||||
Movie(
|
||||
id_=meta["id"],
|
||||
service=self.__class__,
|
||||
name=meta["title"],
|
||||
year=meta.get("year") or None,
|
||||
language=meta.get("originalLanguage"),
|
||||
data=meta,
|
||||
)
|
||||
])
|
||||
|
||||
def get_tracks(self, title: Title_T) -> Tracks:
|
||||
playback = self.session.get(
|
||||
self.config["endpoints"]["playback"].format(id=title.id)
|
||||
).json()
|
||||
return DASH.from_url(
|
||||
url=playback["manifest_url"], session=self.session
|
||||
).to_tracks(language=title.language)
|
||||
|
||||
def get_chapters(self, title: Title_T) -> Chapters:
|
||||
return Chapters()
|
||||
|
||||
def get_widevine_license(self, *, challenge: bytes, title, track):
|
||||
res = self.session.post(self.config["endpoints"]["license"], data=challenge)
|
||||
res.raise_for_status()
|
||||
return res.content
|
||||
```
|
||||
|
||||
```yaml title="services/MYSVC/config.yaml"
|
||||
endpoints:
|
||||
metadata: https://api.myservice.com/v1/metadata/{id}
|
||||
playback: https://api.myservice.com/v1/playback/{id}
|
||||
license: https://api.myservice.com/v1/license/widevine
|
||||
```
|
||||
|
||||
Test it with:
|
||||
|
||||
```bash
|
||||
unshackle dl MYSVC https://myservice.com/watch/abc123
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Folder name, class name, and Click `name=` all match the tag exactly.
|
||||
- [ ] `cli` is a `@staticmethod` returning an instance; `__init__` calls `super().__init__(ctx)`.
|
||||
- [ ] `get_titles`, `get_tracks`, `get_chapters` implemented; titles carry the original `language`.
|
||||
- [ ] Tracks come from a manifest parser with `to_tracks(language=title.language)`; no filtering by quality/language.
|
||||
- [ ] Encrypted tracks are marked with DRM; the license callbacks you need return responses **unmodified**.
|
||||
- [ ] URLs, user agents, and certificates live in `config.yaml`, read via `self.config[...]`.
|
||||
|
||||
Read `unshackle/services/EXAMPLE/` end to end; it annotates every feature
|
||||
touched above in one place.
|
||||
@@ -0,0 +1,311 @@
|
||||
# REST API Authentication
|
||||
|
||||
The unshackle REST API protects every endpoint (with one exception) behind a single API key. Clients present that key in a request header, and the server checks it against the keys defined in your configuration. This page explains how that check works, how to configure keys, how clients send them, and the extra controls that build on top of the key: per-key service allowlists, per-key CDM device access, and session IP binding.
|
||||
|
||||
The API is started with the [`unshackle serve`](../../guide/cli-reference.md) command. Authentication is on by default: if you start the server without an API key configured, it refuses to launch.
|
||||
|
||||
!!! note "Which server am I running?"
|
||||
`unshackle serve` can run in two shapes, and each uses a slightly different piece of code to check your key:
|
||||
|
||||
- The **integrated server** (the default) exposes the REST API *and* the pywidevine / pyplayready CDM HTTP endpoints.
|
||||
- The **API-only server** (`--api-only`, also implied by `--remote-only`) exposes only the REST API.
|
||||
|
||||
The header, the config, and the behaviour you see as a client are the same in both. The differences are internal and are called out in the developer notes below.
|
||||
|
||||
## The short version
|
||||
|
||||
1. Add an `api_secret` to the `serve` section of your `unshackle.yaml`.
|
||||
2. Start the server with `unshackle serve`.
|
||||
3. Send that secret in the `X-Secret-Key` header on every request except `GET /api/health`.
|
||||
|
||||
```bash title="A minimal authenticated request"
|
||||
curl -H "X-Secret-Key: your-secret-key-here" \
|
||||
http://127.0.0.1:8786/api/services
|
||||
```
|
||||
|
||||
## Configuring authentication
|
||||
|
||||
All authentication settings live under the `serve` section of your unshackle config file (`unshackle.yaml`).
|
||||
|
||||
### A single API key
|
||||
|
||||
The simplest setup defines one shared secret with `api_secret`:
|
||||
|
||||
```yaml title="unshackle.yaml"
|
||||
serve:
|
||||
api_secret: "a-long-random-secret-string"
|
||||
```
|
||||
|
||||
This value becomes the one and only key that the server accepts. Requests that present it in the `X-Secret-Key` header are allowed; everything else is rejected.
|
||||
|
||||
!!! warning "The server will not start without a key"
|
||||
If you start `unshackle serve` with no `api_secret` configured, and you have *not* passed `--no-key`, the command fails immediately with:
|
||||
|
||||
```
|
||||
API secret key is not configured. Please add 'api_secret' to the 'serve' section in your config.
|
||||
```
|
||||
|
||||
Choose a long, unpredictable value. The key is the only thing standing between the internet and your CDM and download endpoints.
|
||||
|
||||
### Multiple keys with `users`
|
||||
|
||||
For finer control you can define named users, each with their own key. A user entry can also grant CDM device access and restrict which services that key may touch:
|
||||
|
||||
```yaml title="unshackle.yaml"
|
||||
serve:
|
||||
api_secret: "the-primary-admin-key"
|
||||
users:
|
||||
a-secret-key-for-alice:
|
||||
username: alice
|
||||
devices: ["my_widevine_device"] # Widevine (.wvd) devices this key may use
|
||||
playready_devices: ["my_playready_prd"] # PlayReady (.prd) devices this key may use
|
||||
services: ["AMZN", "NF"] # optional per-key service allowlist
|
||||
a-secret-key-for-bob:
|
||||
username: bob
|
||||
devices: ["my_widevine_device"]
|
||||
```
|
||||
|
||||
The key of each entry under `users` (for example `a-secret-key-for-alice`) *is* the secret that client sends in `X-Secret-Key`. The `username` is a human-readable label used in logs and in job/history metadata.
|
||||
|
||||
!!! info "How `api_secret` and `users` combine (integrated server)"
|
||||
On the integrated server, the configured `api_secret` is automatically added as an extra accepted key alongside everyone in `users`. It is granted access to all loaded Widevine and PlayReady devices and is labelled `api_user`. So `api_secret` behaves as an admin-level key in addition to any named users you define.
|
||||
|
||||
On the API-only server, only `api_secret` is accepted: the single-key model. (See the developer notes for the exact difference.)
|
||||
|
||||
### CDM device access per key
|
||||
|
||||
Under the integrated server, a key can only run DRM licensing through a CDM device that its user entry lists:
|
||||
|
||||
| Config key | Grants access to |
|
||||
|---|---|
|
||||
| `devices` | Widevine `.wvd` devices, by name |
|
||||
| `playready_devices` | PlayReady `.prd` devices, by name |
|
||||
|
||||
!!! warning "PlayReady access is opt-in per user"
|
||||
If a user entry omits `playready_devices`, that key is given **no** PlayReady access, and the server logs a warning at startup. You must list PlayReady devices explicitly for any key that needs them. Widevine and PlayReady device files themselves are auto-loaded from your WVDs and PRDs directories.
|
||||
|
||||
!!! note "Device lists *are* the server-side-decryption switch"
|
||||
There is no separate tier, capability flag, or permission toggle for whether the server will hand back keys (`KID:KEY`) for a session download. The presence of devices on the calling key decides it. With **empty** `devices` and `playready_devices`, the server can only proxy CDM challenges, so the client must run its own CDM. Once those lists are **populated**, the client may request `mode:server_cdm` and receive keys back. Don't go looking for a tier setting; configuring devices is what enables server-side decryption.
|
||||
|
||||
### Disabling authentication
|
||||
|
||||
The `--no-key` flag turns authentication off entirely. Every request is allowed, `X-Secret-Key` is ignored, and no key needs to be configured.
|
||||
|
||||
```bash
|
||||
unshackle serve --no-key
|
||||
```
|
||||
|
||||
!!! danger "Only use `--no-key` on a trusted, isolated network"
|
||||
With `--no-key` there is nothing stopping any client that can reach the port from listing your services, starting downloads, or using your CDM devices. Never expose a `--no-key` server to an untrusted network.
|
||||
|
||||
## How clients present credentials
|
||||
|
||||
Clients authenticate by sending the secret in a single HTTP request header on every call:
|
||||
|
||||
```
|
||||
X-Secret-Key: <your-secret>
|
||||
```
|
||||
|
||||
The value must exactly match one of the configured keys (`api_secret`, or one of the keys under `users`). There is no login step, no session cookie, and no token exchange for API access. The same static key is sent on each request.
|
||||
|
||||
=== "curl"
|
||||
|
||||
```bash
|
||||
curl -H "X-Secret-Key: your-secret-key-here" \
|
||||
http://127.0.0.1:8786/api/services
|
||||
```
|
||||
|
||||
=== "Python (requests)"
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
BASE = "http://127.0.0.1:8786"
|
||||
HEADERS = {"X-Secret-Key": "your-secret-key-here"}
|
||||
|
||||
resp = requests.get(f"{BASE}/api/services", headers=HEADERS, timeout=30)
|
||||
resp.raise_for_status()
|
||||
print(resp.json())
|
||||
```
|
||||
|
||||
=== "JavaScript (fetch)"
|
||||
|
||||
```javascript
|
||||
const resp = await fetch("http://127.0.0.1:8786/api/services", {
|
||||
headers: { "X-Secret-Key": "your-secret-key-here" },
|
||||
});
|
||||
if (!resp.ok) throw new Error(`Server returned ${resp.status}`);
|
||||
console.log(await resp.json());
|
||||
```
|
||||
|
||||
!!! tip "The reference client sets it for you"
|
||||
unshackle's own [remote download](../../guide/downloading.md) client (`RemoteService`) sends `X-Secret-Key` automatically, reading the key from your `remote_services` config. It also sends a `User-Agent` of `unshackle/<version>`. You only deal with the header directly when writing your own client.
|
||||
|
||||
### The health check is always open
|
||||
|
||||
One endpoint is exempt from authentication: `GET /api/health`. It never requires a key, so you can use it for liveness and version probes without embedding a secret in your monitoring:
|
||||
|
||||
```bash title="No key needed"
|
||||
curl http://127.0.0.1:8786/api/health
|
||||
```
|
||||
|
||||
```json title="Response"
|
||||
{
|
||||
"status": "ok",
|
||||
"version": "5.3.0",
|
||||
"update_check": {
|
||||
"update_available": null,
|
||||
"current_version": "5.3.0",
|
||||
"latest_version": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Every other endpoint (including the Swagger UI at `/api/docs/` on the API-only server) requires a valid key.
|
||||
|
||||
## Authentication failures
|
||||
|
||||
When a request is rejected for authentication reasons, the response has an HTTP `401 Unauthorized` status and a small JSON body.
|
||||
|
||||
=== "Missing header"
|
||||
|
||||
Sending a request with no `X-Secret-Key` header:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": 401,
|
||||
"message": "Secret Key is Empty."
|
||||
}
|
||||
```
|
||||
|
||||
=== "Wrong key"
|
||||
|
||||
Sending a key that is not in your config:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": 401,
|
||||
"message": "Secret Key is Invalid."
|
||||
}
|
||||
```
|
||||
|
||||
!!! warning "Auth errors use a different JSON shape than other errors (developer note)"
|
||||
The rest of the API reports errors with a structured body containing `error_code`, `message`, `timestamp`, and other fields (see [Errors](errors.md)). The authentication middleware is different: it returns `{"status": 401, "message": "..."}`, where `status` is an **integer**, and there is no `error_code` or `timestamp`.
|
||||
|
||||
A robust client should treat any `401` as an authentication problem and be prepared to parse *both* shapes rather than assuming the standard error body is always present.
|
||||
|
||||
## Building on the key: access controls
|
||||
|
||||
Once a request is authenticated, several additional controls decide what that specific key is allowed to do.
|
||||
|
||||
### Per-key service allowlists
|
||||
|
||||
You can restrict which services a key may use. Two layers combine:
|
||||
|
||||
- **Global allowlist**: `serve.services`, applied to every key.
|
||||
- **Per-key allowlist**: `services` inside that key's `users` entry.
|
||||
|
||||
The effective allowlist for a request is:
|
||||
|
||||
| Global set? | Per-key set? | Result |
|
||||
|---|---|---|
|
||||
| Yes | Yes | The **intersection** of the two |
|
||||
| Yes | No | The global list |
|
||||
| No | Yes | The per-key list |
|
||||
| No | No | No restriction, all services allowed |
|
||||
|
||||
```yaml title="Restricting services"
|
||||
serve:
|
||||
api_secret: "admin-key"
|
||||
services: ["AMZN", "NF", "DSNP"] # global ceiling for all keys
|
||||
users:
|
||||
limited-key:
|
||||
username: limited
|
||||
services: ["AMZN"] # this key ends up allowed only AMZN
|
||||
```
|
||||
|
||||
Endpoints that name a service (listing services, searching, listing titles/tracks, downloading, profiles, history, and remote sessions) all filter against this allowlist. A request for a service outside the key's allowlist is rejected as an invalid service. Service tags are matched after normalization, so casing and aliases resolve consistently.
|
||||
|
||||
### CDM overrides and per-job credentials
|
||||
|
||||
Two download-time capabilities are gated behind server config and, unless enabled, are refused with `403 Forbidden` even for an authenticated key:
|
||||
|
||||
| Config key | Default | Controls |
|
||||
|---|---|---|
|
||||
| `serve.cdm_overrides` | off | Whether a request may pick a specific CDM device via the `cdm` field. Set to `true` to allow any, or to a list/set of device names to allow only those. |
|
||||
| `serve.allow_job_credentials` | off | Whether a request may pass its own `credential` / `credentials` login details for a download job. |
|
||||
|
||||
```yaml title="Opting in to per-request CDM and credentials"
|
||||
serve:
|
||||
api_secret: "admin-key"
|
||||
cdm_overrides: ["my_widevine_device"] # or true to allow any configured device
|
||||
allow_job_credentials: true
|
||||
```
|
||||
|
||||
!!! tip "When to enable these gates"
|
||||
The two gates are fully independent and can be combined. The intended posture is that a locked-down or public-facing deployment leaves **both** unset (they default off and reject with `403`) so client requests can never steer the server's CDM or feed it their own logins. Enabling them only makes sense for a trusted single-client deployment, where the caller and the operator are effectively the same party.
|
||||
|
||||
!!! note "Client-supplied credentials get isolated token caches"
|
||||
When `allow_job_credentials` is enabled and a download job supplies its own `credential` / `credentials`, each distinct credential is given its **own** isolated token cache on the server. Client-supplied logins are never shared or merged with the server's configured-credential caches, so they cannot cross-contaminate or read the server's own token state, and they don't clobber each other.
|
||||
|
||||
See [Downloads](endpoints.md) for how these fields are used on the download endpoint.
|
||||
|
||||
### Session IP binding (remote sessions)
|
||||
|
||||
The remote-download session endpoints (`/api/session/*`) add an extra check on top of the key. When a session is created, the server records the client's IP address. Any later request for that session must come from the **same** IP, or it is rejected with `403 Forbidden`. This means a session is bound both to the key that created it and to the address it was created from.
|
||||
|
||||
## Transport security (HTTPS)
|
||||
|
||||
The API server speaks plain HTTP. Because your key is sent on every request, you should not expose it over an unencrypted connection on an untrusted network. To serve over HTTPS, run a reverse proxy in front of unshackle. The `serve` command can launch Caddy for you:
|
||||
|
||||
```bash
|
||||
unshackle serve --caddy
|
||||
```
|
||||
|
||||
With `--caddy`, unshackle also starts `caddy run` using the `Caddyfile` located next to your unshackle config, letting you terminate TLS and reverse-proxy to the API. Configuring Caddy itself (certificates, domains, upstream) is done in that `Caddyfile`.
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Auth header | `X-Secret-Key` |
|
||||
| Default bind address | `127.0.0.1:8786` |
|
||||
| Exempt endpoint | `GET /api/health` |
|
||||
| Failure status | `401 Unauthorized` |
|
||||
| Missing-header body | `{"status": 401, "message": "Secret Key is Empty."}` |
|
||||
| Invalid-key body | `{"status": 401, "message": "Secret Key is Invalid."}` |
|
||||
| Disable auth | `--no-key` (all requests allowed) |
|
||||
| Config location | `serve.api_secret` and `serve.users` in `unshackle.yaml` |
|
||||
|
||||
---
|
||||
|
||||
## Developer notes: how the check is implemented
|
||||
|
||||
!!! note "This section is for people writing or debugging the server, not for API consumers."
|
||||
|
||||
### Two middlewares, one header
|
||||
|
||||
The header and config are identical between modes, but the code that enforces them differs:
|
||||
|
||||
- **API-only server** (`--api-only`, or `--remote-only` which forces API-only). An `api_key_authentication` middleware runs on every request. It lets `GET /api/health` through unconditionally, then reads `X-Secret-Key`. An empty header returns the *"Secret Key is Empty."* body; a header that is not a configured user returns *"Secret Key is Invalid."*. In this mode the accepted-keys map contains exactly one entry: your `api_secret` (labelled `api_user`, with no devices). With `--no-key`, this middleware is simply not installed and the users map is empty.
|
||||
|
||||
- **Integrated server** (default). A `serve_authentication` middleware wraps pywidevine's authentication (and pyplayready's for `/playready*` paths), and the REST routes live in the same application, guarded by the same check. Here the accepted-keys map is your full `serve` config: `api_secret` (auto-added with access to all loaded devices) plus every entry under `users`, each with its declared `devices` / `playready_devices`.
|
||||
|
||||
Because the API-only middleware guards *all* non-health paths, the Swagger UI at `/api/docs/` is also behind `X-Secret-Key` there. In `--remote-only` mode the Swagger UI is not mounted at all.
|
||||
|
||||
### CORS and the `Authorization` header
|
||||
|
||||
A `cors_middleware` runs on every response in both modes (and answers `OPTIONS` preflight requests with an empty response). It emits:
|
||||
|
||||
```
|
||||
Access-Control-Allow-Origin: *
|
||||
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
|
||||
Access-Control-Allow-Headers: Content-Type, X-Secret-Key, Authorization
|
||||
Access-Control-Max-Age: 3600
|
||||
```
|
||||
|
||||
The REST API authenticates with `X-Secret-Key`; `Authorization` is advertised because the co-hosted pywidevine / pyplayready CDM endpoints use it. `PUT` appears in the allowed methods list but no REST route uses it; only `GET`, `POST`, and `DELETE` are wired up.
|
||||
|
||||
### Where device access is read
|
||||
|
||||
The map of accepted keys lives at `request.app["config"]["users"]`. For DRM operations, a key's permitted Widevine devices come from its `devices` list and PlayReady devices from `playready_devices`; the first configured device is used when a request does not (or may not) override it. A server-wide `config.cdm[<service>]` mapping, if present, takes precedence over the key's device list when resolving which device to use.
|
||||
@@ -0,0 +1,980 @@
|
||||
# REST API Endpoints
|
||||
|
||||
This is the full endpoint reference for the unshackle REST API. It covers every route: its method and path, the request body it accepts, the response shape it returns, and the status codes it can produce. Routes are grouped into general, download jobs, server management, and remote-dl sessions.
|
||||
|
||||
Start the server with [`unshackle serve`](../../guide/cli-reference.md). By default it binds to `127.0.0.1:8786`, and every route lives under the `/api/` prefix. Authentication, server modes, and the `X-Secret-Key` header are covered in detail on the [Authentication](authentication.md) page. Read that first if you have not already. For an end-to-end walkthrough, see the [Quickstart](quickstart.md).
|
||||
|
||||
!!! note "Base URL"
|
||||
All examples below assume a base URL of `http://127.0.0.1:8786`. Replace the host and port to match your own `unshackle serve -h <host> -p <port>` invocation.
|
||||
|
||||
!!! tip "Interactive docs"
|
||||
A live Swagger UI is mounted at **`/api/docs/`** whenever the server is not running in `--remote-only` mode. Its title is "Unshackle REST API" and its version tracks the installed package version. In API-only mode with a key configured, the Swagger UI itself sits behind `X-Secret-Key`; only `/api/health` is exempt.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Authentication.** Every request except `GET /api/health` requires the `X-Secret-Key` header when a key is configured. Starting the server with `--no-key` disables the check entirely. See [Authentication](authentication.md).
|
||||
- **Content type.** Request bodies are JSON (`Content-Type: application/json`); all success responses are JSON unless a `204 No Content` is documented.
|
||||
- **CORS.** Every response carries permissive CORS headers, and `OPTIONS` preflight requests are answered automatically.
|
||||
- **Compression.** JSON responses of 256 bytes or more are gzip-compressed when the client sends `Accept-Encoding: gzip`.
|
||||
- **Service allowlist.** Service-facing endpoints are filtered by the effective allowlist for your API key (the intersection of the global `serve.services` list and your per-key list). Services you are not allowed to use are treated as unknown.
|
||||
|
||||
!!! warning "Error responses have two shapes"
|
||||
Structured application errors (raised inside a handler) return `{"status": "error", "error_code": "...", "message": "...", "timestamp": "..."}`. The authentication middleware, however, returns a **different** shape: `{"status": 401, "message": "..."}` with an integer `status` and no `error_code`. Clients must handle both. The full error model is documented in the [Error responses](#error-responses) section at the bottom of this page.
|
||||
|
||||
---
|
||||
|
||||
## General endpoints
|
||||
|
||||
These endpoints report server state and enumerate what is available. `GET /api/health`, `GET /api/services`, and `POST /api/search` are also exposed in `--remote-only` mode.
|
||||
|
||||
### `GET /api/health`
|
||||
|
||||
Health check. **This is the only route exempt from authentication**: you can call it without an `X-Secret-Key` header. It reports the running version and a best-effort update check.
|
||||
|
||||
=== "Request"
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8786/api/health
|
||||
```
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"version": "5.3.0",
|
||||
"update_check": {
|
||||
"update_available": false,
|
||||
"current_version": "5.3.0",
|
||||
"latest_version": "5.3.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If the update check fails (for example, no network), `update_available` and `latest_version` are `null` while `current_version` still reports the installed version.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `200` | Server is healthy. |
|
||||
|
||||
### `GET /api/services`
|
||||
|
||||
List the streaming services available on this server, filtered by your allowlist. Each entry describes the service's tag, matching rules, capability flags, and its CLI parameters (useful for building a UI that accepts service-specific options).
|
||||
|
||||
=== "Request"
|
||||
|
||||
```bash
|
||||
curl -H "X-Secret-Key: $KEY" http://127.0.0.1:8786/api/services
|
||||
```
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"services": [
|
||||
{
|
||||
"tag": "EXAMPLE",
|
||||
"aliases": ["EX"],
|
||||
"geofence": ["US"],
|
||||
"title_regex": "^https?://example\\.com/.+",
|
||||
"url": "https://example.com",
|
||||
"help": "EXAMPLE\nAuthorization: Cookies\nSecurity: FHD@L3",
|
||||
"cli_params": [
|
||||
{"name": "title", "kind": "argument", "required": true},
|
||||
{
|
||||
"name": "movie",
|
||||
"kind": "option",
|
||||
"opts": ["-m", "--movie"],
|
||||
"is_flag": true,
|
||||
"default": false,
|
||||
"help": "Title is a movie.",
|
||||
"type": "boolean",
|
||||
"multiple": false
|
||||
}
|
||||
],
|
||||
"needs_auth": true,
|
||||
"has_search": true,
|
||||
"has_drm": true,
|
||||
"auth_methods": ["cookies"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Field notes:
|
||||
|
||||
| Field | Description |
|
||||
| --- | --- |
|
||||
| `tag` | Canonical service tag; use this as the `service` value on other endpoints. |
|
||||
| `aliases` | Alternate tags the service also answers to. |
|
||||
| `geofence` | Region codes the service is geofenced to (may be empty). |
|
||||
| `title_regex` | URL/ID pattern the service matches, as a string or list of strings (or `null`). |
|
||||
| `url` | Service URL, taken from the service's CLI short help. |
|
||||
| `help` | The service module's docstring. |
|
||||
| `cli_params` | Array of parameter descriptors. Arguments have `{name, kind: "argument", required}`; options have `{name, kind: "option", opts, is_flag, default, help, type, multiple}` and, for choice options, a `choices` list. |
|
||||
| `needs_auth` | Whether the service overrides `authenticate()`. |
|
||||
| `has_search` | Whether the service supports `POST /api/search`. |
|
||||
| `has_drm` | Whether the service overrides a Widevine or PlayReady license hook. |
|
||||
| `auth_methods` | Declared or inferred auth methods, e.g. `["cookies"]`, `["credentials"]`. |
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `200` | Service list returned. |
|
||||
| `500` | Unexpected server error (`INTERNAL_ERROR`). |
|
||||
|
||||
### `POST /api/search`
|
||||
|
||||
Search a service for titles by query string. The service must implement `search()` (see `has_search` above).
|
||||
|
||||
**Request body**
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `service` | string | yes | — | Service tag. |
|
||||
| `query` | string | yes | — | Search query. |
|
||||
| `profile` | string | no | `null` | Credential/cookie profile to use. |
|
||||
| `proxy` | string | no | `null` | Proxy URI or country code. |
|
||||
| `no_proxy` | boolean | no | `false` | Force-disable all proxy use. |
|
||||
|
||||
=== "Request"
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:8786/api/search \
|
||||
-H "X-Secret-Key: $KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"service": "EXAMPLE", "query": "planet earth"}'
|
||||
```
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"id": "12345",
|
||||
"title": "Planet Earth",
|
||||
"description": "A nature documentary series.",
|
||||
"label": "Available",
|
||||
"url": "https://example.com/title/12345"
|
||||
}
|
||||
],
|
||||
"count": 1
|
||||
}
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Search completed. |
|
||||
| `400` | `INVALID_PARAMETERS` | Missing `query`. |
|
||||
| `400` | `INVALID_SERVICE` | Unknown or disallowed service. |
|
||||
| `400` | `INVALID_PROXY` | Proxy could not be resolved. |
|
||||
| `502` | `SERVICE_ERROR` | Service failed to initialize, or does not support search. |
|
||||
|
||||
### `POST /api/list-titles`
|
||||
|
||||
Resolve a title ID into its list of titles (episodes, movie, etc.) without downloading anything.
|
||||
|
||||
**Request body**
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `service` | string | yes | Service tag. |
|
||||
| `title_id` | string | yes | Title identifier. |
|
||||
| `profile` | string | no | Credential/cookie profile. |
|
||||
| `proxy` | string | no | Proxy URI or country code. |
|
||||
| `no_proxy` | boolean | no | Force-disable proxy. |
|
||||
| `cdm_type` | string | no | Preferred CDM type. |
|
||||
|
||||
Service-specific CLI options may also be passed as additional keys.
|
||||
|
||||
=== "Request"
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:8786/api/list-titles \
|
||||
-H "X-Secret-Key: $KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"service": "EXAMPLE", "title_id": "12345"}'
|
||||
```
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"titles": [
|
||||
{
|
||||
"type": "episode",
|
||||
"name": "Pilot",
|
||||
"id": "ep-1",
|
||||
"language": "en",
|
||||
"description": "The first episode.",
|
||||
"date": "2020-01-01",
|
||||
"cover_url": "https://example.com/cover.jpg",
|
||||
"year": 2020,
|
||||
"series_title": "Example Show",
|
||||
"season": 1,
|
||||
"number": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Each serialized title carries `type` (`"episode"`, `"movie"`, or `"other"`), `name`, `id`, `language`, `description`, `date`, and `cover_url`. Episodes and movies add `year`; episodes additionally add `series_title`, `season`, and `number`.
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Titles returned. |
|
||||
| `400` | `INVALID_INPUT` | Missing `service` or `title_id`. |
|
||||
| `400` | `INVALID_SERVICE` | Unknown or disallowed service. |
|
||||
| `401` | `AUTH_FAILED` | Authentication with the service failed. |
|
||||
| `404` | `NOT_FOUND` | Title not found. |
|
||||
| `500` | `INTERNAL_ERROR` | Unexpected error. |
|
||||
|
||||
### `POST /api/list-tracks`
|
||||
|
||||
List the video, audio, and subtitle tracks for a title. For series, you can scope to specific episodes.
|
||||
|
||||
**Request body**
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `service` | string | yes | Service tag. |
|
||||
| `title_id` | string | yes | Title identifier. |
|
||||
| `wanted` | string | no | Episode/season range (e.g. `"S01E01-S01E03"`). |
|
||||
| `season` | int/string | no | Season number (combined with `episode`). |
|
||||
| `episode` | int/string | no | Episode number (combined with `season`). |
|
||||
| `profile`, `proxy`, `no_proxy`, `cdm_type` | — | no | As on `list-titles`. |
|
||||
|
||||
When both `season` and `episode` are given, they are combined into a `"{season}x{episode}"` selector.
|
||||
|
||||
=== "Single title / movie: Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"title": { "type": "movie", "name": "Example Movie", "id": "12345", "year": 2021 },
|
||||
"video": [
|
||||
{
|
||||
"id": "v-1", "codec": "H265", "codec_display": "HEVC",
|
||||
"bitrate": 8000, "width": 3840, "height": 2160,
|
||||
"resolution": "2160p", "fps": "24", "range": "HDR10",
|
||||
"range_display": "HDR10", "language": "en", "drm": [], "descriptor": "DASH"
|
||||
}
|
||||
],
|
||||
"audio": [
|
||||
{
|
||||
"id": "a-1", "codec": "EC3", "codec_display": "DD+",
|
||||
"bitrate": 640, "channels": "5.1", "language": "en",
|
||||
"atmos": false, "descriptive": false, "drm": [], "descriptor": "DASH"
|
||||
}
|
||||
],
|
||||
"subtitles": [
|
||||
{ "id": "s-1", "codec": "vtt", "language": "en",
|
||||
"forced": false, "sdh": false, "cc": false, "descriptor": "DASH" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
=== "Multiple episodes: Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"episodes": [
|
||||
{
|
||||
"title": { "type": "episode", "name": "Pilot", "id": "ep-1", "season": 1, "number": 1 },
|
||||
"video": [ ... ],
|
||||
"audio": [ ... ],
|
||||
"subtitles": [ ... ]
|
||||
}
|
||||
],
|
||||
"unavailable_episodes": [
|
||||
{ "title": { "id": "ep-2" }, "error": "..." }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Tracks are sorted by bitrate, descending. Episodes that fail to resolve are collected into `unavailable_episodes` rather than failing the whole request. Track objects here do **not** include a download `url`; segment URLs are only exposed through the remote-dl session endpoints.
|
||||
|
||||
The `drm` array on a track is a list of `{type, pssh?, kids?, content_keys?, license_url?}` objects.
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Tracks returned. |
|
||||
| `400` | `INVALID_INPUT` | Missing `service` or `title_id`. |
|
||||
| `400` | `INVALID_PARAMETERS` | Invalid `wanted` range. |
|
||||
| `400` | `INVALID_SERVICE` | Unknown or disallowed service. |
|
||||
| `404` | `NO_CONTENT` | No matching episodes, or all matches failed. |
|
||||
|
||||
---
|
||||
|
||||
## Download jobs
|
||||
|
||||
Downloads are asynchronous. `POST /api/download` enqueues a job and returns immediately with a `job_id`; the server runs each job in a background worker subprocess. You then poll the job endpoints for progress and results.
|
||||
|
||||
!!! info "Job states"
|
||||
A job moves through `queued` → `downloading` → one of the terminal states `completed`, `failed`, or `cancelled`. Only queued jobs can be prioritized; only terminal jobs can be retried or removed.
|
||||
|
||||
### `POST /api/download`
|
||||
|
||||
Create a download job. Requires `service` and `title_id`; every other field is an optional download parameter with a server-side default. On success this returns **`202 Accepted`**, not `200`.
|
||||
|
||||
=== "Request"
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:8786/api/download \
|
||||
-H "X-Secret-Key: $KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"service": "EXAMPLE",
|
||||
"title_id": "12345",
|
||||
"quality": [2160],
|
||||
"vcodec": "H265",
|
||||
"range": ["HDR10"],
|
||||
"wanted": ["S01E01"]
|
||||
}'
|
||||
```
|
||||
|
||||
=== "Response `202`"
|
||||
|
||||
```json
|
||||
{
|
||||
"job_id": "b0f7c8e2-4a1d-4e9c-9b7e-2f3a1c0d5e6f",
|
||||
"status": "queued",
|
||||
"created_time": "2026-07-03T12:00:00+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
**Common parameters** (all optional). These mirror the [`dl` command](../../guide/downloading.md) flags:
|
||||
|
||||
| Field | Type | Default | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `profile` | string | `null` | Credential/cookie profile. |
|
||||
| `quality` | int[] | `[]` | Target resolution(s); best available if empty. |
|
||||
| `vcodec` | string/string[] | `null` | Video codec filter. |
|
||||
| `acodec` | string/string[] | `null` | Audio codec filter. |
|
||||
| `vbitrate` / `abitrate` | int | `null` | Video / audio bitrate in kbps (positive). |
|
||||
| `vbitrate_range` / `abitrate_range` | string | `null` | `"MIN-MAX"` bitrate window. |
|
||||
| `range` | string[] | `["SDR"]` | Dynamic range(s). |
|
||||
| `channels` | number | `null` | Audio channel count. |
|
||||
| `no_atmos` | boolean | `false` | Exclude Atmos tracks. |
|
||||
| `wanted` | string[] | `[]` | Episode/season selectors. |
|
||||
| `latest_episode` | boolean | `false` | Only the newest episode. |
|
||||
| `lang` / `v_lang` / `a_lang` / `s_lang` | string[] | `["orig"]` / `[]` / `[]` / `["all"]` | Language filters. |
|
||||
| `require_subs` | string[] | `[]` | Required subtitle languages. |
|
||||
| `forced_subs` | boolean | `false` | Include forced subtitles. |
|
||||
| `exact_lang` | boolean | `false` | Exact language matching. |
|
||||
| `sub_format` | string | `null` | Output subtitle format. |
|
||||
| `video_only` / `audio_only` / `subs_only` / `chapters_only` | boolean | `false` | Track-type restriction (at most one). |
|
||||
| `no_subs` / `no_audio` / `no_chapters` / `no_video` | boolean | `false` | Skip a track type. |
|
||||
| `audio_description` | boolean | `false` | Download audio description. |
|
||||
| `slow` | boolean/string | `null` | Randomized delay between downloads. |
|
||||
| `split_audio` | boolean | `null` | Separate files per audio codec. |
|
||||
| `skip_dl` | boolean | `false` | Only fetch keys, do not download. |
|
||||
| `export` | boolean | `false` | Export manifest/keys/subs to JSON. |
|
||||
| `cdm_only` | boolean | `null` | Force CDM-only (`true`) or vault-only (`false`) key retrieval. |
|
||||
| `proxy` / `no_proxy` / `no_proxy_download` | string / bool / bool | `null` / `false` / `false` | Proxy controls. |
|
||||
| `no_folder` / `no_source` / `no_mux` | boolean | `false` | Output/mux controls. |
|
||||
| `workers` | int | `null` | Threads per track. |
|
||||
| `downloads` | int | `1` | Concurrent tracks. |
|
||||
| `worst` | boolean | `false` | Lowest bitrate within quality. |
|
||||
| `best_available` | boolean | `false` | Fall back to best available. |
|
||||
| `repack` | boolean | `false` | Add REPACK tag. |
|
||||
| `tag` | string | `null` | Release group tag. |
|
||||
| `tmdb_id` / `imdb_id` / `animeapi_id` | — | `null` | External ID overrides for tagging. |
|
||||
| `enrich` | boolean | `false` | Override title/year from external source. |
|
||||
| `output_dir` | string | `null` | Override output directory. |
|
||||
| `no_cache` / `reset_cache` | boolean | `false` | Title cache controls. |
|
||||
|
||||
**Validation.** Invalid values return `400 INVALID_PARAMETERS`. Notably: `vcodec` must be one of H264/H265/H.264/H.265/AVC/HEVC/VC1/VC-1/VP8/VP9/AV1; `acodec` one of AAC/AC3/EC3/EAC3/DD/DD+/AC4/OPUS/FLAC/ALAC/VORBIS/OGG/DTS; `range` one of SDR/HDR10/HDR10P/DV/HLG/HYBRID (`HDR10+` is accepted); bitrate/worker/download counts must be positive integers; at most one of the `*_only` flags may be set; `no_subs` cannot be combined with `subs_only`, `no_audio` with `audio_only`, or `s_lang` with `require_subs`.
|
||||
|
||||
!!! warning "Gated parameters (developer)"
|
||||
A few keys are accepted but only permitted when the server config opts in, otherwise the request is rejected with `403 FORBIDDEN`:
|
||||
|
||||
- `cdm` (per-request CDM device name): allowed only if `serve.cdm_overrides` is `true` or a list containing that device name.
|
||||
- `credential` (`"user:pass"`) and `credentials` (a map): allowed only if `serve.allow_job_credentials` is truthy (off by default).
|
||||
|
||||
Server-side, effective parameters are merged as defaults → `serve:` config overrides → service-specific defaults → your request body.
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `202` | — | Job queued. |
|
||||
| `400` | `INVALID_INPUT` | Missing `service`/`title_id` or malformed JSON. |
|
||||
| `400` | `INVALID_PARAMETERS` | A parameter failed validation. |
|
||||
| `400` | `INVALID_SERVICE` | Unknown or disallowed service. |
|
||||
| `403` | `FORBIDDEN` | A gated parameter is not permitted. |
|
||||
|
||||
### `GET /api/download/jobs`
|
||||
|
||||
List download jobs, with optional filtering and sorting.
|
||||
|
||||
**Query parameters**
|
||||
|
||||
| Param | Values | Default | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `status` | `queued`, `downloading`, `completed`, `failed`, `cancelled` | — | Filter by status. |
|
||||
| `service` | service tag | — | Filter by service. |
|
||||
| `sort_by` | `created_time`, `started_time`, `completed_time`, `progress`, `status`, `service` | `created_time` | Sort field. |
|
||||
| `sort_order` | `asc`, `desc` | `desc` | Sort direction. |
|
||||
| `full` | `true`, `false` | `false` | Include full per-job details. |
|
||||
|
||||
=== "Request"
|
||||
|
||||
```bash
|
||||
curl -H "X-Secret-Key: $KEY" \
|
||||
"http://127.0.0.1:8786/api/download/jobs?status=downloading&sort_by=progress&sort_order=desc"
|
||||
```
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"jobs": [
|
||||
{
|
||||
"job_id": "b0f7c8e2-...",
|
||||
"status": "downloading",
|
||||
"created_time": "2026-07-03T12:00:00+00:00",
|
||||
"service": "EXAMPLE",
|
||||
"title_id": "12345",
|
||||
"title": "Example Show S01E01",
|
||||
"progress": 42.5,
|
||||
"phase": "downloading",
|
||||
"current_title": "Episode 1",
|
||||
"completed_tracks": 1,
|
||||
"total_tracks": 3,
|
||||
"active_tracks": ["v-1"],
|
||||
"track_progress": [{"label": "Video", "progress": 42.5, "speed": "5.2 MB/s"}],
|
||||
"segments_done": 120,
|
||||
"segments_total": 300,
|
||||
"speed": "5.2 MB/s",
|
||||
"skipped_subtitles": []
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
With `full=true`, each job additionally includes `parameters` (with secrets redacted), `started_time`, `completed_time`, `output_files`, `error_message`, `error_details`, `error_code`, `error_traceback`, and `worker_stderr`.
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Jobs returned. |
|
||||
| `400` | `INVALID_PARAMETERS` | Invalid `sort_by` or `sort_order`. |
|
||||
|
||||
### `GET /api/download/jobs/{job_id}`
|
||||
|
||||
Fetch a single job with full details (equivalent to a `full=true` list entry).
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Job returned. |
|
||||
| `404` | `JOB_NOT_FOUND` | No such job. |
|
||||
|
||||
### `DELETE /api/download/jobs/{job_id}`
|
||||
|
||||
Cancel or remove a job. The behaviour depends on the job's current state:
|
||||
|
||||
- **Terminal job** (completed/failed/cancelled): removed from the manager, returning **`204 No Content`** with an empty body.
|
||||
- **Queued or downloading job**: cancelled, returning `200` `{"status": "success", "message": "Job cancelled"}`.
|
||||
|
||||
!!! warning
|
||||
A `204` response has no body. Do not attempt to parse JSON from it.
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Job cancelled. |
|
||||
| `204` | — | Terminal job removed (empty body). |
|
||||
| `400` | `INVALID_PARAMETERS` | Job is in a state that cannot be cancelled. |
|
||||
| `404` | `JOB_NOT_FOUND` | No such job. |
|
||||
|
||||
### `POST /api/download/jobs/clear-finished`
|
||||
|
||||
Remove every terminal (completed/failed/cancelled) job from the manager. No request body.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{ "removed": 3 }
|
||||
```
|
||||
|
||||
### `POST /api/download/jobs/{job_id}/retry`
|
||||
|
||||
Enqueue a **new** job that reuses a terminal job's service, title, and parameters. The service allowlist and parameter gates are re-checked. Returns `202` with the new job's ID.
|
||||
|
||||
!!! warning "Retry reuses the original, unredacted parameters"
|
||||
Retry replays the finished job's stored parameters exactly as they were submitted, including any secrets such as `credential`/`credentials`, not the redacted `parameters` view that `GET /api/download/jobs` and `GET /api/history` expose. This is deliberate: it lets a retried job reproduce a credentialed download that the redacted job record alone never could. It also means anyone who can retry a job can re-run it with its original credentials, even though those credentials are masked everywhere else in the API.
|
||||
|
||||
=== "Response `202`"
|
||||
|
||||
```json
|
||||
{
|
||||
"job_id": "9c1e2f3a-...",
|
||||
"status": "queued",
|
||||
"created_time": "2026-07-03T12:05:00+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `202` | — | New job queued. |
|
||||
| `404` | `JOB_NOT_FOUND` | No such job. |
|
||||
| `409` | `CONFLICT` | Job is not in a terminal state. |
|
||||
| `400` | `INVALID_SERVICE` | Service is no longer allowed. |
|
||||
| `403` | `FORBIDDEN` | A gated parameter is no longer permitted. |
|
||||
|
||||
### `POST /api/download/jobs/{job_id}/priority`
|
||||
|
||||
Move a queued job to the front of the download queue.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{ "job_id": "b0f7c8e2-...", "position": "front" }
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Job moved to front. |
|
||||
| `404` | `JOB_NOT_FOUND` | No such job. |
|
||||
| `409` | `CONFLICT` | Job is not queued. |
|
||||
|
||||
---
|
||||
|
||||
## Server management
|
||||
|
||||
Endpoints for profiles, effective configuration, download history, environment checks, and maintenance actions.
|
||||
|
||||
### `GET /api/profiles`
|
||||
|
||||
List the named credential profiles configured per service (usable as the `profile` parameter elsewhere). Only services whose credentials are a mapping of profile name to credential are listed; services with a single unnamed credential are omitted. Filtered by your allowlist.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"profiles": {
|
||||
"EXAMPLE": ["default", "premium"],
|
||||
"OTHER": ["us-account"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `GET /api/config`
|
||||
|
||||
Return a read-only, redacted view of the effective server configuration, suitable for a settings page in a UI. Secrets are never emitted: keys matching `secret`, `password`, `token`, `api_key`, or `credential`, and any userinfo in proxy URLs, are masked.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"config": {
|
||||
"dl": { "...": "redacted default dl parameters" },
|
||||
"serve": {
|
||||
"max_concurrent_downloads": 2,
|
||||
"job_retention_hours": 24,
|
||||
"history_limit": 100,
|
||||
"services": null,
|
||||
"remote_only": false,
|
||||
"cdm_overrides": null,
|
||||
"allow_job_credentials": false
|
||||
},
|
||||
"directories": {
|
||||
"downloads": "/home/user/Downloads/unshackle",
|
||||
"temp": "/home/user/.unshackle/temp",
|
||||
"cache": "/home/user/.unshackle/cache"
|
||||
},
|
||||
"services": ["EXAMPLE", "OTHER"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `GET /api/history`
|
||||
|
||||
Read the persisted history of jobs that reached a terminal state, newest first. History is stored in `api_history.jsonl` in the cache directory; corrupt lines are skipped and a missing file yields an empty list.
|
||||
|
||||
**Query parameters**
|
||||
|
||||
| Param | Type | Default | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `limit` | int (≥ 1) | `100` | Maximum entries to return. |
|
||||
| `service` | string | — | Case-insensitive service filter. |
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"history": [
|
||||
{
|
||||
"job_id": "b0f7c8e2-...",
|
||||
"service": "EXAMPLE",
|
||||
"title_id": "12345",
|
||||
"title": "Example Show S01E01",
|
||||
"status": "completed",
|
||||
"created_time": "2026-07-03T12:00:00+00:00",
|
||||
"completed_time": "2026-07-03T12:08:00+00:00",
|
||||
"output_files": ["/downloads/Example.S01E01.mkv"],
|
||||
"error_message": null,
|
||||
"parameters": { "...": "redacted" }
|
||||
}
|
||||
],
|
||||
"count": 1
|
||||
}
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | History returned. |
|
||||
| `400` | `INVALID_PARAMETERS` | Invalid `limit`. |
|
||||
|
||||
### `DELETE /api/history/{job_id}`
|
||||
|
||||
Remove a single persisted history entry. Returns **`204 No Content`** (empty body) on success.
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `204` | — | Entry removed (empty body). |
|
||||
| `404` | `NOT_FOUND` | No such entry (or outside your allowlist). |
|
||||
|
||||
### `POST /api/maintenance/clear-cache` and `POST /api/maintenance/clear-temp`
|
||||
|
||||
Delete and recreate the cache or temp directory. Neither takes a request body. Both are blocked with `409 CONFLICT` if any job is currently `downloading` (the `details` include the offending `active_jobs`).
|
||||
|
||||
The `409` guard is about active file I/O, not an arbitrary lock: an in-flight job may still be *reading* cached title data under the cache directory, and active download workers are *writing* segment files under temp. Removing either directory mid-download would corrupt those running jobs, so the endpoints refuse until nothing is downloading.
|
||||
|
||||
!!! warning "clear-cache also wipes download history"
|
||||
The persistent history read by [`GET /api/history`](#get-apihistory) lives in `api_history.jsonl` **inside the cache directory**. Because clear-cache deletes and recreates that whole directory, calling it also erases your entire download history, not just cached title data. If you want to keep history, do not clear the cache; remove individual entries with [`DELETE /api/history/{job_id}`](#delete-apihistoryjob_id) instead.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{ "cleared": true, "freed_bytes": 1048576 }
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Directory cleared. |
|
||||
| `409` | `CONFLICT` | A download is currently active. |
|
||||
|
||||
### `POST /api/maintenance/refresh-services`
|
||||
|
||||
git-pull every service repository configured under `directories.services`. `refreshed` is `true` when all repos updated successfully (or none are configured).
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"refreshed": true,
|
||||
"repos": [
|
||||
{ "spec": "github.com/example/services", "updated": true, "changes": ["a1b2c3d Update EXAMPLE"] }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### `GET /api/env/check`
|
||||
|
||||
Report the install status of the external binaries unshackle depends on (the same set `unshackle env check` inspects), with best-effort version detection.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"checks": [
|
||||
{ "name": "ffmpeg", "installed": true, "version": "6.1.1", "required": true },
|
||||
{ "name": "ccextractor", "installed": false, "version": null, "required": false }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Remote-dl sessions
|
||||
|
||||
!!! note "Developer feature"
|
||||
The `/api/session/*` endpoints power unshackle's **remote download** mode, where a thin local client authenticates once against a remote server, then fetches titles, tracks, and segments and proxies DRM licensing back through it. These endpoints are exposed in `--remote-only` mode, and they are what the built-in `RemoteService` client drives. Most end users never call them directly.
|
||||
|
||||
Sessions are held in memory. They expire after `serve.session_ttl` (default 300s), except while awaiting interactive authentication input, which uses a 600s window. A session is bound to the IP that created it, so a request from a different IP is rejected with `403 FORBIDDEN`.
|
||||
|
||||
### Session lifecycle overview
|
||||
|
||||
```text
|
||||
POST /api/session/create → returns session_id, status "authenticating"
|
||||
GET /api/session/{id}/prompt → poll until authenticated (submit answers via POST)
|
||||
GET /api/session/{id}/titles → list titles
|
||||
POST /api/session/{id}/tracks → tracks + chapters (with URLs) for one title
|
||||
POST /api/session/{id}/segments → resolve segment URLs / DRM / headers per track
|
||||
POST /api/session/{id}/license → proxy or server-CDM DRM licensing
|
||||
DELETE /api/session/{id} → tear down, harvest updated cache
|
||||
```
|
||||
|
||||
### `POST /api/session/create`
|
||||
|
||||
Create a session for a service and title. Authentication runs asynchronously in the background. This call returns immediately with `status: "authenticating"`, and you then poll the prompt endpoint. The body accepts `service` and `title_id` (both required) plus a broad set of optional keys (`additionalProperties` is allowed): `credentials` (`{username, password, extra?}`), `cookies` (base64 of zlib-compressed Netscape cookie file), `proxy`, `no_proxy`, `profile`, `cache` (map of forwarded cache files), `client_region`, `cdm_type`, `range_`, `vcodec`, `quality`, `best_available`, and any service CLI options.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "d4e5f6a7-...",
|
||||
"service": "EXAMPLE",
|
||||
"status": "authenticating"
|
||||
}
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Session created; authentication in progress. |
|
||||
| `400` | `INVALID_INPUT` | Missing `service` or `title_id`. |
|
||||
| `400` | `INVALID_SERVICE` | Unknown or disallowed service. |
|
||||
|
||||
### `GET /api/session/{session_id}/prompt`
|
||||
|
||||
Poll for the authentication status and any pending interactive prompt (OTP, PIN, device code). Response variants:
|
||||
|
||||
=== "Authenticating"
|
||||
|
||||
```json
|
||||
{ "status": "authenticating" }
|
||||
```
|
||||
|
||||
=== "Pending input"
|
||||
|
||||
```json
|
||||
{ "status": "pending_input", "prompt": "Enter the code sent to your email:" }
|
||||
```
|
||||
|
||||
=== "Authenticated"
|
||||
|
||||
```json
|
||||
{ "status": "authenticated" }
|
||||
```
|
||||
|
||||
=== "Failed"
|
||||
|
||||
```json
|
||||
{ "status": "failed", "error": "Invalid credentials" }
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Status returned. |
|
||||
| `403` | `FORBIDDEN` | Request IP differs from the session creator. |
|
||||
| `404` | `SESSION_NOT_FOUND` | No such session. |
|
||||
|
||||
### `POST /api/session/{session_id}/prompt`
|
||||
|
||||
Submit a response to a pending prompt. Body: `{ "response": "..." }` (required).
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{ "status": "accepted" }
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Response accepted. |
|
||||
| `400` | `INVALID_INPUT` | Missing `response`, or no prompt is pending. |
|
||||
| `404` | `SESSION_NOT_FOUND` | No such session. |
|
||||
|
||||
### `GET /api/session/{session_id}/titles`
|
||||
|
||||
Fetch the titles for an authenticated session.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "d4e5f6a7-...",
|
||||
"titles": [ { "type": "episode", "name": "Pilot", "id": "ep-1" } ]
|
||||
}
|
||||
```
|
||||
|
||||
Requires an authenticated session; `404 SESSION_NOT_FOUND` otherwise.
|
||||
|
||||
### `POST /api/session/{session_id}/tracks`
|
||||
|
||||
Get tracks and chapters for a specific title within the session. Body: `{ "title_id": "..." }` (required). Unlike `list-tracks`, track objects here **include** download URLs, along with session headers/cookies and manifest data the client needs to download directly.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"title": { "type": "episode", "name": "Pilot", "id": "ep-1" },
|
||||
"video": [ { "id": "v-1", "url": "https://...", "drm": [ ... ] } ],
|
||||
"audio": [ { "id": "a-1", "url": "https://..." } ],
|
||||
"subtitles": [ { "id": "s-1", "url": "https://..." } ],
|
||||
"chapters": [ { "timestamp": "00:00:00.000", "name": "Intro" } ],
|
||||
"attachments": [ { "url": "https://...", "name": "font.ttf", "mime_type": "font/ttf", "description": "" } ],
|
||||
"manifests": [ { "type": "dash", "url": "https://...", "data": "<base64 zlib xml>" } ],
|
||||
"session_headers": { "User-Agent": "..." },
|
||||
"session_cookies": { "sid": "..." },
|
||||
"server_cdm_type": "widevine"
|
||||
}
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Tracks returned. |
|
||||
| `400` | `INVALID_INPUT` | Missing or unknown `title_id`. |
|
||||
| `404` | `SESSION_NOT_FOUND` | No such session. |
|
||||
|
||||
### `POST /api/session/{session_id}/segments`
|
||||
|
||||
Resolve the download URL, DRM info, headers, cookies, and raw track data for selected tracks. Body: `{ "track_ids": ["v-1", "a-1"] }` (required).
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"tracks": {
|
||||
"v-1": { "descriptor": "DASH", "url": "https://...", "drm": [ ... ], "headers": { ... }, "cookies": { ... }, "data": { ... } }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | Segment info returned. |
|
||||
| `400` | `INVALID_INPUT` | Missing `track_ids`. |
|
||||
| `404` | `TRACK_NOT_FOUND` | An unknown track ID was requested. |
|
||||
|
||||
### `POST /api/session/{session_id}/license`
|
||||
|
||||
Acquire DRM keys. Two modes are supported via the `mode` field.
|
||||
|
||||
**Proxy mode** (`mode: "proxy"`, the default) forwards the client's CDM `challenge` to the service's license endpoint and returns the raw license bytes:
|
||||
|
||||
=== "Request"
|
||||
|
||||
```json
|
||||
{ "track_id": "v-1", "challenge": "<base64 CDM challenge>", "drm_type": "widevine" }
|
||||
```
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{ "license": "<base64 license bytes>" }
|
||||
```
|
||||
|
||||
**Server-CDM mode** (`mode: "server_cdm"`) runs the full CDM flow on the server using the device configured for your API key, returning content keys directly. Use `track_id` for a single track or `track_ids` for a batch:
|
||||
|
||||
=== "Single: Response `200`"
|
||||
|
||||
```json
|
||||
{ "keys": { "<kid_hex>": "<key_hex>" } }
|
||||
```
|
||||
|
||||
=== "Batch: Response `200`"
|
||||
|
||||
```json
|
||||
{ "keys": { "v-1": { "<kid_hex>": "<key_hex>" } }, "drm_type": "widevine" }
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
| --- | --- |
|
||||
| `track_id` / `track_ids` | Single track, or a batch of tracks. |
|
||||
| `challenge` | Base64 CDM challenge (proxy mode). |
|
||||
| `pssh` | Base64 PSSH (server-CDM mode). |
|
||||
| `drm_type` | `widevine` (default) or `playready`. |
|
||||
| `mode` | `proxy` (default) or `server_cdm`. |
|
||||
|
||||
| Status | Error code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `200` | — | License/keys returned. |
|
||||
| `400` | `INVALID_INPUT` | Missing challenge/PSSH, or no CDM device configured for the key. |
|
||||
| `400` | `INVALID_PARAMETERS` | Unsupported `drm_type`. |
|
||||
| `404` | `TRACK_NOT_FOUND` | Unknown track ID. |
|
||||
| `404` | `NO_CONTENT` | Server CDM produced no keys. |
|
||||
| `502` | `SERVICE_ERROR` | The service aborted during licensing. |
|
||||
|
||||
### `GET /api/session/{session_id}`
|
||||
|
||||
Return session metadata and validity.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "d4e5f6a7-...",
|
||||
"service": "EXAMPLE",
|
||||
"valid": true,
|
||||
"expires_in": 300,
|
||||
"track_count": 4,
|
||||
"title_count": 10
|
||||
}
|
||||
```
|
||||
|
||||
`404 SESSION_NOT_FOUND` if the session does not exist or has expired.
|
||||
|
||||
### `DELETE /api/session/{session_id}`
|
||||
|
||||
Tear down a session. Its input bridge is cancelled, any updated cache files are harvested, and the session plus its cache directory are deleted.
|
||||
|
||||
=== "Response `200`"
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"cache": { "tokens.json": "<base64 zlib bytes>" }
|
||||
}
|
||||
```
|
||||
|
||||
The `cache` field is present only if the session produced cache files. `404 SESSION_NOT_FOUND` if the session does not exist.
|
||||
|
||||
---
|
||||
|
||||
## Error responses
|
||||
|
||||
Every structured error raised inside a handler is serialized to the same JSON envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "error",
|
||||
"error_code": "INVALID_PARAMETERS",
|
||||
"message": "Invalid video codec: XYZ",
|
||||
"timestamp": "2026-07-03T12:00:00+00:00",
|
||||
"details": { "field": "vcodec" },
|
||||
"retryable": true,
|
||||
"debug_info": { "exception_type": "APIError", "traceback": "..." }
|
||||
}
|
||||
```
|
||||
|
||||
- `details` appears only when non-empty.
|
||||
- `retryable` appears only when the error is retryable (e.g. network errors, rate limiting).
|
||||
- `debug_info` appears only when the server was started with `--debug-api`.
|
||||
|
||||
!!! warning "Authentication errors differ"
|
||||
Errors raised by the API-key middleware use a distinct shape (`{"status": 401, "message": "Secret Key is Empty."}` or `{"status": 401, "message": "Secret Key is Invalid."}`) with an integer `status` and no `error_code`, `timestamp`, or `details`. A malformed JSON body on any POST, by contrast, returns the standard envelope with `error_code: "INVALID_INPUT"`.
|
||||
|
||||
### Error codes and default status
|
||||
|
||||
| Error code | HTTP status |
|
||||
| --- | --- |
|
||||
| `INVALID_INPUT` | 400 |
|
||||
| `INVALID_SERVICE` | 400 |
|
||||
| `INVALID_PROXY` | 400 |
|
||||
| `INVALID_PARAMETERS` | 400 |
|
||||
| `AUTH_FAILED` | 401 |
|
||||
| `FORBIDDEN` | 403 |
|
||||
| `GEOFENCE` | 403 |
|
||||
| `NOT_FOUND` | 404 |
|
||||
| `NO_CONTENT` | 404 |
|
||||
| `JOB_NOT_FOUND` | 404 |
|
||||
| `SESSION_NOT_FOUND` | 404 |
|
||||
| `TRACK_NOT_FOUND` | 404 |
|
||||
| `CONFLICT` | 409 |
|
||||
| `RATE_LIMITED` | 429 |
|
||||
| `INTERNAL_ERROR` | 500 |
|
||||
| `DOWNLOAD_ERROR` | 500 |
|
||||
| `WORKER_ERROR` | 500 |
|
||||
| `SERVICE_ERROR` | 502 |
|
||||
| `DRM_ERROR` | 502 |
|
||||
| `NETWORK_ERROR` | 503 |
|
||||
| `SERVICE_UNAVAILABLE` | 503 |
|
||||
|
||||
When a generic (uncategorized) exception reaches a handler, unshackle inspects its message and type to pick a code. For example, messages mentioning "connection" or "timeout" become `NETWORK_ERROR` (503, retryable), "rate limit" or "429" become `RATE_LIMITED` (429, retryable), and anything unrecognized collapses to `INTERNAL_ERROR` (500).
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- [REST API overview](index.md). What the server is and when to use it.
|
||||
- [Authentication](authentication.md). Keys, server modes, and the `X-Secret-Key` header.
|
||||
- [Quickstart](quickstart.md). A first end-to-end request walkthrough.
|
||||
- [Downloading](../../guide/downloading.md). The `dl` flags mirrored by `POST /api/download`.
|
||||
@@ -0,0 +1,254 @@
|
||||
# Error Responses
|
||||
|
||||
Every endpoint of the unshackle REST API reports failures in a single, predictable
|
||||
shape. Whether a request was rejected because a field was missing, a service refused
|
||||
to authenticate, or the download worker crashed, the server answers with a JSON body
|
||||
that carries a machine-readable **error code**, a human-readable **message**, and a
|
||||
matching **HTTP status**. This page documents that format and catalogues every error
|
||||
code the server can emit.
|
||||
|
||||
If you are building a client against the API, read this page alongside the endpoint
|
||||
reference. Each endpoint lists the specific codes it raises, and the meanings of
|
||||
those codes are defined here.
|
||||
|
||||
!!! note "Where this comes from"
|
||||
The error format is defined in `unshackle/core/api/errors.py`, and the wrapper
|
||||
that turns raised errors into responses lives in `unshackle/core/api/routes.py`.
|
||||
Nothing on this page is invented; it reflects the actual server behaviour.
|
||||
|
||||
## The standard error shape
|
||||
|
||||
A structured error is always a JSON object with `status` set to the literal string
|
||||
`"error"`. The full set of fields:
|
||||
|
||||
```json title="Error response body"
|
||||
{
|
||||
"status": "error",
|
||||
"error_code": "INVALID_PARAMETERS",
|
||||
"message": "vcodec must be one of: H264, H265, ...",
|
||||
"timestamp": "2026-07-03T12:34:56.789012+00:00",
|
||||
"details": { "field": "vcodec" },
|
||||
"retryable": true,
|
||||
"debug_info": {
|
||||
"exception_type": "APIError",
|
||||
"traceback": "..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Always present? | Meaning |
|
||||
|---|---|---|---|
|
||||
| `status` | string | Yes | Always the literal `"error"`. Lets a client distinguish an error body from a success body at a glance. |
|
||||
| `error_code` | string | Yes | A stable, machine-readable code from the [catalogue below](#error-code-catalogue). Match on this, not on `message`. |
|
||||
| `message` | string | Yes | A human-readable description of what went wrong. Wording may change between versions, so do not parse it. |
|
||||
| `timestamp` | string | Yes | ISO 8601 timestamp (UTC) of when the error response was built. |
|
||||
| `details` | object | Only when non-empty | Structured, error-specific context (for example the invalid field, a list of available titles, or the set of currently active jobs). |
|
||||
| `retryable` | boolean | Only when `true` | Present and set to `true` when retrying the same request may succeed later (network blips, rate limiting, transient outages). Absent otherwise. |
|
||||
| `debug_info` | object | Only with `--debug-api` | Technical diagnostics; see [Debug information](#debug-information). |
|
||||
|
||||
!!! tip "Match on `error_code`, branch on HTTP status"
|
||||
Treat `error_code` as the primary signal your code reacts to, and the HTTP status
|
||||
as a coarse fallback for anything you do not recognise. The two are consistent:
|
||||
every code maps to a default status (see the catalogue), and the pairing is
|
||||
stable across releases.
|
||||
|
||||
### The `details` object
|
||||
|
||||
When present, `details` carries extra context specific to the error. It is only
|
||||
included when it is non-empty, so a client must treat a missing `details` as "no
|
||||
extra information". Some concrete examples you may encounter:
|
||||
|
||||
- A validation failure may include the offending field or the list of valid values.
|
||||
- A "title not found" error on a remote-dl session includes `available_titles`.
|
||||
- A maintenance conflict includes `active_jobs`, the jobs blocking the operation.
|
||||
- Categorised exceptions add a `reason` tag, such as `"network_connectivity"` or
|
||||
`"geofence_restriction"`.
|
||||
|
||||
### The `retryable` hint
|
||||
|
||||
The `retryable` field appears **only when it is `true`**. It is set for conditions
|
||||
the server believes are transient: network errors, rate limiting, and temporary
|
||||
service unavailability. When you see it, backing off and retrying is reasonable; when
|
||||
it is absent, assume the request will keep failing until you change something.
|
||||
|
||||
!!! example "A retryable network error"
|
||||
```json
|
||||
{
|
||||
"status": "error",
|
||||
"error_code": "NETWORK_ERROR",
|
||||
"message": "Network error occurred: Connection timed out",
|
||||
"timestamp": "2026-07-03T12:00:00+00:00",
|
||||
"details": { "reason": "network_connectivity" },
|
||||
"retryable": true
|
||||
}
|
||||
```
|
||||
|
||||
## Error code catalogue
|
||||
|
||||
Every code below is a member of the `APIErrorCode` enumeration. The **Status** column
|
||||
is the default HTTP status the server returns for that code. A handler can override
|
||||
the status in special cases (for example, a network error is always forced to `503`),
|
||||
but the defaults hold for the overwhelming majority of responses.
|
||||
|
||||
### Client errors (4xx)
|
||||
|
||||
| Code | Status | Meaning |
|
||||
|---|---|---|
|
||||
| `INVALID_INPUT` | 400 | A required field is missing, or the request body could not be parsed. Also used as the generic "your input was malformed" code. |
|
||||
| `INVALID_SERVICE` | 400 | The named service is unknown, or is not permitted for your API key. |
|
||||
| `INVALID_PROXY` | 400 | The supplied proxy specification could not be resolved or is malformed. |
|
||||
| `INVALID_PARAMETERS` | 400 | One or more download or query parameters failed validation (bad codec, bitrate, sort field, and so on). |
|
||||
| `AUTH_FAILED` | 401 | Authentication with the streaming service failed (bad credentials or cookies). |
|
||||
| `FORBIDDEN` | 403 | The action is not allowed. Raised by server-side gates (per-key restrictions on CDM or credential overrides) and by session IP binding. |
|
||||
| `GEOFENCE` | 403 | The content is not available in the applicable region. |
|
||||
| `NOT_FOUND` | 404 | A requested resource (title, history entry, and so on) does not exist. |
|
||||
| `NO_CONTENT` | 404 | The request was valid but produced nothing: no matching titles, tracks, episodes, or keys. |
|
||||
| `JOB_NOT_FOUND` | 404 | The referenced download job does not exist. |
|
||||
| `SESSION_NOT_FOUND` | 404 | The remote-dl session does not exist or has expired. |
|
||||
| `TRACK_NOT_FOUND` | 404 | A referenced track ID is not present in the session. |
|
||||
| `CONFLICT` | 409 | The target resource is in a state that disallows the action (retrying a running job, prioritising a non-queued job, clearing cache during an active download). |
|
||||
| `RATE_LIMITED` | 429 | The streaming service is rate-limiting requests. Retryable. |
|
||||
|
||||
### Server errors (5xx)
|
||||
|
||||
| Code | Status | Meaning |
|
||||
|---|---|---|
|
||||
| `INTERNAL_ERROR` | 500 | An unexpected server-side error. This is the fallback for any exception the server could not categorise. |
|
||||
| `DOWNLOAD_ERROR` | 500 | The download process itself failed. |
|
||||
| `WORKER_ERROR` | 500 | The download worker subprocess errored out. |
|
||||
| `SERVICE_ERROR` | 502 | The streaming service's API returned an error, or the service does not support the requested operation. |
|
||||
| `DRM_ERROR` | 502 | DRM or license acquisition failed. Retryable. |
|
||||
| `NETWORK_ERROR` | 503 | A network connectivity problem (connection, timeout, DNS, TLS). Retryable. |
|
||||
| `SERVICE_UNAVAILABLE` | 503 | The service is temporarily unavailable or under maintenance. Retryable. |
|
||||
|
||||
!!! note "Fallback status"
|
||||
If a code is ever emitted without a mapped status, the server falls back to
|
||||
`500`. In practice every code above has an explicit mapping.
|
||||
|
||||
## How errors are produced
|
||||
|
||||
Understanding where an error comes from helps interpret it.
|
||||
|
||||
**Explicitly raised errors.** Most endpoints raise a structured `APIError` directly.
|
||||
For example, a missing `service` field yields `INVALID_INPUT`, and a disallowed
|
||||
service yields `INVALID_SERVICE`. These carry a precise code, message, and often a
|
||||
populated `details` object.
|
||||
|
||||
**Categorised exceptions.** When an endpoint calls into a streaming service and an
|
||||
uncategorised exception bubbles up, the server runs it through a classifier that
|
||||
inspects the exception type and message and maps it to the most appropriate code.
|
||||
The classification is keyword-based, checked in order:
|
||||
|
||||
| If the error mentions… | It becomes | Retryable |
|
||||
|---|---|---|
|
||||
| auth, login, credential, unauthorized, forbidden, token | `AUTH_FAILED` (401) | no |
|
||||
| connection, timeout, network, unreachable, socket, dns, resolve (or `ConnectionError`, `TimeoutError`, `URLError`, `SSLError`) | `NETWORK_ERROR` (503) | yes |
|
||||
| geofence, region, "not available in", territory | `GEOFENCE` (403) | no |
|
||||
| "not found", 404, "does not exist", "invalid id" | `NOT_FOUND` (404) | no |
|
||||
| rate limit, too many requests, 429, throttle | `RATE_LIMITED` (429) | yes |
|
||||
| drm, license, widevine, playready, decrypt | `DRM_ERROR` (502) | no |
|
||||
| "service unavailable", 503, maintenance, "temporarily unavailable" | `SERVICE_UNAVAILABLE` (503) | yes |
|
||||
| invalid, malformed, validation (or `ValueError`, `ValidationError`) | `INVALID_INPUT` (400) | no |
|
||||
| anything else | `INTERNAL_ERROR` (500) | no |
|
||||
|
||||
Categorised errors include a `reason` tag inside `details` (such as
|
||||
`"network_connectivity"` or `"drm_failure"`) so a client can react without parsing
|
||||
the message text.
|
||||
|
||||
**Invalid JSON bodies.** If a `POST` endpoint receives a body that is not valid JSON,
|
||||
the server returns `INVALID_INPUT` (400) with the parse error in
|
||||
`details.error`:
|
||||
|
||||
```json title="Malformed request body"
|
||||
{
|
||||
"status": "error",
|
||||
"error_code": "INVALID_INPUT",
|
||||
"message": "Invalid JSON request body",
|
||||
"timestamp": "2026-07-03T12:00:00+00:00",
|
||||
"details": { "error": "Expecting value: line 1 column 1 (char 0)" }
|
||||
}
|
||||
```
|
||||
|
||||
## The authentication error exception
|
||||
|
||||
There is one place where the response does **not** follow the standard shape: the API
|
||||
key authentication middleware. When you send a request to a protected endpoint
|
||||
without a valid `X-Secret-Key` header, the middleware short-circuits with a legacy
|
||||
body that uses an integer `status` and no `error_code` or `timestamp`.
|
||||
|
||||
=== "Missing key"
|
||||
|
||||
```json
|
||||
{ "status": 401, "message": "Secret Key is Empty." }
|
||||
```
|
||||
|
||||
=== "Invalid key"
|
||||
|
||||
```json
|
||||
{ "status": 401, "message": "Secret Key is Invalid." }
|
||||
```
|
||||
|
||||
!!! warning "Handle both shapes"
|
||||
A robust client must expect two error formats: the standard structured error
|
||||
(string `status` of `"error"`, with `error_code`), and this authentication error
|
||||
(integer `status`, no `error_code`). The `GET /api/health` endpoint is exempt
|
||||
from authentication, so it never returns the key-error shape.
|
||||
|
||||
## Responses with no body
|
||||
|
||||
Not every failure or completion has a JSON body. Some endpoints answer with
|
||||
**`204 No Content`** and an empty body, notably deleting a terminal download job and
|
||||
deleting a history entry.
|
||||
|
||||
!!! warning "Do not parse `204` bodies"
|
||||
A `204` response has no body. Clients must check the status code before attempting
|
||||
to decode JSON, or the decode will fail on an empty string.
|
||||
|
||||
## Debug information
|
||||
|
||||
By default, error responses never include tracebacks or internal diagnostics. If the
|
||||
server was started with the `--debug-api` flag, every structured error gains a
|
||||
`debug_info` object:
|
||||
|
||||
```json title="Error with --debug-api enabled"
|
||||
{
|
||||
"status": "error",
|
||||
"error_code": "INTERNAL_ERROR",
|
||||
"message": "An unexpected error occurred: ...",
|
||||
"timestamp": "2026-07-03T12:00:00+00:00",
|
||||
"debug_info": {
|
||||
"exception_type": "KeyError",
|
||||
"traceback": "Traceback (most recent call last): ..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `debug_info` object always carries `exception_type` and, for exception-derived
|
||||
errors, a formatted `traceback`. Some endpoints add extra diagnostics such as
|
||||
captured worker stderr.
|
||||
|
||||
!!! warning "Developer use only"
|
||||
`--debug-api` exposes internal tracebacks in responses. Enable it only for local
|
||||
debugging, never on a server reachable by untrusted clients.
|
||||
|
||||
## Client handling recommendations
|
||||
|
||||
For anyone writing a consumer of the API, a durable strategy is:
|
||||
|
||||
1. **Read the HTTP status first.** It tells you the broad class of outcome and lets
|
||||
you handle the `204`/no-body and authentication-error cases before touching the
|
||||
body.
|
||||
2. **Branch on `error_code`.** For structured errors, this is the stable identifier
|
||||
to switch on. Do not match on `message`.
|
||||
3. **Respect `retryable`.** When it is `true`, back off and retry; the reference
|
||||
client retries `429`, `500`, `502`, `503`, and `504` with exponential backoff. When
|
||||
absent, fix the request rather than repeating it.
|
||||
4. **Fall back gracefully.** Unrecognised codes should be surfaced using the
|
||||
`message` plus the HTTP status.
|
||||
|
||||
!!! note "Reference consumer"
|
||||
unshackle's own remote-download client treats any status of `400` or above as
|
||||
fatal: it logs `Server error [<error_code>]: <message>` and exits. It also wraps
|
||||
requests in a retry adapter (five attempts, backing off on `429`, `500`, `502`,
|
||||
`503`, and `504`). This is a reasonable template for your own client's error
|
||||
handling.
|
||||
@@ -0,0 +1,237 @@
|
||||
# REST API Overview
|
||||
|
||||
unshackle ships with a built-in HTTP server. Alongside the command-line tool, you
|
||||
can run `unshackle serve` and drive the same download engine over a REST API:
|
||||
searching services, listing titles and tracks, queueing downloads, watching live
|
||||
progress, and browsing history, all as JSON over HTTP.
|
||||
|
||||
This page explains what the API is for, how to start the server, where it listens,
|
||||
and how the endpoints are organised. Later pages cover each area in depth.
|
||||
|
||||
!!! info "Who this section is for"
|
||||
The REST API is aimed at people building automation, dashboards, or remote
|
||||
clients on top of unshackle. If you only download the occasional title, the
|
||||
[command-line tool](../../guide/cli-reference.md) does everything the API does
|
||||
and is simpler to use. Reach for the API when you want a long-running service, a
|
||||
job queue, or a web front-end.
|
||||
|
||||
## What the API is for
|
||||
|
||||
The server exposes the same capabilities as the `dl` command, but as discrete HTTP
|
||||
calls you can script against:
|
||||
|
||||
- **Discovery**: list the services available on the server, run a service's search,
|
||||
and enumerate the titles and tracks behind a title ID.
|
||||
- **Downloading**: submit a download as a background *job*, then poll its status and
|
||||
live progress, cancel it, retry it, or bump it to the front of the queue.
|
||||
- **History and housekeeping**: read the log of finished jobs, inspect the server's
|
||||
effective (redacted) configuration, check external tool versions, and clear caches
|
||||
or temp files.
|
||||
- **Remote-download sessions**: a thin local client can authenticate against a
|
||||
service *on the server*, then pull titles, tracks, and segment info back, and proxy
|
||||
DRM licensing. This is how one machine with the credentials and CDMs can serve
|
||||
downloads to another. See [Remote Sessions](remote-sessions.md).
|
||||
|
||||
Because every download runs as a queued job in a separate worker process, the API is
|
||||
well suited to running unattended: submit work, come back later, and read the result.
|
||||
|
||||
## Running the server
|
||||
|
||||
Start the server with the `serve` command:
|
||||
|
||||
```shell title="Start the REST API on the default host and port"
|
||||
unshackle serve
|
||||
```
|
||||
|
||||
By default the server binds to `127.0.0.1:8786` and requires an API key on every
|
||||
request. It also serves the local Widevine and PlayReady CDM endpoints (for
|
||||
pywidevine / pyplayready clients) in addition to the REST API.
|
||||
|
||||
!!! warning "An API key is required by default"
|
||||
Unless you pass `--no-key`, the server needs an API secret configured under the
|
||||
`serve` section of your config. If none is set, `unshackle serve` refuses to
|
||||
start. See [Authentication](authentication.md) for how to set `api_secret` and
|
||||
how the `X-Secret-Key` header works.
|
||||
|
||||
### Common options
|
||||
|
||||
The `serve` command accepts the following flags:
|
||||
|
||||
| Option | Default | Effect |
|
||||
|---|---|---|
|
||||
| `-h`, `--host` | `127.0.0.1` | Address to bind to. Use `0.0.0.0` to accept connections from other machines. |
|
||||
| `-p`, `--port` | `8786` | Port to bind to. |
|
||||
| `--api-only` | off | Serve only the REST API, without the Widevine/PlayReady CDM HTTP endpoints. |
|
||||
| `--no-widevine` | off | Disable the Widevine CDM endpoints. Cannot be combined with `--api-only`. |
|
||||
| `--no-playready` | off | Disable the PlayReady CDM endpoints. Cannot be combined with `--api-only`. |
|
||||
| `--no-key` | off | Disable API-key authentication entirely; every request is allowed. |
|
||||
| `--remote-only` | off | Expose only the remote-session subset of routes. Implies `--api-only`. |
|
||||
| `--debug-api` | off | Include tracebacks and worker stderr in error responses. |
|
||||
| `--debug` | off | Enable debug-level logging. |
|
||||
| `--caddy` | off | Also launch [Caddy](https://caddyserver.com/) as a reverse proxy, using the `Caddyfile` in your user-config directory. |
|
||||
|
||||
```shell title="Serve the REST API only, on all interfaces"
|
||||
unshackle serve --api-only --host 0.0.0.0 --port 8786
|
||||
```
|
||||
|
||||
!!! danger "Binding to a public interface"
|
||||
`--no-key` disables authentication completely. Anyone who can reach the port can
|
||||
search, download, and read your configuration. Only combine it with a host like
|
||||
`127.0.0.1`, or keep the default key-based auth when binding to `0.0.0.0`. Put a
|
||||
TLS-terminating reverse proxy (Caddy, nginx) in front for anything internet-facing.
|
||||
|
||||
### Server modes
|
||||
|
||||
The set of routes that is actually mounted depends on how you start the server:
|
||||
|
||||
=== "Default (CDM + REST)"
|
||||
|
||||
Both the pywidevine/pyplayready CDM endpoints and the full REST API are mounted.
|
||||
This is the mode used when you run `unshackle serve` with no mode flags.
|
||||
|
||||
=== "`--api-only`"
|
||||
|
||||
Only the REST API is mounted; the CDM HTTP endpoints are omitted. Authentication
|
||||
uses the single configured API key (or none, with `--no-key`).
|
||||
|
||||
=== "`--remote-only`"
|
||||
|
||||
Only the remote-session subset of routes is exposed (health, services, search,
|
||||
and everything under `/api/session/*`). This mode implies `--api-only`, and the
|
||||
interactive Swagger UI is **not** mounted.
|
||||
|
||||
## Base URL
|
||||
|
||||
All REST endpoints live under the `/api` prefix. With the default bind address the
|
||||
base URL is:
|
||||
|
||||
```
|
||||
http://127.0.0.1:8786/api
|
||||
```
|
||||
|
||||
Every path in this documentation is relative to the host and port. For example, the
|
||||
health check is `GET http://127.0.0.1:8786/api/health`.
|
||||
|
||||
!!! tip "Interactive API docs"
|
||||
Unless you started the server with `--remote-only`, an interactive Swagger UI is
|
||||
available at [`/api/docs/`](http://127.0.0.1:8786/api/docs/). It is generated from
|
||||
the same route table the server actually serves, so it stays in sync with the
|
||||
running build. Note that in `--api-only` mode the docs page is itself behind the
|
||||
API key; only `/api/health` is exempt from authentication.
|
||||
|
||||
A quick smoke test that needs no key:
|
||||
|
||||
```shell title="Confirm the server is up"
|
||||
curl http://127.0.0.1:8786/api/health
|
||||
```
|
||||
|
||||
```json title="Response"
|
||||
{
|
||||
"status": "ok",
|
||||
"version": "5.3.0",
|
||||
"update_check": {
|
||||
"update_available": false,
|
||||
"current_version": "5.3.0",
|
||||
"latest_version": "5.3.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Endpoint map
|
||||
|
||||
Every REST path uses JSON request and response bodies and lives under `/api`. The
|
||||
table below is a map of the surface; the [Endpoints](endpoints.md) page documents each
|
||||
request and response in full.
|
||||
|
||||
### Discovery
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `GET` | `/api/health` | Liveness and version check. Never requires a key. |
|
||||
| `GET` | `/api/services` | List the services available on this server. |
|
||||
| `POST` | `/api/search` | Run a service's search for a query. |
|
||||
| `POST` | `/api/list-titles` | List the titles behind a service title ID. |
|
||||
| `POST` | `/api/list-tracks` | List the video, audio, and subtitle tracks for a title. |
|
||||
|
||||
### Downloads and jobs
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `POST` | `/api/download` | Queue a download; returns a `job_id`. |
|
||||
| `GET` | `/api/download/jobs` | List jobs, with filtering and sorting. |
|
||||
| `POST` | `/api/download/jobs/clear-finished` | Remove all finished jobs. |
|
||||
| `GET` | `/api/download/jobs/{job_id}` | Full detail for one job. |
|
||||
| `DELETE` | `/api/download/jobs/{job_id}` | Cancel a running job, or remove a finished one. |
|
||||
| `POST` | `/api/download/jobs/{job_id}/retry` | Re-queue a finished job's parameters as a new job. |
|
||||
| `POST` | `/api/download/jobs/{job_id}/priority` | Move a queued job to the front. |
|
||||
|
||||
### History and server info
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `GET` | `/api/history` | Read the persisted log of finished jobs. |
|
||||
| `DELETE` | `/api/history/{job_id}` | Delete a history entry. |
|
||||
| `GET` | `/api/profiles` | Named credential profiles per service. |
|
||||
| `GET` | `/api/config` | The server's effective, redacted configuration. |
|
||||
| `GET` | `/api/env/check` | Presence and versions of external tools. |
|
||||
|
||||
### Maintenance
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `POST` | `/api/maintenance/clear-cache` | Clear the cache directory. |
|
||||
| `POST` | `/api/maintenance/clear-temp` | Clear the temp directory. |
|
||||
| `POST` | `/api/maintenance/refresh-services` | `git pull` each configured service repository. |
|
||||
|
||||
### Remote-download sessions
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `POST` | `/api/session/create` | Start a server-side, authenticated service session. |
|
||||
| `GET` | `/api/session/{session_id}` | Session info and time-to-live. |
|
||||
| `DELETE` | `/api/session/{session_id}` | Tear down a session. |
|
||||
| `GET` | `/api/session/{session_id}/titles` | Titles for the session's title ID. |
|
||||
| `POST` | `/api/session/{session_id}/tracks` | Tracks (with URLs) for a chosen title. |
|
||||
| `POST` | `/api/session/{session_id}/segments` | Per-track segment/download details. |
|
||||
| `POST` | `/api/session/{session_id}/license` | Proxy or server-side DRM licensing. |
|
||||
| `GET` | `/api/session/{session_id}/prompt` | Poll for interactive auth status/prompt. |
|
||||
| `POST` | `/api/session/{session_id}/prompt` | Submit an answer to an auth prompt. |
|
||||
|
||||
!!! note "Remote-only routes"
|
||||
The health, services, search, and `/api/session/*` routes are the ones exposed
|
||||
when the server runs with `--remote-only`. Everything else (downloads, jobs,
|
||||
history, config, maintenance) is only available in the full or `--api-only` modes.
|
||||
|
||||
## Authentication in one line
|
||||
|
||||
With the default configuration, send your API secret in the `X-Secret-Key` header on
|
||||
every request except `/api/health`:
|
||||
|
||||
```shell title="An authenticated request"
|
||||
curl -X POST http://127.0.0.1:8786/api/search \
|
||||
-H "X-Secret-Key: your-api-secret" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"service": "NF", "query": "example"}'
|
||||
```
|
||||
|
||||
A missing or unknown key returns `401`. The full model, including per-key service
|
||||
allowlists, is covered in [Authentication](authentication.md).
|
||||
|
||||
## Where to go next
|
||||
|
||||
<div class="grid cards" markdown>
|
||||
|
||||
- **[Quick Start](quickstart.md)**: start the server, set a key, and run your first
|
||||
end-to-end download over the API.
|
||||
- **[Authentication](authentication.md)**: the `X-Secret-Key` header, `--no-key`,
|
||||
per-key service allowlists, and CORS.
|
||||
- **[Endpoints](endpoints.md)**: every request body, query parameter, and response
|
||||
shape, endpoint by endpoint.
|
||||
- **[Job Lifecycle](job-lifecycle.md)**: how a download moves from `queued` to a
|
||||
terminal state, live progress, cancellation, and retries.
|
||||
- **[Remote Sessions](remote-sessions.md)**: the `/api/session/*` flow for driving a
|
||||
service from a thin remote client.
|
||||
- **[Error Responses](errors.md)**: the standard error envelope, error codes, and
|
||||
their HTTP statuses.
|
||||
|
||||
</div>
|
||||
@@ -0,0 +1,412 @@
|
||||
# Download Job Lifecycle
|
||||
|
||||
When you submit a download over the REST API, the server does not block the request
|
||||
until the file is written. Instead it creates a **job**, hands it back to you with an
|
||||
ID, and works through it in the background. Everything you do afterwards (watching
|
||||
progress, cancelling, retrying, reading the result) is done by referring to that job
|
||||
ID.
|
||||
|
||||
This page explains the states a job moves through, how live progress is reported, how
|
||||
to cancel or retry, and, in clearly marked developer sections, how the worker and
|
||||
manager behind the queue actually run the download.
|
||||
|
||||
!!! info "Who this page is for"
|
||||
The first half is for anyone driving the API: the states, the fields you can read,
|
||||
and the calls that move a job around. The **Under the hood** sections near the end
|
||||
describe the internal worker/manager model and are aimed at contributors and people
|
||||
debugging the server. You do not need them to use the API.
|
||||
|
||||
For the full request and response shape of each endpoint, see
|
||||
[Endpoints](endpoints.md). For how errors are reported, see
|
||||
[Error Responses](errors.md).
|
||||
|
||||
## The five states
|
||||
|
||||
A job is always in exactly one of five states, exposed as the `status` field:
|
||||
|
||||
| Status | Meaning |
|
||||
|---|---|
|
||||
| `queued` | Created and waiting for a free worker. No download has started yet. |
|
||||
| `downloading` | A worker has picked the job up and the download is in progress. |
|
||||
| `completed` | The download finished successfully; `output_files` lists the results. |
|
||||
| `failed` | The download stopped with an error; `error_message` and `error_code` explain why. |
|
||||
| `cancelled` | The job was cancelled before it could finish, from either the queued or downloading state. |
|
||||
|
||||
The last three (`completed`, `failed`, and `cancelled`) are **terminal**: once a
|
||||
job reaches one of them it never changes state again. Terminal jobs can be removed,
|
||||
retried, or read from history, but they will not resume.
|
||||
|
||||
## How a job moves between states
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> queued: POST /api/download
|
||||
queued --> downloading: a worker picks it up
|
||||
queued --> cancelled: DELETE (while queued)
|
||||
downloading --> completed: download succeeds
|
||||
downloading --> failed: download errors
|
||||
downloading --> cancelled: DELETE (while downloading)
|
||||
completed --> [*]
|
||||
failed --> [*]
|
||||
cancelled --> [*]
|
||||
|
||||
completed --> queued: POST .../retry (as a NEW job)
|
||||
failed --> queued: POST .../retry (as a NEW job)
|
||||
cancelled --> queued: POST .../retry (as a NEW job)
|
||||
```
|
||||
|
||||
The transitions in prose:
|
||||
|
||||
- **`queued` → `downloading`** happens automatically. As soon as one of the server's
|
||||
workers is free, it takes the next job off the queue and starts it. You do not
|
||||
trigger this; you only observe it.
|
||||
- **`downloading` → `completed`** is the happy path: the download engine wrote all the
|
||||
requested files and reported success.
|
||||
- **`downloading` → `failed`** is any error during the download: a service failure, a
|
||||
network problem, a DRM error, and so on. The job records what went wrong.
|
||||
- **`queued`/`downloading` → `cancelled`** happens when you send a `DELETE` for a job
|
||||
that has not finished (see [Cancelling a job](#cancelling-a-job)).
|
||||
- **Retry** does *not* move a terminal job back to `queued`. Instead it creates a
|
||||
brand-new job that reuses the old job's service, title, and parameters. The original
|
||||
job stays in its terminal state (see [Retrying a job](#retrying-a-job)).
|
||||
|
||||
!!! note "Priority does not change status"
|
||||
Moving a job to the front of the queue with the priority endpoint reorders *when*
|
||||
it will start, but it stays `queued` until a worker actually picks it up. It is not
|
||||
a state change.
|
||||
|
||||
## Creating a job
|
||||
|
||||
A download is queued with `POST /api/download`. The request needs at least a `service`
|
||||
and a `title_id`, plus any download options you want to set. On success the server
|
||||
replies with **`202 Accepted`** and the new job's identity:
|
||||
|
||||
```shell title="Queue a download"
|
||||
curl -X POST http://127.0.0.1:8786/api/download \
|
||||
-H "X-Secret-Key: your-api-secret" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"service": "NF",
|
||||
"title_id": "81234567",
|
||||
"quality": ["1080"],
|
||||
"range": ["HDR10"]
|
||||
}'
|
||||
```
|
||||
|
||||
```json title="202 Accepted"
|
||||
{
|
||||
"job_id": "b1e6c7de-6d0a-4f2f-9d1e-3a2b4c5d6e7f",
|
||||
"status": "queued",
|
||||
"created_time": "2026-07-03T12:00:00.000000"
|
||||
}
|
||||
```
|
||||
|
||||
The job starts in `queued`. Workers auto-start on the first job submitted, so there is
|
||||
nothing else to switch on. Keep the `job_id`; every later call is addressed to it.
|
||||
|
||||
!!! tip "The full parameter set"
|
||||
`POST /api/download` accepts the same download options as the `dl` command
|
||||
(quality, codecs, language selection, `wanted` episode ranges, and many more), with
|
||||
server-side defaults filling in anything you omit. The complete list, their
|
||||
defaults, and validation rules are documented on the [Endpoints](endpoints.md) page.
|
||||
|
||||
## Reading a job's status and progress
|
||||
|
||||
There are two ways to look at jobs.
|
||||
|
||||
=== "One job in full"
|
||||
|
||||
`GET /api/download/jobs/{job_id}` returns the complete detail for a single job,
|
||||
including parameters (with secrets redacted), timing, output files, and error
|
||||
information if it failed.
|
||||
|
||||
```shell
|
||||
curl http://127.0.0.1:8786/api/download/jobs/b1e6c7de-6d0a-4f2f-9d1e-3a2b4c5d6e7f \
|
||||
-H "X-Secret-Key: your-api-secret"
|
||||
```
|
||||
|
||||
=== "All jobs (summaries)"
|
||||
|
||||
`GET /api/download/jobs` lists every job the manager currently holds. By default
|
||||
each entry is a **summary**. Pass `full=true` to get the same full detail as the
|
||||
single-job endpoint for every job.
|
||||
|
||||
```shell
|
||||
curl "http://127.0.0.1:8786/api/download/jobs?status=downloading&sort_by=progress&sort_order=desc" \
|
||||
-H "X-Secret-Key: your-api-secret"
|
||||
```
|
||||
|
||||
The list can be filtered by `status` and `service`, and ordered with `sort_by`
|
||||
(`created_time`, `started_time`, `completed_time`, `progress`, `status`, or
|
||||
`service`) and `sort_order` (`asc` / `desc`). See [Endpoints](endpoints.md) for the
|
||||
query-parameter reference.
|
||||
|
||||
### The progress fields
|
||||
|
||||
While a job is `downloading`, the following fields update roughly twice a second so a
|
||||
client can poll and render live progress. They are present on both the summary and the
|
||||
full detail:
|
||||
|
||||
| Field | Type | Meaning |
|
||||
|---|---|---|
|
||||
| `progress` | number `0` to `100` | Overall percent complete. |
|
||||
| `phase` | string | Human-readable phase, e.g. `downloading video 1080p HDR10, audio en 5.1`. |
|
||||
| `title` | string | Title being processed. |
|
||||
| `current_title` | string | The specific episode/movie in progress (for a batch). |
|
||||
| `completed_tracks` | int | Number of tracks finished so far. |
|
||||
| `total_tracks` | int | Total tracks selected for the download. |
|
||||
| `active_tracks` | list of strings | Labels of the tracks downloading right now. |
|
||||
| `track_progress` | list | Per-active-track `{label, progress, speed}` entries. |
|
||||
| `segments_done` | number | Segments downloaded for the track in progress. |
|
||||
| `segments_total` | number | Total segments for the track in progress. |
|
||||
| `speed` | string | Transfer speed of the track in progress, e.g. `4.20 MB/s`. |
|
||||
| `skipped_subtitles` | list | Subtitles that were skipped as non-fatal failures (when `skip_subtitle_errors` is set). |
|
||||
|
||||
The full detail adds `started_time`, `completed_time`, `output_files`, the redacted
|
||||
`parameters`, and, for failed jobs, `error_message`, `error_details`, `error_code`,
|
||||
`error_traceback`, and `worker_stderr`.
|
||||
|
||||
!!! example "A typical polling loop"
|
||||
```shell
|
||||
JOB=b1e6c7de-6d0a-4f2f-9d1e-3a2b4c5d6e7f
|
||||
while true; do
|
||||
curl -s "http://127.0.0.1:8786/api/download/jobs/$JOB" \
|
||||
-H "X-Secret-Key: your-api-secret" \
|
||||
| python -c 'import sys,json; d=json.load(sys.stdin); print(d["status"], d["progress"], d.get("phase"))'
|
||||
sleep 1
|
||||
done
|
||||
```
|
||||
|
||||
!!! note "What the percentage means"
|
||||
`progress` climbs from `0` toward `90` while tracks download, then the final stretch
|
||||
to `100` covers post-download work such as decryption cleanup, repackaging, and
|
||||
muxing. A job that has finished downloading but is still muxing can sit near `90`
|
||||
for a moment before completing. On success the manager sets it to exactly `100`.
|
||||
|
||||
!!! warning "For a batch, `progress` is per-episode, not whole-batch"
|
||||
A multi-episode request (a `wanted` range) is a **single job** that processes its
|
||||
episodes sequentially. As each new episode begins, `progress` and the track counts
|
||||
(`completed_tracks` / `total_tracks`) **reset to zero** and count up again for that
|
||||
episode alone. The top-level bar therefore tracks the *current* episode, not the
|
||||
batch as a whole. To derive true overall completion, compare the growing
|
||||
`output_files` array against `parameters.wanted` rather than trusting `progress`.
|
||||
|
||||
## Terminal states in detail
|
||||
|
||||
### Completed
|
||||
|
||||
A `completed` job has its `output_files` populated with the paths of the muxed
|
||||
results, and `progress` is `100`. This is the state you are waiting for.
|
||||
|
||||
### Failed
|
||||
|
||||
A `failed` job carries the error. `error_message` is the short human message,
|
||||
`error_code` is one of the API's structured error codes (for example `NETWORK_ERROR`,
|
||||
`DRM_ERROR`, `AUTH_FAILED`, or `DOWNLOAD_ERROR`), and `error_details` gives more
|
||||
context. When the server was started with `--debug-api`, `error_traceback` and
|
||||
`worker_stderr` are also filled in for diagnosis.
|
||||
|
||||
!!! tip "Secrets are scrubbed from error text"
|
||||
A service can echo a credential or a proxy URL into its output. The server masks
|
||||
any known secret values out of `error_message`, `error_details`, `error_traceback`,
|
||||
and `worker_stderr` before returning them, so it is safe to log these fields.
|
||||
|
||||
The error-code catalogue and how each maps to an HTTP status is on the
|
||||
[Error Responses](errors.md) page.
|
||||
|
||||
### Cancelled
|
||||
|
||||
A `cancelled` job was stopped on request before finishing. It may have partial output,
|
||||
but it is not considered complete. See below.
|
||||
|
||||
## Cancelling a job
|
||||
|
||||
Send `DELETE /api/download/jobs/{job_id}`. The behaviour depends on the job's current
|
||||
state, and this is important because the response differs:
|
||||
|
||||
| Job state when you DELETE | What happens | Response |
|
||||
|---|---|---|
|
||||
| `queued` or `downloading` | The job is cancelled. A running download's worker process is terminated. | `200` with `{"status": "success", "message": "Job cancelled"}` |
|
||||
| `completed`, `failed`, or `cancelled` (terminal) | The job is **removed** from the manager entirely. | `204 No Content`, empty body |
|
||||
| Not found | — | `404` with error code `JOB_NOT_FOUND` |
|
||||
|
||||
In other words, `DELETE` on a live job *cancels* it, while `DELETE` on an already
|
||||
finished job *deletes it from the list*. This dual meaning is deliberate: one verb
|
||||
both stops in-flight work and tidies up afterwards.
|
||||
|
||||
!!! warning "Do not parse the body of a 204"
|
||||
Removing a terminal job returns `204 No Content` with an empty body. A client must
|
||||
not try to decode JSON from it; check the status code first.
|
||||
|
||||
```shell title="Cancel a running job"
|
||||
curl -X DELETE http://127.0.0.1:8786/api/download/jobs/b1e6c7de-6d0a-4f2f-9d1e-3a2b4c5d6e7f \
|
||||
-H "X-Secret-Key: your-api-secret"
|
||||
```
|
||||
|
||||
To remove *all* terminal jobs at once, use `POST /api/download/jobs/clear-finished`,
|
||||
which returns `{"removed": <count>}`.
|
||||
|
||||
## Retrying a job
|
||||
|
||||
`POST /api/download/jobs/{job_id}/retry` re-runs a finished job by creating a **new**
|
||||
job with the same service, title, and parameters. The original job is untouched and
|
||||
stays in its terminal state; the response gives you the new job's ID with `202`.
|
||||
|
||||
- Only terminal jobs can be retried. Retrying a `queued` or `downloading` job returns
|
||||
`409` with error code `CONFLICT`.
|
||||
- A missing job returns `404` (`JOB_NOT_FOUND`).
|
||||
- The service allowlist and any download gates are re-checked for the new job, exactly
|
||||
as they were for the original submission, so a retry can be rejected with
|
||||
`INVALID_SERVICE` or `FORBIDDEN` if permissions changed.
|
||||
|
||||
```json title="202 Accepted, a fresh job"
|
||||
{
|
||||
"job_id": "c2f7d8ef-7e1b-5a3a-ae2f-4b3c5d6e7f80",
|
||||
"status": "queued",
|
||||
"created_time": "2026-07-03T12:30:00.000000"
|
||||
}
|
||||
```
|
||||
|
||||
## Prioritising a job
|
||||
|
||||
`POST /api/download/jobs/{job_id}/priority` moves a **queued** job to the front of the
|
||||
line, so it will be the next one a free worker picks up.
|
||||
|
||||
- The job must be `queued`. Prioritising a job in any other state returns `409`
|
||||
(`CONFLICT`); a missing job returns `404`.
|
||||
- On success the response is `{"job_id": "<id>", "position": "front"}`.
|
||||
|
||||
This only reorders the queue; the job's status stays `queued` until a worker starts
|
||||
it.
|
||||
|
||||
## History and retention
|
||||
|
||||
Every job that reaches a terminal state is also written once to a persistent history
|
||||
log, so it survives even after the job is removed from the live list.
|
||||
|
||||
- Read it with `GET /api/history` (newest first; supports `limit` and `service`
|
||||
filters).
|
||||
- Delete a single entry with `DELETE /api/history/{job_id}` (returns `204`).
|
||||
|
||||
The live in-memory jobs are separate from this on-disk history. A background cleanup
|
||||
task periodically drops old terminal jobs from the live list based on a retention
|
||||
period; the history file keeps them for later reference. See [Endpoints](endpoints.md)
|
||||
for the history response shape.
|
||||
|
||||
!!! note "Why the history file lives under the cache directory"
|
||||
`api_history.jsonl` is placed under the cache directory on purpose, so it stays out
|
||||
of the config/data tree. It is disposable runtime state, not user configuration or
|
||||
curated data, which is exactly why clearing the cache is a safe operation against
|
||||
it.
|
||||
|
||||
---
|
||||
|
||||
## Under the hood: the worker/manager model
|
||||
|
||||
!!! info "Developer section"
|
||||
Everything below describes server internals. It is useful for contributors and for
|
||||
anyone debugging the server, but it is not needed to *use* the API.
|
||||
|
||||
### The manager and its queue
|
||||
|
||||
A single process-wide **download queue manager** owns all jobs. It holds them in
|
||||
memory keyed by ID, backs the wait list with a FIFO queue, and keeps a separate
|
||||
priority deque for jobs that have been bumped to the front. On the first job, it starts
|
||||
a fixed pool of worker tasks plus one cleanup task; starting is idempotent, so later
|
||||
jobs reuse the same pool.
|
||||
|
||||
- **Concurrency** is capped by `max_concurrent_downloads` (default **2**). That many
|
||||
jobs can be in `downloading` at once; the rest wait in `queued`.
|
||||
- **Retention** is governed by `job_retention_hours` (default **24**). An hourly
|
||||
cleanup task removes terminal jobs whose completion time (or, if never started,
|
||||
creation time) is older than the retention window.
|
||||
|
||||
!!! note "Two concurrency knobs that are easy to conflate"
|
||||
`max_concurrent_downloads` caps how many separate download **jobs** run at once.
|
||||
The separate `serve.downloads` setting controls how many **tracks** download in
|
||||
parallel *within* a single job. They are independent, and raising one does not affect
|
||||
the other.
|
||||
|
||||
!!! note "Concurrency and retention config keys"
|
||||
The manager reads these as top-level config attributes
|
||||
(`config.max_concurrent_downloads` and `config.download_job_retention_hours`).
|
||||
Those keys are not part of the standard config schema, so in practice the built-in
|
||||
defaults of **2** and **24** apply. The live values are surfaced by
|
||||
[`GET /api/config`](endpoints.md). The related `serve.history_limit` (default
|
||||
**100**) instead controls how many entries the on-disk history file keeps.
|
||||
|
||||
Each worker loop drains the priority deque first, then the FIFO queue. A job that was
|
||||
promoted has its stale FIFO copy skipped so it does not run twice. When a worker takes
|
||||
a job it flips it to `downloading`, stamps `started_time`, runs it, and on finish
|
||||
stamps `completed_time` and records history.
|
||||
|
||||
### Each job runs in its own subprocess
|
||||
|
||||
The manager does **not** run the download in the server's event loop or even in the
|
||||
server process. For each job it spawns a separate Python subprocess:
|
||||
|
||||
```text
|
||||
python -m unshackle.core.api.download_worker <payload.json> <result.json> <progress.json>
|
||||
```
|
||||
|
||||
Three temporary files bridge the parent and the worker:
|
||||
|
||||
- **`payload.json`**: written by the manager before launch; contains the job ID,
|
||||
service, title ID, and merged parameters.
|
||||
- **`progress.json`**: written continuously by the worker as the download proceeds.
|
||||
- **`result.json`**: written by the worker at the end.
|
||||
|
||||
Running each download out-of-process isolates the server from crashes, memory growth,
|
||||
and blocking work inside the download engine, and makes hard cancellation (killing the
|
||||
process) reliable.
|
||||
|
||||
The worker's exit codes are `0` for success, `1` for a download error, and `2` for bad
|
||||
arguments. Its `result.json` is either `{"status": "success", "output_files": [...]}`
|
||||
or `{"status": "error", "message", "error_details", "error_code", "traceback"}`. If the
|
||||
process exits non-zero, or the result is missing or not a success, the parent marks the
|
||||
job `failed` and adopts the worker's `error_code` (or derives one by categorising the
|
||||
exception).
|
||||
|
||||
### How progress reaches the API
|
||||
|
||||
While the worker runs, the manager polls `progress.json` every **0.5 s** and copies the
|
||||
fields it finds onto the live `DownloadJob`. That is what the `GET` endpoints return.
|
||||
|
||||
Inside the worker, progress is aggregated across all selected tracks. Each track's
|
||||
completion fraction is weighted by its bitrate, so video and audio dominate the bar and
|
||||
subtitles barely move it (subtitles get a small fixed weight). Track fractions are kept
|
||||
**monotonic**: the percentage only ever climbs. The download portion is scaled to fill
|
||||
`0` to `90`; the remaining `90` to `100` is driven by the download engine as it repackages
|
||||
and muxes. A track counts as done when it reaches a terminal state such as
|
||||
`Downloaded`, `Decrypted`, or a skipped subtitle.
|
||||
|
||||
### How cancellation actually works
|
||||
|
||||
Each job carries a cancellation event. When you `DELETE` a live job:
|
||||
|
||||
- If it is **`queued`**, it flips straight to `cancelled`, its history is recorded
|
||||
immediately (it never reaches a worker), and the worker loop skips it if it later
|
||||
pops it off the queue.
|
||||
- If it is **`downloading`**, the manager sets the cancellation event and flips the job
|
||||
to `cancelled`. The polling loop notices the event, `terminate()`s the worker
|
||||
subprocess (escalating to `kill()` if it does not exit within 5 seconds), and raises
|
||||
a cancellation so the task unwinds. The subprocess model is what makes this a clean,
|
||||
immediate stop rather than a cooperative one.
|
||||
|
||||
### Secret handling in serialized jobs
|
||||
|
||||
Job parameters may carry secrets: a raw `user:pass` credential, or a proxy URL with
|
||||
embedded userinfo. Whenever the manager serializes a job (for an API response or the
|
||||
history file), it masks the sensitive keys (`credential`, `credentials`, `password`,
|
||||
`token`, `api_key`) and strips userinfo from proxy URLs. Free-text fields that a
|
||||
service might have leaked a secret into (`error_message`, `error_details`,
|
||||
`error_traceback`, `worker_stderr`, and worker stdout/stderr in the logs) are scrubbed
|
||||
of the job's known secret values, longest-match first, before they are ever returned or
|
||||
logged.
|
||||
|
||||
## See also
|
||||
|
||||
- [Endpoints](endpoints.md): the full request/response reference for every job endpoint.
|
||||
- [Error Responses](errors.md): the error envelope, codes, and HTTP statuses a failed job reports.
|
||||
- [Authentication](authentication.md): the `X-Secret-Key` header and per-key service allowlists that gate job creation and retry.
|
||||
- [Downloading](../../guide/downloading.md): the equivalent options on the `dl` command line.
|
||||
@@ -0,0 +1,437 @@
|
||||
# REST API Quick Start
|
||||
|
||||
Unshackle ships with a built-in REST API that exposes the same download engine you drive from the command line. It is served by the `unshackle serve` command and lets you queue downloads, poll their progress, browse services, and manage history over HTTP instead of the terminal.
|
||||
|
||||
The full loop runs end to end: start the server, authenticate, submit a download job, and poll it to completion, using real `curl` commands built directly from the API routes.
|
||||
|
||||
!!! note "Who this page is for"
|
||||
This is a **developer** guide. It assumes you are comfortable with HTTP, JSON, and a tool like `curl`. If you just want to download something, the [command line](../../index.md) is simpler. The REST API is meant for building tools, UIs, and automation on top of unshackle.
|
||||
|
||||
---
|
||||
|
||||
## Before you begin
|
||||
|
||||
You need a working unshackle install with services and CDM devices already configured, exactly as you would for a normal command-line download. If `unshackle dl` works for you, the API will too, since it runs the identical download pipeline.
|
||||
|
||||
=== "Installed as a tool"
|
||||
|
||||
```shell
|
||||
uv tool install git+https://github.com/unshackle-dl/unshackle.git
|
||||
unshackle --help
|
||||
```
|
||||
|
||||
=== "From a clone"
|
||||
|
||||
```shell
|
||||
uv run unshackle --help
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Configure an API secret
|
||||
|
||||
Every request (except the health check) is authenticated with a secret key sent in the `X-Secret-Key` header. The server reads this key from the `serve` section of your `unshackle.yaml` config.
|
||||
|
||||
At minimum you must set `api_secret`:
|
||||
|
||||
```yaml title="unshackle.yaml"
|
||||
serve:
|
||||
api_secret: "choose-a-long-random-secret"
|
||||
```
|
||||
|
||||
If you want the API to be able to license and download DRM-protected content, you also need to grant CDM devices to the key. Configure that under `serve.users`, keyed by the secret:
|
||||
|
||||
```yaml title="unshackle.yaml"
|
||||
serve:
|
||||
api_secret: "choose-a-long-random-secret"
|
||||
users:
|
||||
choose-a-long-random-secret:
|
||||
devices: ["your_widevine_device"] # Widevine .wvd stems
|
||||
playready_devices: ["your_playready_device"] # PlayReady .prd stems
|
||||
username: api_user
|
||||
```
|
||||
|
||||
!!! warning "The secret must exist"
|
||||
If `api_secret` is not set and you do **not** pass `--no-key`, `unshackle serve` refuses to start and raises an error. Treat the secret like a password. Anyone who has it can queue downloads on your machine.
|
||||
|
||||
!!! tip "Disabling auth for local testing"
|
||||
Passing `--no-key` to `serve` turns authentication off entirely: every request is allowed and the `X-Secret-Key` header is ignored. This is convenient for a throwaway local test, but never expose a `--no-key` server to a network you do not fully trust.
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Start the server
|
||||
|
||||
Start the REST API with the `serve` command:
|
||||
|
||||
```shell
|
||||
unshackle serve --api-only
|
||||
```
|
||||
|
||||
By default the server binds to `127.0.0.1:8786`. The `--api-only` flag serves just the REST API and omits the Widevine/PlayReady CDM HTTP endpoints; drop it if you also want to expose those.
|
||||
|
||||
Common options:
|
||||
|
||||
| Option | Default | Effect |
|
||||
| --- | --- | --- |
|
||||
| `-h, --host` | `127.0.0.1` | Address to bind to |
|
||||
| `-p, --port` | `8786` | Port to bind to |
|
||||
| `--api-only` | off | Serve only the REST API, no CDM endpoints |
|
||||
| `--no-key` | off | Disable API-key authentication (allow all requests) |
|
||||
| `--debug-api` | off | Include tracebacks and stderr in error responses |
|
||||
| `--debug` | off | Enable DEBUG-level logging |
|
||||
| `--remote-only` | off | Expose only the remote-session subset of routes |
|
||||
| `--caddy` | off | Also launch Caddy as a reverse proxy (for HTTPS) |
|
||||
|
||||
Once it is up you will see the API mounted under `/api/` and, unless you started in remote-only mode, an interactive Swagger UI:
|
||||
|
||||
```text
|
||||
REST API endpoints available at http://127.0.0.1:8786/api/
|
||||
Swagger UI available at http://127.0.0.1:8786/api/docs/
|
||||
```
|
||||
|
||||
!!! note "Swagger UI is also behind auth"
|
||||
In `--api-only` mode (without `--no-key`), the Swagger UI at `/api/docs/` requires the `X-Secret-Key` header just like every other route. Only `/api/health` is exempt from authentication.
|
||||
|
||||
For the rest of this page we will assume the server is reachable at `http://127.0.0.1:8786` and store the secret in a shell variable:
|
||||
|
||||
```shell
|
||||
export SECRET="choose-a-long-random-secret"
|
||||
export BASE="http://127.0.0.1:8786"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Confirm the server is alive
|
||||
|
||||
The health endpoint is the one route that never requires a key. Use it to confirm the server is running and to read its version:
|
||||
|
||||
```shell
|
||||
curl "$BASE/api/health"
|
||||
```
|
||||
|
||||
```json title="200 OK"
|
||||
{
|
||||
"status": "ok",
|
||||
"version": "5.3.0",
|
||||
"update_check": {
|
||||
"update_available": false,
|
||||
"current_version": "5.3.0",
|
||||
"latest_version": "5.3.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `update_check` block is best effort. If unshackle cannot reach the update source, `update_available` and `latest_version` come back as `null` rather than failing the request.
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Authenticate a real request
|
||||
|
||||
Every other endpoint needs the `X-Secret-Key` header. A quick way to prove your key works is to list the services the server can see:
|
||||
|
||||
```shell
|
||||
curl "$BASE/api/services" \
|
||||
-H "X-Secret-Key: $SECRET"
|
||||
```
|
||||
|
||||
```json title="200 OK (truncated)"
|
||||
{
|
||||
"services": [
|
||||
{
|
||||
"tag": "EXAMPLE",
|
||||
"aliases": ["EX"],
|
||||
"geofence": [],
|
||||
"title_regex": null,
|
||||
"url": "https://example.com",
|
||||
"help": "EXAMPLE service documentation...",
|
||||
"needs_auth": true,
|
||||
"has_search": true,
|
||||
"has_drm": true,
|
||||
"auth_methods": ["cookies", "credentials"],
|
||||
"cli_params": []
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
If the key is missing or wrong, you get a `401` with a plain body:
|
||||
|
||||
```json title="401 Unauthorized"
|
||||
{ "status": 401, "message": "Secret Key is Invalid." }
|
||||
```
|
||||
|
||||
!!! warning "Two error shapes exist"
|
||||
The authentication middleware returns the short `{"status": 401, "message": "..."}` shape shown above. Every **other** error the API produces uses the richer structured shape (`error_code`, `timestamp`, `details`, …) described in [Error handling](#error-handling). A robust client should handle both.
|
||||
|
||||
The `tag` value for the service you want is the identifier you pass as `service` in the calls below.
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Submit a download job
|
||||
|
||||
Downloads are asynchronous. You `POST` a job to `/api/download`, the server immediately hands back a `job_id`, and the actual download runs in the background so your request never blocks.
|
||||
|
||||
The only required fields are `service` and `title_id`. Everything else mirrors the flags of the `dl` command and has a sensible default, so a minimal job looks like this:
|
||||
|
||||
```shell
|
||||
curl -X POST "$BASE/api/download" \
|
||||
-H "X-Secret-Key: $SECRET" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"service": "EXAMPLE",
|
||||
"title_id": "12345"
|
||||
}'
|
||||
```
|
||||
|
||||
A more realistic job pins the quality and codecs and asks for specific episodes:
|
||||
|
||||
```shell
|
||||
curl -X POST "$BASE/api/download" \
|
||||
-H "X-Secret-Key: $SECRET" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"service": "EXAMPLE",
|
||||
"title_id": "12345",
|
||||
"quality": [1080],
|
||||
"vcodec": "H265",
|
||||
"range": ["SDR"],
|
||||
"wanted": ["S01E01", "S01E02"]
|
||||
}'
|
||||
```
|
||||
|
||||
On success the server responds with **`202 Accepted`** and the identifier you will poll:
|
||||
|
||||
```json title="202 Accepted"
|
||||
{
|
||||
"job_id": "3f9c1e0a-8d2b-4e6f-9a1c-77b0f2a1c4d5",
|
||||
"status": "queued",
|
||||
"created_time": "2026-07-03T12:00:00.000000+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
Capture the `job_id` for the next step:
|
||||
|
||||
```shell
|
||||
JOB=$(curl -s -X POST "$BASE/api/download" \
|
||||
-H "X-Secret-Key: $SECRET" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"service":"EXAMPLE","title_id":"12345","quality":[1080]}' \
|
||||
| python -c "import sys, json; print(json.load(sys.stdin)['job_id'])")
|
||||
```
|
||||
|
||||
### Frequently used download fields
|
||||
|
||||
The request body accepts the full download parameter set. A few of the most useful:
|
||||
|
||||
| Field | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `service` | — | Service tag (required) |
|
||||
| `title_id` | — | Title identifier (required) |
|
||||
| `profile` | `null` | Named credential/cookie profile to use |
|
||||
| `quality` | `[]` | Resolution(s), e.g. `[1080]`; empty means best available |
|
||||
| `vcodec` | `null` | Video codec, e.g. `"H265"` |
|
||||
| `acodec` | `null` | Audio codec, e.g. `"EC3"` |
|
||||
| `range` | `["SDR"]` | Dynamic range: `SDR`, `HDR10`, `HDR10+`, `DV`, `HLG`, `HYBRID` |
|
||||
| `wanted` | `[]` | Episodes to fetch, e.g. `["S01E01"]`; empty means all |
|
||||
| `lang` | `["orig"]` | Video/audio language(s) |
|
||||
| `s_lang` | `["all"]` | Subtitle language(s) |
|
||||
| `downloads` | `1` | Number of tracks to download concurrently |
|
||||
| `proxy` | `null` | Proxy URI or country code |
|
||||
|
||||
Validation is strict: invalid codecs, subtitle formats, dynamic-range names, non-positive counts, or mutually exclusive flags (for example both `video_only` and `audio_only`) come back as `400` with error code `INVALID_PARAMETERS`. See the full reference on the [download reference page](endpoints.md) for every accepted field.
|
||||
|
||||
!!! note "Gated fields"
|
||||
Passing a per-request CDM (`cdm`) or inline credentials (`credential` / `credentials`) is refused with `403 FORBIDDEN` unless the server operator has explicitly opted in via `serve.cdm_overrides` and `serve.allow_job_credentials`. Both are off by default.
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Poll the job to completion
|
||||
|
||||
A job moves through these states:
|
||||
|
||||
```text
|
||||
queued → downloading → completed | failed | cancelled
|
||||
```
|
||||
|
||||
The last three are **terminal**. Poll the job detail endpoint until you reach one of them. This endpoint always returns the full job record:
|
||||
|
||||
```shell
|
||||
curl "$BASE/api/download/jobs/$JOB" \
|
||||
-H "X-Secret-Key: $SECRET"
|
||||
```
|
||||
|
||||
While the download is running you will see `status: "downloading"` and a `progress` value climbing from `0` toward `100`:
|
||||
|
||||
```json title="200 OK (in progress)"
|
||||
{
|
||||
"job_id": "3f9c1e0a-8d2b-4e6f-9a1c-77b0f2a1c4d5",
|
||||
"status": "downloading",
|
||||
"progress": 42.7,
|
||||
"phase": "downloading",
|
||||
"service": "EXAMPLE",
|
||||
"title_id": "12345",
|
||||
"title": "Example Show S01E01",
|
||||
"current_title": "Example Show S01E01",
|
||||
"completed_tracks": 1,
|
||||
"total_tracks": 3,
|
||||
"segments_done": 512,
|
||||
"segments_total": 1200,
|
||||
"speed": "8.4 MiB/s",
|
||||
"created_time": "2026-07-03T12:00:00.000000+00:00",
|
||||
"started_time": "2026-07-03T12:00:03.000000+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
When it finishes you will see `status: "completed"` and the produced files under `output_files`:
|
||||
|
||||
```json title="200 OK (completed)"
|
||||
{
|
||||
"job_id": "3f9c1e0a-8d2b-4e6f-9a1c-77b0f2a1c4d5",
|
||||
"status": "completed",
|
||||
"progress": 100.0,
|
||||
"output_files": ["/path/to/downloads/Example Show S01E01.mkv"],
|
||||
"completed_time": "2026-07-03T12:04:31.000000+00:00"
|
||||
}
|
||||
```
|
||||
|
||||
A simple polling loop in shell:
|
||||
|
||||
```shell title="poll.sh"
|
||||
while true; do
|
||||
STATUS=$(curl -s "$BASE/api/download/jobs/$JOB" \
|
||||
-H "X-Secret-Key: $SECRET" \
|
||||
| python -c "import sys, json; print(json.load(sys.stdin)['status'])")
|
||||
echo "job status: $STATUS"
|
||||
case "$STATUS" in
|
||||
completed|failed|cancelled) break ;;
|
||||
esac
|
||||
sleep 3
|
||||
done
|
||||
```
|
||||
|
||||
!!! tip "Polling cadence"
|
||||
A few seconds between polls is plenty. The server writes progress updates roughly twice a second internally, but there is no push/streaming channel; you read the current snapshot each time you ask.
|
||||
|
||||
If a job ends in `failed`, the full record includes `error_message`, `error_code`, and (when the server was started with `--debug-api`) `error_traceback` and `worker_stderr` to help you diagnose it. Sensitive values in these fields are scrubbed before they are returned.
|
||||
|
||||
---
|
||||
|
||||
## Step 7: Clean up
|
||||
|
||||
Once you have collected the output you can remove the finished job. `DELETE` on a job has two behaviors depending on its state:
|
||||
|
||||
```shell
|
||||
curl -X DELETE "$BASE/api/download/jobs/$JOB" \
|
||||
-H "X-Secret-Key: $SECRET"
|
||||
```
|
||||
|
||||
- If the job is **terminal** (completed/failed/cancelled) it is removed from the manager and you get **`204 No Content`** with an empty body.
|
||||
- If the job is still **queued or downloading** it is cancelled instead, and you get `200` with `{"status": "success", "message": "Job cancelled"}`.
|
||||
|
||||
!!! warning "Do not parse JSON on 204"
|
||||
A `204` response has no body. This applies to terminal-job removal here and to deleting history entries; check the status code before attempting to decode JSON.
|
||||
|
||||
To clear every finished job at once instead of one at a time:
|
||||
|
||||
```shell
|
||||
curl -X POST "$BASE/api/download/jobs/clear-finished" \
|
||||
-H "X-Secret-Key: $SECRET"
|
||||
```
|
||||
|
||||
```json title="200 OK"
|
||||
{ "removed": 3 }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Managing the queue
|
||||
|
||||
A handful of endpoints round out job control. All require the `X-Secret-Key` header.
|
||||
|
||||
| Method | Path | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `GET` | `/api/download/jobs` | List all jobs, with optional filtering and sorting |
|
||||
| `GET` | `/api/download/jobs/{job_id}` | Full detail for one job |
|
||||
| `DELETE` | `/api/download/jobs/{job_id}` | Cancel a running job, or remove a terminal one |
|
||||
| `POST` | `/api/download/jobs/{job_id}/retry` | Re-queue a new job from a terminal one |
|
||||
| `POST` | `/api/download/jobs/{job_id}/priority` | Move a queued job to the front of the queue |
|
||||
| `POST` | `/api/download/jobs/clear-finished` | Remove all terminal jobs |
|
||||
|
||||
Listing supports query parameters for filtering by `status` or `service`, sorting via `sort_by` (`created_time`, `started_time`, `completed_time`, `progress`, `status`, `service`) and `sort_order` (`asc` / `desc`), and requesting full per-job detail with `full=true`:
|
||||
|
||||
```shell
|
||||
curl "$BASE/api/download/jobs?status=downloading&sort_by=progress&sort_order=desc" \
|
||||
-H "X-Secret-Key: $SECRET"
|
||||
```
|
||||
|
||||
```json title="200 OK"
|
||||
{
|
||||
"jobs": [
|
||||
{
|
||||
"job_id": "3f9c1e0a-8d2b-4e6f-9a1c-77b0f2a1c4d5",
|
||||
"status": "downloading",
|
||||
"service": "EXAMPLE",
|
||||
"title_id": "12345",
|
||||
"title": "Example Show S01E01",
|
||||
"progress": 42.7,
|
||||
"phase": "downloading",
|
||||
"completed_tracks": 1,
|
||||
"total_tracks": 3
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Retrying reuses the original job's service, title, and parameters to enqueue a brand new job (returning `202` with a fresh `job_id`); it only works on terminal jobs and returns `409 CONFLICT` otherwise. Prioritizing only works on jobs still in the `queued` state.
|
||||
|
||||
---
|
||||
|
||||
## Beyond a single download
|
||||
|
||||
The download loop above is the core of the API, but the same server exposes more:
|
||||
|
||||
- **Discovery**: `POST /api/search` finds titles by query, and `POST /api/list-titles` / `POST /api/list-tracks` inspect what a title offers before you commit to downloading it. See [list titles and tracks](endpoints.md).
|
||||
- **History**: `GET /api/history` reads a persisted log of jobs that reached a terminal state (newest first), and `DELETE /api/history/{job_id}` removes a single entry. See [history](endpoints.md).
|
||||
- **Introspection**: `GET /api/profiles` lists configured credential profiles per service, `GET /api/config` returns a redacted view of the effective server config, and `GET /api/env/check` reports which external binaries are installed.
|
||||
- **Maintenance**: `POST /api/maintenance/clear-cache`, `.../clear-temp`, and `.../refresh-services` perform housekeeping (the cache/temp clears are blocked with `409` while a download is active).
|
||||
- **Remote sessions**: the `/api/session/*` endpoints drive the thin-client "remote download" flow, where the server authenticates and licenses on behalf of a local client. See [remote sessions](remote-sessions.md).
|
||||
|
||||
The interactive **Swagger UI at `/api/docs/`** documents every route and lets you try them in the browser (send your `X-Secret-Key` there too).
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
Apart from the auth middleware's short form, every error the API returns uses a single structured envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "error",
|
||||
"error_code": "INVALID_PARAMETERS",
|
||||
"message": "Human-readable description",
|
||||
"timestamp": "2026-07-03T12:00:00Z",
|
||||
"details": { }
|
||||
}
|
||||
```
|
||||
|
||||
`details` appears only when there is something to add, `retryable: true` appears only for transient failures, and a `debug_info` block (exception type and traceback) is included only when the server runs with `--debug-api`.
|
||||
|
||||
The `error_code` maps to an HTTP status. The ones you will meet most often:
|
||||
|
||||
| Error code | HTTP | Typical cause |
|
||||
| --- | --- | --- |
|
||||
| `INVALID_INPUT` | 400 | Malformed JSON body or missing required field |
|
||||
| `INVALID_SERVICE` | 400 | Unknown or disallowed service tag |
|
||||
| `INVALID_PARAMETERS` | 400 | A download field failed validation |
|
||||
| `AUTH_FAILED` | 401 | Service login/credential failure |
|
||||
| `FORBIDDEN` | 403 | A gated field was used without server opt-in |
|
||||
| `NOT_FOUND` | 404 | Title or resource does not exist |
|
||||
| `JOB_NOT_FOUND` | 404 | No job with that `job_id` |
|
||||
| `CONFLICT` | 409 | Illegal state transition (e.g. retry a running job) |
|
||||
| `RATE_LIMITED` | 429 | Upstream throttling (retryable) |
|
||||
| `INTERNAL_ERROR` | 500 | Unexpected server-side failure |
|
||||
| `SERVICE_ERROR` | 502 | The service raised an error |
|
||||
| `NETWORK_ERROR` | 503 | Connection/timeout reaching the service (retryable) |
|
||||
|
||||
!!! tip "Retry politely"
|
||||
Codes marked retryable (`NETWORK_ERROR`, `RATE_LIMITED`, `SERVICE_UNAVAILABLE`) reflect transient conditions. Back off and retry rather than hammering the server.
|
||||
@@ -0,0 +1,431 @@
|
||||
# Remote Service Sessions
|
||||
|
||||
A **remote service session** lets one machine do the hard, service-specific work
|
||||
(authenticating to a streaming service, listing titles, resolving tracks and
|
||||
manifests, and, optionally, DRM licensing) while another machine runs the actual
|
||||
download, decryption, and muxing locally.
|
||||
|
||||
This is the mechanism behind unshackle's client/server "remote download" workflow.
|
||||
A thin local client (`RemoteService`) talks to an unshackle `serve` instance over
|
||||
HTTP. The server keeps an authenticated service instance alive between requests so
|
||||
the client can authenticate once and then make many follow-up calls against the
|
||||
same session.
|
||||
|
||||
!!! note "Why you'd use this"
|
||||
- The server has service accounts, cookies, or a CDM you don't want to copy to
|
||||
every machine.
|
||||
- You want to run downloads on a fast/local machine but keep credentials and
|
||||
licensing centralized.
|
||||
- A service is region-locked to where the server lives, and the client isn't.
|
||||
|
||||
---
|
||||
|
||||
## The big picture
|
||||
|
||||
```
|
||||
┌──────────────────────────┐ HTTP (X-Secret-Key) ┌──────────────────────────┐
|
||||
│ Local machine (client) │ ───────────────────────────────▶ │ unshackle serve │
|
||||
│ │ │ │
|
||||
│ unshackle dl --server │ POST /api/session/create │ authenticated Service │
|
||||
│ │ GET /api/session/{id}/titles │ instance, kept alive in │
|
||||
│ • track selection │ POST /api/session/{id}/tracks │ an in-memory session │
|
||||
│ • download / decrypt │ POST /api/session/{id}/segments │ store (TTL-based) │
|
||||
│ • mux │ POST /api/session/{id}/license │ │
|
||||
│ │ DELETE /api/session/{id} │ │
|
||||
└──────────────────────────┘ └──────────────────────────┘
|
||||
```
|
||||
|
||||
The client implements the same interface `dl` expects from a normal service, but
|
||||
every service-facing method proxies to the server. Everything after track
|
||||
selection (downloading segments, decrypting, and muxing) runs locally.
|
||||
|
||||
!!! note "Migration: `remote_dl` is gone"
|
||||
The `RemoteService` adapter lives inside the ordinary `dl` command and replaces
|
||||
the old standalone `remote_dl` command. If you have prior scripts or docs that
|
||||
invoke `remote_dl`, use `dl` with a configured remote server instead. The
|
||||
functionality now rides on `dl` via `RemoteService`.
|
||||
|
||||
For the auth-facing endpoints, header format, and error shapes referenced below,
|
||||
see [Authentication](authentication.md) and the endpoint reference in
|
||||
[the REST API index](index.md).
|
||||
|
||||
---
|
||||
|
||||
## Configuring a remote server (client side)
|
||||
|
||||
On the **client**, add a `remote_services` block to your `unshackle.yaml`. Each
|
||||
entry names a server and provides its URL and API key.
|
||||
|
||||
```yaml title="unshackle.yaml (client)"
|
||||
remote_services:
|
||||
my_server:
|
||||
url: "https://my-box.example:8786"
|
||||
api_key: "your-api-key"
|
||||
# Optional: let the server's CDM do the licensing instead of a local CDM
|
||||
server_cdm: false
|
||||
# Optional: per-service config overrides applied locally (title_map, cdm, etc.)
|
||||
services:
|
||||
AMZN:
|
||||
title_map:
|
||||
"0ABCDEF": "The Show (Renamed Locally)"
|
||||
```
|
||||
|
||||
| Key | Required | Meaning |
|
||||
|---|---|---|
|
||||
| `url` | yes | Base URL of the `unshackle serve` instance (no trailing slash needed) |
|
||||
| `api_key` | no | Sent as the `X-Secret-Key` header on every request |
|
||||
| `server_cdm` | no | When `true`, DRM keys are resolved by the server's CDM rather than a local device (default `false`) |
|
||||
| `services` | no | Per-service local config overrides, keyed by service tag |
|
||||
|
||||
!!! warning "Port default"
|
||||
The `unshackle serve` default port is **`8786`** (not `8080`). Use the port
|
||||
your server actually binds. Some older example snippets show `8080`, which is
|
||||
not the default.
|
||||
|
||||
### Selecting a server
|
||||
|
||||
If exactly one server is configured, unshackle uses it implicitly:
|
||||
|
||||
```bash
|
||||
unshackle dl AMZN 0ABCDEF
|
||||
```
|
||||
|
||||
If **more than one** server is configured, you must pick one with `--server`:
|
||||
|
||||
```bash
|
||||
unshackle dl --server my_server AMZN 0ABCDEF
|
||||
```
|
||||
|
||||
With no `remote_services` configured at all, unshackle raises a clear error telling
|
||||
you to add the block shown above.
|
||||
|
||||
---
|
||||
|
||||
## What happens during a download
|
||||
|
||||
From your point of view the command looks like an ordinary download. Under the
|
||||
hood the client walks a session through its lifecycle.
|
||||
|
||||
=== "1. Create + authenticate"
|
||||
|
||||
The client calls `POST /api/session/create`, forwarding whatever it can so the
|
||||
server doesn't have to prompt:
|
||||
|
||||
- Local **credentials** for the service/profile (`{username, password, extra?}`)
|
||||
- Local **cookies**, compressed and base64-encoded
|
||||
- A resolved **proxy**, or, if you didn't set one, your detected
|
||||
**client region** so the server can auto-proxy to match
|
||||
- Track-selection hints (`range_`, `vcodec`, `quality`, `best_available`) so
|
||||
the server fetches the right manifests
|
||||
- Your local per-service **cache files** (e.g. refreshed tokens)
|
||||
|
||||
The server responds immediately with a session ID and a status. Authentication
|
||||
runs in the background on the server.
|
||||
|
||||
=== "2. Interactive prompts (if needed)"
|
||||
|
||||
Some services need a one-time code, PIN, or device confirmation. When the
|
||||
server's auth flow asks for input, the session enters `pending_input`. The
|
||||
client polls, displays the prompt to you locally, collects your answer, and
|
||||
sends it back. The server's auth thread resumes with your response.
|
||||
|
||||
```text
|
||||
Enter the 6-digit code sent to your email: 483920
|
||||
```
|
||||
|
||||
=== "3. Titles + tracks"
|
||||
|
||||
Once authenticated, the client fetches the title list, then the tracks for the
|
||||
chosen title. The tracks come back with playback URLs, and DASH/ISM manifests
|
||||
are shipped as compressed XML so the client can re-parse them locally for
|
||||
downloading. Any `session_headers` / `session_cookies` the server used are
|
||||
merged into the client's local HTTP session.
|
||||
|
||||
=== "4. Licensing"
|
||||
|
||||
For DRM content the client either **proxies** its CDM challenge through the
|
||||
server, or, if `server_cdm: true`, asks the server to run the full CDM flow
|
||||
and return the content keys directly. See
|
||||
[Server-side vs. proxied CDM](#server-side-vs-proxied-cdm) below.
|
||||
|
||||
=== "5. Download + close"
|
||||
|
||||
The client downloads, decrypts, and muxes locally. On completion it deletes
|
||||
the session. If the server has updated cache files (e.g. a refreshed token),
|
||||
they're returned on delete and saved locally so the **next** session can skip
|
||||
interactive auth.
|
||||
|
||||
!!! tip "Renaming remote titles locally"
|
||||
You can rename titles for a remote service you don't have installed locally by
|
||||
adding a `title_map` under that service in your client `remote_services.<name>.services`
|
||||
config. The server sends raw titles; your local map wins.
|
||||
|
||||
This is deliberate: the server does **no** `title_map` remapping of its own and
|
||||
sends titles exactly as the service returns them. All remapping happens on the
|
||||
client, applied to the titles the server sends back. It keeps the final output
|
||||
name fully under the client's control and lets you rename titles even for
|
||||
services you have no local install of.
|
||||
|
||||
---
|
||||
|
||||
## Server-side vs. proxied CDM
|
||||
|
||||
There are two ways DRM keys get resolved, chosen by the client's `server_cdm` flag.
|
||||
|
||||
=== "Proxied CDM (default)"
|
||||
|
||||
Your **local CDM** builds the license challenge. The client sends that challenge
|
||||
to `POST /api/session/{id}/license`, the server forwards it to the service, and
|
||||
the raw license bytes come back for your local CDM to parse.
|
||||
|
||||
- Keeps your CDM local; the server just relays the license request.
|
||||
- Used when `server_cdm` is `false` (the default).
|
||||
|
||||
=== "Server-side CDM (server_cdm: true)"
|
||||
|
||||
The **server's CDM** does everything. The client sends track IDs (or a PSSH),
|
||||
the server checks its key vaults, loads the device configured for your API key,
|
||||
runs the CDM flow, and returns `KID:KEY` pairs directly. No CDM is needed on the
|
||||
client.
|
||||
|
||||
- Enable with `server_cdm: true` in the server entry.
|
||||
- The server tells the client which DRM type it actually used
|
||||
(`widevine` or `playready`).
|
||||
|
||||
---
|
||||
|
||||
## Session lifecycle and expiry
|
||||
|
||||
Sessions live in an **in-memory store on the server** and expire on a timer. You
|
||||
generally never touch this directly, but it explains behavior you might observe.
|
||||
|
||||
| Behavior | Value | Notes |
|
||||
|---|---|---|
|
||||
| Idle session TTL | **300s** (5 min) default | Refreshed on every request to the session |
|
||||
| Max concurrent sessions | **100** default | Oldest (least recently used) is evicted when full |
|
||||
| Auth/input timeout | **600s** (10 min) | Sessions still authenticating or awaiting a prompt use this longer window instead of the TTL |
|
||||
| Cleanup sweep | every 60s | Expired sessions are removed and their input prompts cancelled |
|
||||
|
||||
!!! note "Auth is never rushed"
|
||||
A session that is still `authenticating` or waiting on `pending_input` is **not**
|
||||
subject to the short 300s TTL; it gets the full 600s auth window. This gives you
|
||||
time to enter an interactive code without the session vanishing mid-prompt.
|
||||
|
||||
The TTL and max-session limits are read from server config:
|
||||
|
||||
```yaml title="unshackle.yaml (server)"
|
||||
serve:
|
||||
session_ttl: 300 # seconds a session may sit idle
|
||||
max_sessions: 100 # cap on concurrent sessions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security properties worth knowing
|
||||
|
||||
- **API key required.** Every session request carries `X-Secret-Key`; requests
|
||||
without a valid key are rejected (health check aside). See
|
||||
[Authentication](authentication.md).
|
||||
- **IP binding.** A session records the creator's IP at create time. If a later
|
||||
request for the same session comes from a different IP, the server returns
|
||||
`403 FORBIDDEN`. Sessions are not portable between hosts.
|
||||
- **Namespaced, isolated cache.** Each session gets its own cache directory,
|
||||
namespaced by a hash of the API key and the session ID, so sessions can't read
|
||||
each other's cached tokens. The directory is deleted when the session ends.
|
||||
- **Secrets are scrubbed from logs.** Session IDs, service tags, and other user
|
||||
values are sanitized before logging.
|
||||
|
||||
---
|
||||
|
||||
## Developer reference
|
||||
|
||||
!!! info "This section is for developers"
|
||||
The material below documents the server-side implementation and the HTTP
|
||||
contract. End users configuring a client can stop at the sections above.
|
||||
|
||||
### Session endpoints
|
||||
|
||||
All routes are exposed even in `--remote-only` server mode. Paths use the
|
||||
`session_id` returned by create.
|
||||
|
||||
!!! note "Why `--remote-only` exists"
|
||||
`--remote-only` narrows the server to just the health, services, search, and
|
||||
session subset (and emits CORS headers) precisely so it is safe to sit behind
|
||||
Cloudflare or serve cross-origin browser clients. That trimmed surface, rather
|
||||
than the full `--api-only` mode, is what makes a CORS/Cloudflare-fronted
|
||||
deployment practical: reach for it when the server is public-facing or accessed
|
||||
from a browser origin, not when you simply want the local HTTP API.
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `POST` | `/api/session/create` | Build + authenticate a service instance; returns a session ID immediately |
|
||||
| `GET` | `/api/session/{id}/titles` | Fetch and cache the title list |
|
||||
| `POST` | `/api/session/{id}/tracks` | Resolve tracks, manifests, chapters, headers/cookies for a title |
|
||||
| `POST` | `/api/session/{id}/segments` | Resolve per-segment/track download descriptors |
|
||||
| `POST` | `/api/session/{id}/license` | DRM licensing (proxy or server CDM) |
|
||||
| `GET` | `/api/session/{id}/prompt` | Poll interactive auth status / pending prompt |
|
||||
| `POST` | `/api/session/{id}/prompt` | Submit an answer to a pending prompt |
|
||||
| `GET` | `/api/session/{id}` | Session info (validity, TTL, counts) |
|
||||
| `DELETE` | `/api/session/{id}` | Close the session, return updated cache, clean up |
|
||||
|
||||
### Create: request and response
|
||||
|
||||
`POST /api/session/create` requires `service` and `title_id`. It also accepts
|
||||
`credentials`, `cookies` (base64 of zlib-compressed Netscape cookie file), `proxy`,
|
||||
`no_proxy`, `profile`, `cache` (a map of `filename → base64(zlib(bytes))`),
|
||||
`client_region`, `cdm_type`, and the track-selection hints `range_`, `vcodec`,
|
||||
`quality`, `best_available`, plus arbitrary service CLI options
|
||||
(`additionalProperties: true`).
|
||||
|
||||
The response returns **before** authentication finishes:
|
||||
|
||||
```json
|
||||
{ "session_id": "…uuid4…", "service": "AMZN", "status": "authenticating" }
|
||||
```
|
||||
|
||||
Authentication runs on a background thread (`asyncio.to_thread(authenticate, …)`).
|
||||
The session starts in `AUTHENTICATING`; the client must poll the prompt endpoint
|
||||
until it reaches `authenticated` (or `failed`).
|
||||
|
||||
### Interactive auth: the `InputBridge`
|
||||
|
||||
When the service calls its input function during `authenticate()` on the server
|
||||
thread, an `InputBridge` pauses that thread and exposes the prompt to the HTTP
|
||||
layer.
|
||||
|
||||
- `AuthStatus` values: `authenticating`, `pending_input`, `authenticated`, `failed`.
|
||||
- `InputBridge.request_input(prompt, timeout=600)` blocks the sync auth thread on a
|
||||
`threading.Event` until `submit_response()` or `cancel()` fires.
|
||||
- A timeout raises `TimeoutError` and marks the session `FAILED`.
|
||||
- `AUTH_INPUT_TIMEOUT = 600.0` seconds. This is also the TTL granted to
|
||||
`AUTHENTICATING` / `PENDING_INPUT` sessions in the store.
|
||||
|
||||
`GET /api/session/{id}/prompt` returns one of:
|
||||
|
||||
```json
|
||||
{ "status": "authenticated" }
|
||||
{ "status": "authenticating" }
|
||||
{ "status": "pending_input", "prompt": "Enter code: " }
|
||||
{ "status": "failed", "error": "…message…" }
|
||||
```
|
||||
|
||||
A missing session returns `404 SESSION_NOT_FOUND`; an IP mismatch returns
|
||||
`403 FORBIDDEN`. `POST /api/session/{id}/prompt` takes `{ "response": "…" }` and
|
||||
returns `{ "status": "accepted" }`; posting with no pending prompt is an
|
||||
`INVALID_INPUT` error.
|
||||
|
||||
### `SessionStore` internals
|
||||
|
||||
Source: `unshackle/core/api/session_store.py`. A singleton obtained via
|
||||
`get_session_store()`.
|
||||
|
||||
- **Config-driven limits.** `serve.session_ttl` (default `300`) and
|
||||
`serve.max_sessions` (default `100`) are read as properties, so config changes
|
||||
take effect without recreating the store.
|
||||
- **`create()`** evicts the least-recently-accessed session when at capacity,
|
||||
then stores a new `SessionEntry` (defaulting `auth_status` to `AUTHENTICATED`;
|
||||
the create handler overrides it to `AUTHENTICATING`).
|
||||
- **`get()`** refreshes `last_accessed` via `touch()`. It returns `None` (and
|
||||
deletes the entry) if an authenticated session has been idle longer than the TTL.
|
||||
Sessions in `AUTHENTICATING` / `PENDING_INPUT` are exempt from TTL expiry.
|
||||
- **`cleanup_expired()`** runs every 60s: authenticated sessions expire at `ttl`,
|
||||
in-flight-auth sessions expire at `AUTH_INPUT_TIMEOUT`. Removing a session
|
||||
cancels its `InputBridge` and deletes its cache directory (pruning empty parent
|
||||
dirs up to, but not including, the cache root).
|
||||
- **`cancel_all_bridges()`** is called on server shutdown to unblock any waiting
|
||||
auth threads.
|
||||
|
||||
`SessionEntry` fields:
|
||||
|
||||
| Field | Description |
|
||||
|---|---|
|
||||
| `session_id` | UUID4 string |
|
||||
| `service_tag` | Normalized service tag |
|
||||
| `service_instance` | The authenticated service object kept alive between calls |
|
||||
| `titles` / `title_map` | Result of `get_titles()` and a `title_id → Title` map |
|
||||
| `tracks` / `tracks_by_title` / `chapters_by_title` | Cached resolved tracks and chapters |
|
||||
| `creator_ip` | IP recorded at create time for IP-binding checks |
|
||||
| `cache_tag` | Per-session cache directory tag |
|
||||
| `input_bridge` | `InputBridge` for interactive auth, if any |
|
||||
| `auth_status` / `auth_error` | Current `AuthStatus` and last error message |
|
||||
| `created_at` / `last_accessed` | Timestamps; `last_accessed` drives TTL and LRU eviction |
|
||||
|
||||
### Per-session cache namespacing
|
||||
|
||||
The create handler builds a `Cacher` namespaced as:
|
||||
|
||||
```text
|
||||
_sessions/<pbkdf2_hmac(sha256, X-Secret-Key, "unshackle-session-ns", 100000)[:12]>/<session_id>/<service>
|
||||
```
|
||||
|
||||
Forwarded `cache` files are written into that directory before authentication.
|
||||
On `DELETE`, the handler harvests updated cache files (compressing each with zlib
|
||||
and base64-encoding, **excluding** `titles_*` files) and returns them under a
|
||||
`cache` key so the client can persist refreshed tokens:
|
||||
|
||||
```json
|
||||
{ "status": "ok", "cache": { "tokens": "…base64(zlib(bytes))…" } }
|
||||
```
|
||||
|
||||
### Session info response
|
||||
|
||||
`GET /api/session/{id}`:
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "…",
|
||||
"service": "AMZN",
|
||||
"valid": true,
|
||||
"expires_in": 300,
|
||||
"track_count": 12,
|
||||
"title_count": 40
|
||||
}
|
||||
```
|
||||
|
||||
!!! warning "`expires_in` is the configured TTL, not time remaining"
|
||||
The `expires_in` value reports the store's configured `session_ttl`, not the
|
||||
seconds left before this specific session expires.
|
||||
|
||||
### Reference client: `RemoteService`
|
||||
|
||||
Source: `unshackle/core/remote_service.py`. This is the canonical consumer of the
|
||||
session API and a good template for any client.
|
||||
|
||||
- **`RemoteClient._request`** sets `X-Secret-Key` and `User-Agent: unshackle/<version>`,
|
||||
uses a 120s timeout for `POST` and 30s for `GET`/`DELETE`, and treats any
|
||||
`status_code >= 400` as fatal: it logs `Server error [<error_code>]: <message>`
|
||||
and raises `SystemExit(1)`.
|
||||
- **Retries.** The download-side HTTP session mounts an adapter with
|
||||
`Retry(total=5, backoff_factor=0.2, status_forcelist=[429, 500, 502, 503, 504])`.
|
||||
- **Flow.** `authenticate()` → `create` (+ poll `prompt` every 2s up to a 600s
|
||||
deadline, answering `pending_input` prompts) → `get_titles()` → `get_tracks()`
|
||||
(merging returned `session_headers`/`session_cookies`, re-parsing `manifests`) →
|
||||
license via proxy or `server_cdm` → `close()` (`DELETE`, saving any returned
|
||||
`cache`).
|
||||
- **Server resolution.** `resolve_server()` reads `config.remote_services.<name>`
|
||||
into `{url, api_key, services, server_cdm}`, injecting the `server_cdm` flag into
|
||||
the services map as `_server_cdm`.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
!!! example "\"Could not connect to remote server … Is it running?\""
|
||||
The client couldn't reach the URL. Confirm `unshackle serve` is running on the
|
||||
server, the `url`/port in `remote_services` are correct (default port `8786`),
|
||||
and any firewall or reverse proxy allows the connection.
|
||||
|
||||
!!! example "`403 FORBIDDEN` mid-download"
|
||||
The request came from a different IP than the one that created the session.
|
||||
Sessions are IP-bound. Don't move between networks (or NAT egress IPs) during a
|
||||
remote download.
|
||||
|
||||
!!! example "Auth times out or the prompt never resolves"
|
||||
Interactive auth allows up to 600s. If you miss that window the session is
|
||||
marked `failed` and cleaned up; just re-run the download. If the server never
|
||||
prompts you, check that credentials/cookies forwarded from the client are valid
|
||||
for the service.
|
||||
|
||||
!!! example "\"Multiple remote services configured. Use --server …\""
|
||||
You have more than one entry under `remote_services`. Pass `--server <name>`
|
||||
to pick one.
|
||||
@@ -0,0 +1,452 @@
|
||||
# Service Architecture
|
||||
|
||||
This page is the conceptual map of the internal object model that a **service author**
|
||||
works against. It ties together the moving parts: the `Service` base class, the
|
||||
`Title` hierarchy a service returns, the `Track` model those titles carry, the
|
||||
manifest parsers that produce tracks, the DRM hooks that license and decrypt them,
|
||||
and the request/session objects wired up for you at construction time.
|
||||
|
||||
!!! info "Who this page is for"
|
||||
This is a **developer reference**. If you only want to *run* downloads, you never
|
||||
touch these classes directly; see [Downloading](../guide/downloading.md) and the
|
||||
[CLI reference](../guide/cli-reference.md) instead. If you are writing or reading a
|
||||
service plugin, this page explains what each object is and how the pieces connect.
|
||||
For a guided, step-by-step build, see [Creating a Service](creating-a-service.md);
|
||||
for exact signatures, see the auto-generated [Service API](../reference/api/service.md),
|
||||
[Titles API](../reference/api/titles.md), and [Tracks API](../reference/api/tracks.md).
|
||||
|
||||
## The shape of a download
|
||||
|
||||
A service is a translator between one streaming platform and unshackle's generic
|
||||
pipeline. It never downloads or decrypts anything itself; it just answers questions.
|
||||
The `dl` command drives the whole flow and only ever calls three required methods on
|
||||
your service, in order:
|
||||
|
||||
```text
|
||||
Service.get_titles → which movies / episodes / songs exist
|
||||
│
|
||||
Service.get_tracks → fetch a manifest for one title …
|
||||
│
|
||||
manifest.to_tracks → … parsed into Video / Audio / Subtitle tracks
|
||||
│
|
||||
Track.download → fetch + merge segments (core-driven)
|
||||
│
|
||||
DRM license + decrypt → service license hooks → CDM → decrypt tool
|
||||
│
|
||||
Tracks.mux → mkvmerge → tagged .mkv
|
||||
```
|
||||
|
||||
1. **`get_titles()`** returns a `Titles` collection: the movies, episodes, or songs
|
||||
the platform exposes for the requested ID.
|
||||
2. **`get_tracks(title)`** returns a `Tracks` object for a single chosen title. In
|
||||
practice this means fetching a manifest and handing it to a parser, which builds
|
||||
the `Video`/`Audio`/`Subtitle` objects.
|
||||
3. **`get_chapters(title)`** returns any chapter markers.
|
||||
|
||||
Everything after that (track selection, licensing, decryption, repacking, subtitle
|
||||
conversion, and muxing) is handled by the core using the objects your service
|
||||
returned. Your job is to produce accurate `Title` and `Track` objects; the framework
|
||||
does the rest.
|
||||
|
||||
## The `Service` base class
|
||||
|
||||
Every service subclasses `unshackle.core.service.Service`, an abstract base class.
|
||||
Two kinds of members matter: **class variables** that describe the service, and
|
||||
**methods** that the pipeline calls.
|
||||
|
||||
### Class variables
|
||||
|
||||
Declared at the top of your service class, these are static descriptors:
|
||||
|
||||
| Class variable | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `ALIASES` | `tuple[str, ...]` | Alternate tags the service answers to (resolved by `Services.get_tag`). |
|
||||
| `GEOFENCE` | `tuple[str, ...]` | ISO region codes the service requires; empty means no geofence. The first entry is treated as the "main region" for automatic proxy selection. |
|
||||
| `VAULT_TAG` | `Optional[str]` | Overrides the key-vault namespace; defaults to the service's own tag. |
|
||||
| `AUTH_METHODS` | `Optional[tuple[str, ...]]` | Accepted auth methods (`"cookies"` / `"credentials"`). When `None`, the REST `/services` endpoint infers them from `authenticate()`. |
|
||||
|
||||
### Required methods
|
||||
|
||||
These are abstract; a service will not load unless it implements all three:
|
||||
|
||||
- `get_titles() -> Titles_T`: resolve the requested ID into a title collection.
|
||||
- `get_tracks(title) -> Tracks`: build the track set for one title.
|
||||
- `get_chapters(title) -> Chapters`: return chapter markers (may be empty).
|
||||
|
||||
### Optional methods and hooks
|
||||
|
||||
- **`authenticate(cookies=None, credential=None)`**: the base implementation loads a
|
||||
`CookieJar` into `self.session.cookies` and stores `self.credential`. Override it to
|
||||
perform a real login handshake.
|
||||
- **`search()`**: raises `NotImplementedError` by default; implement it to support the
|
||||
`search` command.
|
||||
- **DRM license hooks**: `get_widevine_service_certificate`, `get_widevine_license`,
|
||||
`get_playready_license` (delegates to Widevine by default), and
|
||||
`get_clearkey_license` (returns `None` by default). See [DRM hooks](#drm-hooks) below.
|
||||
- **`request_input(prompt)`**: prompts the user; routed through the serve-mode input
|
||||
bridge or the rich console.
|
||||
- **Event callbacks**: `on_segment_downloaded`, `on_track_downloaded`,
|
||||
`on_track_decrypted`, `on_track_repacked`, `on_track_multiplex` let a service react to
|
||||
pipeline events (for example, MonaLisa decrypts each segment in `on_segment_downloaded`).
|
||||
- **`resolve_server_keys(title)`**: an escape hatch for services whose decryption keys are
|
||||
resolved server-side *ahead of* the normal per-track license flow. Unlike everything else
|
||||
here it is **not declared on the `Service` base class at all**: `dl.py` probes for it with
|
||||
`hasattr` and calls it (once, after track selection) only if your service defines it. Use
|
||||
it when keys must be fetched or injected in bulk before download, separate from the
|
||||
per-track license callbacks below.
|
||||
|
||||
### What `__init__` wires up for you
|
||||
|
||||
You rarely override `__init__`; the base constructor takes the Click context and sets
|
||||
up the objects your methods rely on:
|
||||
|
||||
| Attribute | What it is |
|
||||
|---|---|
|
||||
| `self.config` | The per-service config slice (see [Request and session objects](#request-and-session-objects)). |
|
||||
| `self.log` | A `logging.Logger` named after your class. |
|
||||
| `self.session` | A ready-to-use HTTP session (headers, retries, and proxy applied). |
|
||||
| `self.cache` | A generic key/value `Cacher` namespaced to the service. |
|
||||
| `self.title_cache` | A region- and account-aware `TitleCacher`. |
|
||||
| `self.cache_dir` | `config.directories.cache / <ServiceName>`. |
|
||||
| `self.credential` | The active `Credential`, or `None` until authenticated. |
|
||||
| `self.track_request` | A `TrackRequest` describing the requested codecs, ranges, and best-available flag (see below). |
|
||||
|
||||
The constructor also resolves proxies: it reads `--proxy` / `--proxy-query` /
|
||||
`--proxy-provider`, consults the per-service `proxy_map`, and, if `GEOFENCE` is set and
|
||||
no explicit proxy was given, does a live IP check and auto-fetches a proxy to
|
||||
`GEOFENCE[0]` when it detects a geoblock. See [Proxies and VPN](../guide/proxies-and-vpn.md).
|
||||
|
||||
!!! note "`TrackRequest`"
|
||||
`self.track_request` is a small dataclass built from the CLI: `codecs`
|
||||
(from `--vcodec`, empty means "accept any"), `ranges` (from `--range`, default
|
||||
`[Video.Range.SDR]`), and `best_available`. Use it in `get_tracks` to decide which
|
||||
variants to fetch.
|
||||
|
||||
## Titles and the `Title` hierarchy
|
||||
|
||||
`get_titles()` returns a **title collection**, one of the `Titles_T` types. Each
|
||||
collection holds one or more `Title` objects (`Title_T`). There are three parallel
|
||||
families, one per media kind:
|
||||
|
||||
| Media kind | Item (`Title_T`) | Collection (`Titles_T`) |
|
||||
|---|---|---|
|
||||
| Movies | `Movie` | `Movies` |
|
||||
| TV | `Episode` | `Series` |
|
||||
| Music | `Song` | `Album`, and `Music` (the base collection) |
|
||||
|
||||
All items subclass `Title`, which carries the shared identity: a **unique `id`**, the
|
||||
originating `service` class, an optional original `language`, arbitrary `data`, and a
|
||||
freshly-constructed `self.tracks = Tracks()` that `get_tracks` will fill. Collections
|
||||
are sorted lists, so titles stay in a stable order.
|
||||
|
||||
!!! warning "The unique ID is validated"
|
||||
`Title.__init__` rejects a falsy `id_`, and rejects any ID shorter than 4 characters
|
||||
(`"The unique ID is not large enough, clash likely."`). Pick a stable, sufficiently
|
||||
unique identifier from the platform.
|
||||
|
||||
### Constructors at a glance
|
||||
|
||||
Each subclass adds its own required fields on top of `id_` and `service`:
|
||||
|
||||
=== "Movie"
|
||||
|
||||
```python
|
||||
Movie(id_, service, name, year=None, language=None, data=None, description=None)
|
||||
```
|
||||
|
||||
`name` is required; `year` is optional but must be a positive integer if given.
|
||||
|
||||
=== "Episode"
|
||||
|
||||
```python
|
||||
Episode(id_, service, title, season, number,
|
||||
name=None, year=None, language=None, data=None,
|
||||
description=None, air_date=None)
|
||||
```
|
||||
|
||||
`title` is the **show** name; `season` and `number` are required (0 is allowed).
|
||||
An `air_date` is normalized to a `date` so the `{date}` naming token stays clean,
|
||||
and a placeholder episode `name` like `"Episode 4"` is discarded automatically.
|
||||
|
||||
=== "Song"
|
||||
|
||||
```python
|
||||
Song(id_, service, name, artist, album, track, disc, year,
|
||||
language=None, data=None)
|
||||
```
|
||||
|
||||
`name`, `artist`, `album`, `track`, `disc`, and `year` are all required and validated.
|
||||
|
||||
### Title remapping
|
||||
|
||||
After titles are fetched (and after any cache), the framework applies
|
||||
`remap_titles(titles, title_map)` using the per-service `title_map` config. This does an
|
||||
**exact-string** rename (`Episode`s are matched on their show `title`, `Movie`s and
|
||||
`Song`s on their `name`), so a platform's odd naming can be normalized to what your
|
||||
library expects. Titles are also cache-wrapped via `get_titles_cached()`, which is region-
|
||||
and account-aware; see the [TitleCacher](#request-and-session-objects) notes.
|
||||
|
||||
The ordering here is deliberate on both sides. `title_map` is applied **after** the title
|
||||
cache so that editing the map takes effect immediately, without forcing a cache reset, and
|
||||
**before** metadata enrichment (`--enrich`) so that an explicit enrich override still wins
|
||||
over a mapped name.
|
||||
|
||||
## Tracks and the `Track` model
|
||||
|
||||
A `Tracks` object is the container `get_tracks` returns. It has typed buckets
|
||||
(`videos`, `audio`, `subtitles`, `chapters`, and `attachments`) plus a `manifest_url`.
|
||||
Iterating a `Tracks` yields **only** video, audio, and subtitle tracks (not chapters or
|
||||
attachments), and the canonical display/mux order is Video → Audio → Subtitle → Chapter
|
||||
→ Attachment.
|
||||
|
||||
!!! warning "Track IDs must be deterministic"
|
||||
`Tracks.add` refuses duplicate IDs. Track IDs are CRC32-derived from the track's
|
||||
identity (URL, codec, language, bitrate) and **must be stable across requests**, never
|
||||
random. The manifest parsers already do this correctly; if you build tracks by hand,
|
||||
keep IDs deterministic or `add` will raise.
|
||||
|
||||
### The base `Track`
|
||||
|
||||
`Track` is the base for `Video`, `Audio`, and `Subtitle`. (`Chapter`, `Chapters`, and
|
||||
`Attachment` are standalone classes, **not** `Track` subclasses.) Every track knows how
|
||||
it is delivered, via `Track.Descriptor`:
|
||||
|
||||
| Descriptor | Delivery |
|
||||
|---|---|
|
||||
| `URL` | A single direct URL. |
|
||||
| `HLS` | HTTP Live Streaming. |
|
||||
| `DASH` | Dynamic Adaptive Streaming over HTTP. |
|
||||
| `ISM` | Smooth Streaming. |
|
||||
|
||||
Key attributes include `url` (a string or list of strings), `language` (a
|
||||
`langcodes.Language`), `drm` (an optional iterable of DRM objects), `needs_repack`, and
|
||||
descriptor-specific state stashed in `track.data["hls"]` / `track.data["dash"]`. When
|
||||
`name` is omitted, a human-readable name is derived from the language automatically.
|
||||
|
||||
### `Video`
|
||||
|
||||
`Video` adds `codec`, `range`, `bitrate`, `width`, `height`, `fps`, `scan_type`, and
|
||||
closed-caption descriptors. The relevant enums:
|
||||
|
||||
- **`Video.Codec`**: `AVC` (H.264), `HEVC` (H.265), `VC1`, `VP8`, `VP9`, `AV1`.
|
||||
- **`Video.Range`**: `SDR`, `HLG`, `HDR10`, `HDR10P` (HDR10+), `DV` (Dolby Vision), and
|
||||
`HYBRID`. `HYBRID` is special: it tells the pipeline to select both an HDR10(+) base and
|
||||
a DV track and merge the DV RPU onto the HDR10 stream with `dovi_tool` (see
|
||||
[Dolby Vision hybrid](#dolby-vision-and-hybrid)).
|
||||
|
||||
!!! note "Bitrate units"
|
||||
Both `Video` and `Audio` store `bitrate` in **bytes per second** (rounded up), even
|
||||
though the string form displays kb/s.
|
||||
|
||||
### `Audio`
|
||||
|
||||
`Audio` adds `bitrate`, `channels`, `joc` (Joint Object Coding, the Atmos indicator),
|
||||
and `descriptive`. `Audio.Codec` covers `AAC`, `AC3` (`DD`), `EC3` (`DD+`), `AC4`,
|
||||
`OPUS`, `OGG` (Vorbis), `DTS`, `ALAC`, and `FLAC`. The `atmos` property is `True` when
|
||||
`joc` is set (or when `extra` flags it).
|
||||
|
||||
### `Subtitle`
|
||||
|
||||
`Subtitle` adds three mutually-constrained flags: `cc` (closed captions), `sdh`
|
||||
(subtitles for the deaf and hard-of-hearing), and `forced`. A track cannot be both `cc`
|
||||
and `sdh`, and `forced` cannot combine with either. `Subtitle.Codec` includes `SubRip`
|
||||
(SRT), `WebVTT` (VTT), `SubStationAlpha`/`SubStationAlphav4` (SSA/ASS),
|
||||
`TimedTextMarkupLang` (TTML), `SAMI`, and the box-encapsulated DASH formats `fTTML`
|
||||
(STPP) and `fVTT` (WVTT). Subtitles receive post-download normalization
|
||||
(segment merging, WebVTT sanitization, format conversion, and optional SDH stripping),
|
||||
all documented for users in [Subtitles](../guide/subtitles.md).
|
||||
|
||||
### Chapters and attachments
|
||||
|
||||
- **`Chapter(timestamp, name=None)`**: a single marker; the timestamp accepts
|
||||
`HH:MM:SS[.mmm]`, integer milliseconds, or float seconds.
|
||||
- **`Chapters`**: a sorted collection that auto-inserts a `00:00:00.000` chapter and
|
||||
reads/writes OGM Simple format.
|
||||
- **`Attachment`**: a font or image attached to the mux (e.g. fonts an ASS subtitle
|
||||
references), built from a local `path` or a `url`.
|
||||
|
||||
### Selection, sorting, and muxing
|
||||
|
||||
`Tracks` is also the selection engine the `dl` command uses. It offers
|
||||
`sort_videos` / `sort_audio` / `sort_subtitles` (P2P-standard ordering), predicate-based
|
||||
`select_*` filters, `by_resolutions`, and `by_language`. When downloads finish, the same
|
||||
object multiplexes everything: **`Tracks.mux(title, ...)`** builds and runs an
|
||||
`mkvmerge` command, applying language, default-track, forced, and hearing-impaired flags,
|
||||
attaching chapters and fonts, and writing a `.muxed.mkv` (or `.mka` / `.mks` for
|
||||
audio- or subtitle-only sets). As a service author you never call `mux` yourself; you
|
||||
just return correct tracks.
|
||||
|
||||
## Manifest parsers
|
||||
|
||||
Most services do not build tracks by hand; they hand a manifest to one of three parsers
|
||||
in `unshackle.core.manifests`. Each parser has the same three-part contract:
|
||||
|
||||
| Format | Class | Notes |
|
||||
|---|---|---|
|
||||
| MPEG-DASH (`.mpd`) | `DASH` | Parses `<MPD>`; enumerates tracks from the first content period. |
|
||||
| HLS (`.m3u8`) | `HLS` | Requires a **variant (master) playlist**, not a media playlist. |
|
||||
| Smooth Streaming (`.ism`) | `ISM` | Synthesizes init segments from `CodecPrivateData`. |
|
||||
|
||||
### The parser contract
|
||||
|
||||
1. **Construct**: `DASH.from_url(url, session=...)` or `DASH.from_text(text, url)`
|
||||
(and likewise for `HLS` / `ISM`). A `url` is always required, even for `from_text`,
|
||||
because relative segment paths are resolved against it.
|
||||
2. **`to_tracks(...)`**: parses the manifest into a `Tracks` container of `Video`,
|
||||
`Audio`, and `Subtitle` objects, each carrying descriptor-specific state.
|
||||
3. **`download_track(...)`**: a staticmethod the core calls to fetch, decrypt, and merge
|
||||
segments. You do not call it; you just return the tracks it will operate on.
|
||||
|
||||
A typical `get_tracks` therefore looks like fetching the manifest text with
|
||||
`self.session` and returning `DASH.from_text(mpd, url).to_tracks(language=title.language)`.
|
||||
|
||||
!!! tip "Pass the original language"
|
||||
DASH and HLS `to_tracks` raise `ValueError` if a track's language cannot be derived
|
||||
and no valid fallback was supplied. Always pass the title's original recorded
|
||||
language. For HLS this fallback is used directly for the primary track, since HLS
|
||||
variants carry no language.
|
||||
|
||||
### Segment-filtering hooks
|
||||
|
||||
- **HLS**: set `track.OnSegmentFilter(segment)` to drop unwanted segments (ads/bumpers).
|
||||
- **DASH**: pass a `period_filter` callable to `to_tracks` to drop whole periods.
|
||||
Amazon caching-breadth periods and non-`content` segment types are dropped automatically.
|
||||
|
||||
### Format gotchas worth knowing
|
||||
|
||||
- **DASH** enumerates tracks from only the *first* content period, but downloads segments
|
||||
from *all* content periods matching the representation. `SegmentTemplate`,
|
||||
`SegmentList`, and `SegmentBase` addressing and `$Number%0Nd$` padding are handled.
|
||||
- **HLS** re-fetches the media playlist at download time, recovers `SUPPLEMENTAL-CODECS`
|
||||
from raw text (python-m3u8 drops it) to detect Dolby Vision composite streams, and uses
|
||||
the media-sequence-number IV for AES-128 keys that omit an explicit IV.
|
||||
- **ISM** has no segment list and no init segments: segment URLs are synthesized from the
|
||||
`<c>` fragment timeline, and a `moov`/`ftyp` init box is rebuilt from `CodecPrivateData`
|
||||
(supported for AVC, HEVC, DV-over-HEVC, AAC, EC-3, and TTML).
|
||||
|
||||
## DRM hooks
|
||||
|
||||
If a track carries DRM, the framework licenses and decrypts it, but it needs your
|
||||
service to talk to the platform's license server. unshackle recognizes five DRM systems:
|
||||
|
||||
| DRM class | System | License flow |
|
||||
|---|---|---|
|
||||
| `Widevine` | Widevine | CDM challenge → license server → content keys (`.wvd` device) |
|
||||
| `PlayReady` | PlayReady | CDM challenge (WRM header) → license server → content keys (`.prd` device) |
|
||||
| `ClearKeyCENC` | W3C `org.w3.clearkey` over CENC | JSON KID request → JWK set; no CDM |
|
||||
| `ClearKey` | HLS AES-128 | key fetched directly from a URI; no CDM, no challenge |
|
||||
| `MonaLisa` | Proprietary WASM DRM | keys extracted locally; no license server |
|
||||
|
||||
!!! warning "`ClearKey` vs `ClearKeyCENC`"
|
||||
Despite the shared prefix these are entirely different systems. `ClearKey` is HLS
|
||||
AES-128, decrypted in-process. `ClearKeyCENC` is the W3C EME JWK flow over MPEG-CENC,
|
||||
decrypted with shaka-packager or mp4decrypt like Widevine.
|
||||
|
||||
### The service-side hooks
|
||||
|
||||
The framework supplies the CDM and the decryption tool; your service provides the network
|
||||
calls that the CDM's challenge must be sent through. Override the hooks that your platform
|
||||
needs:
|
||||
|
||||
| Hook | Return | Purpose |
|
||||
|---|---|---|
|
||||
| `get_widevine_service_certificate(title, track, challenge)` | cert bytes | Optional privacy-mode service certificate. |
|
||||
| `get_widevine_license(title, track, challenge)` | license bytes | POST the Widevine challenge to the license server, return the response. |
|
||||
| `get_playready_license(title, track, challenge)` | license | Same for PlayReady; defaults to delegating to Widevine. |
|
||||
| `get_clearkey_license(title, track, challenge)` | dict / str / bytes / `None` | For `ClearKeyCENC`; return `None` to let the framework POST to the manifest `Laurl`. |
|
||||
|
||||
You do not open CDM sessions or run shaka-packager yourself. The `Widevine`/`PlayReady`
|
||||
classes drive the CDM handshake and invoke the decryption tool. Your hook is just the
|
||||
"send this challenge, give me back the license" step.
|
||||
|
||||
### How a CDM is chosen and keys are found
|
||||
|
||||
CDM selection is config-driven: the `cdm` map resolves a service tag (and optionally the
|
||||
track quality or DRM type) to a device name, which `load_cdm` instantiates as a local
|
||||
`.wvd`/`.prd` or a remote CDM. Before ever requesting a license, the framework tries an
|
||||
in-process key cache and the configured **key vaults**; only unresolved KIDs trigger an
|
||||
actual license request, and newly licensed keys are written back to every vault. This is
|
||||
why the same key is rarely re-licensed twice. Users configure all of this in
|
||||
[DRM and CDM](../guide/drm-and-cdm.md) and [Key vaults](../guide/vaults.md); the API
|
||||
surface is documented in the [DRM API reference](../reference/api/drm.md).
|
||||
|
||||
### Dolby Vision and hybrid
|
||||
|
||||
Two DV mechanisms sit on top of the track model. **DV fixup** re-exposes DV signaling on
|
||||
HLS streams that carry a valid DV RPU but advertise plain HEVC (`dv_compatible_bitstream`).
|
||||
**Hybrid** processing (`Video.Range.HYBRID`) selects an HDR10(+) base and a DV donor,
|
||||
extracts the RPU with `dovi_tool`, and injects it onto the base layer, producing a single
|
||||
stream with both HDR10 and DV metadata. Both are automatic once the right tracks are
|
||||
selected; a service only needs to expose accurate ranges.
|
||||
|
||||
## Request and session objects
|
||||
|
||||
Everything a service does over the network flows through objects the base constructor
|
||||
prepared, all derived from the Click context (`ctx`) that `dl` passes in.
|
||||
|
||||
- **`self.config`**: your service's slice of `unshackle.yaml`, i.e.
|
||||
`config.services[<ServiceName>]`. This is a plain dict; use it for per-service settings
|
||||
like base URLs, an API key, or a `title_map`. See
|
||||
[Configuration](../getting-started/configuration-file.md).
|
||||
- **`self.session`**: a `requests.Session` with `config.headers` applied, a retry policy
|
||||
(`total=5`, backoff, retry on 429/5xx), and any resolved proxy already mounted. Pass this
|
||||
session into the manifest parsers and DRM calls so headers, cookies, and proxies flow
|
||||
everywhere. (unshackle also ships an rnet-based TLS-fingerprinting session used by the
|
||||
newer `session()` factory and configured via the `network` config key.)
|
||||
- **`self.credential`**: the active `Credential` (`username:password[:extra]`), populated
|
||||
by `authenticate`. Its `sha1` is used as the account hash for cache keys.
|
||||
- **`self.cache`**: a generic `Cacher`: JSON key/value storage under the service's cache
|
||||
dir, with CRC32 integrity checks, versioning, and optional expiry (auto-derived from a
|
||||
JWT `exp` claim when present). Good for auth tokens and API responses.
|
||||
- **`self.title_cache`**: a `TitleCacher` that wraps `get_titles` results, keyed by title
|
||||
ID, region, and account hash. It serves fresh data within `title_cache_time`, falls back
|
||||
to stale data within `title_cache_max_retention` when an API call fails, and is bypassed
|
||||
by `--no-cache`.
|
||||
- **`self.track_request`**: the `TrackRequest` describing what the user asked for
|
||||
(codecs, ranges, best-available), for use inside `get_tracks`.
|
||||
|
||||
!!! example "A minimal service skeleton"
|
||||
```python title="services/EXMP/__init__.py"
|
||||
from unshackle.core.service import Service
|
||||
from unshackle.core.manifests import DASH
|
||||
from unshackle.core.titles import Movie, Movies
|
||||
|
||||
|
||||
class EXMP(Service):
|
||||
ALIASES = ("example",)
|
||||
GEOFENCE = ("us",)
|
||||
|
||||
def get_titles(self) -> Movies:
|
||||
data = self.session.get(f"https://api.example.com/title/{self.title}").json()
|
||||
return Movies([
|
||||
Movie(
|
||||
id_=data["id"],
|
||||
service=self.__class__,
|
||||
name=data["name"],
|
||||
year=data.get("year"),
|
||||
language=data.get("language"),
|
||||
)
|
||||
])
|
||||
|
||||
def get_tracks(self, title) -> "Tracks":
|
||||
mpd_url = self.session.get(...).json()["manifest"]
|
||||
mpd = self.session.get(mpd_url).text
|
||||
return DASH.from_text(mpd, mpd_url).to_tracks(language=title.language)
|
||||
|
||||
def get_chapters(self, title):
|
||||
return []
|
||||
|
||||
def get_widevine_license(self, *, challenge, title, track):
|
||||
return self.session.post("https://lic.example.com/wv", data=challenge).content
|
||||
```
|
||||
|
||||
The directory name (`EXMP`) must exactly match the class name; that stem is the
|
||||
service tag. Put your service in one of the `directories.services` paths.
|
||||
|
||||
## Where to go next
|
||||
|
||||
- **[Creating a Service](creating-a-service.md)**: a guided walkthrough that builds on this model.
|
||||
- **[Service API](../reference/api/service.md)**: exact signatures for the `Service` base class.
|
||||
- **[Titles API](../reference/api/titles.md)** and **[Tracks API](../reference/api/tracks.md)**: the full object reference.
|
||||
- **[DRM API](../reference/api/drm.md)**: the DRM and CDM classes in detail.
|
||||
- **[Downloading](../guide/downloading.md)**: how the flags a user passes map onto track selection.
|
||||
Reference in New Issue
Block a user