From c4b25756ebee1a7bdab86c8511aae2ab723d19e6 Mon Sep 17 00:00:00 2001 From: imSp4rky Date: Tue, 4 Aug 2026 10:25:32 -0600 Subject: [PATCH] docs: split configuration reference into per-area pages --- docs/dev/creating-a-service.md | 2 +- docs/getting-started/configuration-file.md | 10 +- docs/getting-started/installation.md | 2 +- docs/guide/cli-reference.md | 2 +- docs/guide/downloading.md | 2 +- docs/guide/drm-and-cdm.md | 2 +- docs/guide/output-and-naming.md | 11 +- docs/guide/subtitles.md | 4 +- docs/index.md | 2 +- docs/reference/api/config.md | 2 +- docs/reference/configuration.md | 999 +------------------- docs/reference/configuration/directories.md | 88 ++ docs/reference/configuration/download.md | 253 +++++ docs/reference/configuration/drm.md | 157 +++ docs/reference/configuration/example.md | 51 + docs/reference/configuration/index.md | 42 + docs/reference/configuration/misc.md | 109 +++ docs/reference/configuration/network.md | 86 ++ docs/reference/configuration/output.md | 149 +++ docs/reference/configuration/services.md | 137 +++ docs/reference/configuration/vaults.md | 47 + mkdocs.yml | 14 +- 22 files changed, 1169 insertions(+), 1002 deletions(-) create mode 100644 docs/reference/configuration/directories.md create mode 100644 docs/reference/configuration/download.md create mode 100644 docs/reference/configuration/drm.md create mode 100644 docs/reference/configuration/example.md create mode 100644 docs/reference/configuration/index.md create mode 100644 docs/reference/configuration/misc.md create mode 100644 docs/reference/configuration/network.md create mode 100644 docs/reference/configuration/output.md create mode 100644 docs/reference/configuration/services.md create mode 100644 docs/reference/configuration/vaults.md diff --git a/docs/dev/creating-a-service.md b/docs/dev/creating-a-service.md index 242d06da..a2578d89 100644 --- a/docs/dev/creating-a-service.md +++ b/docs/dev/creating-a-service.md @@ -8,7 +8,7 @@ downloading, decryption, muxing, naming) is handled by the unshackle core. !!! info "Who this page is for" This is a **developer guide**. If you only want to *use* services, see [downloading](../guide/downloading.md) and - [configuration](../reference/configuration.md). Nothing here is required to run + [configuration](../reference/configuration/index.md). Nothing here is required to run the CLI. Service code is deliberately kept separate from the core so it can live in its own private repository. diff --git a/docs/getting-started/configuration-file.md b/docs/getting-started/configuration-file.md index 6196aa80..04c6a4ac 100644 --- a/docs/getting-started/configuration-file.md +++ b/docs/getting-started/configuration-file.md @@ -3,7 +3,7 @@ Almost everything about how unshackle behaves is controlled by a single YAML file named `unshackle.yaml`: where it saves downloads, which CDM it uses, your credentials, proxies, filename templates, and more. This page covers where that file lives, how unshackle finds it, the shape of its contents, the directory layout unshackle builds around it, and how to edit values from the command line. !!! note "Where the exhaustive key list lives" - This page is an overview. For the complete, key-by-key breakdown of every setting, its type, and its default, see the [Configuration Reference](../reference/configuration.md). + This page is an overview. For the complete, key-by-key breakdown of every setting, its type, and its default, see the [Configuration Reference](../reference/configuration/index.md). ## The config filename @@ -89,7 +89,7 @@ The top-level keys group loosely into these areas: | Behavior & logging | `update_checks`, `redact_paths`, `debug`, `unicode_filenames` | | Paths | `directories`, `filenames` | -For the full list with types and defaults, see the [Configuration Reference](../reference/configuration.md). +For the full list with types and defaults, see the [Configuration Reference](../reference/configuration/index.md). !!! warning "Unknown keys are silently ignored" unshackle does not validate your config against a schema. If you misspell a key, it is simply skipped and the default is used. You will not get an error. Double-check key names (and their nesting) if a setting does not seem to apply. @@ -126,7 +126,7 @@ services: !!! tip "A few keys are named after the flag's internal name" Most keys are obvious, but set these exact ones: `range` (`-r`), `list` (`--list`), `tmdb_id` (`--tmdb`), `imdb_id` (`--imdb`), `tvdb_id` (`--tvdb`), `no_atmos` (`--noatmos`), and `output_dir` - (`-o`). The [Configuration Reference](../reference/configuration.md#dl) has the full list + (`-o`). The [Configuration Reference](../reference/configuration/download.md#dl) has the full list and every available key. ## The directory layout @@ -177,7 +177,7 @@ unshackle searches entries in the order listed, and **the first source to define ## The `filenames` key -Alongside `directories`, the `filenames` key lets you override the templates unshackle uses when naming its own working and log files (for example the log filename or the temporary chapters file). Most users never need to touch this. The available names and their default templates are listed in the [Configuration Reference](../reference/configuration.md#filenames). +Alongside `directories`, the `filenames` key lets you override the templates unshackle uses when naming its own working and log files (for example the log filename or the temporary chapters file). Most users never need to touch this. The available names and their default templates are listed in the [Configuration Reference](../reference/configuration/directories.md#filenames). ## Editing the config with `unshackle cfg` @@ -223,6 +223,6 @@ When it writes, `unshackle cfg` targets the config file that was loaded. If none ## Next steps -- Browse the [Configuration Reference](../reference/configuration.md) for every key and default. +- Browse the [Configuration Reference](../reference/configuration/index.md) for every key and default. - Set up your first download in [Downloading](../guide/downloading.md). - Learn how services are discovered and updated in [Creating a Service](../dev/creating-a-service.md). diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index af9d8d52..f993ae91 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -108,7 +108,7 @@ These must be present for core downloading, decryption, and muxing to work: !!! warning "shaka-packager is the default decryptor" unshackle decrypts with shaka-packager unless you explicitly configure a different decryptor. If it is missing, protected downloads will fail at the - decryption step. See the [configuration reference](../reference/configuration.md#decryption) for the + decryption step. See the [configuration reference](../reference/configuration/drm.md#decryption) for the `decryption` key if you want to switch to `mp4decrypt`. ### Recommended and optional tools diff --git a/docs/guide/cli-reference.md b/docs/guide/cli-reference.md index 09473416..e6a1c8f9 100644 --- a/docs/guide/cli-reference.md +++ b/docs/guide/cli-reference.md @@ -103,7 +103,7 @@ unshackle dl [OPTIONS] SERVICE [SERVICE ARGS...] !!! tip "These flags select languages, they do not order them" Naming languages with `-l` or `-sl` removes the others. To keep every language but put your preferred ones first, use `audio.language_priority` and - [`subtitle.language_priority`](../reference/configuration.md#subtitle) in your + [`subtitle.language_priority`](../reference/configuration/download.md#subtitle) in your configuration file. The special language tokens `orig`, `all`, and `best` are honoured everywhere a language is expected. diff --git a/docs/guide/downloading.md b/docs/guide/downloading.md index 6686a9d4..047b4be1 100644 --- a/docs/guide/downloading.md +++ b/docs/guide/downloading.md @@ -272,7 +272,7 @@ alphabetically, after the title's original language. Nothing is removed. Audio w same way, but the languages you leave out keep their bitrate and codec order instead of an alphabetical one. -See [`subtitle.language_priority`](../reference/configuration.md#subtitle) for the details. +See [`subtitle.language_priority`](../reference/configuration/download.md#subtitle) for the details. ### Exact vs fuzzy matching diff --git a/docs/guide/drm-and-cdm.md b/docs/guide/drm-and-cdm.md index 69c691e5..27824cf8 100644 --- a/docs/guide/drm-and-cdm.md +++ b/docs/guide/drm-and-cdm.md @@ -516,5 +516,5 @@ debug_requests: true - [Downloading](downloading.md): the `dl` command and its DRM-related flags. - [Key vaults](vaults.md): caching content keys across downloads. -- [Configuration](../reference/configuration.md): the full `unshackle.yaml` reference. +- [Configuration](../reference/configuration/index.md): the full `unshackle.yaml` reference. - [Dependencies](../getting-started/installation.md): installing shaka-packager and mp4decrypt. diff --git a/docs/guide/output-and-naming.md b/docs/guide/output-and-naming.md index 5b8b1647..9b412d05 100644 --- a/docs/guide/output-and-naming.md +++ b/docs/guide/output-and-naming.md @@ -102,6 +102,7 @@ Every variable below is valid in both output and folder templates. Values are de | Variable | Meaning | Example | |---|---|---| | `title` | Title name (movie/show/song name; `$` is rendered as `S`) | `The Show` | +| `title_type` | Media kind of the title | `movie`, `series`, `music` | | `year` | Release year | `2024` | | `source` | Service tag / class name (empty with `--no-source`) | `EXAMPLE` | | `quality` | Resolution with scan suffix | `1080p`, `2160p`, `576i` | @@ -138,7 +139,7 @@ Every variable below is valid in both output and folder templates. Values are de | `audio_channels` | Channel layout | `5.1`, `2.0` | | `audio_full` | Codec + channels combined | `DDP5.1` | | `atmos` | `Atmos` when any track carries JOC | `Atmos` | -| `dual` | `DUAL` for two audio languages (see [`dual_multi_mode`](../reference/configuration.md#dual_multi_mode)) | `DUAL` | +| `dual` | `DUAL` for two audio languages (see [`dual_multi_mode`](../reference/configuration/download.md#dual_multi_mode)) | `DUAL` | | `multi` | `MULTi` for more than two audio languages | `MULTi` | | `dubbed` | `DUBBED` for a single audio language that is not the original (strict mode only) | `DUBBED` | @@ -234,6 +235,10 @@ The `--no-mux` flag skips muxing entirely and writes the individual track files, ## Tags & group naming The release-group tag is the `-TAG` portion at the end of scene-style names, produced by the `{tag}` variable. +It can be swapped per release with [`tag_rules`](../reference/configuration/output.md#tag_rules): ordered rules that +match on the release-attribute variables only (`quality`, `resolution`, `hdr`, `source`, `lang_tag`, +`title_type`, and so on), first match wins. The title's own fields, such as `title` and `season_episode`, +cannot be matched. See the [reference page](../reference/configuration/output.md#tag_rules) for the exact list. ```yaml title="unshackle.yaml" tag: "MYGROUP" @@ -405,7 +410,7 @@ how the optional (`?`) variables appear and disappear. === "Multi-language" Place `{dual?}`, `{multi?}` and `{dubbed?}` next to each other. At most one of them is - ever set (see [`dual_multi_mode`](../reference/configuration.md#dual_multi_mode)), and the + ever set (see [`dual_multi_mode`](../reference/configuration/download.md#dual_multi_mode)), and the empty ones collapse: ```yaml @@ -439,7 +444,7 @@ how the optional (`?`) variables appear and disappear. ``` For finer control (e.g. a `SUBBED` tag driven by subtitle languages), use `{lang_tag?}` - with [`language_tags` rules](../reference/configuration.md#language_tags) instead. + with [`language_tags` rules](../reference/configuration/download.md#language_tags) instead. === "Daily shows" diff --git a/docs/guide/subtitles.md b/docs/guide/subtitles.md index 62d43835..96159d23 100644 --- a/docs/guide/subtitles.md +++ b/docs/guide/subtitles.md @@ -154,8 +154,8 @@ flags drive selection, sort order, output filenames, and the Matroska track flag By default this type order is the primary sort, so all forced tracks come first, then all normal tracks, then all SDH/CC tracks, each block sorted by language. To keep each language next to its own variants instead, set - [`subtitle.group_by`](../reference/configuration.md#subtitle) to `language`. Use - [`subtitle.type_priority`](../reference/configuration.md#subtitle) to change the + [`subtitle.group_by`](../reference/configuration/download.md#subtitle) to `language`. Use + [`subtitle.type_priority`](../reference/configuration/download.md#subtitle) to change the Forced → Normal → SDH/CC order itself. A single track cannot be both CC and SDH, and a forced track cannot also be flagged diff --git a/docs/index.md b/docs/index.md index 5eb5fcb3..64cdd785 100644 --- a/docs/index.md +++ b/docs/index.md @@ -86,7 +86,7 @@ install to your first file. - **[Installation](getting-started/installation.md)**. Install unshackle and the external tools it depends on. - **[Quickstart](getting-started/quickstart.md)**. Configure `unshackle.yaml` and run your first download. - **[Downloading](guide/downloading.md)**. The full `dl` command guide: quality, codecs, languages, track selection, and output. -- **[Configuration](getting-started/configuration-file.md)**. Where `unshackle.yaml` lives, the directory layout, and how to edit values. Full key list in the [Configuration Reference](reference/configuration.md). +- **[Configuration](getting-started/configuration-file.md)**. Where `unshackle.yaml` lives, the directory layout, and how to edit values. Full key list in the [Configuration Reference](reference/configuration/index.md). - **[REST API](dev/rest-api/index.md)**. Run the `serve` HTTP server and drive downloads programmatically. !!! tip "Getting help" diff --git a/docs/reference/api/config.md b/docs/reference/api/config.md index 3961a558..9e29b85e 100644 --- a/docs/reference/api/config.md +++ b/docs/reference/api/config.md @@ -2,7 +2,7 @@ Auto-generated reference for the [`Config`][unshackle.core.config.Config] object that models the `unshackle` configuration file. For a human-friendly walkthrough of every key, see the -[Configuration Reference](../configuration.md). +[Configuration Reference](../configuration/index.md). ::: unshackle.core.config options: diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index cab4e4ff..e6bfa177 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -1,987 +1,18 @@ -# Configuration Reference +# Configuration Reference (moved) -This page documents every key you can set in -`unshackle.yaml`. For each key you will find its name, expected type, default value, and -what it does. Keys are grouped by area so you can jump straight to the part of unshackle -you are configuring. +This page has moved. The configuration reference is now split by area under +[`docs/reference/configuration/`](configuration/index.md), or on the docs site at +[docs.unshackle.dev/reference/configuration](https://docs.unshackle.dev/reference/configuration/). -If you are new to the config file (where it lives, how unshackle discovers it, and how to -edit it), start with [The Configuration File](../getting-started/configuration-file.md). -This page assumes you already know the basics and want precise details. - -!!! info "How defaults work" - Every key is optional except [`output_template`](#output-naming), which `dl` requires. - Without it, `dl` aborts with an error. If you omit any other key, unshackle uses - the built-in default listed here. There is **no schema validation**: unknown keys are - silently ignored (they never become settings), and the file is parsed with - `yaml.safe_load`. An empty or missing file means every other key falls back to its - default. - -!!! warning "Two keys behave specially on load" - - `curl_impersonate` is a **deprecated** alias of [`network`](#network-proxy) and emits a - `DeprecationWarning`. - - `scene_naming` has been **removed**. If it is present at all, unshackle exits - immediately with an error telling you to use [`output_template`](#output-naming) - instead. - - See [Deprecated & removed keys](#deprecated-removed-keys) at the end of this page. - ---- - -## Quick index - -| Area | Keys | +| Area | Page | |------|------| -| [Directories](#directories) | `directories`, `filenames` | -| [Services & authentication](#services-auth) | `services`, `credentials`, `firefox_cookies`, `remote_services`, `serve` | -| [Download & processing](#download-processing) | `dl`, `subtitle`, `audio`, `muxing`, `language_tags`, `dual_multi_mode` | -| [Output & naming](#output-naming) | `output_template`, `tag`, `tag_group_name`, `tag_imdb_tmdb`, `chapter_fallback_name`, `unicode_filenames` | -| [DRM & CDM](#drm-cdm) | `cdm`, `remote_cdm`, `decryption` | -| [Network & proxy](#network-proxy) | `network`, `headers`, `proxy_providers` | -| [Key vaults](#key-vaults) | `key_vaults`, `vault_timeout` | -| [External API keys](#external-api-keys) | `imdb_api_enabled`, `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `decrypt_labs_api_key`, `ipinfo_api_key` | -| [Caching & updates](#caching-updates) | `title_cache_enabled`, `title_cache_time`, `title_cache_max_retention`, `update_checks`, `update_check_interval` | -| [Logging, privacy & debug](#logging-privacy-debug) | `redact_paths`, `debug`, `debug_keys`, `debug_requests`, `set_terminal_bg` | -| [Deprecated & removed](#deprecated-removed-keys) | `curl_impersonate`, `downloader`, `scene_naming` | - ---- - -## Directories { #directories } - -The `directories` key is a **dict** mapping directory names to filesystem paths. Only the -names listed below are honoured; a handful are **protected** and cannot be moved (attempts -to override them are silently skipped). All defaults are computed relative to the installed -package, and every user-settable path is passed through `Path(...).expanduser()`, so `~` -works. - -```yaml title="unshackle.yaml" -directories: - downloads: ~/Media/unshackle - temp: /mnt/fast-scratch/unshackle-temp - wvds: ~/.unshackle/WVDs - prds: ~/.unshackle/PRDs -``` - -| Key | Type | Default | Overridable | Purpose | -|-----|------|---------|:-----------:|---------| -| `downloads` | path | `/downloads` | ✅ | Default output directory for finished files. | -| `temp` | path | `/temp` | ✅ | Temporary working files during download/decrypt/mux. | -| `cache` | path | `/cache` | ✅ | Generic cache, title cache, and the update-check store. | -| `cookies` | path | `/cookies` | ✅ | Per-service cookie files (and VPN cookie/token files). | -| `logs` | path | `/logs` | ✅ | Log files. | -| `exports` | path | `/exports` | ✅ | Export JSON files. | -| `wvds` | path | `/WVDs` | ✅ | Widevine device files (`.wvd`). | -| `prds` | path | `/PRDs` | ✅ | PlayReady device files (`.prd`). | -| `dcsl` | path | `/DCSL` | ✅ | DCSL data. | -| `commands` | path | `unshackle/commands` | ✅ | CLI command modules. | -| `services` | list \| path | `[unshackle/services]` | ✅ | Service search paths and/or remote repo specs (see below). | -| `vaults` | path | `unshackle/vaults` | ✅ | Vault backend modules. | -| `fonts` | path | `unshackle/fonts` | ✅ | Bundled fonts. | -| `user_configs` | path | `unshackle/` | ❌ protected | Where `unshackle.yaml` lives. | -| `data` | path | `unshackle/` | ❌ protected | Base for the data subdirectories above. | -| `core_dir` | path | `unshackle/core` | ❌ protected | Package core. | -| `namespace_dir` | path | `unshackle/` | ❌ protected | Package root. | -| `app_dirs` | - | `AppDirs("unshackle", False)` | ❌ protected | Internal AppDirs instance. | - -!!! note "The `services` directory is special" - `services` may be a **list**, and each entry can be either a local directory or a - **repository spec**: a git URL (`https://...`, `ssh://...`, `git@...`, or anything ending in - `.git`) or `owner/repo` shorthand. Repo specs are cloned and updated automatically; plain - paths are used as-is. **List order is priority**: the first source to define a service - tag wins, so put your own local service directories **last** to make them fallbacks. - - ```yaml - directories: - services: - - you/your-services # cloned from GitHub - - https://example.com/private.git@stable - - ~/my-services # local, lowest priority - ``` - -!!! info "How git-backed service repos are handled" - unshackle's use of git is **read-only on the remote**: it only ever performs a shallow - `clone`, `fetch`, `pull`, and a local `reset`; **nothing is ever pushed**. Private repos - are authenticated through your existing git credential helper, and unshackle stores no - tokens of its own. Clones live under `/_repos//` (or - the bundled `unshackle/services`), and **nothing is written to the cache directory**. After - the first clone unshackle re-pulls **at most once every 24 h**, so it does not touch the - network on every run. - -!!! warning "The two refresh paths diverge on purpose" - The **automatic** 24 h-TTL refresh that happens during a normal `dl`/`search` run will - deliberately **refuse to refresh and exit** (naming the offending clone) if that clone - has uncommitted changes to tracked files or unpushed local commits, so a background pull can - never clobber work in progress. Untracked files (new service folders, `__pycache__`) never - block it. The **manual** `unshackle util refresh-services` command does the opposite: it - **hard-resets** the clone to upstream, discarding any local edits. Reach for the manual - command only when you actually want to throw local changes away. - -!!! warning "Read-only installs and reinstalls" - If the services directory lives inside the installed package, a reinstall can delete the - `_repos` clones. They are simply re-cloned on next use. On **read-only installs** you must - point `services` at a writable path, or cloning will fail. - -### Filenames { #filenames } - -The `filenames` key is a **dict** of templated file/name patterns. Each value is used -verbatim (no path processing). Braced fields like `{time}` and `{service}` are filled in at -runtime. - -| Key | Type | Default | Notes | -|-----|------|---------|-------| -| `log` | str | `"unshackle_{name}_{time}.log"` | Written under `directories.logs`. | -| `debug_log` | str | `"unshackle_debug_{service}_{time}.jsonl"` | Structured debug log, under `directories.logs`. | -| `config` | str | `"config.yaml"` | Per-service config file, under that service's directory. | -| `root_config` | str | `"unshackle.yaml"` | The main config filename itself. | -| `chapters` | str | `"Chapters_{title}_{random}.txt"` | Under `directories.temp`. | -| `subtitle` | str | `"Subtitle_{id}_{language}.srt"` | Under `directories.temp`. | - ---- - -## Services & authentication { #services-auth } - -### `services` - -- **Type:** `dict` keyed by service tag  ·  **Default:** `{}` - -Per-service configuration, keyed by the canonical service tag (for example `EXAMPLE1`, `EXAMPLE2`, -`EXAMPLE3`). unshackle reads several well-known sub-keys out of each service's block: - -- **`proxy_map`**: a dict mapping `"provider:query"` or `"query"` to a proxy value, used to - auto-select a proxy for that service. -- **`title_map`**: an exact-match rename map applied to fetched titles (source name → - desired name), so a service that names a title differently from your library still matches. - -Individual services may read any additional keys they define. The whole block is handed to -the service as `self.config`. - -!!! tip "Why put secrets here and not in the service's own `config.yaml`?" - A service ships a per-service `config.yaml` inside its own folder, and that file is - typically shared/committed and **not** meant to be edited by you. So anything user- or - device-specific (API keys, account IDs, device attributes) belongs in this `services` - block of `unshackle.yaml` instead, where it is **merged over** the service's `config.yaml` - at load time. That is why the two config locations exist: one for shared service defaults, - one for your private overrides. - -```yaml -services: - EXAMPLE1: - proxy_map: - default: us - title_map: - "My Show: The Movie": "My Show Presents The Movie" -``` - -### `credentials` - -- **Type:** `dict` keyed by service tag  ·  **Default:** `{}` - -Per-service login credentials. Each value is a `username:password[:extra]` string (a list or -dict form is also accepted where a service supports it). These are parsed into `Credential` -objects; the credential's SHA-1 is also used as an account hash for cache keys. - -```yaml -credentials: - EXAMPLE2: user@example.com:hunter2 - EXAMPLE1: - - primary@example.com:pw1 - - secondary@example.com:pw2 -``` - -!!! tip "Cookies vs credentials" - Cookies are stored as files under [`directories.cookies`](#directories), not in this key. - A service's `authenticate()` accepts cookies, credentials, or both. - -### `firefox_cookies` - -- **Type:** `dict`  ·  **Default:** `{}` - -Settings for extracting cookies directly from a local Firefox profile. A service block is -expected to provide `hosts` (a list of cookie hostnames; entries shorter than 3 characters -are ignored to prevent dumping the whole cookie store) and an optional `local_storage` -boolean to also pull matching local-storage entries. Extraction is read-only. - -!!! note "Firefox does not need to be closed" - The extractor copies **both** `cookies.sqlite` **and** its WAL file into a `0700` temp - directory; the WAL copy exists to capture writes Firefox has not yet flushed to the main - DB, so the copy is current even against a running browser. The only real constraint is that - Firefox must not hold an exclusive write lock at the exact instant of the copy. The live - profile is otherwise left untouched. - -!!! tip "When you actually need `local_storage: true`" - Enable it only for services that keep their auth tokens in the browser's **localStorage** - (`webappsstore.sqlite`) rather than in HTTP cookies. If a service authenticates with plain - cookies, leaving it off is correct. - -!!! warning "Extraction fails silently to file cookies" - If extraction yields no cookies or fails for **any** reason, unshackle silently falls back - to the normal file-based cookie path (`cookies/.txt` or - `cookies//.txt`). This is a safety net, but it also means that if you - expected Firefox cookies and file cookies are being used instead, extraction quietly failed. - Check that first when troubleshooting. - -### `remote_services` - -- **Type:** `dict`  ·  **Default:** `{}` - -Definitions of remote unshackle service servers, used by the `--remote` mode. Each entry -carries a `services` sub-dict of remote service tags, which unshackle turns into synthetic -CLI commands that run against the remote server. - -### `serve` - -- **Type:** `dict`  ·  **Default:** `{}` - -Configuration for the `serve` command (the built-in REST API server). The full server guide -is the [REST API](../dev/rest-api/index.md) section; these are the config keys. - -| Sub-key | Type | Default | Description | -|---------|------|---------|-------------| -| `api_secret` | str | *(unset)* | Master secret accepted in the `X-Secret-Key` header. | -| `users` | dict | `{}` | Per-user API keys and their allowlists (see below). | -| `services` | list | *(unset)* | Global service allowlist. Omit to allow all. | -| `remote_only` | bool | `false` | Expose only the remote service session endpoints (health, services, search, session) and disable the rest of the REST API. | -| `session_ttl` | int (s) | `300` | Lifetime of an interactive auth session. | -| `max_sessions` | int | `100` | Maximum concurrent sessions. | -| `history_limit` | int | `100` | How many finished jobs to retain in history. | -| `compression_level` | int | `1` | gzip level for responses. | -| `global_speed_limit` | str | *(unlimited)* | Server-wide download speed cap, e.g. `10M`, `1.5G` or plain bytes/sec (same format as `speed_limit`). One shared budget across all concurrent jobs; per-job speed limits are ignored while it is set. | -| `cdm_overrides` | dict | *(unset)* | Allowed per-request CDM overrides. | -| `allow_job_credentials` | bool | `false` | Permit clients to supply credentials per job. | -| `devices` | list | *(auto)* | Widevine devices offered; auto-filled from `directories.wvds`. | -| `playready_devices` | list | *(auto)* | PlayReady devices; auto-filled from `directories.prds`. | - -Each entry under `users` is keyed by that user's API key and may set its own `services`, -`devices`, and `playready_devices` allowlists, narrowing the global ones. - -```yaml -serve: - api_secret: change-me - remote_only: true - services: [EXAMPLE1, EXAMPLE2] - # server-wide download defaults (same keys as the `dl:` block) - downloads: 3 - best_available: true - users: - a1b2c3d4: # this user's API key - services: [EXAMPLE1] # may only use EXAMPLE1 -``` - -!!! tip "Declaring `dl` defaults under `serve`" - You can set most [`dl`](#dl) flag keys (`downloads`, `workers`, `best_available`, and so on) - directly inside `serve`; the server recognises a fixed subset of download parameters, so a - few CLI-only flags are ignored here. This raises concurrency and behaviour defaults - **server-wide** (every request the server handles picks them up) without changing - each client call individually. - ---- - -## Download & processing { #download-processing } - -### `dl` - -- **Type:** `dict`  ·  **Default:** `{}` - -Persistent defaults for the `dl` command. This block is used as Click's `default_map`, so -**any** `dl` flag can be given a default here and it applies to every download without you -typing it. See [Downloading](../guide/downloading.md) and the -[CLI Reference](../guide/cli-reference.md) for what each flag does. - -**Key naming.** A key is the flag's long name with dashes replaced by underscores -(`--best-available` → `best_available`, `--sub-format` → `sub_format`). A few keys are named -after the flag's internal destination rather than its visible name; set these exact keys: - -| Flag | Config key | -|------|-----------| -| `-r` / `--range` | `range` | -| `--list` | `list` | -| `--tmdb` | `tmdb_id` | -| `--imdb` | `imdb_id` | -| `--animeapi` | `animeapi_id` | -| `-naa` / `--noatmos` | `no_atmos` | -| `-o` / `--output` | `output_dir` | -| `--cdm-only` / `--vaults-only` | `cdm_only` | - -**Precedence** (highest first): an explicit CLI flag → a per-service -[`services..dl`](#services-auth) override → this global `dl:` block → the built-in -default. In other words, config only fills in options you did not set on the command line. - -Common keys (a useful subset; every `dl` flag works): - -| Key | Type | Default | Sets | -|-----|------|---------|------| -| `lang` | list | `["orig"]` | Video/audio language(s); `orig` = original language. | -| `a_lang` / `v_lang` | list | `[]` | Audio- / video-only language override. | -| `s_lang` | list | `["all"]` | Subtitle language(s). | -| `quality` | list | `[]` (best) | Resolution(s), e.g. `[1080]`. | -| `vcodec` | list | `[]` (any) | Video codec(s), e.g. `[H265]`. | -| `acodec` | list | `[]` (any) | Audio codec(s), e.g. `[EC3]`. | -| `range` | list | `["SDR"]` | Colour range(s): `SDR`, `HDR10`, `HDR10+`, `DV`. | -| `channels` | float | *(unset)* | Audio channels, e.g. `6` for 5.1. | -| `sub_format` | str | *(unset)* | Convert subtitles to this format (`srt`, `vtt`, `original`, ...). | -| `forced_subs` | bool | `false` | Include forced subtitle tracks. | -| `forced_s_lang` | list | `[]` | Forced subtitle language(s); implies `forced_subs`. | -| `no_subs` / `no_audio` / `no_chapters` | bool | `false` | Skip that track type. | -| `downloads` | int | `1` | Tracks downloaded concurrently. | -| `workers` | int | *(downloader default)* | Threads per track. | -| `speed_limit` | str | *(unlimited)* | Total download speed cap across all downloads combined, e.g. `500k`, `5M`, `1.5G` or plain bytes/sec. Values are bytes, not bits (`5M` = 5.0 MB/s). | -| `slow` | str | *(unset)* | Inter-title delay, e.g. `"20-40"`. | -| `best_available` | bool | `false` | Fall back to best quality if the request is unavailable. | -| `proxy` | str | *(unset)* | Default proxy URI or 2-letter country. | -| `no_folder` | bool | `false` | Do not create a per-title folder. | - -```yaml title="Global download defaults" -dl: - lang: [en] - quality: [1080] - range: [HDR10] - vcodec: [H265] - sub_format: srt - downloads: 2 -``` - -!!! tip "Per-service overrides" - Nest a `dl` block under a service in [`services`](#services-auth) to override these defaults - for just that service. It uses the same keys and wins over the global `dl:` block (but still - loses to an explicit CLI flag). - - ```yaml - dl: - lang: [en] # global default - services: - EXAMPLE1: - dl: - lang: [en, ja] # EXAMPLE1 downloads both; everything else stays English - range: [DV] - ``` - -!!! note "`serve` uses these same keys" - The [`serve`](#serve) block accepts the same `dl` option keys at its top level to set - server-wide download defaults for every request it handles. - -### `subtitle` - -- **Type:** `dict`  ·  **Default:** `{}` - -Subtitle handling: SDH stripping, format conversion, and whether subtitles are muxed in or -written as sidecar files. See [Subtitles](../guide/subtitles.md) for the full guide. - -| Sub-key | Type | Default | Description | -|---------|------|---------|-------------| -| `strip_sdh` | bool | `true` | Strip SDH/CC cues (hearing-impaired annotations) into a clean track. | -| `sdh_method` | str | `"auto"` | SDH-stripping backend to use. | -| `preserve_formatting` | bool | `true` | Keep styling/positioning when converting or stripping. | -| `convert_before_strip` | bool | `true` | Convert to the working format before stripping SDH. | -| `conversion_method` | str | `"auto"` | Subtitle conversion backend. | -| `preferred_conversion_method` | str | *(unset)* | Preferred converter when several are available. | -| `output_mode` | str | `"mux"` | `mux` embeds subtitles in the MKV; `sidecar` writes separate files; `both` writes sidecars and still muxes. | -| `sidecar_format` | str | `"srt"` | Format for sidecar files when `output_mode: sidecar`. | -| `type_priority` | list | *(unset)* | Ordered ranking of subtitle types (`forced`, `normal`, `sdh`; CC counts as SDH). Types you leave out fall to the end. | -| `group_by` | str | `"type"` | Which key groups the subtitle tracks. `type` puts all forced tracks together, then all normal, then all SDH. `language` keeps each language next to its own variants. | -| `language_priority` | list | *(unset)* | Languages to sort to the top, in the order you give. The rest keep the default order: original language first, then alphabetical. Nothing is removed. | - -```yaml -subtitle: - strip_sdh: true - output_mode: sidecar - sidecar_format: srt -``` - -!!! note "`group_by` puts a language next to its own SDH track" - The default `type` order groups by subtitle type first, so every normal track comes - before every SDH track. In a title with many languages, a track ends up far from its - own SDH version, for example Finnish at track 5 and Finnish SDH at track 42. Set - `group_by: language` to keep them together: - - ```yaml - subtitle: - group_by: language - ``` - - | `group_by` | Result | - |---|---| - | `type` | Danish, Finnish, French, … Danish SDH, Finnish SDH, French SDH | - | `language` | Danish, Danish SDH, Finnish, Finnish SDH, French, French SDH | - - `type_priority` still sets the order of the variants, so with `group_by: language` it - controls the order inside each language. - -!!! note "`language_priority` sorts languages, it does not select them" - By default the title's original language comes first and every other language follows - alphabetically. For an English title that gives English, Arabic, Bulgarian, and the - rest. Give `language_priority` the languages you want first: - - ```yaml - subtitle: - group_by: language - language_priority: [en, es, fr] - ``` - - | | Result for an English title | - |---|---| - | Default | English, Arabic, Bulgarian, Spanish, French, Japanese | - | With the list above | English, Spanish, French, Arabic, Bulgarian, Japanese | - - This is only an order. Every other language is still downloaded, which is what makes - it different from `-sl`: `-sl en,es,fr` **removes** all other subtitles. - - Use `orig` in the list for the title's original language. A base tag also covers its - regional forms, so `en` sorts `en-US` and `en-GB`, but `zh` does **not** cover - `zh-Hant`, which you must name. The sort ignores entries that match no track, and it - ignores entries that are not valid language tags. - - `--exact-lang` applies here too: with it, `en` sorts only `en`, and `en-US` falls to - the alphabetical tail with the rest. Name each tag you want when you use that flag. - -!!! note "`type_priority` also picks the default subtitle" - At mux time the first track in the default subtitle language gets the `default` flag. - The built-in order is `forced`, `normal`, `sdh`, which means a forced track becomes - default whenever one exists. If you want the full dialogue track as default instead, - set: - - ```yaml - subtitle: - type_priority: [normal, sdh, forced] - ``` - -### `audio` - -- **Type:** `dict`  ·  **Default:** `{}` - -Audio handling options. The most notable sub-key is **`codec_priority`**, an ordered list of -audio codecs used to break ties in track selection. **`language_priority`** is an ordered list -of languages that sort to the top of the audio tracks. - -```yaml -audio: - codec_priority: - - EAC3 - - AC3 - language_priority: - - orig - - en -``` - -!!! note "`codec_priority` is a soft priority: nothing is dropped" - `codec_priority` outranks bitrate: any listed codec sorts above every unlisted one, and among - listed codecs the list order decides. Bitrate order survives only within a single codec rank. - Nothing is ever removed. Two higher-level rules still override it: **Atmos - tracks always take precedence over `codec_priority`**, and **audio-description tracks are - always forced to the end** regardless of it. When `codec_priority` is unset, tracks sort by - bitrate alone, with those same Atmos/descriptive rules still applied. - -### `muxing` - -- **Type:** `dict`  ·  **Default:** `{}` - -Matroska (MKV) muxing options. - -| Sub-key | Type | Default | Description | -|---------|------|---------|-------------| -| `set_title` | bool | `true` | Write a human-readable title into the MKV container. | -| `default_language` | dict | `{}` | Force which language is flagged *default* per track type, e.g. `{audio: en, subtitle: en}`. | -| `merge_audio` | bool | `true` | Merge all audio into one file. `--split-audio` on the CLI flips this off. | -| `merge_video` | bool | `false` | Merge video tracks that share height, range, and codec into one file, so only language varies inside a file. `--merge-video` on the CLI flips this on. | - -```yaml -muxing: - set_title: true - default_language: - audio: en - subtitle: en -``` - -### `language_tags` - -- **Type:** `dict`  ·  **Default:** `{}` - -Language-tag remapping and the rule engine behind the `{lang_tag}` filename variable. When a -`rules` list is present, each rule is evaluated in order and the first match's `tag` is used. -A rule may test: - -| Condition | Meaning | -|-----------|---------| -| `audio` | Matches against the audio track languages. | -| `subs_contain` | At least one subtitle language matches. | -| `subs_contain_all` | Every listed language must be present (scalar or list). | - -```yaml -language_tags: - rules: - - audio: [ja] - subs_contain: [en] - tag: "SUBBED" -``` - -### `dual_multi_mode` - -- **Type:** `str`  ·  **Default:** `strict` - -Rule set for the `{dual}`, `{multi}`, and `{dubbed}` filename variables. - -=== "`strict` (default)" - - | Variable | Set when | - |----------|----------| - | `{dual}` | Exactly two audio languages **and one is the title's original language**. | - | `{multi}` | Three or more audio languages, even without the original. | - | `{dubbed}` | A single audio language that is not the title's original language (requires a known original). | - -=== "`count` (legacy)" - - | Variable | Set when | - |----------|----------| - | `{dual}` | Exactly two audio languages (original language ignored). | - | `{multi}` | Three or more audio languages. | - | `{dubbed}` | Never set. | - -```yaml -dual_multi_mode: strict -``` - -!!! note "Dialects count as one language" - Regional variants of the same base language (e.g. `en-US` and `en-GB`) collapse to a single - language in both modes, so they never trigger `{dual}` on their own. - ---- - -## Output & naming { #output-naming } - -For the full picture of how templates render, see -[Output & naming](../guide/output-and-naming.md). The keys are summarised here. - -### `output_template` - -- **Type:** `dict`  ·  **Required** (no usable default; `dl` exits with an error if this is missing) - -Filename templates keyed by media kind, using `{variable}` placeholders. Recognised kinds are -`movies`, `series`, and `songs` (and `albums` for folder templates). A trailing `?` on a -variable (e.g. `{quality?}`) marks it optional: if empty, it and one adjacent separator are -dropped. - -```yaml title="unshackle.yaml" -output_template: - movies: "{title}.{year}.{quality?}.{source}-{tag}" - series: "{title}.{season_episode}.{episode_name?}.{quality?}.{source}-{tag}" - songs: "{track_number}. {title}" - folder: # special nested key, see below - movies: "{title} ({year})" - series: "{title} ({year})" -``` - -**The nested `folder` sub-key** controls output subfolders and is popped out of -`output_template`: - -- A **dict** sets per-kind folder templates (kinds: `movies`, `series`, `songs`, `albums`). -- A **str** sets a single folder template for everything. - -unshackle validates templates on load and emits warnings for unknown variables, non-string -values, filesystem-unsafe characters (`< > : " / \ | ? *`), empty templates, and unknown -folder-kind keys. - -??? example "All valid template variables" - `title`, `year`, `season`, `episode`, `season_episode`, `episode_name`, `date`, - `quality`, `resolution`, `source`, `tag`, `track_number`, `artist`, `album_artist`, - `album`, `disc`, `track_total`, `disc_total`, `release_type`, `genre`, `explicit`, - `isrc`, `upc`, `label`, `audio`, `audio_channels`, `audio_full`, `atmos`, `dual`, - `multi`, `dubbed`, `video`, `hdr`, `hfr`, `edition`, `repack`, `lang_tag`. - -### Tagging & naming keys - -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| `tag` | str | `""` | Release-group tag appended to filenames and written into MKV tags. | -| `tag_group_name` | bool | `true` | Include the group name (`tag`) in the MKV `Group` tag. | -| `tag_imdb_tmdb` | bool | `true` | Write IMDb/TMDB/TVDB external-ID tags into the MKV (needs metadata providers). | -| `chapter_fallback_name` | str | `""` | Fallback chapter-name template, e.g. `"Chapter {i:02}"`. | -| `unicode_filenames` | bool | `false` | Allow Unicode in filenames; when `false`, names are ASCII-sanitised. | - ---- - -## DRM & CDM { #drm-cdm } - -The full conceptual guide lives at [DRM & CDM](../guide/drm-and-cdm.md). These are the config -keys. Device files themselves live in [`directories.wvds`](#directories) (`.wvd`) and -`directories.prds` (`.prd`). - -### `cdm` - -- **Type:** `dict`  ·  **Default:** `{}` - -Maps a service tag to the CDM (Widevine/PlayReady device) to use, with a `default` fallback. -Resolution is case-insensitive: an explicit CLI override wins, then the per-service entry, -then `default`. - -```yaml -cdm: - default: chromecdm_l3 - EXAMPLE2: android_l1 -``` - -A value may also be a **nested dict** for advanced selection by quality, DRM system, or -profile: - -```yaml -cdm: - EXAMPLE1: - ">=1080": android_l1 # by track height - "<1080": chromecdm_l3 - widevine: android_l1 # by DRM system - playready: sl3000 - default: chromecdm_l3 -``` - -Quality keys support `1080`, `>=1080`, `>720`, `<=576`, `<480` style comparisons; DRM keys -are `widevine` / `playready`. - -### `remote_cdm` - -- **Type:** `list[dict]`  ·  **Default:** `[]` - -A list of remote CDM server definitions. Each entry is matched by its `name` (referenced from -[`cdm`](#cdm)), and its `type` selects the backend: - -| `type` | Backend | Key fields | -|--------|---------|-----------| -| `decrypt_labs` | Decrypt Labs KeyXtractor | `host` (default `https://keyxtractor.decryptlabs.com`), `device_name`, `secret` (falls back to `decrypt_labs_api_key`) | -| `custom_api` | Fully YAML-configurable remote API | `device`, `auth`, `endpoints`, `request_mapping`, `response_mapping`, `caching`, `timeout` | -| *(none)*, `Device Type: PLAYREADY` | pyplayready `RemoteCdm` | `host`, `secret`, `device_name`, `security_level` (default 3000) | -| *(none)*, otherwise | pywidevine `RemoteCdm` | `host`, `secret`, `device_name`, `device_type`, `system_id`, `security_level` (default 3000) | - -```yaml -remote_cdm: - - name: keyxtractor - type: decrypt_labs - device_name: L1 - - name: my_wv_server - host: https://cdm.example.com - secret: s3cr3t - device_name: android_l1 - device_type: ANDROID - system_id: 26830 - security_level: 1 -``` - -### `decryption` - -- **Type:** `str` or `dict`  ·  **Default:** `"shaka"` - -Selects the tool used to physically decrypt CENC content. The value is compared -case-insensitively; `"mp4decrypt"` selects Bento4's mp4decrypt, and **anything else** -(including `"shaka"`) selects shaka-packager. - -=== "Global" - - ```yaml - decryption: shaka - ``` - -=== "Per-service" - - ```yaml - decryption: - default: shaka - EXAMPLE1: mp4decrypt - ``` - ---- - -## Network & proxy { #network-proxy } - -### `network` - -- **Type:** `dict`  ·  **Default:** `{}` - -TLS-fingerprinting and HTTP client settings for the rnet-based session. - -| Sub-key | Type | Default | Description | -|---------|------|---------|-------------| -| `browser` | str | `"Chrome131"` | Impersonation preset. Must be an exact preset name (e.g. `Chrome131`, `Firefox135`, `Edge101`, `Safari18`, `OkHttp4_12`, `OkHttp5`, `Opera118`); unknown names raise an error. | -| `http1_only` | bool | *(unset)* | Force HTTP/1.1. | -| `http2_only` | bool | *(unset)* | Force HTTP/2. | -| `pool_max_idle_per_host` | int | *(unset)* | Connection-pool tuning. | -| `pool_max_size` | int | *(unset)* | Connection-pool tuning. | -| `tcp_nodelay` | bool | *(unset)* | Disable Nagle's algorithm. | - -```yaml -network: - browser: Firefox135 - http2_only: true -``` - -!!! note "When to force `http1_only`" - HTTP/2 is the right default on large CDNs, so leave it on. Force HTTP/1.1 only for a host that - throttles per-connection, or a slow origin stuck behind HTTP/2 flow control. In benchmarks - this won 30 to 50% on such hosts but **cost up to 27% on fast CDNs**, so enable it only when - your own measurements justify it. - -!!! warning "Renamed from `curl_impersonate`" - The old `curl_impersonate` section is a deprecated alias. If you still use it, unshackle - honours it (only when `network` is absent) but emits a `DeprecationWarning`. Rename it to - `network`. - -### `headers` - -- **Type:** `dict`  ·  **Default:** `{}` - -Default HTTP headers merged into every session unshackle creates. - -```yaml -headers: - Accept-Language: en-US,en;q=0.9 -``` - -!!! warning "Don't set `Accept-Encoding` (and similar) here" - Keep this block to sane cross-service defaults such as `Accept-Language` and `User-Agent`. - Compatibility headers like `Accept-Encoding` are set for you by the rnet HTTP backend as - part of its browser-impersonation profile, and overriding them can break the impersonation - fingerprint. Anything useful to only some services belongs in that service's config, not - here. - -### `proxy_providers` - -- **Type:** `dict`  ·  **Default:** `{}` - -Proxy/VPN provider configuration. Each sub-key names a provider, and its block is passed -straight to that provider's constructor. See [Proxies & VPN](../guide/proxies-and-vpn.md) for -the full provider guide. Recognised providers and their exit ports: - -| Provider | Config key | Credentials | Proxy scheme/port | -|----------|-----------|-------------|-------------------| -| Basic (static) | `basic` | country → URI(s) | as specified | -| NordVPN | `nordvpn` | 48-char **service** credentials | `https://...:89` | -| Surfshark | `surfsharkvpn` | 48-char **service** credentials | `https://...:443` | -| Windscribe | `windscribevpn` | service credentials | `https://...:443` | -| ExpressVPN | `expressvpn` | device login (`enable: true`) / token cache | `https://cat:...@...:443` | -| ProtonVPN | `protonvpn` | TV login or exported cookies | `https://...:4443` (or `:443` Secure Core) | -| Gluetun | `gluetun` | per-VPN keys/creds | `http://localhost:{port}` (local Docker) | -| Hola | *(none, auto)* | none | `http://...:{peer}` | - -```yaml -proxy_providers: - basic: - us: http://user:pass@1.2.3.4:8080 - de: - - http://a.example:8080 - - socks5://b.example:1080 - nordvpn: - username: <48-char service username> - password: <48-char service password> -``` - -!!! note "Provider loading differs between CLI and REST server" - The `dl` CLI loads all providers, including `windscribevpn` and `gluetun`. The REST API / - remote-client path uses a separate resolver that does **not** load `windscribevpn` or - `gluetun`. ExpressVPN and ProtonVPN also auto-load when their cached session exists, and Hola - auto-loads whenever the `hola-proxy` binary is present. - ---- - -## Key vaults { #key-vaults } - -The full guide is at [Vaults](../guide/vaults.md). Two keys configure them. - -### `key_vaults` - -- **Type:** `list[dict]`  ·  **Default:** `[]` - -An ordered list of key-vault backends. unshackle queries them in order and reuses content -keys instead of re-licensing. Each entry needs a `type` (the backend module name) and a -`name`, plus backend-specific keys. - -| `type` | Purpose | Required keys | Notes | -|--------|---------|--------------|-------| -| `SQLite` | Local SQLite database | `name`, `path` | Loaded **critically**; a failure aborts the run. | -| `MySQL` | Remote MySQL database | `name`, `host`, `database`, `username` | Extra keys (e.g. `password`, `port`) forwarded to pymysql. | -| `API` | RESTful JSON API | `name`, `uri`, `token` | Honours `vault_timeout`. | -| `HTTP` | HTTP API with modes | `name`, `host`, one of `password`/`api_key` | `api_mode`: `query` (default), `json`, `decrypt_labs`. | - -```yaml -key_vaults: - - type: SQLite - name: Local - path: ~/.unshackle/keys.db - - type: MySQL - name: Team - host: db.example.com - database: keys - username: unshackle - password: hunter2 - no_push: false -``` - -!!! tip "Common per-entry options" - - `no_push: true` makes a vault read-only (keys are fetched but never written to it). - - A vault of `type: API` whose `name` contains `decrypt_labs` auto-fills its `token` from - [`decrypt_labs_api_key`](#external-api-keys) when not set inline. Vault `type` values are - case-sensitive module names. - - An all-zero content key (32 zeros) is treated as "no key" everywhere and is never - stored. - -### `vault_timeout` - -- **Type:** `float`  ·  **Default:** `10.0` - -Timeout in seconds for vault operations. Injected automatically into any backend whose -constructor accepts a `timeout` parameter (a per-vault `timeout` still wins). - ---- - -## External API keys { #external-api-keys } - -All default to an empty string and enable optional metadata/geolocation features. - -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| `imdb_api_enabled` | bool | `false` | Use the free IMDxAPI (api.tiffara.com) metadata provider, which needs no API key. Off by default since the site has been unreliable. | -| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a more reliable alternative to IMDxAPI. Free keys are available on the OMDb site. | -| `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. | -| `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. | -| `tvdb_api_key` | str | `""` | [TheTVDB v4](https://thetvdb.com/api-information) API key; a fallback source to TMDB, strongest on TV series. Free keys are available on the TVDB site. | -| `tvdb_pin` | str | `""` | Subscriber PIN, only needed for a user-supported TVDB key. Leave empty for a normal project key. | -| `tvdb_order` | str | `""` | Default for `--tvdb-order`: renumber episodes to a TVDB season order (`official`, `dvd`, `absolute`, `alternate`, `regional`). Empty keeps the numbering the service gives. | -| `metadata_providers` | list | *(see below)* | Metadata providers to use, in the order they are tried. | -| `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). | -| `ipinfo_api_key` | str | `""` | ipinfo.io API key for IP/region lookups. | - -!!! tip "Choosing and ordering metadata providers" - A lookup goes to each provider in turn and stops at the first good match, so the order - decides which source wins. `metadata_providers` sets that order: - - ```yaml - metadata_providers: [tvdb, tmdb, simkl] - ``` - - Names not in the list are never used, so this both orders and filters. Unknown names are - ignored with a warning. Leave it unset for the default order: - - ```yaml - metadata_providers: [imdbapi, omdb, simkl, tmdb, tvdb] - ``` - - A provider is skipped when its API key is missing, whatever the order. Available names are - `imdbapi`, `omdb`, `simkl`, `tmdb`, and `tvdb`. Put `tvdb` first to make TheTVDB your - primary source; leave it last to use it only as a fallback for TMDB. - -!!! note "`ipinfo_api_key` never touches your service sessions" - The token is only ever sent to `api.ipinfo.io` as a per-request `Authorization` header; it - is **never** attached to your service session, so it cannot leak to a streaming provider. - Lookups degrade gracefully through a fallback chain: the authenticated Lite endpoint (higher - rate limits and richer fields) → anonymous ipinfo → `ip-api.in` as a last resort. - ---- - -## Caching & updates { #caching-updates } - -### Title cache - -unshackle caches fetched title metadata (region- and account-aware) to avoid repeat API -calls. - -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| `title_cache_enabled` | bool | `true` | Master switch for the title cache. | -| `title_cache_time` | int (seconds) | `1800` (30 min) | Lifetime of fresh cached titles. | -| `title_cache_max_retention` | int (seconds) | `86400` (24 h) | How long stale cached data may still be served as a fallback when a live fetch fails. | - -### Update checks - -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| `update_checks` | bool | `true` | Whether to check for new unshackle releases. | -| `update_check_interval` | int (hours) | `24` | Minimum hours between update checks. | - -!!! note - Update checks query the GitHub releases API with a fixed 5-second timeout and cache the - result in `directories.cache/update_check.json`. - ---- - -## Logging, privacy & debug { #logging-privacy-debug } - -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| `redact_paths` | bool | `true` | Mask install-root / venv / home prefixes in logged paths (``, ``, `~`). Set `false` to show full paths. | -| `debug` | bool | `false` | Global debug mode. | -| `debug_keys` | bool | `false` | Log decryption keys. | -| `debug_requests` | bool | `false` | Log HTTP requests. | -| `set_terminal_bg` | bool | `false` | Append the theme's background colour to output styling. | - -!!! tip "When to enable `set_terminal_bg`" - When on, the theme's background colour is appended to the foreground styles so the full - colour palette renders correctly on terminals whose *default* background differs from - unshackle's theme. Turn it on if the ASCII-art banner or coloured output **looks off** on - your terminal. Otherwise leave it off. - -!!! danger "Key exposure" - Decryption content keys are written to the structured debug log at INFO level during the - DRM handshake. Treat `debug`, `debug_keys`, and the `unshackle_debug_*.jsonl` files as - sensitive, and keep `redact_paths` enabled when sharing logs. - - `debug_keys` affects **only** content-encryption keys (the `content_key`/`key` fields). - Passwords, tokens, cookies, and session tokens are **always** redacted regardless of this - setting, and KIDs, key counts, and other metadata are always logged either way. - ---- - -## Deprecated & removed keys { #deprecated-removed-keys } - -| Key | Status | Behaviour | -|-----|--------|-----------| -| `curl_impersonate` | **Deprecated** → use [`network`](#network-proxy) | Emits a `DeprecationWarning`; still honoured only if `network` is absent. | -| `downloader` | **Deprecated** | Any value other than `"requests"` emits a `DeprecationWarning`; the value is otherwise ignored (the unified requests downloader is always used). | -| `scene_naming` | **Removed** | If present at all, unshackle exits with an error directing you to configure [`output_template`](#output-naming) instead. | - ---- - -## A complete annotated example - -The following config sets a handful of common keys. Every key not shown keeps its default. - -```yaml title="unshackle.yaml" -directories: - downloads: ~/Media/unshackle - temp: /mnt/scratch/unshackle - -network: - browser: Chrome131 - -headers: - Accept-Language: en-US,en;q=0.9 - -cdm: - default: chromecdm_l3 - EXAMPLE2: android_l1 - -decryption: shaka - -credentials: - EXAMPLE2: user@example.com:hunter2 - -output_template: - movies: "{title}.{year}.{quality?}.{source}-{tag}" - series: "{title}.{season_episode}.{quality?}.{source}-{tag}" - folder: - movies: "{title} ({year})" - series: "{title} ({year})" - -tag: MYGRP - -key_vaults: - - type: SQLite - name: Local - path: ~/.unshackle/keys.db - -proxy_providers: - basic: - us: http://user:pass@1.2.3.4:8080 - -omdb_api_key: "your-omdb-key" -tmdb_api_key: "your-tmdb-key" -tvdb_api_key: "your-tvdb-key" -title_cache_time: 3600 -redact_paths: true -``` - -For how these interact at runtime, continue to [Downloading](../guide/downloading.md), -[DRM & CDM](../guide/drm-and-cdm.md), and [Output & naming](../guide/output-and-naming.md). +| Overview & quick index | [configuration/index.md](configuration/index.md) | +| Directories & filenames | [configuration/directories.md](configuration/directories.md) | +| Services & authentication | [configuration/services.md](configuration/services.md) | +| Download & processing | [configuration/download.md](configuration/download.md) | +| Output & naming | [configuration/output.md](configuration/output.md) | +| DRM & CDM | [configuration/drm.md](configuration/drm.md) | +| Network & proxy | [configuration/network.md](configuration/network.md) | +| Key vaults | [configuration/vaults.md](configuration/vaults.md) | +| Other settings | [configuration/misc.md](configuration/misc.md) | +| Complete annotated example | [configuration/example.md](configuration/example.md) | diff --git a/docs/reference/configuration/directories.md b/docs/reference/configuration/directories.md new file mode 100644 index 00000000..363fec81 --- /dev/null +++ b/docs/reference/configuration/directories.md @@ -0,0 +1,88 @@ +# Directories { #directories } + +The `directories` key is a **dict** mapping directory names to filesystem paths. Only the +names listed below are honoured; a handful are **protected** and cannot be moved (attempts +to override them are silently skipped). All defaults are computed relative to the installed +package, and every user-settable path is passed through `Path(...).expanduser()`, so `~` +works. + +```yaml title="unshackle.yaml" +directories: + downloads: ~/Media/unshackle + temp: /mnt/fast-scratch/unshackle-temp + wvds: ~/.unshackle/WVDs + prds: ~/.unshackle/PRDs +``` + +| Key | Type | Default | Overridable | Purpose | +|-----|------|---------|:-----------:|---------| +| `downloads` | path | `/downloads` | ✅ | Default output directory for finished files. | +| `temp` | path | `/temp` | ✅ | Temporary working files during download/decrypt/mux. | +| `cache` | path | `/cache` | ✅ | Generic cache, title cache, and the update-check store. | +| `cookies` | path | `/cookies` | ✅ | Per-service cookie files (and VPN cookie files). | +| `logs` | path | `/logs` | ✅ | Log files. | +| `exports` | path | `/exports` | ✅ | Export JSON files. | +| `wvds` | path | `/WVDs` | ✅ | Widevine device files (`.wvd`). | +| `prds` | path | `/PRDs` | ✅ | PlayReady device files (`.prd`). | +| `dcsl` | path | `/DCSL` | ✅ | DCSL data. | +| `commands` | path | `unshackle/commands` | ✅ | CLI command modules. | +| `services` | list \| path | `[unshackle/services]` | ✅ | Service search paths and/or remote repo specs (see below). | +| `vaults` | path | `unshackle/vaults` | ✅ | Vault backend modules. | +| `fonts` | path | `unshackle/fonts` | ✅ | Bundled fonts. | +| `user_configs` | path | `unshackle/` | ❌ protected | Where `unshackle.yaml` lives. | +| `data` | path | `unshackle/` | ❌ protected | Base for the data subdirectories above. | +| `core_dir` | path | `unshackle/core` | ❌ protected | Package core. | +| `namespace_dir` | path | `unshackle/` | ❌ protected | Package root. | +| `app_dirs` | - | `AppDirs("unshackle", False)` | ❌ protected | Internal AppDirs instance. | + +!!! note "The `services` directory is special" + `services` may be a **list**, and each entry can be either a local directory or a + **repository spec**: a git URL (`https://...`, `ssh://...`, `git@...`, or anything ending in + `.git`) or `owner/repo` shorthand. Repo specs are cloned and updated automatically; plain + paths are used as-is. **List order is priority**: the first source to define a service + tag wins. + + ```yaml + directories: + services: + - you/your-services # cloned from GitHub + - https://example.com/private.git@stable + - ~/my-services # local, lowest priority + ``` + +!!! info "How git-backed service repos are handled" + unshackle's use of git is **read-only on the remote**: it only ever performs a shallow + `clone`, `fetch`, `pull`, and a local `reset`; **nothing is ever pushed**. Private repos + are authenticated through your existing git credential helper, and unshackle stores no + tokens of its own. Clones live under `/_repos/____/` + (or the bundled `unshackle/services`), and **nothing is written to the cache directory**. After + the first clone unshackle re-pulls **at most once every 24 h**, so it does not touch the + network on every run. + +!!! warning "Automatic and manual refresh behave differently" + The **automatic** 24 h-TTL refresh that happens during a normal `dl`/`search` run + **refuses to refresh and exits** (naming the offending clone) if that clone has + uncommitted changes to tracked files or unpushed local commits. Untracked files (new + service folders, `__pycache__`) never block it. The **manual** + `unshackle util refresh-services` command instead **hard-resets** the clone to upstream, + discarding any local edits. + +!!! warning "Read-only installs and reinstalls" + If the services directory lives inside the installed package, a reinstall can delete the + `_repos` clones. They are re-cloned on next use. On **read-only installs** you must + point `services` at a writable path, or cloning will fail. + +## Filenames { #filenames } + +The `filenames` key is a **dict** of templated file/name patterns. Each value is used +verbatim (no path processing). Braced fields like `{time}` and `{service}` are filled in at +runtime. + +| Key | Type | Default | Notes | +|-----|------|---------|-------| +| `log` | str | `"unshackle_{name}_{time}.log"` | Written under `directories.logs`. | +| `debug_log` | str | `"unshackle_debug_{service}_{time}.jsonl"` | Structured debug log, under `directories.logs`. | +| `config` | str | `"config.yaml"` | Per-service config file, under that service's directory. | +| `root_config` | str | `"unshackle.yaml"` | The main config filename itself. | +| `chapters` | str | `"Chapters_{title}_{random}.txt"` | Under `directories.temp`. | +| `subtitle` | str | `"Subtitle_{id}_{language}.srt"` | Under `directories.temp`. | diff --git a/docs/reference/configuration/download.md b/docs/reference/configuration/download.md new file mode 100644 index 00000000..b85b27f3 --- /dev/null +++ b/docs/reference/configuration/download.md @@ -0,0 +1,253 @@ +# Download & processing { #download-processing } + +## `dl` + +- **Type:** `dict`  ·  **Default:** `{}` + +Persistent defaults for the `dl` command. The block is used as Click's `default_map`, so any +`dl` flag can be given a default here. See [Downloading](../../guide/downloading.md) and the +[CLI Reference](../../guide/cli-reference.md) for what each flag does. + +**Key naming.** A key is the flag's long name with dashes replaced by underscores +(`--best-available` → `best_available`, `--sub-format` → `sub_format`). A few keys are named +after the flag's internal destination rather than its visible name; set these exact keys: + +| Flag | Config key | +|------|-----------| +| `-r` / `--range` | `range` | +| `--list` | `list` | +| `--tmdb` | `tmdb_id` | +| `--imdb` | `imdb_id` | +| `--animeapi` | `animeapi_id` | +| `-naa` / `--noatmos` | `no_atmos` | +| `-o` / `--output` | `output_dir` | +| `--cdm-only` / `--vaults-only` | `cdm_only` | + +**Precedence** (highest first): an explicit CLI flag → a per-service +[`services..dl`](services.md) override → this global `dl:` block → the built-in +default. In other words, config only fills in options you did not set on the command line. + +Common keys (a useful subset; every `dl` flag works): + +| Key | Type | Default | Sets | +|-----|------|---------|------| +| `lang` | list | `["orig"]` | Video/audio language(s); `orig` = original language. | +| `a_lang` / `v_lang` | list | `[]` | Audio- / video-only language override. | +| `s_lang` | list | `["all"]` | Subtitle language(s). | +| `quality` | list | `[]` (best) | Resolution(s), e.g. `[1080]`. | +| `vcodec` | list | `[]` (any) | Video codec(s), e.g. `[H265]`. | +| `acodec` | list | `[]` (any) | Audio codec(s), e.g. `[EC3]`. | +| `range` | list | `["SDR"]` | Colour range(s): `SDR`, `HLG`, `HDR10`, `HDR10P`, `DV`, `HYBRID`. | +| `channels` | float | *(unset)* | Audio channels, e.g. `6` for 5.1. | +| `sub_format` | str | *(unset)* | Convert subtitles to this format (`srt`, `vtt`, `original`, ...). | +| `forced_subs` | bool | `false` | Include forced subtitle tracks. | +| `forced_s_lang` | list | `[]` | Forced subtitle language(s); implies `forced_subs`. | +| `no_subs` / `no_audio` / `no_chapters` | bool | `false` | Skip that track type. | +| `downloads` | int | `1` | Tracks downloaded concurrently. | +| `workers` | int | *(downloader default)* | Threads per track. | +| `speed_limit` | str | *(unlimited)* | Total download speed cap across all downloads combined, e.g. `500k`, `5M`, `1.5G` or plain bytes/sec. Values are bytes, not bits (`5M` = 5.0 MB/s). | +| `slow` | str | *(unset)* | Inter-title delay, e.g. `"20-40"`. | +| `best_available` | bool | `false` | Fall back to best quality if the request is unavailable. | +| `proxy` | str | *(unset)* | Default proxy URI or 2-letter country. | +| `no_folder` | bool | `false` | Do not create a per-title folder. | + +```yaml title="Global download defaults" +dl: + lang: [en] + quality: [1080] + range: [HDR10] + vcodec: [H265] + sub_format: srt + downloads: 2 +``` + +!!! note "Per-service overrides" + A `dl` block nested under a service in [`services`](services.md) takes the same keys and + applies to that service only. + + ```yaml + dl: + lang: [en] + services: + EXAMPLE1: + dl: + lang: [en, ja] + range: [DV] + ``` + +!!! note "`serve` uses these same keys" + The [`serve`](services.md#serve) block accepts most of the same `dl` option keys at its top level + to set server-wide download defaults for every request it handles. Keys the API does not know + are ignored there (`speed_limit`, for example; `serve` has its own `global_speed_limit`). + +## `subtitle` + +- **Type:** `dict`  ·  **Default:** `{}` + +Subtitle handling: SDH stripping, format conversion, and whether subtitles are muxed in or +written as sidecar files. See [Subtitles](../../guide/subtitles.md) for the full guide. + +| Sub-key | Type | Default | Description | +|---------|------|---------|-------------| +| `strip_sdh` | bool | `true` | Strip SDH/CC cues (hearing-impaired annotations) into a clean track. | +| `sdh_method` | str | `"auto"` | SDH-stripping backend to use. | +| `preserve_formatting` | bool | `true` | Keep styling/positioning when converting or stripping. | +| `convert_before_strip` | bool | `true` | Convert to the working format before stripping SDH. | +| `conversion_method` | str | `"auto"` | Subtitle conversion backend. | +| `output_mode` | str | `"mux"` | `mux` embeds subtitles in the MKV; `sidecar` writes separate files; `both` writes sidecars and still muxes. | +| `sidecar_format` | str | `"srt"` | Format for sidecar files when `output_mode` is `sidecar` or `both`. | +| `type_priority` | list | *(unset)* | Ordered ranking of subtitle types (`forced`, `normal`, `sdh`; CC counts as SDH). Types you leave out fall to the end. | +| `group_by` | str | `"type"` | Which key groups the subtitle tracks. `type` puts all forced tracks together, then all normal, then all SDH. `language` keeps each language next to its own variants. | +| `language_priority` | list | *(unset)* | Languages to sort to the top, in the order you give. The rest keep the default order: original language first, then alphabetical. Nothing is removed. | + +```yaml +subtitle: + strip_sdh: true + output_mode: sidecar + sidecar_format: srt +``` + +!!! note "`group_by` track order" + | `group_by` | Result | + |---|---| + | `type` | Danish, Finnish, French, … Danish SDH, Finnish SDH, French SDH | + | `language` | Danish, Danish SDH, Finnish, Finnish SDH, French, French SDH | + + `type_priority` still sets the order of the variants, so with `group_by: language` it + controls the order inside each language. + +!!! note "`language_priority` sorts languages, it does not select them" + ```yaml + subtitle: + group_by: language + language_priority: [en, es, fr] + ``` + + | | Result for an English title | + |---|---| + | Default | English, Arabic, Bulgarian, Spanish, French, Japanese | + | With the list above | English, Spanish, French, Arabic, Bulgarian, Japanese | + + Nothing is removed, unlike `-sl`, which drops every subtitle language it does not name. + + Use `orig` in the list for the title's original language. A base tag also covers its + regional forms, so `en` sorts `en-US` and `en-GB`, but `zh` does **not** cover + `zh-Hant`, which you must name. The sort ignores entries that match no track, and it + ignores entries that are not valid language tags. + + `--exact-lang` applies here too: with it, `en` sorts only `en`, and `en-US` falls to + the alphabetical tail with the rest. Name each tag you want when you use that flag. + +!!! note "`type_priority` picks the default subtitle only with `muxing.default_language`" + At mux time the `default` flag goes to every forced subtitle whose language matches the + first audio track, so a full dialogue track never becomes default on its own. + `type_priority` alone does not change that. Name a subtitle language in + [`muxing.default_language`](#muxing) and the first track in that language gets the flag + instead, which is where `type_priority` decides which track that is: + + ```yaml + muxing: + default_language: + subtitle: en + subtitle: + type_priority: [normal, sdh, forced] + ``` + +## `audio` + +- **Type:** `dict`  ·  **Default:** `{}` + +Audio handling options. The most notable sub-key is **`codec_priority`**, an ordered list of +audio codecs used to break ties in track selection. **`language_priority`** is an ordered list +of languages that sort to the top of the audio tracks. + +```yaml +audio: + codec_priority: + - EAC3 + - AC3 + language_priority: + - orig + - en +``` + +!!! note "`codec_priority` is a soft priority: nothing is dropped" + `codec_priority` outranks bitrate: any listed codec sorts above every unlisted one, and among + listed codecs the list order decides. Bitrate order survives only within a single codec rank. + Nothing is ever removed. Two higher-level rules still override it: **Atmos + tracks always take precedence over `codec_priority`**, and **audio-description tracks are + always forced to the end** regardless of it. When `codec_priority` is unset, tracks sort by + bitrate alone, with those same Atmos/descriptive rules still applied. + +## `muxing` + +- **Type:** `dict`  ·  **Default:** `{}` + +Matroska (MKV) muxing options. + +| Sub-key | Type | Default | Description | +|---------|------|---------|-------------| +| `set_title` | bool | `true` | Write a human-readable title into the MKV container. | +| `default_language` | dict | `{}` | Force which language is flagged *default* per track type, e.g. `{audio: en, subtitle: en}`. | +| `merge_audio` | bool | `true` | Merge all audio into one file. `--split-audio` on the CLI flips this off. | +| `merge_video` | bool | `false` | Merge video tracks that share height, range, and codec into one file, so only language varies inside a file. `--merge-video` on the CLI flips this on. | + +```yaml +muxing: + set_title: true + default_language: + audio: en + subtitle: en +``` + +## `language_tags` + +- **Type:** `dict`  ·  **Default:** `{}` + +Language-tag remapping and the rule engine behind the `{lang_tag}` filename variable. When a +`rules` list is present, each rule is evaluated in order and the first match's `tag` is used. +A rule may test: + +| Condition | Meaning | +|-----------|---------| +| `audio` | Matches against the audio track languages. | +| `subs_contain` | At least one subtitle language matches. | +| `subs_contain_all` | Every listed language must be present (scalar or list). | + +```yaml +language_tags: + rules: + - audio: [ja] + subs_contain: [en] + tag: "SUBBED" +``` + +## `dual_multi_mode` + +- **Type:** `str`  ·  **Default:** `strict` + +Rule set for the `{dual}`, `{multi}`, and `{dubbed}` filename variables. + +=== "`strict` (default)" + + | Variable | Set when | + |----------|----------| + | `{dual}` | Exactly two audio languages **and one is the title's original language**. | + | `{multi}` | Three or more audio languages, even without the original. | + | `{dubbed}` | A single audio language that is not the title's original language (requires a known original). | + +=== "`count` (legacy)" + + | Variable | Set when | + |----------|----------| + | `{dual}` | Exactly two audio languages (original language ignored). | + | `{multi}` | Three or more audio languages. | + | `{dubbed}` | Never set. | + +```yaml +dual_multi_mode: strict +``` + +!!! note "Dialects count as one language" + Regional variants of the same base language (e.g. `en-US` and `en-GB`) collapse to a single + language in both modes, so they never trigger `{dual}` on their own. diff --git a/docs/reference/configuration/drm.md b/docs/reference/configuration/drm.md new file mode 100644 index 00000000..b4c2f0f1 --- /dev/null +++ b/docs/reference/configuration/drm.md @@ -0,0 +1,157 @@ +# DRM & CDM { #drm-cdm } + +These are the config keys. For concepts, device provisioning, and the `wvd`/`prd` commands see +the [DRM & CDM guide](../../guide/drm-and-cdm.md). Device files themselves live in +[`directories.wvds`](directories.md) (`.wvd`) and `directories.prds` (`.prd`). + +## `cdm` + +- **Type:** `dict`  ·  **Default:** `{}` + +Maps a service tag to the CDM (Widevine/PlayReady device) to use, with a `default` fallback. +Resolution is case-insensitive: a per-request API override wins, then the per-service entry, +then `default`. + +```yaml +cdm: + default: chromecdm_l3 + EXAMPLE2: android_l1 +``` + +A value may also be a **nested dict** for advanced selection by quality, DRM system, or +profile: + +```yaml +cdm: + EXAMPLE1: + ">=1080": android_l1 # by track height + "<1080": chromecdm_l3 + widevine: android_l1 # by DRM system + playready: sl3000 + default: chromecdm_l3 +``` + +Quality keys support `1080`, `>=1080`, `>720`, `<=576`, `<480` style comparisons; DRM keys +are `widevine` / `playready`. Quality keys are matched first, but only when a video height is +known; if the value is still a dict after that, `widevine`/`playready` are used when present, +otherwise the credential profile name, then `default`. + +The resolved name is looked up in [`remote_cdm`](#remote_cdm) by `name` first. If no entry +matches, it is loaded as a local device file, in this order: `.prd` in +`directories.prds`, `.prd` in `directories.wvds`, then `.wvd` in +`directories.wvds`. + +## `remote_cdm` + +- **Type:** `list[dict]`  ·  **Default:** `[]` + +A list of remote CDM server definitions. Each entry is matched by its `name` (referenced from +[`cdm`](#cdm)), and its `type` selects the backend: + +| `type` | Backend | Key fields | +|--------|---------|-----------| +| `decrypt_labs` | Decrypt Labs KeyXtractor | `host`, `device_name`, `secret` ([fields](#remote-cdm-decrypt-labs)) | +| `custom_api` | Fully YAML-configurable remote API | `host` (required), `device`, `auth`, `endpoints`, `request_mapping`, `response_mapping`, `caching`, `timeout` ([fields](#remote-cdm-custom-api)) | +| *(none)*, `Device Type: PLAYREADY` | pyplayready `RemoteCdm` | `host`, `secret`, `device_name`, `security_level` (default 3000) | +| *(none)*, otherwise | pywidevine `RemoteCdm` | `host`, `secret`, `device_name`, `device_type`, `system_id`, `security_level` (default 3000) | + +```yaml +remote_cdm: + - name: keyxtractor + type: decrypt_labs + device_name: L1 + - name: my_wv_server + host: https://cdm.example.com + secret: s3cr3t + device_name: android_l1 + device_type: ANDROID + system_id: 26830 + security_level: 1 +``` + +For the two pywidevine/pyplayready backends, field names are read case-insensitively in both +styles: `Device Type`/`device_type`, `System ID`/`system_id`, `Security Level`/`security_level`, +`Host`/`host`, `Secret`/`secret`, `Device Name`/`device_name`. + +!!! warning "PlayReady `host` needs a `/playready` suffix" + pyplayready's `RemoteCdm` treats `host` as a base URL and appends its own endpoint paths, + so a PlayReady entry's `host` must include the trailing `/playready` segment (for example + `https://cdm.example.com/playready`). Without it the server returns `404` rather than a + configuration error. + +### `decrypt_labs` fields { #remote-cdm-decrypt-labs } + +| Field | Default | Notes | +|-------|---------|-------| +| `host` | `https://keyxtractor.decryptlabs.com` | | +| `device_name` | `ChromeCDM` | `ChromeCDM`, `L1`, `L2` (Widevine) or `SL2`, `SL3` (PlayReady) | +| `secret` | from [`decrypt_labs_api_key`](misc.md#external-api-keys) | Sent as the `decrypt-labs-api-key` header. An error is raised if neither is set. | +| `system_id` | `26830` (Widevine), `0` (PlayReady) | | +| `security_level` | Widevine `3`; PlayReady `2000` for `SL2`, else `3000` | | + +PlayReady mode is used when `device_type` is `PLAYREADY` or `device_name` starts with `SL`. + +### `custom_api` fields { #remote-cdm-custom-api } + +| Field | Type | Notes | +|-------|------|-------| +| `host` | str | Required. Base URL of the API. | +| `device` | dict | `name`, `type` (`CHROME`, `ANDROID`, `PLAYREADY`), `system_id`, `security_level` | +| `auth` | dict | `type` (default `header`): `header` uses `header_name` (default `Authorization`) and `key`; `bearer` uses `bearer_token` or `key`; `basic` uses `username` and `password`. `custom_headers` is merged in for any type. | +| `endpoints` | dict | `get_request` and `decrypt_response`, each `{path, method, timeout}` | +| `request_mapping` | dict | Per endpoint: `param_names`, `static_params`, `conditional_params`, `transforms`, `nested_params`, `exclude_params` | +| `response_mapping` | dict | Per endpoint: `fields`, `transforms`, `response_types`, `success_conditions`, `error_fields`, `key_fields` | +| `caching` | dict | `enabled`, `use_vaults`, `check_cached_first` | +| `legacy` | dict | Legacy mode options. | +| `timeout` | int | Default `30`. Request timeout in seconds. | + +Transforms cover base64/hex/JSON encoding and `kid:key` parsing. PlayReady mode is used when +`device.type` is `PLAYREADY` or the device name is `SL2`/`SL3`. + +```yaml +remote_cdm: + - name: my_custom + type: custom_api + host: https://cdm.example.com + device: + name: ChromeCDM + type: CHROME + system_id: 26830 + security_level: 3 + auth: + type: bearer + key: your-token + endpoints: + get_request: + path: /get-challenge + method: POST + decrypt_response: + path: /get-keys + method: POST + timeout: 30 +``` + +## `decryption` + +- **Type:** `str` or `dict`  ·  **Default:** `"shaka"` + +Selects the tool used to physically decrypt CENC content. The value is compared +case-insensitively; `"mp4decrypt"` selects Bento4's mp4decrypt, and **anything else** +(including `"shaka"`) selects shaka-packager. When the value is a dict, the service key is +matched case-insensitively and the `default` entry is the fallback. + +HLS AES-128 ClearKey is decrypted in-process and ignores this setting. + +=== "Global" + + ```yaml + decryption: shaka + ``` + +=== "Per-service" + + ```yaml + decryption: + default: shaka + EXAMPLE1: mp4decrypt + ``` diff --git a/docs/reference/configuration/example.md b/docs/reference/configuration/example.md new file mode 100644 index 00000000..382cca61 --- /dev/null +++ b/docs/reference/configuration/example.md @@ -0,0 +1,51 @@ +# A complete annotated example + +The following config sets a handful of common keys. Every key not shown keeps its default. + +```yaml title="unshackle.yaml" +directories: + downloads: ~/Media/unshackle + temp: /mnt/scratch/unshackle + +network: + browser: Chrome131 + +headers: + Accept-Language: en-US,en;q=0.9 + +cdm: + default: chromecdm_l3 + EXAMPLE2: android_l1 + +decryption: shaka + +credentials: + EXAMPLE2: user@example.com:hunter2 + +output_template: + movies: "{title}.{year}.{quality?}.{source}-{tag}" + series: "{title}.{season_episode}.{quality?}.{source}-{tag}" + folder: + movies: "{title} ({year})" + series: "{title} ({year})" + +tag: MYGRP + +key_vaults: + - type: SQLite + name: Local + path: ~/.unshackle/keys.db + +proxy_providers: + basic: + us: http://user:pass@1.2.3.4:8080 + +omdb_api_key: "your-omdb-key" +tmdb_api_key: "your-tmdb-key" +tvdb_api_key: "your-tvdb-key" +title_cache_time: 3600 +redact_paths: true +``` + +For how these interact at runtime, continue to [Downloading](../../guide/downloading.md), +[DRM & CDM](../../guide/drm-and-cdm.md), and [Output & naming](../../guide/output-and-naming.md). diff --git a/docs/reference/configuration/index.md b/docs/reference/configuration/index.md new file mode 100644 index 00000000..c435059a --- /dev/null +++ b/docs/reference/configuration/index.md @@ -0,0 +1,42 @@ +# Configuration Reference + +Every key you can set in `unshackle.yaml`, with its name, type, default value, and behaviour, +split into one page per area. A worked config is on the [annotated example](example.md) page. + +Where the file lives, how unshackle discovers it, and how to edit it: +[The Configuration File](../../getting-started/configuration-file.md). + +!!! info "How defaults work" + Every key is optional except [`output_template`](output.md), which `dl` requires. + Without it, `dl` aborts with an error. If you omit any other key, unshackle uses + the built-in default listed here. There is **no schema validation**: unknown keys are + silently ignored (they never become settings), and the file is parsed with + `yaml.safe_load`. An empty or missing file means every other key falls back to its + default. + +!!! warning "Two keys behave specially on load" + - `curl_impersonate` is a **deprecated** alias of [`network`](network.md) and emits a + `DeprecationWarning`. + - `scene_naming` has been **removed**. If it is set to any value, unshackle exits + immediately with an error telling you to use [`output_template`](output.md) + instead. + + See [Deprecated & removed keys](misc.md#deprecated-removed-keys). + +--- + +## Quick index + +| Area | Keys | +|------|------| +| [Directories](directories.md) | `directories`, `filenames` | +| [Services & authentication](services.md) | `services`, `credentials`, `firefox_cookies`, `remote_services`, `serve` | +| [Download & processing](download.md) | `dl`, `subtitle`, `audio`, `muxing`, `language_tags`, `dual_multi_mode` | +| [Output & naming](output.md) | `output_template`, `tag`, `tag_rules`, `tag_group_name`, `tag_imdb_tmdb`, `chapter_fallback_name`, `unicode_filenames` | +| [DRM & CDM](drm.md) | `cdm`, `remote_cdm`, `decryption` | +| [Network & proxy](network.md) | `network`, `headers`, `proxy_providers` | +| [Key vaults](vaults.md) | `key_vaults`, `vault_timeout` | +| [External API keys](misc.md#external-api-keys) | `imdb_api_enabled`, `omdb_api_key`, `tmdb_api_key`, `tvdb_api_key`, `tvdb_pin`, `tvdb_order`, `simkl_client_id`, `metadata_providers`, `decrypt_labs_api_key`, `ipinfo_api_key` | +| [Caching & updates](misc.md#caching-updates) | `title_cache_enabled`, `title_cache_time`, `title_cache_max_retention`, `update_checks`, `update_check_interval` | +| [Logging, privacy & debug](misc.md#logging-privacy-debug) | `redact_paths`, `debug`, `debug_keys`, `debug_requests`, `set_terminal_bg` | +| [Deprecated & removed](misc.md#deprecated-removed-keys) | `curl_impersonate`, `downloader`, `scene_naming` | diff --git a/docs/reference/configuration/misc.md b/docs/reference/configuration/misc.md new file mode 100644 index 00000000..ce3b5c28 --- /dev/null +++ b/docs/reference/configuration/misc.md @@ -0,0 +1,109 @@ +# Other Settings + +Smaller configuration areas: external metadata API keys, caching and update checks, logging and debug flags, and deprecated keys. + +## External API keys { #external-api-keys } + +All are unset by default and enable optional metadata/geolocation features. + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `imdb_api_enabled` | bool | `false` | Use the free IMDxAPI (api.tiffara.com) metadata provider, which needs no API key. Off by default since the site has been unreliable. | +| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a more reliable alternative to IMDxAPI. Free keys are available on the OMDb site. | +| `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. | +| `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. | +| `tvdb_api_key` | str | `""` | [TheTVDB v4](https://thetvdb.com/api-information) API key; a fallback source to TMDB, strongest on TV series. Free keys are available on the TVDB site. | +| `tvdb_pin` | str | `""` | Subscriber PIN, only needed for a user-supported TVDB key. Leave empty for a normal project key. | +| `tvdb_order` | str | `""` | Default for `--tvdb-order`: renumber episodes to a TVDB season order (`official`, `dvd`, `absolute`, `alternate`, `regional`). Empty keeps the numbering the service gives. | +| `metadata_providers` | list | *(see below)* | Metadata providers to use, in the order they are tried. | +| `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). | +| `ipinfo_api_key` | str | `""` | ipinfo.io API key for IP/region lookups. | + +!!! note "How `metadata_providers` is applied" + A lookup goes to each provider in turn and stops at the first good match, so the order + decides which source wins. `metadata_providers` sets that order: + + ```yaml + metadata_providers: [tvdb, tmdb, simkl] + ``` + + Names not in the list are never used, so this both orders and filters. Unknown names are + ignored with a warning. Leave it unset for the default order: + + ```yaml + metadata_providers: [imdbapi, omdb, simkl, tmdb, tvdb] + ``` + + Whatever the order, a provider is skipped when its API key is missing (or, for + `imdbapi`, when `imdb_api_enabled` is `false`). Available names are + `imdbapi`, `omdb`, `simkl`, `tmdb`, and `tvdb`. + +!!! note "`ipinfo_api_key` never touches your service sessions" + The token is only ever sent to `api.ipinfo.io` as a per-request `Authorization` header; it + is **never** attached to your service session, so it cannot leak to a streaming provider. + Lookups degrade gracefully through a fallback chain: the authenticated Lite endpoint (higher + rate limits, and ASN/continent data, but no city or region) → anonymous ipinfo → + `ip-api.in` as a last resort. + +--- + +## Caching & updates { #caching-updates } + +### Title cache + +unshackle caches fetched title metadata (region- and account-aware) to avoid repeat API +calls. + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `title_cache_enabled` | bool | `true` | Master switch for the title cache. | +| `title_cache_time` | int (seconds) | `1800` (30 min) | Lifetime of fresh cached titles. | +| `title_cache_max_retention` | int (seconds) | `86400` (24 h) | Total retention window for a cached title, counted from when it was cached. Past `title_cache_time` the stale copy is only served as a fallback when a live fetch fails. | + +### Update checks + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `update_checks` | bool | `true` | Whether to check for new unshackle releases. | +| `update_check_interval` | int (hours) | `24` | Minimum hours between update checks. | + +!!! note + Update checks query the GitHub releases API with a fixed 5-second timeout and cache the + result in `directories.cache/update_check.json`. + +--- + +## Logging, privacy & debug { #logging-privacy-debug } + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `redact_paths` | bool | `true` | Mask install-root / venv / home prefixes in logged paths (``, ``, `~`). Set `false` to show full paths. | +| `debug` | bool | `false` | Global debug mode. | +| `debug_keys` | bool | `false` | Log decryption keys. | +| `debug_requests` | bool | `false` | Log HTTP requests. | +| `set_terminal_bg` | bool | `false` | Append the theme's background colour to output styling. | + +!!! note "Effect of `set_terminal_bg`" + When on, the theme's background colour is appended to the foreground styles, which makes + the full colour palette render correctly on terminals whose *default* background differs + from unshackle's theme. When off, such terminals can render the ASCII-art banner and + coloured output incorrectly. + +!!! danger "Key exposure" + With `debug_keys` on, decryption content keys are written to the structured debug log at + INFO level during the DRM handshake. Treat `debug`, `debug_keys`, and the `unshackle_debug_*.jsonl` files as + sensitive, and keep `redact_paths` enabled when sharing logs. + + `debug_keys` affects **only** content-encryption keys (the `content_key`/`key` fields). + Passwords, tokens, cookies, and session tokens are **always** redacted regardless of this + setting, and KIDs, key counts, and other metadata are always logged either way. + +--- + +## Deprecated & removed keys { #deprecated-removed-keys } + +| Key | Status | Behaviour | +|-----|--------|-----------| +| `curl_impersonate` | **Deprecated** → use [`network`](network.md) | Emits a `DeprecationWarning`; still honoured only if `network` is absent. | +| `downloader` | **Deprecated** | Any value other than `"requests"` emits a `DeprecationWarning`; the value is otherwise ignored (the unified requests downloader is always used). | +| `scene_naming` | **Removed** | If set to any non-null value, unshackle exits with an error directing you to configure [`output_template`](output.md) instead. | diff --git a/docs/reference/configuration/network.md b/docs/reference/configuration/network.md new file mode 100644 index 00000000..652a000e --- /dev/null +++ b/docs/reference/configuration/network.md @@ -0,0 +1,86 @@ +# Network & proxy { #network-proxy } + +## `network` + +- **Type:** `dict`  ·  **Default:** `{}` + +TLS-fingerprinting and HTTP client settings for the rnet-based session. + +| Sub-key | Type | Default | Description | +|---------|------|---------|-------------| +| `browser` | str | `"Chrome131"` | Impersonation preset. Must be an exact preset name (e.g. `Chrome131`, `Firefox135`, `Edge101`, `Safari18`, `OkHttp4_12`, `OkHttp5`, `Opera118`); unknown names raise an error. | +| `http1_only` | bool | *(unset)* | Force HTTP/1.1. | +| `http2_only` | bool | *(unset)* | Force HTTP/2. | +| `pool_max_idle_per_host` | int | *(unset)* | Connection-pool tuning. | +| `pool_max_size` | int | *(unset)* | Connection-pool tuning. | +| `tcp_nodelay` | bool | *(unset)* | Disable Nagle's algorithm. | + +```yaml +network: + browser: Firefox135 + http2_only: true +``` + +!!! note "Measured effect of `http1_only`" + Both `http1_only` and `http2_only` are unset by default. In benchmarks, forcing HTTP/1.1 + gained 30 to 50% on hosts that throttle per-connection or stall behind HTTP/2 flow control, + and **cost up to 27% on fast CDNs**. + +!!! warning "Renamed from `curl_impersonate`" + The old `curl_impersonate` section is a deprecated alias. If you still use it, unshackle + honours it (only when `network` is absent) but emits a `DeprecationWarning`. Rename it to + `network`. + +## `headers` + +- **Type:** `dict`  ·  **Default:** `{}` + +Default HTTP headers merged into every session unshackle creates. + +```yaml +headers: + Accept-Language: en-US,en;q=0.9 +``` + +!!! warning "Don't set `Accept-Encoding` (and similar) here" + Compatibility headers such as `Accept-Encoding` are set by the rnet HTTP backend as part of + its browser-impersonation profile; overriding them breaks the impersonation fingerprint. + This block is for cross-service defaults only (for example `Accept-Language`, + `User-Agent`); per-service headers belong in that service's own config. + +## `proxy_providers` + +- **Type:** `dict`  ·  **Default:** `{}` + +Proxy/VPN provider configuration. Each sub-key names a provider, and its block is passed +straight to that provider's constructor. See [Proxies & VPN](../../guide/proxies-and-vpn.md) for +the full provider guide. Recognised providers and their exit ports: + +| Provider | Config key | Credentials | Proxy scheme/port | +|----------|-----------|-------------|-------------------| +| Basic (static) | `basic` | country → URI(s) | as specified | +| NordVPN | `nordvpn` | **service** credentials (48 chars combined) | `https://...:89` | +| Surfshark | `surfsharkvpn` | **service** credentials (48 chars combined) | `https://...:443` | +| Windscribe | `windscribevpn` | service credentials | `https://...:443` | +| ExpressVPN | `expressvpn` | device login (`enable: true`) / token cache | `https://cat:...@...:443` | +| ProtonVPN | `protonvpn` | TV login or exported cookies | `https://...:4443` (or `:443` Secure Core) | +| Gluetun | `gluetun` | per-VPN keys/creds | `http://localhost:{port}` (local Docker) | +| Hola | *(none, auto)* | none | `http://...:{peer}` | + +```yaml +proxy_providers: + basic: + us: http://user:pass@1.2.3.4:8080 + de: + - http://a.example:8080 + - socks5://b.example:1080 + nordvpn: + username: + password: +``` + +!!! note "Provider loading differs between CLI and REST server" + The `dl` CLI loads all providers, including `windscribevpn` and `gluetun`. The REST API / + remote-client path uses a separate resolver that does **not** load `windscribevpn` or + `gluetun`. ExpressVPN and ProtonVPN also auto-load when their cached session exists, and Hola + auto-loads whenever the `hola-proxy` binary is present. diff --git a/docs/reference/configuration/output.md b/docs/reference/configuration/output.md new file mode 100644 index 00000000..5f696937 --- /dev/null +++ b/docs/reference/configuration/output.md @@ -0,0 +1,149 @@ +# Output & naming { #output-naming } + +Keys that control the output filename, the per-title subfolder, release-group tagging, and +filename character handling. For template cookbook material and worked naming examples, see +the [Output & naming guide](../../guide/output-and-naming.md). + +## `output_template` + +- **Type:** `dict`  ·  **Required** (no usable default; `dl` exits with an error if this is missing) + +Filename templates keyed by media kind, using `{variable}` placeholders. Recognised kinds are +`movies`, `series`, and `songs` (and `albums` for folder templates). A trailing `?` on a +variable (e.g. `{quality?}`) marks it optional: if empty, it and one adjacent separator are +dropped. + +```yaml title="unshackle.yaml" +output_template: + movies: "{title}.{year}.{quality?}.{source}-{tag}" + series: "{title}.{season_episode}.{episode_name?}.{quality?}.{source}-{tag}" + songs: "{track_number}. {title}" + folder: # special nested key, see below + movies: "{title} ({year})" + series: "{title} ({year})" +``` + +**The nested `folder` sub-key** controls output subfolders and is popped out of +`output_template`: + +- A **dict** sets per-kind folder templates (kinds: `movies`, `series`, `songs`, `albums`). +- A **str** sets a single folder template for everything. + +unshackle validates templates on load and emits warnings for unknown variables, non-string +values, filesystem-unsafe characters (`< > : " / \ | ? *`), empty templates, and unknown +folder-kind keys. + +??? example "All valid template variables" + `title`, `year`, `season`, `episode`, `season_episode`, `episode_name`, `date`, + `quality`, `resolution`, `source`, `tag`, `track_number`, `artist`, `album_artist`, + `album`, `disc`, `track_total`, `disc_total`, `release_type`, `genre`, `explicit`, + `isrc`, `upc`, `label`, `audio`, `audio_channels`, `audio_full`, `atmos`, `dual`, + `multi`, `dubbed`, `video`, `hdr`, `hfr`, `edition`, `repack`, `lang_tag`, + `title_type`. + +## Tagging & naming keys + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `tag` | str | `""` | Release-group tag. Fills the `{tag}` template variable and is written into MKV/audio tags. | +| `tag_group_name` | bool | `true` | Include the group name (`tag`) in the MKV `Group` tag. | +| `tag_imdb_tmdb` | bool | `true` | Write IMDb/TMDB/TVDB external-ID tags into the MKV (needs metadata providers). | +| `chapter_fallback_name` | str | `""` | Fallback chapter-name template, e.g. `"Chapter {i:02}"`. | +| `unicode_filenames` | bool | `false` | Allow Unicode in filenames; when `false`, names are ASCII-sanitised. | + +## `tag_rules` + +- **Type:** `list`  ·  **Default:** `[]` + +Conditionally replaces [`tag`](#tagging-naming-keys) for a given download. Each rule pairs a `when` +dict of conditions with the `tag` to use when they hold. Rules are evaluated in order and the +first match wins; if no rule matches, the global `tag` is kept. + +```yaml title="unshackle.yaml" +tag_rules: + - when: { title_type: movie, lang_tag: SUBBED } + tag: SUBMOVIES + - when: { quality: 2160p, hdr: [DV, DV.HDR, DV.HDR10P, HDR, HDR10P] } + tag: UHDGROUP +``` + +Rule semantics: + +- Every condition in `when` must hold (they are AND-ed). +- A value may be a scalar or a list; a list matches if **any** entry matches. +- Comparison is case-insensitive string equality. +- The keys of `when` are the filename template variables that describe the release itself: + `title_type`, `quality`, `resolution`, `source`, `video`, `hdr`, `hfr`, `audio`, + `audio_channels`, `audio_full`, `atmos`, `dual`, `multi`, `dubbed`, `edition`, `repack`, + and `lang_tag`. Only these are available. The title's own fields (`title`, `year`, `season`, + `episode`, and so on) are added after the rules run and cannot be matched. +- An unknown key in `when` is a mistake: unshackle logs a warning that names the key and skips + that rule. A rule without conditions or without a `tag` is also warned about and skipped. + +Put the more specific rules first, because the first match ends the search. + +!!! warning "Match `hdr` on the exact value" + Matching is exact, and `hdr` is a composite label for Dolby Vision titles: a DV title with + an HDR10 base layer gives `DV.HDR`, and with an HDR10+ base layer `DV.HDR10P`. Plain `DV` + does **not** match those. List every variant you want, as the example above does. Note + also that HDR10 gives `HDR`, not `HDR10`. The full set of values is `HDR`, `HDR10P`, `DV`, + `DV.HDR`, `DV.HDR10P`, and `HLG`, as listed in the + [variable table](../../guide/output-and-naming.md#template-variables). + +### Operators + +A condition value may start with a comparison operator, which is applied to the rest of the +value: + +```yaml title="unshackle.yaml" +tag_rules: + - when: { resolution: "<1080" } + tag: SDGROUP + - when: { quality: ">=2160" } + tag: UHDGROUP + - when: { video: "!=H.265" } + tag: AVCGROUP +``` + +- `>`, `<`, `>=`, `<=` compare numbers. unshackle reads the first number out of the actual + value, so `">=2160"` matches a quality of `2160p`. If the actual value has no number, the + condition fails. +- `=`, `==`, and `!=` compare text, case-insensitively, when the operand is not a number. + With a numeric operand they compare numbers like the operators above. A value with no number + in it is never equal to a number, so `"!=2160"` holds for it and `"==2160"` does not. +- An ordering operator with a text operand (for example `">DV"`) is a mistake: unshackle logs + a warning and the condition fails. So does an operator with nothing after it (`">="`). +- A value with no operator keeps the plain equality behaviour described above. + +!!! warning "Quote operator values in YAML" + Two of these operators break an unquoted YAML value, and both stop the config from + loading: + + - `>` opens a folded block scalar, so `quality: >=2160` fails with a "while scanning a + block scalar" error. + - `!` starts a tag, so `hdr: !=DV` fails with a "could not determine a constructor" + error. + + Quote every operator value: `quality: ">=2160"`, `resolution: "<1080"`, `video: "!=H.265"`. + +!!! note "Keep a negation out of lists" + A list matches if any entry matches, so two or more `!=` entries in one list are always + true. Give a negation as a single value, not as a list entry. + +`title_type` is the media kind of the title: `movie`, `series`, or `music`. It is useful to +give each kind its own group: + +```yaml +tag_rules: + - when: { title_type: movie } + tag: MOVIEGRP + - when: { title_type: series } + tag: TVGRP +``` + +The resolved tag applies to the `{tag}` filename variable only. The `Group` metadata tag +written into MKV and music files always uses [`tag`](#tagging-naming-keys). + +!!! note "`--tag` wins" + An explicit `dl --tag` on the command line disables the rules completely and uses the + given tag. diff --git a/docs/reference/configuration/services.md b/docs/reference/configuration/services.md new file mode 100644 index 00000000..3ba1043a --- /dev/null +++ b/docs/reference/configuration/services.md @@ -0,0 +1,137 @@ +# Services & authentication { #services-auth } + +## `services` + +- **Type:** `dict` keyed by service tag  ·  **Default:** `{}` + +Per-service configuration, keyed by the canonical service tag (for example `EXAMPLE1`, `EXAMPLE2`, +`EXAMPLE3`). unshackle reads several well-known sub-keys out of each service's block: + +- **`proxy_map`**: remaps the region query you passed to `-p/--proxy` for this service. A key is + that query, or `"provider:query"` when the flag also named a provider; the value is the query + asked of the proxy provider instead. It has no effect when `-p` was given a full proxy URL. +- **`title_map`**: an exact-match rename map applied to fetched titles (source name → + desired name), so a service that names a title differently from your library still matches. +- **`dl`**: per-service download defaults, using the same keys as the global + [`dl`](download.md#dl) block. + +Individual services may read any additional keys they define. The merged result (the service's +own `config.yaml` with this block layered on top) is handed to the service as `self.config`. + +User- or device-specific values (API keys, account IDs, device attributes) belong in this block +rather than in the service's own `config.yaml`, which holds shared defaults; see +[creating a service](../../dev/creating-a-service.md). + +```yaml +services: + EXAMPLE1: + proxy_map: + us: us-nyc + "nordvpn:ca": ca-toronto + title_map: + "My Show: The Movie": "My Show Presents The Movie" +``` + +## `credentials` + +- **Type:** `dict` keyed by service tag  ·  **Default:** `{}` + +Per-service login credentials. Each value is a `username:password[:extra]` string, the same +data as a `[username, password]` (or `[username, password, extra]`) list, or a dict of profile +name to either of those. With the dict form, `-p/--profile` selects the entry, and the +`default` key is used when no profile was given or the named profile is missing. These are +parsed into `Credential` objects; the credential's SHA-1 is also used as an account hash for +cache keys. + +```yaml +credentials: + EXAMPLE2: user@example.com:hunter2 + EXAMPLE1: + default: primary@example.com:pw1 + second: secondary@example.com:pw2 +``` + +!!! tip "Cookies vs credentials" + Cookies are stored as files under [`directories.cookies`](directories.md), not in this key. + A service's `authenticate()` accepts cookies, credentials, or both. + +## `firefox_cookies` + +- **Type:** `dict` keyed by service tag  ·  **Default:** `{}` + +Settings for extracting cookies directly from a local Firefox profile. A service block is +expected to provide `hosts` (a list of cookie hostnames; entries shorter than 3 characters +are ignored to prevent dumping the whole cookie store) and an optional `local_storage` +boolean to also pull matching entries from `webappsstore.sqlite`, which only services that +keep auth tokens in localStorage rather than in HTTP cookies need. Extraction is read-only. + +!!! note "Firefox does not need to be closed" + The extractor copies **both** `cookies.sqlite` **and** its WAL file into a `0700` temp + directory, so writes Firefox has not yet flushed to the main DB are included. Extraction + fails only if Firefox holds an exclusive write lock at the instant of the copy. The live + profile is not modified. + +!!! warning "Extraction falls back silently to file cookies" + If extraction yields no cookies or fails for **any** reason, unshackle silently falls back + to the normal file-based cookie path (`cookies/.txt` or + `cookies//.txt`), with no error reported. + +## `remote_services` + +- **Type:** `dict`  ·  **Default:** `{}` + +Definitions of remote unshackle service servers, used by the `--remote` mode. Each entry is +named by you (pick it with `--server`, or omit that flag when only one is configured) and gives +the server's `url` (required), an optional `api_key` sent as `X-Secret-Key`, an optional +`server_cdm` boolean, and an optional `services` sub-dict of per-service local overrides such +as `title_map`. In `--remote` mode unshackle turns the server's service list into synthetic CLI +commands that run against it, falling back to the tags in that `services` sub-dict when the +list cannot be fetched. See [remote sessions](../../dev/rest-api/remote-sessions.md) for the +full setup. + +## `serve` + +- **Type:** `dict`  ·  **Default:** `{}` + +Configuration for the `serve` command (the built-in REST API server). The full server guide +is the [REST API](../../dev/rest-api/index.md) section; these are the config keys. + +| Sub-key | Type | Default | Description | +|---------|------|---------|-------------| +| `api_secret` | str | *(unset)* | Master secret accepted in the `X-Secret-Key` header. Required unless the server is started with `--no-key`. | +| `users` | dict | `{}` | Per-user API keys and their allowlists (see below). | +| `services` | list | *(unset)* | Global service allowlist. Omit to allow all. | +| `remote_only` | bool | `false` | Expose only the remote service session endpoints (health, services, search, session) and disable the rest of the REST API. | +| `session_ttl` | int (s) | `300` | Lifetime of an interactive auth session. | +| `max_sessions` | int | `100` | Maximum concurrent sessions. | +| `history_limit` | int | `100` | How many finished jobs to retain in history. | +| `compression_level` | int | `1` | gzip level for responses. | +| `global_speed_limit` | str | *(unlimited)* | Server-wide download speed cap, e.g. `10M`, `1.5G` or plain bytes/sec (same format as `speed_limit`). One shared budget across all concurrent jobs; per-job speed limits are ignored while it is set. | +| `cdm_overrides` | list or bool | *(unset)* | Allowed per-request CDM overrides: a list of permitted device names, or `true` for any. Unset rejects every override. | +| `allow_job_credentials` | bool | `false` | Permit clients to supply credentials per job. | +| `devices` | list | *(auto)* | Widevine devices offered; auto-filled from `directories.wvds`. | +| `playready_devices` | list | *(auto)* | PlayReady devices; auto-filled from `directories.prds`. | + +Each entry under `users` is keyed by that user's API key and may set its own `services`, +`devices`, and `playready_devices` allowlists, narrowing the global ones, plus an optional +`username` used as the log label for that key (defaults to a truncated form of the key). A +user with no `playready_devices` key gets no PlayReady access at all, not the global list. + +```yaml +serve: + api_secret: change-me + remote_only: true + services: [EXAMPLE1, EXAMPLE2] + # server-wide download defaults (same keys as the `dl:` block) + downloads: 3 + best_available: true + users: + a1b2c3d4: # this user's API key + services: [EXAMPLE1] # may only use EXAMPLE1 +``` + +!!! note "`dl` keys inside `serve`" + Most [`dl`](download.md#dl) flag keys (`downloads`, `workers`, `best_available`, and so on) + can be set directly inside `serve`, where they apply to every request the server handles. + The server recognises a fixed subset of download parameters, so a few CLI-only flags are + ignored here. diff --git a/docs/reference/configuration/vaults.md b/docs/reference/configuration/vaults.md new file mode 100644 index 00000000..2ef9cbd1 --- /dev/null +++ b/docs/reference/configuration/vaults.md @@ -0,0 +1,47 @@ +# Key vaults { #key-vaults } + +The full guide is at [Vaults](../../guide/vaults.md). Two keys configure them. + +## `key_vaults` + +- **Type:** `list[dict]`  ·  **Default:** `[]` + +An ordered list of key-vault backends. unshackle queries them in order and reuses content +keys instead of re-licensing. Each entry needs a `type` (the backend module name) and a +`name`, plus backend-specific keys. + +| `type` | Purpose | Required keys | Notes | +|--------|---------|--------------|-------| +| `SQLite` | Local SQLite database | `name`, `path` | Loaded **critically**; a failure aborts the run. | +| `MySQL` | Remote MySQL database | `name`, `host`, `database`, `username` | Extra keys (e.g. `password`, `port`) forwarded to pymysql. | +| `API` | RESTful JSON API | `name`, `uri`, `token` | Optional `headers` map is sent with every request. Honours `vault_timeout`. | +| `HTTP` | HTTP API with modes | `name`, `host`, one of `password`/`api_key`, and `username` in query mode | `api_mode`: `query` (default), `json`, `decrypt_labs`. Honours `vault_timeout`. | + +```yaml +key_vaults: + - type: SQLite + name: Local + path: ~/.unshackle/keys.db + - type: MySQL + name: Team + host: db.example.com + database: keys + username: unshackle + password: hunter2 + no_push: false +``` + +!!! note "Per-entry options" + - `no_push: true` makes a vault read-only (keys are fetched but never written to it). + - A vault of `type: API` whose `name` contains `decrypt_labs` auto-fills its `token` from + [`decrypt_labs_api_key`](misc.md#external-api-keys) when not set inline. Vault `type` values are + case-sensitive module names. + - An all-zero content key (32 zeros) is treated as "no key" everywhere and is never + stored. + +## `vault_timeout` + +- **Type:** `float`  ·  **Default:** `10.0` + +Timeout in seconds for vault operations. Injected automatically into any backend whose +constructor accepts a `timeout` parameter (a per-vault `timeout` still wins). diff --git a/mkdocs.yml b/mkdocs.yml index c7977bef..8fa8d712 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -5,6 +5,9 @@ repo_url: https://github.com/unshackle-dl/unshackle repo_name: unshackle-dl/unshackle docs_dir: docs +exclude_docs: | + reference/configuration.md + theme: name: material custom_dir: overrides @@ -117,7 +120,16 @@ nav: - Subtitles: guide/subtitles.md - Troubleshooting: guide/troubleshooting.md - Configuration: - - Reference: reference/configuration.md + - Overview: reference/configuration/index.md + - Directories: reference/configuration/directories.md + - Services & Authentication: reference/configuration/services.md + - Download & Processing: reference/configuration/download.md + - Output & Naming: reference/configuration/output.md + - DRM & CDM: reference/configuration/drm.md + - Network & Proxy: reference/configuration/network.md + - Key Vaults: reference/configuration/vaults.md + - Other Settings: reference/configuration/misc.md + - Annotated Example: reference/configuration/example.md - Developer: - Creating a Service: dev/creating-a-service.md - Service Architecture: dev/service-architecture.md