kenzuya/unshackle
1
0
Fork 2
mirror of https://github.com/unshackle-dl/unshackle.git synced 2026-11-13 12:35:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(proxies): add Control D proxy provider

Browse Source 
Runs a loopback forwarder per region-named profile (unshackle-<region>), so
--proxy and --proxy-download can exit in different countries

Thanks to Orni_ for the original code and the test account.

Co-authored-by: AntCardFr <24798811+AntCardFR@users.noreply.github.com>
This commit is contained in:
imSp4rky
2026-09-23 13:23:50 -06:00
co-authored by AntCardFr
parent ed3159378f
commit 783c8ae1c2
16 changed files with 1194 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/dev/rest-api/endpoints.md
+4 -4
View File
@@ -141,7 +141,7 @@ Find titles in a service by query string. The service must have a `search()` met
| `service` | string | yes | - | Service tag. |
| `query` | string | yes | - | Search query. |
| `profile` | string | no | `null` | Credential/cookie profile to use. |
| `proxy` | string | no | `null` | Full proxy URI, or a country code when the API key has `server_proxy`. |
| `proxy` | string | no | `null` | Full proxy URI, a Control D resolver as `controld://<resolver>@dns.controld.com`, or a country code when the API key has `server_proxy`. |
| `no_proxy` | boolean | no | `false` | Force-disable all proxy use. |
| `credentials`, `cookies`, `cache` | - | no | `null` | Your own login material, on a `--remote-only` server only. See [Client cache on a remote-only server](#client-cache-on-a-remote-only-server). |
@@ -190,7 +190,7 @@ Get the list of titles behind a title ID, for example episodes or a movie, witho
| `service` | string | yes | Service tag. |
| `title_id` | string | yes | Title identifier. |
| `profile` | string | no | Credential/cookie profile. |
| `proxy` | string | no | Full proxy URI, or a country code when the API key has `server_proxy`. |
| `proxy` | string | no | Full proxy URI, a Control D resolver as `controld://<resolver>@dns.controld.com`, or a country code when the API key has `server_proxy`. |
| `no_proxy` | boolean | no | Force-disable proxy. |
| `cdm_type` | string | no | Preferred CDM type. |
| `credentials`, `cookies`, `cache` | - | no | Your own login material, on a `--remote-only` server only. See [Client cache on a remote-only server](#client-cache-on-a-remote-only-server). |
@@ -445,7 +445,7 @@ Make a download job. It requires `service` and `title_id`. Every other field is
| `skip_dl` | boolean | `false` | Only fetch keys, do not download. |
| `export` | boolean | `false` | Export manifest/keys/subs to JSON. |
| `cdm_only` | boolean | `null` | Force CDM-only (`true`) or vault-only (`false`) key retrieval. |
| `proxy` / `no_proxy` / `no_proxy_download` / `proxy_download` | string / bool / bool / string | `null` / `false` / `false` / `null` | Proxy controls. `proxy` and `proxy_download` must be full proxy URIs unless the API key has `server_proxy`. |
| `proxy` / `no_proxy` / `no_proxy_download` / `proxy_download` | string / bool / bool / string | `null` / `false` / `false` / `null` | Proxy controls. `proxy` and `proxy_download` must be full proxy URIs unless the API key has `server_proxy`. A Control D resolver, `controld://<resolver>@dns.controld.com`, counts as a full proxy URI: the server runs a forwarder for it. |
| `no_folder` / `no_source` / `no_mux` | boolean | `false` | Output/mux controls. |
| `workers` | int | `null` | Threads per track. |
| `adaptive_workers` | boolean | `false` | Scale segment workers to measured CDN throughput, up to the `workers` cap. |
@@ -891,7 +891,7 @@ Make a remote session for a service and title. Authentication runs asynchronousl
Service CLI options also travel in a nested `service_params` object, which wins over a flat key with the same name. `profile` at the top level always means the credentials profile, never a service's own `--profile` option.
The `proxy` value must be a full proxy URI, unless the operator gives your API key `server_proxy` in the `serve.users` config. Without it, the server does not resolve country codes with its own proxy providers. It rejects the request with `INVALID_PROXY` when no `proxy` is set and the reported `client_region` differs from the server's region. A request that reports no `client_region` is not blocked. Pass `proxy` with your own proxy, or `no_proxy` to accept the server's own connection. With `server_proxy`, the server resolves a country code and picks a proxy for your `client_region` itself.
The `proxy` value must be a full proxy URI, unless the operator gives your API key `server_proxy` in the `serve.users` config. Without `server_proxy`, the server does not resolve country codes with its own proxy providers. It rejects the request with `INVALID_PROXY` when no `proxy` is set and the reported `client_region` differs from the server's region. A request that reports no `client_region` is not blocked. Pass `proxy` with your own proxy, or `no_proxy` to accept the server's own connection. With `server_proxy`, the server resolves a country code and picks a proxy for your `client_region` itself. A Control D resolver, `controld://<resolver>@dns.controld.com`, counts as a full proxy URI: the server runs a forwarder for it and needs no Control D configuration of its own.
=== "Response `200`"
docs/dev/rest-api/remote-sessions.md
+8 -2
View File
@@ -121,7 +121,12 @@ hood the client walks a remote session through its lifecycle.
it sits in a different region from the reported one, asking the client to
pass `--proxy` with its own proxy, or `--no-proxy` to accept the server's
own connection. With `server_proxy` the server picks a proxy that matches
the client region itself
the client region itself. For `--proxy controld:ca` the client points its
Control D profile at the region and sends only the resolver, as
`controld://<resolver>@dns.controld.com`. The server counts it as a full proxy
URI and runs its own forwarder for it. A bare region skips Control D, because its
proxy is a forwarder on the client. See
[Control D](../../guide/proxies-and-vpn.md#control-d)
- Track-selection hints (`range_`, `vcodec`, `quality`, `best_available`) so
the server fetches the right manifests
- Your language and audio codec selection (`-l`, `-vl`, `-al`, `-a`, `-fs`)
@@ -145,7 +150,8 @@ hood the client walks a remote session through its lifecycle.
`GET /api/services` advertises the regions those accounts cover. When your own
region is not one of them and you set no `--proxy`, the client resolves a proxy for
the first advertised region itself, so both sides sit in a region the account
works in.
works in. This is a bare region query, so it skips Control D. A server-account
service does not accept a `controld://` resolver from the client.
The server responds immediately with a session ID and a status. Authentication
runs in the background on the server.
docs/guide/cli-reference.md
+2 -2
View File
@@ -195,7 +195,7 @@ order the service used, then renumbers the episodes into the order you asked for
| Flag | Description |
|---|---|
| `--proxy` | Proxy URI, a 2-letter country code resolved from configured providers, or `provider:region` (e.g. `nordvpn:ca`, `gluetun:us`, `protonvpn:de:berlin`). |
| `--proxy` | Proxy URI, a country or location code resolved from configured providers, or `provider:region` (e.g. `nordvpn:ca`, `gluetun:us`, `protonvpn:de:berlin`, `controld:yul`). |
| `--no-proxy` | Force-disable all proxy use. |
| `--no-proxy-download` | Bypass the proxy for **all downloads** (manifest, licence, and auth stay proxied). |
| `--proxy-download` | Use a different proxy for **all downloads**, in the same forms as `--proxy` (manifest, licence, and auth stay on `--proxy`). |
@@ -235,7 +235,7 @@ Each service defines its own query syntax. unshackle prints the results as a tre
| Option | Description |
|---|---|
| `-p`, `--profile` | Profile for credentials and cookies. |
| `--proxy` | Proxy URI, 2-letter country code, or `provider:region`. |
| `--proxy` | Proxy URI, country or location code, or `provider:region` (e.g. `controld:ca`). |
| `--no-proxy` | Force-disable all proxy use. |
!!! example
docs/guide/downloading.md
+4 -2
View File
@@ -722,12 +722,14 @@ unshackle dl --tag MYGRP --repack EXAMPLE 81234567
## Proxies
`--proxy` accepts a full proxy URI, a 2-letter country code (resolved through your
configured proxy providers), or a `provider:region` form.
`--proxy` accepts a full proxy URI, a country or location code (resolved through your
configured proxy providers), or a `provider:region` form. See
[Proxies & VPN](proxies-and-vpn.md#how-proxy-resolution-works) for the exact grammar.
```shell title="Proxy forms"
unshackle dl --proxy us EXAMPLE 81234567
unshackle dl --proxy nordvpn:ca EXAMPLE 81234567
unshackle dl --proxy controld:yul EXAMPLE 81234567
unshackle dl --proxy 'http://user:pass@host:8080' EXAMPLE 81234567
```
docs/guide/proxies-and-vpn.md
+189 -9
View File
@@ -9,7 +9,7 @@ unshackle pick a working server for you.
The `--proxy` flag controls proxying, and the proxy providers unshackle supports change
a query to a real proxy URL: [basic static proxies](#basic-static-proxies),
[Gluetun](#gluetun) (documented in the most detail, since it is the most flexible),
[Control D](#control-d), [Gluetun](#gluetun) (documented in the most detail, since it is the most flexible),
[ExpressVPN](#expressvpn), [NordVPN](#nordvpn), [Proton VPN](#proton-vpn),
[Hola](#hola), [Surfshark](#surfshark), and [Windscribe](#windscribe).
@@ -106,23 +106,36 @@ flowchart TD
asks each loaded proxy provider **in order** and uses the first proxy any of them
returns.
unshackle compares the query against a region grammar: a two-letter country code,
optionally followed by a server number and/or a `:city` or `-city` part. It then
unshackle compares the query against a region grammar: a country or location code of two
to four letters, with an optional `_code` part (`res_yyz`), then an optional server
number and/or a `:city` or `-city` part. A `host:port` value never matches. It then
lowercases the query and hands it to the proxy provider. Its section below gives the
exact forms each proxy provider accepts.
A Control D resolver, `controld://<resolver>@dns.controld.com`, is a fourth form. A
`--remote` client sends it to a server in place of a proxy URI, and the server starts a
forwarder for it. See [Control D](#control-d).
### Provider load order
When you do a download without `--no-proxy`, unshackle instantiates every configured
proxy provider once, in this fixed order. It uses that same order for bare-query
resolution:
**Basic → ExpressVPN → NordVPN → Proton VPN → Surfshark → Windscribe → Gluetun → Hola**
**Basic → ExpressVPN → NordVPN → Proton VPN → Surfshark → Windscribe → Gluetun → Hola → Control D**
So if you have both Basic and NordVPN configured and use `--proxy us`, a `us` entry in
your Basic config wins, because unshackle tries Basic first. To skip ahead to a specific
proxy provider, prefix the query (`--proxy nordvpn:us`).
!!! note "A bare region reaches Control D last"
unshackle asks Control D for a bare region such as `--proxy us` only when no other
loaded proxy provider has a proxy for it. Hola loads automatically when the
`hola-proxy` binary is on your `PATH`, and it comes before Control D. When Control D answers, it changes your Control D
account: it points a profile at that country, and it can create an `unshackle-us`
profile and endpoint. The geofence auto-proxy uses the same order. To use Control D
directly, prefix the query (`--proxy controld:us`).
!!! note "Auto-loading providers"
Most providers only load when you configure them under `proxy_providers:`. Three are
special: **ExpressVPN** and **Proton VPN** also load automatically once their cache
@@ -190,6 +203,167 @@ proxy_providers:
unshackle dl --proxy de2 EXAMPLE 81234567
```
## Control D
[Control D](https://controld.com) is a DNS service, not a proxy. It answers the hostnames
it redirects with the IP of a transparent proxy in the region you chose, and that proxy
relays onwards by SNI. There is no proxy URI to hand to Requests, so this proxy provider runs a
small **forwarder on loopback**: it resolves each connection through your Control D
resolver and relays the bytes. Everything downstream (the service session, the downloads,
`--proxy-download`) sees an ordinary HTTP proxy.
unshackle picks the region itself, by pointing a profile's **default rule** at it. A profile
holds one region at a time, so unshackle gives each region in use a profile of its own,
each with its own endpoint and its own forwarder port. `--proxy controld:yyz
--proxy-download controld:yul` therefore uses two profiles, when two are free: Toronto for
the service and Montreal for the downloads. When no second profile is free, the second
region moves the first profile (see [step 5](#how-unshackle-chooses-a-profile)).
Keep both locations in one country. Some CDNs also check the country of the download IP,
and refuse a `--proxy-download` exit in another country with a 403. This applies to every
proxy provider.
`unshackle search --proxy controld:ca` works the same way as `dl`.
### What you need
- **A plan with traffic redirection.** On the personal plans that is *Full Control*;
*Some Control* has no redirection, and every redirect fails. The 14-day trial includes
it. See [Control D's plans](https://controld.com/plans) for current prices.
- **An API token of type *Write*.** A *Read* token cannot change a profile's default rule.
Create it in the dashboard under *API*. If you set *Allowed IPs* on the token, include
the address of each machine that runs unshackle with it.
```yaml title="unshackle.yaml"
proxy_providers:
controld:
token: api.xxxxxxxxxxxx # a Write API token, from the Control D dashboard
```
```shell
unshackle dl --proxy controld:ca EXAMPLE 81234567
```
The token is the only required key. You do not need to create a profile or an endpoint,
or to give a `profile` and `resolver`: unshackle creates an `unshackle-<region>` profile
and endpoint on your account the first time it needs one, and uses it again on the next
run. Give `profile` and `resolver` only to use a profile that you made yourself (see
[How unshackle chooses a profile](#how-unshackle-chooses-a-profile)).
The returned proxy is HTTP on loopback,
`http://127.0.0.1:{port}`. The locations themselves are read from Control D at startup
(107 of them across 68 countries, at the time of writing), so there is nothing to list:
| Query | Meaning |
|---|---|
| `ca` | One of Control D's Canadian locations, at random. |
| `yul` | That exact location, by its code. |
| `res_yyz` | A residential location, by its code. A country pick never uses a residential location, so name it this way. |
The [configuration reference](../reference/configuration/network.md#controld) lists
every `controld` config key.
### How unshackle chooses a profile
Each region query gets a profile and an endpoint named after it: `--proxy controld:ca` uses
`unshackle-ca`. For each region, unshackle uses the first match in this list:
1. A profile that this process already pointed at that region.
2. The endpoint named `unshackle-<region>` on your account, made by an earlier run.
3. A profile from your config that this process does not use yet. These are optional:
```yaml title="unshackle.yaml"
proxy_providers:
controld:
token: api.xxxxxxxxxxxx
profile: abcd123 # a profile kept for unshackle, and nothing else
resolver: efgh456 # the resolver ID of an endpoint using that profile
profiles: # more pairs, in the same shape
- profile: ijkl789
resolver: mnop012
max_profiles: 4 # the most profiles unshackle uses, yours included (default 4)
```
The resolver ID is the last part of the endpoint's DoH URL,
`https://dns.controld.com/<resolver>`. You can also give the full URL.
4. A new profile and endpoint named `unshackle-<region>`, while your config pairs plus the
`unshackle-*` endpoints on the account are fewer than `max_profiles`. unshackle keeps
them for the next run. `max_profiles` limits only this step: unshackle uses every pair
in your config, even more than `max_profiles`.
5. When no profile is free, a profile moves to the new region, with everything that still
uses it. unshackle first moves a profile that this process has not pointed at a region,
which another process can still use. Next, it moves the profile that this process
pointed at a region first. An `unshackle-*` profile that moves, and its endpoint, get the
new region's name. unshackle logs `Every Control D profile is in use; moving profile … to …`,
and connections already open keep the old exit until they close.
Because the names live on the account, every unshackle process agrees on them: two runs,
or two jobs on a `serve` instance, that ask for `ca` both use `unshackle-ca`. Two processes
can still move a profile under each other when they run out of profiles (step 5).
unshackle changes only the profiles in your config and the profiles of `unshackle-*`
endpoints. It skips an `unshackle-*` endpoint whose profile an endpoint without that name
also uses, so unshackle never redirects a profile that serves your own devices. Leave the `unshackle-*`
profiles empty: no services, no custom rules. unshackle sets their default rule to
`REDIRECT`, and the *Services* and *Custom Rules* tabs stay at zero by design. To remove
them, delete the endpoints and profiles in the dashboard. unshackle creates new ones when
it next needs them.
!!! warning "Business accounts pay per endpoint"
Personal plans allow an unlimited number of profiles and endpoints. Business plans bill
each endpoint every month, so each `unshackle-<country>` endpoint adds to the bill. On a business
account, configure your own pairs, as many as the countries you use at the same time,
and set `max_profiles` to that number. unshackle then never creates a profile.
!!! note "Everything goes through the proxy, including the segments"
With the default rule redirecting, the whole download leaves through Control D's
transparent proxy. Control D serves it on a best-effort basis. Pair it with `--no-proxy-download` to
keep the geo-check on the manifest and the licence while pulling the files over your
own connection.
!!! note "The switch is not instant"
unshackle drops its own DNS cache when it changes region, but Control D's edge may
still serve the previous answer for the remainder of the record's TTL.
!!! note "Same machine, same IP, over IPv4"
Control D authorises its proxies by source IP: the address that queried DNS is the one
allowed to use the proxy. The forwarder resolves and connects from the same host, so
this holds. Control D also answers a redirected name only to a query over IPv4, so the
forwarder asks over IPv4 even on a host with IPv6, and ignores `HTTPS_PROXY` for these
queries.
The forwarder refuses a connection to an IP address with a `502`, because an IP address
skips Control D's DNS and connects from your own address. A request that names a host
by IP address fails with this proxy provider.
!!! note "With `--remote`, the server resolves"
unshackle does not send your API token to a server. For `--proxy controld:ca` with
`--remote`, unshackle points `unshackle-ca` at Canada with your token, then sends the
server only that endpoint's resolver ID, as `controld://<resolver>@dns.controld.com`.
The server runs its own forwarder with it, from its own address, and needs no Control D
configuration. The resolver ID lets anyone who has it use that endpoint's proxy, so treat
it as a secret: unshackle masks it in logs and in stored job parameters.
The server must run a version of unshackle that knows `controld://`. An older server
rejects it with `INVALID_PROXY`: `Proxy provider 'controld' not found` when your API key
has `server_proxy`, else a message that asks for a full proxy URI. A bare region, such
as `--proxy ca`, skips Control D with `--remote`, because the proxy it gives is on your
machine. A service whose accounts the server manages does not accept a proxy from the
client, and that includes a Control D resolver.
### Troubleshooting Control D
| Symptom | Cause and fix |
|---|---|
| `Proxy check failed`, because the exit IP is your own IP. | The profile does not redirect. Make sure that the plan includes traffic redirection (*Full Control*, not *Some Control*). After a region change, wait for the TTL of the previous answer to end. |
| A `502 Bad Gateway` for one host. With `-d`, the log shows `Control D returned no A record for …`. | Control D gave no IPv4 address for that host. |
| A `502 Bad Gateway` for one host only. With `-d`, the log shows `Refused …, an address that Control D cannot redirect`. | The request names the host by IP address. The forwarder refuses it. See the note on IPv4 above. |
| An HTTP `401` or `403` from `api.controld.com`. | The token is a *Read* token, or its *Allowed IPs* do not include this machine. Use a *Write* token. |
| `Control D needs both a profile and a resolver` or `Control D needs max_profiles of 1 or more` at startup. | The `controld` config is not valid. Give `profile` and `resolver` together, and a `max_profiles` of 1 or more. Every run fails until you correct it, also a run that does not use Control D. |
| `Could not read Control D's locations`. | unshackle could not get the location list. It tries again at the next Control D query. |
| A `403` for the segments only, with `--proxy-download` in another country. | The CDN checks the country of the download IP. Keep `--proxy-download` in the same country as `--proxy`. |
## Gluetun
Gluetun is unshackle's most flexible proxy backend, and the one to reach for if you want
@@ -651,16 +825,22 @@ The returned proxy is HTTPS on **port 443**.
unshackle has **two** proxy-resolution code paths that are intentionally not
identical:
- `unshackle/commands/dl.py` inlines its own logic and loads **all eight** providers,
- `unshackle/commands/dl.py` inlines its own logic and loads **all nine** providers,
including Windscribe and Gluetun.
- `unshackle/core/proxies/resolve.py::initialize_proxy_providers()` is the shared
resolver used by the REST API handlers and the remote-service client. It **omits**
Windscribe and Gluetun.
Both paths accept the same query grammar (a direct URI, a `provider:query`, or a bare
country query tried against providers in order), but the set of available providers
differs. Keep this in mind when you add a proxy provider: an entry in `dl.py` does not
make it available over the API, and the opposite is also true.
Both paths accept the same query grammar (a direct URI, a `provider:query`, a bare
country query tried against providers in order, or a `controld://` resolver), but the
set of available providers differs. Keep this in mind when you add a proxy provider:
an entry in `dl.py` does not make it available over the API, and the opposite is also
true.
The remote-service client resolves `--proxy` with its own
`resolve_remote_proxy_arg()`. It changes a `controld:` query to a `controld://`
resolver for the server with `get_remote_proxy()`, and it removes Control D from a bare
query, because a loopback forwarder is of no use to a server on another machine.
!!! note "Implementing a new provider (developers)"
Every proxy provider subclasses `unshackle.core.proxies.proxy.Proxy` and has three
docs/index.md
+1 -1
View File
@@ -41,7 +41,7 @@ at a title and it will:
| Audio | AAC, AC-3, E-AC-3, AC-4, Opus, Vorbis, DTS, DTS-X, ALAC, FLAC; Dolby Atmos handling; per-channel-layout selection |
| Subtitles | SRT, WebVTT, ASS/SSA, TTML and more, with optional SDH stripping and format conversion |
| Key vaults | SQLite, MySQL, and HTTP/API vaults to store and share content keys |
| Proxies | Basic proxies plus NordVPN, ProtonVPN, Surfshark, Windscribe, ExpressVPN, Gluetun, and Hola providers |
| Proxies | Basic proxies plus Control D, NordVPN, ProtonVPN, Surfshark, Windscribe, ExpressVPN, Gluetun, and Hola providers |
| Services | A plugin API for the service modules you write. unshackle includes none |
| Automation | A REST API with a job queue, live progress, history, and remote-download sessions |
docs/reference/configuration/network.md
+14
View File
@@ -67,6 +67,7 @@ for the full proxy provider guide. Recognised proxy providers and their exit por
| Provider | Config key | Credentials | Proxy scheme/port |
|----------|-----------|-------------|-------------------|
| Basic (static) | `basic` | country → URI(s) | as specified |
| Control D | `controld` | Write API token (profile/resolver pairs optional) | `http://127.0.0.1:{port}` (local forwarder) |
| NordVPN | `nordvpn` | **service** credentials (48 chars combined) | `https://...:89` |
| Surfshark | `surfsharkvpn` | **service** credentials (48 chars combined) | `https://...:443` |
| Windscribe | `windscribevpn` | service credentials | `https://...:443` |
@@ -87,6 +88,19 @@ proxy_providers:
password: <service password>
```
### `controld`
Config keys of the Control D proxy provider. See
[Control D](../../guide/proxies-and-vpn.md#control-d) for how unshackle uses them.
| Key | Type | Default | Meaning |
|---|---|---|---|
| `token` | `str` | required | A Control D API token of type *Write*. |
| `profile` | `str` | none | Optional. The ID of a profile you made for unshackle. Give it together with `resolver`. Without it, unshackle creates `unshackle-<region>` profiles itself. |
| `resolver` | `str` | none | Optional. The resolver ID of an endpoint that uses `profile`, or its DoH URL `https://dns.controld.com/<resolver>`. |
| `profiles` | `list` | `[]` | Optional. More pairs, each a mapping with `profile` and `resolver`. |
| `max_profiles` | `int` | `4` | The most profiles unshackle uses: your pairs plus the `unshackle-*` endpoints on the account. It must be 1 or more. unshackle creates an `unshackle-<region>` profile and endpoint only while the total is lower. |
!!! note "Provider loading differs between CLI and REST server"
The `dl` CLI loads all providers, including `windscribevpn` and `gluetun`. The REST API /
remote-client path uses a separate resolver that does **not** load `windscribevpn` or