mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-05-18 06:49:27 +00:00
feat(api): sync /api/download with dl CLI flags and add serve.* defaults
- Wire --no-proxy-download through download_manager + handlers + swagger - Add tag/proxy/tmdb_id/animeapi_id/enrich/worst to DEFAULT_DOWNLOAD_PARAMS - Normalize `slow` (bool/"MIN-MAX" string/list) to tuple before invoking dl.result - Overlay any /api/download flag declared under `serve:` in unshackle.yaml as a default (downloads, workers, best_available, etc.); request body still wins - Quiet successful worker stderr from `warning` to `debug` (kept under job.worker_stderr for ?full=true) - Include HYBRID in range validator - Document new flags + overlay layering + max_concurrent_downloads / download_job_retention_hours
This commit is contained in:
@@ -46,6 +46,22 @@ unshackle serve --remote-only # Only expose remote service session en
|
|||||||
- `username` - Internal logging name for the user (not visible to users)
|
- `username` - Internal logging name for the user (not visible to users)
|
||||||
- `services` - Optional per-user service allowlist. Effective access is the intersection of global and per-user allowlists.
|
- `services` - Optional per-user service allowlist. Effective access is the intersection of global and per-user allowlists.
|
||||||
|
|
||||||
|
#### Server-side `dl` defaults
|
||||||
|
|
||||||
|
Any key accepted by `/api/download` (see `docs/API.md`) can also be declared directly under `serve:` and the REST API will treat it as a default. Per-request bodies still win. Use this to raise concurrency, force `best_available`, etc. without each client repeating the values:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
serve:
|
||||||
|
api_secret: "..."
|
||||||
|
users: { ... }
|
||||||
|
downloads: 4 # parallel tracks per job
|
||||||
|
workers: 16 # threads per track
|
||||||
|
best_available: true
|
||||||
|
no_proxy_download: false
|
||||||
|
```
|
||||||
|
|
||||||
|
Layering: built-in defaults < `serve.*` overrides < service-specific defaults < request body.
|
||||||
|
|
||||||
For example,
|
For example,
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
@@ -88,6 +104,26 @@ See [API.md](API.md) for full REST API documentation with endpoints, parameters,
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## max_concurrent_downloads (int)
|
||||||
|
|
||||||
|
Maximum number of `/api/download` jobs the serve queue manager will execute in parallel. Each job runs the full `dl` pipeline (authenticate, fetch tracks, decrypt, mux) in its own worker subprocess. This is independent of `serve.downloads`, which controls parallel tracks **inside** a single job. Default: `2`.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
max_concurrent_downloads: 4
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## download_job_retention_hours (int)
|
||||||
|
|
||||||
|
How long completed, failed, or cancelled download jobs remain queryable via `/api/download/jobs/{job_id}` before the periodic cleanup loop drops them. Default: `24`.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
download_job_retention_hours: 48
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## debug (bool)
|
## debug (bool)
|
||||||
|
|
||||||
Enables comprehensive debug logging. Default: `false`
|
Enables comprehensive debug logging. Default: `false`
|
||||||
|
|||||||
22
docs/API.md
22
docs/API.md
@@ -77,6 +77,22 @@ There is no separate "tier" flag. Whether the server can return KID:KEY for a se
|
|||||||
|
|
||||||
Per-service CDM type can be pinned via `config.cdm` (`widevine`/`playready`) or per-service `cdm_type`; otherwise the server picks the type the user has devices for.
|
Per-service CDM type can be pinned via `config.cdm` (`widevine`/`playready`) or per-service `cdm_type`; otherwise the server picks the type the user has devices for.
|
||||||
|
|
||||||
|
### Server-side `dl` defaults
|
||||||
|
|
||||||
|
Any flag accepted by `/api/download` (see the table below) can be declared under `serve:` in `unshackle.yaml` and the API will apply it as a default. Request-body values still win. Useful for raising concurrency without changing every client call:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
serve:
|
||||||
|
api_secret: "..."
|
||||||
|
users: { ... }
|
||||||
|
downloads: 4 # parallel tracks per download job
|
||||||
|
workers: 16 # threads per track segment fetch
|
||||||
|
best_available: true
|
||||||
|
no_proxy_download: false
|
||||||
|
```
|
||||||
|
|
||||||
|
Layering order: built-in defaults < `serve.*` overrides < service-specific click defaults < request body.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Endpoint Map
|
## Endpoint Map
|
||||||
@@ -300,10 +316,12 @@ Start a download job. Returns immediately with a job ID (HTTP 202). Disabled in
|
|||||||
| `profile` | string | `null` | Profile for credentials/cookies |
|
| `profile` | string | `null` | Profile for credentials/cookies |
|
||||||
| `proxy` | string | `null` | Proxy URI or country code |
|
| `proxy` | string | `null` | Proxy URI or country code |
|
||||||
| `no_proxy` | boolean | `false` | Disable all proxy use |
|
| `no_proxy` | boolean | `false` | Disable all proxy use |
|
||||||
|
| `no_proxy_download` | boolean | `false` | Bypass proxy for segment downloads only. Manifest, license, and auth still use proxy |
|
||||||
| `workers` | int | `null` | Max threads per track download |
|
| `workers` | int | `null` | Max threads per track download |
|
||||||
| `downloads` | int | `1` | Concurrent track downloads |
|
| `downloads` | int | `1` | Concurrent track downloads |
|
||||||
| `slow` | boolean | `false` | Add 60-120s delay between titles |
|
| `slow` | boolean or string | `null` | Add randomized delay between titles. `true` = 60-120s, or `"MIN-MAX"` string (e.g., `"20-40"`). Min must be >= 20 |
|
||||||
| `best_available` | boolean | `false` | Continue if requested quality unavailable |
|
| `best_available` | boolean | `false` | Continue if requested quality unavailable |
|
||||||
|
| `worst` | boolean | `false` | Select the lowest bitrate track within the specified quality. Requires `quality` |
|
||||||
| `skip_dl` | boolean | `false` | Skip download, only get decryption keys |
|
| `skip_dl` | boolean | `false` | Skip download, only get decryption keys |
|
||||||
| `export` | boolean | `false` | Export manifest, track URLs, keys, and subtitles to JSON in the exports directory |
|
| `export` | boolean | `false` | Export manifest, track URLs, keys, and subtitles to JSON in the exports directory |
|
||||||
| `cdm_only` | boolean | `null` | Only use CDM (`true`) or only vaults (`false`) |
|
| `cdm_only` | boolean | `null` | Only use CDM (`true`) or only vaults (`false`) |
|
||||||
@@ -758,6 +776,6 @@ queued -> cancelled
|
|||||||
downloading -> cancelled
|
downloading -> cancelled
|
||||||
```
|
```
|
||||||
|
|
||||||
Jobs are retained for 24 hours after completion. The server supports up to 2 concurrent downloads by default.
|
Jobs are retained for 24 hours after completion (override via top-level `download_job_retention_hours` in `unshackle.yaml`). The server runs up to 2 concurrent download jobs by default; override via top-level `max_concurrent_downloads`. This is independent of `serve.downloads`, which controls parallel tracks **within** a single job.
|
||||||
|
|
||||||
Remote sessions are managed by `SessionStore` (`unshackle/core/api/session_store.py`); idle sessions and their `InputBridge` instances are cleaned up by a background loop started/stopped with the app lifecycle.
|
Remote sessions are managed by `SessionStore` (`unshackle/core/api/session_store.py`); idle sessions and their `InputBridge` instances are cleaned up by a background loop started/stopped with the app lifecycle.
|
||||||
|
|||||||
@@ -102,6 +102,7 @@ to a CLI option on the `dl` command. CLI arguments always take priority over con
|
|||||||
| `require_subs` | list | `[]` | Required subtitle languages (skip title if missing) |
|
| `require_subs` | list | `[]` | Required subtitle languages (skip title if missing) |
|
||||||
| `forced_subs` | bool | `false` | Include forced subtitle tracks |
|
| `forced_subs` | bool | `false` | Include forced subtitle tracks |
|
||||||
| `exact_lang` | bool | `false` | Exact language matching (no regional variants) |
|
| `exact_lang` | bool | `false` | Exact language matching (no regional variants) |
|
||||||
|
| `latest_episode` | bool | `false` | Download only the single most recent episode of a series |
|
||||||
|
|
||||||
**Track selection:**
|
**Track selection:**
|
||||||
|
|
||||||
@@ -147,6 +148,7 @@ to a CLI option on the `dl` command. CLI arguments always take priority over con
|
|||||||
| `downloads` | int | `1` | Concurrent track downloads |
|
| `downloads` | int | `1` | Concurrent track downloads |
|
||||||
| `workers` | int | `min(16, cpu_count + 4)` | Max threads per track download (segments / ranged parts) |
|
| `workers` | int | `min(16, cpu_count + 4)` | Max threads per track download (segments / ranged parts) |
|
||||||
| `slow` | bool or `MIN-MAX` | `false` | Randomized delay between titles. `true` uses 60-120s; pass `MIN-MAX` (e.g., `20-40`) for a custom range |
|
| `slow` | bool or `MIN-MAX` | `false` | Randomized delay between titles. `true` uses 60-120s; pass `MIN-MAX` (e.g., `20-40`) for a custom range |
|
||||||
|
| `no_proxy_download` | bool | `false` | Bypass proxy for segment downloads only. Manifest, license, and auth still use proxy |
|
||||||
| `skip_dl` | bool | `false` | Skip download, only get decryption keys |
|
| `skip_dl` | bool | `false` | Skip download, only get decryption keys |
|
||||||
| `cdm_only` | bool | `null` | Only use CDM (`true`) or only vaults (`false`) |
|
| `cdm_only` | bool | `null` | Only use CDM (`true`) or only vaults (`false`) |
|
||||||
|
|
||||||
|
|||||||
@@ -150,6 +150,21 @@ def _perform_download(
|
|||||||
if params.get("export"):
|
if params.get("export"):
|
||||||
params["export"] = bool(params["export"])
|
params["export"] = bool(params["export"])
|
||||||
|
|
||||||
|
# Normalize slow: accept string "MIN-MAX", list/tuple, or True (default 60-120)
|
||||||
|
slow_raw = params.get("slow")
|
||||||
|
if slow_raw is not None and not isinstance(slow_raw, tuple):
|
||||||
|
if isinstance(slow_raw, bool):
|
||||||
|
params["slow"] = (60, 120) if slow_raw else None
|
||||||
|
elif isinstance(slow_raw, list) and len(slow_raw) == 2:
|
||||||
|
params["slow"] = (int(slow_raw[0]), int(slow_raw[1]))
|
||||||
|
elif isinstance(slow_raw, str):
|
||||||
|
from unshackle.core.utils.click_types import SLOW_DELAY_RANGE
|
||||||
|
|
||||||
|
try:
|
||||||
|
params["slow"] = SLOW_DELAY_RANGE.convert(slow_raw, None, None)
|
||||||
|
except click.BadParameter as exc:
|
||||||
|
raise Exception(f"Invalid slow parameter: {exc}")
|
||||||
|
|
||||||
# Convert wanted episode strings to internal "SxE" format
|
# Convert wanted episode strings to internal "SxE" format
|
||||||
# Accepts: "S01E01", "S01-S03", "s1e1", "1x1", or already-parsed format
|
# Accepts: "S01E01", "S01-S03", "s1e1", "1x1", or already-parsed format
|
||||||
wanted_raw = params.get("wanted")
|
wanted_raw = params.get("wanted")
|
||||||
@@ -180,6 +195,7 @@ def _perform_download(
|
|||||||
ctx.params = {
|
ctx.params = {
|
||||||
"proxy": params.get("proxy"),
|
"proxy": params.get("proxy"),
|
||||||
"no_proxy": params.get("no_proxy", False),
|
"no_proxy": params.get("no_proxy", False),
|
||||||
|
"no_proxy_download": params.get("no_proxy_download", False),
|
||||||
"profile": params.get("profile"),
|
"profile": params.get("profile"),
|
||||||
"repack": params.get("repack", False),
|
"repack": params.get("repack", False),
|
||||||
"tag": params.get("tag"),
|
"tag": params.get("tag"),
|
||||||
@@ -318,6 +334,7 @@ def _perform_download(
|
|||||||
export=params.get("export"),
|
export=params.get("export"),
|
||||||
cdm_only=params.get("cdm_only"),
|
cdm_only=params.get("cdm_only"),
|
||||||
no_proxy=params.get("no_proxy", False),
|
no_proxy=params.get("no_proxy", False),
|
||||||
|
no_proxy_download=params.get("no_proxy_download", False),
|
||||||
no_folder=params.get("no_folder", False),
|
no_folder=params.get("no_folder", False),
|
||||||
no_source=params.get("no_source", False),
|
no_source=params.get("no_source", False),
|
||||||
no_mux=params.get("no_mux", False),
|
no_mux=params.get("no_mux", False),
|
||||||
@@ -653,8 +670,11 @@ class DownloadQueueManager:
|
|||||||
if stdout.strip():
|
if stdout.strip():
|
||||||
log.debug(f"Worker stdout for job {job.job_id}: {stdout.strip()}")
|
log.debug(f"Worker stdout for job {job.job_id}: {stdout.strip()}")
|
||||||
if stderr.strip():
|
if stderr.strip():
|
||||||
log.warning(f"Worker stderr for job {job.job_id}: {stderr.strip()}")
|
|
||||||
job.worker_stderr = stderr.strip()
|
job.worker_stderr = stderr.strip()
|
||||||
|
if returncode != 0:
|
||||||
|
log.warning(f"Worker stderr for job {job.job_id}: {stderr.strip()}")
|
||||||
|
else:
|
||||||
|
log.debug(f"Worker stderr for job {job.job_id}: {stderr.strip()}")
|
||||||
|
|
||||||
result_data: Optional[Dict[str, Any]] = None
|
result_data: Optional[Dict[str, Any]] = None
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -58,7 +58,9 @@ DEFAULT_DOWNLOAD_PARAMS = {
|
|||||||
"skip_dl": False,
|
"skip_dl": False,
|
||||||
"export": False,
|
"export": False,
|
||||||
"cdm_only": None,
|
"cdm_only": None,
|
||||||
|
"proxy": None,
|
||||||
"no_proxy": False,
|
"no_proxy": False,
|
||||||
|
"no_proxy_download": False,
|
||||||
"no_folder": False,
|
"no_folder": False,
|
||||||
"no_source": False,
|
"no_source": False,
|
||||||
"no_mux": False,
|
"no_mux": False,
|
||||||
@@ -67,7 +69,11 @@ DEFAULT_DOWNLOAD_PARAMS = {
|
|||||||
"worst": False,
|
"worst": False,
|
||||||
"best_available": False,
|
"best_available": False,
|
||||||
"repack": False,
|
"repack": False,
|
||||||
|
"tag": None,
|
||||||
|
"tmdb_id": None,
|
||||||
"imdb_id": None,
|
"imdb_id": None,
|
||||||
|
"animeapi_id": None,
|
||||||
|
"enrich": False,
|
||||||
"output_dir": None,
|
"output_dir": None,
|
||||||
"no_cache": False,
|
"no_cache": False,
|
||||||
"reset_cache": False,
|
"reset_cache": False,
|
||||||
@@ -1026,7 +1032,7 @@ def validate_download_parameters(data: Dict[str, Any]) -> Optional[str]:
|
|||||||
return "Cannot use both s_lang and require_subs"
|
return "Cannot use both s_lang and require_subs"
|
||||||
|
|
||||||
if "range" in data and data["range"]:
|
if "range" in data and data["range"]:
|
||||||
valid_ranges = ["SDR", "HDR10", "HDR10+", "DV", "HLG"]
|
valid_ranges = ["SDR", "HDR10", "HDR10+", "DV", "HLG", "HYBRID"]
|
||||||
if isinstance(data["range"], list):
|
if isinstance(data["range"], list):
|
||||||
for r in data["range"]:
|
for r in data["range"]:
|
||||||
if r.upper() not in valid_ranges:
|
if r.upper() not in valid_ranges:
|
||||||
@@ -1097,8 +1103,17 @@ async def download_handler(data: Dict[str, Any], request: Optional[web.Request]
|
|||||||
|
|
||||||
# Create download job with filtered parameters (exclude service and title_id as they're already passed)
|
# Create download job with filtered parameters (exclude service and title_id as they're already passed)
|
||||||
filtered_params = {k: v for k, v in data.items() if k not in ["service", "title_id"]}
|
filtered_params = {k: v for k, v in data.items() if k not in ["service", "title_id"]}
|
||||||
# Merge defaults with provided parameters (user params override service defaults, which override global defaults)
|
# Overlay any dl-relevant keys from `serve:` config (e.g. downloads, workers) so the API
|
||||||
params_with_defaults = {**DEFAULT_DOWNLOAD_PARAMS, **service_specific_defaults, **filtered_params}
|
# respects server-side defaults without each client having to send them.
|
||||||
|
serve_overrides = {
|
||||||
|
k: v for k, v in (config.serve or {}).items() if k in DEFAULT_DOWNLOAD_PARAMS and v is not None
|
||||||
|
}
|
||||||
|
params_with_defaults = {
|
||||||
|
**DEFAULT_DOWNLOAD_PARAMS,
|
||||||
|
**serve_overrides,
|
||||||
|
**service_specific_defaults,
|
||||||
|
**filtered_params,
|
||||||
|
}
|
||||||
job = manager.create_job(normalized_service, title_id, **params_with_defaults)
|
job = manager.create_job(normalized_service, title_id, **params_with_defaults)
|
||||||
|
|
||||||
return web.json_response(
|
return web.json_response(
|
||||||
|
|||||||
@@ -630,8 +630,10 @@ async def download(request: web.Request) -> web.Response:
|
|||||||
type: boolean
|
type: boolean
|
||||||
description: Download audio description tracks (default - false)
|
description: Download audio description tracks (default - false)
|
||||||
slow:
|
slow:
|
||||||
type: boolean
|
oneOf:
|
||||||
description: Add 60-120s delay between downloads (default - false)
|
- type: boolean
|
||||||
|
- type: string
|
||||||
|
description: Add randomized delay between downloads. `true` for default 60-120s, or `"MIN-MAX"` string (e.g., `"20-40"`). Min must be >= 20 (default - null)
|
||||||
split_audio:
|
split_audio:
|
||||||
type: boolean
|
type: boolean
|
||||||
description: Create separate output files per audio codec instead of merging all audio (default - null)
|
description: Create separate output files per audio codec instead of merging all audio (default - null)
|
||||||
@@ -650,6 +652,9 @@ async def download(request: web.Request) -> web.Response:
|
|||||||
no_proxy:
|
no_proxy:
|
||||||
type: boolean
|
type: boolean
|
||||||
description: Force disable all proxy use (default - false)
|
description: Force disable all proxy use (default - false)
|
||||||
|
no_proxy_download:
|
||||||
|
type: boolean
|
||||||
|
description: Bypass proxy for segment downloads only. Manifest, license, and auth still use proxy (default - false)
|
||||||
tag:
|
tag:
|
||||||
type: string
|
type: string
|
||||||
description: Set the group tag to be used (default - None)
|
description: Set the group tag to be used (default - None)
|
||||||
@@ -680,6 +685,9 @@ async def download(request: web.Request) -> web.Response:
|
|||||||
best_available:
|
best_available:
|
||||||
type: boolean
|
type: boolean
|
||||||
description: Continue with best available if requested quality unavailable (default - false)
|
description: Continue with best available if requested quality unavailable (default - false)
|
||||||
|
worst:
|
||||||
|
type: boolean
|
||||||
|
description: Select the lowest bitrate track within the specified quality. Requires `quality` (default - false)
|
||||||
repack:
|
repack:
|
||||||
type: boolean
|
type: boolean
|
||||||
description: Add REPACK tag to the output filename (default - false)
|
description: Add REPACK tag to the output filename (default - false)
|
||||||
|
|||||||
@@ -522,6 +522,14 @@ serve:
|
|||||||
# playready_devices: # PlayReady device paths (auto-populated from directories.prds)
|
# playready_devices: # PlayReady device paths (auto-populated from directories.prds)
|
||||||
# - '/path/to/device.prd'
|
# - '/path/to/device.prd'
|
||||||
|
|
||||||
|
# Optional: any /api/download flag can be set here as a server-side default.
|
||||||
|
# Per-request body values still win. Useful for raising concurrency without
|
||||||
|
# changing every client call. Full list of accepted keys: see docs/API.md.
|
||||||
|
# downloads: 4 # parallel tracks per download job
|
||||||
|
# workers: 16 # threads per track segment fetch
|
||||||
|
# best_available: true
|
||||||
|
# no_proxy_download: false
|
||||||
|
|
||||||
# Remote Services Configuration
|
# Remote Services Configuration
|
||||||
# Connect to a remote unshackle server (unshackle serve) to use its services
|
# Connect to a remote unshackle server (unshackle serve) to use its services
|
||||||
# without needing the service code locally. Use with: unshackle dl --remote
|
# without needing the service code locally. Use with: unshackle dl --remote
|
||||||
|
|||||||
Reference in New Issue
Block a user