Commit Graph
52 Commits
Author SHA1 Message Date
imSp4rky d7a2bb6b8e fix(subs): stop segmented WebVTT merge from stripping formatting
Merge cues at the string level instead of round-tripping through
pycaption, which dropped inline tags and cue settings. STYLE and
REGION blocks are kept, duplicate boundary cues splice correctly,
and a bad timing line now skips the cue instead of crashing.

Also rank pysubs2 above SubtitleEdit for ASS to TTML/VTT since
SubtitleEdit flattens inline styling there. Documented the measured
backend fidelity table and added regression tests for the merge.
2026-07-12 09:39:39 -06:00
imSp4rky 8769f678b1 fix(tags): point IMDbApi provider at api.tiffara.com (IMDxAPI)
Upstream migrated the free IMDxAPI from imdbapi.dev to api.tiffara.com.
Endpoints and JSON schema are unchanged, so only BASE_URL and the
user-facing docs/docstrings needed updating.
2026-07-12 07:34:47 -06:00
imSp4rky 6434fc2081 fix(dl): make audio/video track selection consistent and crash-safe
Share one best-audio selector across the episode and music paths so best/all works for songs, descriptive audio is capped to one track, and codec_priority is honored. Guard --channels and 16:9 canvas math against tracks missing channel/width metadata. Extend --best-available to video languages and full resolution misses in both normal and hybrid mode, and count only deliverable ranges when reporting missing resolutions. Rank video by resolution before bitrate, and trim group audio selections to one track per song with a warning.

BREAKING CHANGE: a missing resolution with -r HYBRID now errors without --best-available, matching non-hybrid behavior.
2026-07-11 18:41:41 -06:00
imSp4rky c47aad6fa1 feat(tags): add OMDb metadata provider with omdb_api_key config 2026-07-11 11:46:58 -06:00
imSp4rky 20a4e6e2eb feat(tags): add imdb_api_enabled toggle, off by default 2026-07-11 09:42:52 -06:00
imSp4rky 847696b301 feat(proxies)!: rewrite ExpressVPN provider to Android device login
Swap cookie/PKCE auth for the OAuth 2.0 device authorization grant with a cached rotating refresh token. Fix the connection-token key (`cat`) and move the cache to {cache}/vpn/ to match ProtonVPN.

BREAKING CHANGE: cookie-file auth removed; run once with `enable: true` to approve a device code.
2026-07-09 11:20:38 -06:00
imSp4rky 4ebe41fafd feat(tracks): add subtitle.type_priority config for variant ordering
Optional ranking of forced/normal/sdh per language, mirroring audio.codec_priority; first track gets the default flag, so this also picks the default subtitle. Unset = existing behavior.
2026-07-07 17:36:58 -06:00
imSp4rky 84d9b92530 docs: expand config reference and genericize service names 2026-07-06 22:50:38 -06:00
imSp4rky 1d8cec6d2b docs: restructure documentation into MkDocs Material site 2026-07-03 19:24:11 -06:00
imSp4rky ef2e880c7f perf(downloader): cut per-segment overhead, bound stalled connections
Segment tmp-file+rename scheme, request timeouts, hedged tail requests, streamed ISM/DASH merges, rnet client knobs, curl_impersonate -> network.
2026-07-02 09:20:03 -06:00
imSp4rky 7fdd514e18 feat(api): add job retry/priority/clear, history, config and maintenance endpoints
Persistent job history (capped by serve.history_limit, stored under cache), profiles/config reads, maintenance + env-check endpoints, and 409 conflict code.
2026-07-01 22:42:48 -06:00
imSp4rky 96da1ac0b0 docs: fix inaccuracies in service-creation guide 2026-06-30 16:04:12 -06:00
imSp4rky 59f3cb4fbe feat(titles): add air_date for date-based episode naming
Optional Episode.air_date names daily/sports content by date instead of SxxExx (no template change). Presentation-only; `{year}` dropped only when it duplicates the date's year. Round-trips via export/import.
2026-06-30 15:45:57 -06:00
imSp4rky 987de239ee docs: add "Creating a Service" guide 2026-06-30 15:45:57 -06:00
imSp4rky 2f92ffe1a2 ci(docs): serve site from docs.unshackle.dev custom domain 2026-06-30 10:40:43 -06:00
imSp4rky ecbcdff314 docs: add MkDocs Material documentation site 2026-06-30 10:28:48 -06:00
imSp4rky 39f5bc2a9e feat(proxies): add ProtonVPN TV login, drop cookie refresh
Self-sustaining TV-code auth; cookies are now access-only and re-exported on expiry.
2026-06-28 18:42:11 -06:00
imSp4rky e96e9be143 feat(output): support nested directories in folder templates
Treat `/` in folder templates as a path separator (sanitize per segment) and let movies use a folder when a `movies` template is set.
2026-06-25 10:40:49 -06:00
imSp4rky 14c96371b9 feat(proxies): add Proton VPN proxy provider
Cookie-based ProtonVPN provider with country/city/server-number selection
2026-06-24 22:11:51 -06:00
imSp4rky 66638ead6b feat(services): load service plugins from git repos
Allow directories.services entries to be git repo specs (URL/owner-repo/ @branch) alongside local paths: clone + TTL-pull as cache, list-order priority with a duplicate-ignored summary, a refresh-services command that force-syncs and reports the per-service diff, and a dirty-clone guard that protects local edits on the automated path. Adds binaries.Git and a redact_paths config toggle.
2026-06-24 08:59:03 -06:00
imSp4rky 6294566823 fix(config): match per-service cdm keys case-insensitively
cdm lookups used the raw service key while decryption upper-cases its keys, so a case-mismatched cdm entry silently used the default. Add a shared ci_get helper, route cdm/decryption through resolvers, drop a redundant decryption mutation and dead downloader writes, and add a regression bench.
2026-06-22 16:50:59 -06:00
imSp4rky ce69e8a028 docs(config): make docs/ canonical, slim unshackle-example.yaml to a starter 2026-06-21 21:23:03 -06:00
imSp4rky 605c0e65cb feat(docs): add per-vault network timeout configuration for remote vaults 2026-06-21 20:57:17 -06:00
imSp4rky ba69bc7d61 feat(dl): add --merge-video to merge video language variants
Group selected videos by (resolution, range, codec) and mux each group into one MKV; only language is collapsed, so ranges/codecs/resolutions stay in separate files. Adds --merge-video flag + muxing.merge_video config (global and per-service), docs, and tests.
2026-06-17 13:31:21 -06:00
imSp4rky 2e1ffebec0 docs: rename to match other docs. 2026-06-15 20:55:27 -06:00
imSp4rky 466bf610cc feat(drm): add native DASH ClearKey (org.w3.clearkey) support
unshackle's DASH parser only recognised Widevine and PlayReady ContentProtection, so services using W3C EME ClearKey had to fake a Widevine object and monkey-patch get_content_keys. Add a first-class ClearKeyCENC DRM type so services just implement a license callback.

- ClearKeyCENC (core/drm/clearkey_cenc.py): KID-based, no CDM/PSSH; builds the W3C JSON license request (unpadded base64url), parses the JWK Set response (dict/str/bytes), falls back to POSTing the manifest Laurl when the service returns None, decrypts via the same shaka/ mp4decrypt CENC path as Widevine
- DASH.get_drm emits ClearKeyCENC for scheme e2719d58-...; KID from own or sibling mp4protection cenc:default_KID, Laurl across dashif/legacy/ bare namespaces
- track.download dispatches prepare_drm for ClearKeyCENC; dl.prepare_drm gains a clearkey branch (cache/vault lookup, license-failure tolerated when content_keys pre-populated, vault push, export)
- Service.get_clearkey_license base callback (default None -> Laurl); drm_from_dict reconstructs ClearKeyCENC for export/import round-trip
- EXAMPLE service + config demo the callback
- Tests: tests/core/test_clearkey_cenc.py and an export round-trip case
- Docs: DRM_CONFIG.md ClearKey section
2026-06-11 12:26:57 -06:00
imSp4rky 394ed67e28 feat(logging): expand debug-logging coverage, add primitives, redaction & tests
Instrument the full download pipeline in the structured JSONL debug logger and make adding logging to new features a one-liner.

Coverage:
- DRM license request/response, content keys (incl. remote-CDM seam) and decrypt timing across Widevine/PlayReady/ClearKey
- DASH/HLS/ISM manifest fetch + parse milestones (HLS.to_tracks also covers the m3u8_parser path used by iTunes/ATV-style services)
- Per-backend vault get/add via the Vaults manager, track selection, subtitle conversion, repackage, normalize_vui, and full mkvmerge mux (command, duration, output size, warnings)
- All external tooling (ffmpeg, ffprobe, mkvmerge, mkvpropedit, dovi_tool, SubtitleEdit, ccextractor) via a unified `tool_run` op, centralised in run_step/ffprobe + log_tool_run

DX:
- Add log_event() / timed_operation() primitives (no-op when disabled); retrofit ~91 guard/timing blocks onto them
- Fix message= collision in log_drm_operation/log_vault_query/log_service_call that raised TypeError on the live decrypt path

Redaction (redact_all = redact_text -> redact_url -> redact_path):
- Collapse content/CDN/api URLs to `redacted[.ext]`
- Strip local path prefixes (install root -> <unshackle>, venv -> <venv>, home -> ~)
- Apply to every logged string so shared logs leak no URLs, paths or usernames
- Drop per-request service_call logging (manifest parse is the request seam)
2026-06-09 22:12:05 -06:00
imSp4rky 246ff528f5 fix(dl): apply per-service dl config overrides for all options
services.<TAG>.dl values only applied when the key was also set in the global dl: section (equality check against config.dl missed Click's declared defaults). Gate on Click's ParameterSource instead: CLI/env > service dl > global dl > defaults, converted via each option's own type.

- record parameter sources on serve's hand-built context so client values are never clobbered
- accept range/list as natural keys for range_/list_
- harden QualityList (YAML int) and SlowDelayRange (YAML bool) converts
2026-06-09 18:17:29 -06:00
imSp4rky 29232925d5 feat(subtitle): data-driven conversion registry + SubtitleEdit 5 support
Replace the hardcoded conversion if/elif in Subtitle.convert with a capability-matrix backend registry (subtitle_convert.py): each backend declares the source->target pairs it supports plus a rank, and run_conversion tries them in order as a real fallback chain. conversion_method pins a backend but still falls back (pin-then-fallback).

- Detect the cross-platform SubtitleEdit 5+ CLI (seconv) and use its --flag syntax for convert, SDH stripping, and reverse-RTL
- Protect styled ASS/SSA from automatic SRT downconversion; honor an explicit --sub-format / sidecar_format
- Read segmented fVTT (wvtt) and fTTML (stpp) directly from fragmented MP4
- Improve ASS/SSA font detection: inline \fn overrides, Format-located Fontname column, @-prefix strip, case-insensitive de-dup; covers SSA too
- Update SUBTITLE_CONFIG.md, example yaml, README; add regression tests and a backend benchmark script
2026-06-07 22:21:25 -06:00
imSp4rky 1cb0e4b766 feat(naming): per-service title_map remapping (#106)
Add services.<TAG>.title_map exact-match dict to rewrite service-provided titles before naming/output. Shared remap_titles helper applied on local dl, import, and client-side dl --remote (server stays raw so clients can override names for services they don't have installed locally).
2026-05-26 14:08:52 -06:00
imSp4rky 13f924f825 feat(dl): add real bitrate probing (-rvb/-rab)
Add --real-video-bitrate/-rvb and --real-audio-bitrate/-rab to measure a track's true bitrate from actual media size instead of the manifest's declared value, which is often inflated. Useful for accurate track listings and --vbitrate/--vbitrate-range selection.
Single-file tracks are measured exactly; segmented HLS/DASH/ISM are sampled. Only the top renditions per quality tier are probed, in parallel, so it stays fast. Without the flags, behaviour is unchanged.
2026-05-24 17:30:40 -06:00
imSp4rky 684e56eb97 feat(tracks): configurable audio codec priority for tie-breaking
Adds optional `audio.codec_priority` list in unshackle.yaml to define preferred audio codec order when tracks share the same bitrate and language. Listed codecs rank in the order given; unlisted codecs retain bitrate-based order and fall after the listed group (soft priority - nothing dropped). Atmos and descriptive rules still apply last.
2026-05-18 11:08:55 -06:00
imSp4rky 61fe16e8d7 feat(api): sync /api/download with dl CLI flags and add serve.* defaults
- Wire --no-proxy-download through download_manager + handlers + swagger
- Add tag/proxy/tmdb_id/animeapi_id/enrich/worst to DEFAULT_DOWNLOAD_PARAMS
- Normalize `slow` (bool/"MIN-MAX" string/list) to tuple before invoking dl.result
- Overlay any /api/download flag declared under `serve:` in unshackle.yaml as a default (downloads, workers, best_available, etc.); request body still wins
- Quiet successful worker stderr from `warning` to `debug` (kept under job.worker_stderr for ?full=true)
- Include HYBRID in range validator
- Document new flags + overlay layering + max_concurrent_downloads / download_job_retention_hours
2026-05-17 11:54:02 -06:00
imSp4rky c0dc1eb91f docs(output): document muxing.default_language override 2026-05-15 08:22:45 -06:00
imSp4rky 7fb88e9a97 docs: update docs to match current codebase 2026-05-08 17:54:45 -06:00
imSp4rky a7af898617 feat(ip-info): consolidate IP lookup, add ipinfo.io token support
Replace get_ip_info + get_cached_ip_info pair with a single unshackle.core.utils.ip_info module providing a normalized return shape across providers. Adds optional ipinfo_api_key config for the ipinfo.io Lite endpoint (higher rate limits, ASN/org/continent data), swaps the ipapi.co fallback for ip-api.in, and migrates all callers (service, gluetun, remote_service, api/handlers, DSNP, YT) to the new import path. Auth token is sent per-request and never attached to the shared session headers.
2026-05-04 22:20:43 -06:00
imSp4rky bb0a800ab6 docs(api): update --export from string path to boolean flag
Update API docs, Swagger schema, handlers, and example config to reflect --export as a boolean flag that auto-generates export files in the configurable exports directory.
2026-04-02 11:27:39 -06:00
Andy e323f6f3b3 feat(template): add configurable folder naming via output_template.folder (#94)
Adds an optional `folder` key under `output_template` to customize output folder names using the same template variables as file naming.
2026-03-25 21:42:47 -06:00
Andy 4c55f7af5b docs: update API and configuration documentation with example service tags 2026-03-19 12:55:39 -06:00
Sp5rky 313efe5b03 Rename remote-services-flow.md to REMOTE-SERVICES-FLOW.md 2026-03-19 12:13:45 -06:00
Andy b1a5f8f37f feat(remote): add server-CDM mode, manifest transfer, and region-aware proxy
Server-side:
- Add server_cdm mode: server handles full CDM licensing using its own devices, returns KID:KEY pairs instead of raw license bytes
- Support batch license resolution for multiple tracks in one request
- Extract DRM from manifest ContentProtection when track.drm is empty
- Serialize DASH/ISM manifest XML as base64 in /tracks response
- Include session cookies/headers and server_cdm_type in /tracks response
- Detect server CDM type from actual track DRM + configured devices
- Check server region against client_region to skip unnecessary proxy
- Support decrypt_labs and custom_api remote CDMs for both WV and PR

Client-side:
- Re-parse DASH/ISM manifests locally from base64 to populate track.data
- Match remote tracks to re-parsed tracks by ID with attribute fallback
- Copy DRM objects from re-parsed manifests to remote tracks
- Pre-fetch keys via resolve_server_keys() before downloads start
- Fallback per-track licensing via _proxy_license during download
- Apply session cookies/headers from server for CDN access
- Apply downloader/decryption config directly for remote services
- Preserve pre-injected content_keys during DASH DRM override
- Skip redundant CDM calls when all KIDs already have keys

Docs:
- Add comprehensive remote-services-flow.md with Mermaid diagrams covering proxy mode, server-CDM mode, manifest transfer, and config
2026-03-18 19:30:59 -06:00
Andy f86d4e7937 docs(dl): add comprehensive list of available dl keys and their descriptions 2026-03-17 09:27:52 -06:00
Andy 572a894620 feat(dl): add --animeapi and --enrich options for anime metadata and tagging
Add AnimeAPI integration to resolve anime database IDs (MAL, AniList, Kitsu, etc.) to TMDB/IMDB/TVDB for MKV tagging. The --enrich flag overrides show title and fills in year when missing from the service.

- Add animeapi-py dependency for cross-platform anime ID resolution
- Add --animeapi option (e.g. mal:12345, anilist:98765, defaults to MAL)
- Add --enrich flag to override title/year from external sources
- Remove --tmdb-name and --tmdb-year in favor of unified --enrich
- Update REST API params and docs to match
2026-02-28 12:51:14 -07:00
Andy 5bd03c67cf fix(api): resolve Sentinel serialization, missing params, and add search endpoint (#80)
Fix multiple issues with the REST API that caused downloads to fail:
- Filter Click Sentinel.UNSET enum values from service parameter defaults that caused "Object of type Sentinel is not JSON serializable" errors
- Add missing select_titles and no_video args to dl.result() call
- Fix wanted param unpacking for list-tracks SeasonRange.parse_tokens()
- Add enum conversion for vcodec, range, sub_format, and export params that were passed as strings but expected as enums by dl.result()
- Add missing dl command params: split_audio, repack, imdb_id, output_dir, no_cache, reset_cache to DEFAULT_DOWNLOAD_PARAMS and download worker
- Expand vcodec/acodec/sub_format validation to cover all supported values
- Add POST /api/search endpoint for searching services by query
- Update Swagger docs with all new params and correct type definitions
- Add comprehensive REST API documentation (docs/API.md)
- Update ADVANCED_CONFIG.md with serve CLI options and API reference
2026-02-27 19:17:15 -07:00
Andy c8883a5404 feat(templates): add configurable language tagging rule engine (#58) 2026-02-26 21:17:46 -07:00
Andy 6ce7b6c4d3 feat(templates)!: add customizable output filename templates (#12)
BREAKING CHANGE: The 'scene_naming' config option has been removed.
Users must configure 'output_template' in unshackle.yaml with movies, series, and songs templates. See unshackle-example.yaml for examples.
2026-02-26 18:23:18 -07:00
Andy 820db5f179 refactor(providers): extract metadata providers into modular system
- Create `unshackle/core/providers/` package with abstract base class, IMDBApi (free, no key), SIMKL, and TMDB provider implementations
- Add consensus-based ID enrichment: cross-references IMDB IDs with TMDB and SIMKL, drops all data from providers that disagree on tmdb_id (likely resolved to wrong title)
- Cache enriched IDs alongside raw provider data so they survive cache round-trips
- Genericize TitleCacher with `cache_provider()`/`get_cached_provider()` replacing provider-specific methods; respect `--no-cache` flag
- Add `--imdb` CLI flag to dl command for direct IMDB ID lookup
2026-02-25 19:02:18 -07:00
Andy 86a7a95b6f docs: update and correct configuration documentation 2026-02-24 23:50:33 -07:00
Andy 5cf488cb34 docs: add configuration documentation WIP 2026-01-30 16:34:49 -07:00
Andy 4b30090d87 feat(gluetun): improve VPN connection display and Windscribe support 2026-01-24 13:36:04 -07:00