Commit Graph
54 Commits
Author SHA1 Message Date
imSp4rky c06b00ec63 fix(import): read an HLS export that names no title language or master key
Probe each media playlist for its KIDs and refuse a manifest unshackle cannot read.
2026-09-23 16:20:49 -06:00
imSp4rky d014c69d00 feat(export): separate export file name parts with dots
Titles, scope and quality parts use dots and never follow output_template.
2026-09-23 15:26:46 -06:00
imSp4rky a19c3e3879 fix(export): name the file from the first write 2026-09-23 15:11:15 -06:00
imSp4rky 347668eafa docs(export): describe the export file name and its messages 2026-09-23 15:11:15 -06:00
imSp4rky 80088fe4f6 feat(import): build tracks from direct-file export rows
Imports unidl json_manifest exports made of whole files.
2026-09-23 15:04:22 -06:00
imSp4rky 06866eb12e feat(export): record each track's KIDs
An import reads them first, so a track gets its own keys without a probe.
2026-09-23 14:35:58 -06:00
imSp4rky 84bb07f589 fix(import): stop clearly when no export or vault key exists
The bad-key retry now requests each dropped KID.
2026-09-23 14:33:34 -06:00
imSp4rkyandAntCardFr 783c8ae1c2 feat(proxies): add Control D proxy provider
Runs a loopback forwarder per region-named profile (unshackle-<region>), so
--proxy and --proxy-download can exit in different countries

Thanks to Orni_ for the original code and the test account.

Co-authored-by: AntCardFr <24798811+AntCardFR@users.noreply.github.com>
2026-09-23 13:23:50 -06:00
imSp4rky 30ea001cb3 Merge branch 'feat/new-export' into dev 2026-09-23 11:12:19 -06:00
imSp4rky a254830121 fix(export): import side-loaded subtitles and keep conflicting keys apart
The importer builds subtitle tracks from track rows that carry a url, which the format promised and no reader did. The writer keeps a conflicting KID instead of overwriting it, identifies a manifest by its whole URL, writes subtitle rows as srt or vtt rather than enum names, and survives an export file it cannot read.
2026-09-20 10:39:13 -06:00
imSp4rky 746385101b feat(export): write mediaexport files and read legacy exports through the package
--export writes the mediaexport format with DRM, content keys and manifests at title level, and unshackle's own track dicts under x-unshackle. unshackle import reads mediaexport, unidl v1 and unshackle v2 through the shared reader, applies the export's request headers to the import session, and rebuilds DRM from the title-level pool. mediaexport is a git dependency on unshackle-dl/mediaexport.
2026-09-20 00:07:04 -06:00
imSp4rky cf2906f120 fix(cli): make the click param types the one source of codec spellings
--acodec now lists its codecs in --help, --vcodec drops the duplicate entries, and the REST API derives its allowlists from the same types, so AC-4, DDP, VORB and sub_format "original" stop being rejected and H264 stops being silently dropped. Adds shell completion for --acodec and repairs MultipleChoice completion. Raises the click floor to 8.2.
2026-09-17 19:22:19 -06:00
imSp4rky a0515f8d57 feat(dl): add --proxy-download for a separate download proxy
Manifest, licence and auth stay on --proxy; track, subtitle and attachment downloads use the second proxy. Also exposed as proxy_download on the REST download job with the same server_proxy gate. The download session now ignores HTTP(S)_PROXY and copies rnet cookies set by name.
2026-09-13 14:41:01 -06:00
imSp4rky a7d6791136 feat(remote): verify server-CDM vault keys on the client and report bad pairs
The licence response lists KIDs a server vault supplied, the client proves them with the decode check, and a bad pair goes to the new session keys/bad route so the server flags its own row. Client vault keys go first over --remote, and the dashboard health warns when no SQLite vault can store flags.
2026-09-12 15:13:08 -06:00
imSp4rky d36797641d feat(dl): add --no-postscript to skip post-scripts for one run 2026-09-06 21:02:58 -06:00
imSp4rky 85503fd87c test(downloader): cover probe judging and the submission window
Assert that a probe is judged on the trailing tick, that a backoff is temporary and climbs back to the cap, and that a worker always finds its next segment staged. Documents adaptive-workers behaviour in the downloading guide.
2026-09-06 20:03:46 -06:00
imSp4rky dffb72e41d fix(audio): accept immersive channel layouts like 5.1.4
parse_channels rejected a three-figure layout, so a remote download crashed rebuilding the track from JSON. Add channel_total so --channels can compare either form.
2026-09-04 19:27:39 -06:00
imSp4rky 235839abc8 feat(audio): support DTS:X Profile 2 with MP4 muxing
Route titles with a dtsx/dtsy sample entry to MP4 via MP4Box, since Matroska has no CodecID for DTS-UHD. Add per-rendition HLS audio codec selection, fix merge_segments progress and its cleanup path.
2026-09-04 14:19:29 -06:00
imSp4rky 4ef577c1d1 fix(tracks): match resolutions per range and codec
An exact height match in one range denied the 16:9 canvas fallback to the others, so a cropped HDR10 or SDR track was dropped whenever an uncropped DV track matched the requested height. Also sweep the hybrid L5 temp files so a killed run cannot leave a stale RPU behind.
2026-08-27 14:32:36 -06:00
imSp4rky b94f338619 docs(post-scripts): use post-script as the one term for hooks
Add the post-script row to the ste-writing terminology contract and
replace hook/postscript/post-download script across docs and docstrings.
2026-08-25 19:57:41 -06:00
imSp4rky e804d09f0a feat(dl): add --cdm to pin one CDM device for a run
Overrides the cdm config mapping (and its quality/DRM sub-entries) for the whole run; the REST API cdm field now flows through the same param.
2026-08-24 16:37:48 -06:00
imSp4rky 8c43250c9b feat(dl): add --no-attachments to skip cover art and subtitle fonts
Drops service attachments at track selection and skips the font attach pass; exposed as no_attachments in the REST dl params and documented.
2026-08-24 16:21:52 -06:00
imSp4rky d8739b3561 feat(dl): add --require-audio/--require-video and --warn-only alias
Gate a title on audio or video languages that must exist, independent of what -l/-al/-vl select. --require-subs joins the same gate: its --s-lang conflict and download-all side effect are gone, so -sl now decides what to keep. Requiring a language that is also excluded fails at parse time.
2026-08-23 12:24:45 -06:00
imSp4rky 8b10b793e2 docs: phase 2 of the prose style across docs and docstrings 2026-08-19 20:59:09 -06:00
imSp4rky 66971230c3 docs: apply simplified technical english across docs and docstrings. 2026-08-19 16:31:06 -06:00
imSp4rky 68c7d95173 refactor(music)!: make Song an ordinary Title on the generic path
Delete the dead grouped-download block, the planner, and the music service hooks; Song now flows through the one download loop, gets tagged via tag_file, and answers -w with {disc}x{track} keys.

BREAKING CHANGE: GROUP_AUDIO_DOWNLOADS, get_music_track_options and get_music_collection_label are gone. A music service returns one Audio track per quality and lets the framework choose.
2026-08-18 15:05:30 -06:00
imSp4rky 89acfab279 feat(dl): run user post-download scripts with metadata variables
New post_scripts config and repeatable --postscript flag run a command per output, season/album folder, or run, substituting filename-template variables into pre-tokenized argv. The REST API rejects hook commands.

Closes #150
2026-08-18 08:26:14 -06:00
imSp4rky 619c86cd10 feat(downloader): resume failed downloads from completed segments across runs
Fingerprint-gated reuse of atomically-published segment files (DASH/ISM/HLS) behind a continue_downloads config option and one-off dl --continue-downloads flag.
2026-08-16 19:38:06 -06:00
imSp4rky ee9d3edfe3 Merge branch 'dev' into feat/downloader-speed
# Conflicts:
#	unshackle/core/api/download_manager.py
#	unshackle/core/downloaders/requests.py
#	unshackle/core/manifests/dash.py
#	unshackle/core/session.py
#	unshackle/core/utils/webvtt.py
2026-08-16 12:43:01 -06:00
imSp4rky 4a21dbebb5 feat(dl): add daily date-based episodes with tvdb air-date fill
--daily flag plus Service.DAILY/Title.daily, -w selection by ISO air
date or colon date range, and a pre-1970 air-date guard.
2026-08-11 17:50:22 -06:00
imSp4rky e4f6ddec85 feat(dl): add absolute episode numbers filled from tvdb absolute order
Episode gains an additive absolute field services can set, an {absolute}
naming token (3-wide), and an --enrich fill from TVDB's absolute order
that never rewrites season or episode numbers.
2026-08-11 16:42:07 -06:00
imSp4rky 72581ba5d2 feat(metadata)!: replace animeapi with a first-class anilist provider
--anilist supersedes --animeapi (REST: anilist_id); the keyless AniList
GraphQL provider joins the registry with title search, mal: ID lookup,
an ANILIST mkv tag, and the anilist_title_language config option.
Services mark anime via Service.ANIME / Title.anime for anilist-first
metadata resolution with fall-through.
2026-08-11 16:42:05 -06:00
imSp4rky ebb4e74318 feat(metadata)!: authoritative id resolver and keyless imdb provider
Supplied IDs resolve directly via a shared resolve_by_ids and never fall back to title search; metadata_providers gains per-kind ordering and a disable_metadata switch stops all automatic lookups while explicit IDs keep working.
BREAKING: --tmdb/--imdb/--tvdb are now mutually exclusive and require a usable provider; --enrich also overwrites original language; the imdbapi provider and imdb_api_enabled are replaced by a keyless imdb provider.
2026-08-11 15:07:24 -06:00
imSp4rky 79ad70a9cf feat(dl): exclude languages with a - prefix on -l, -vl, -al, -sl and -fsl 2026-08-10 12:18:30 -06:00
imSp4rky 8a7022a091 feat(dl): allow --tvdb as an --enrich metadata source 2026-08-09 11:55:06 -06:00
imSp4rky 8d3785b460 feat(enrich): fill in the original language from metadata providers
--enrich now sets Title.language when the service left it unset, so video tracks stop falling back to und. Adds original_language to MetadataResult (TMDB, TVDB, OMDb, IMDxAPI) and falls back to OMDb for --imdb, which previously did nothing while imdb_api_enabled defaults to false.
2026-08-08 13:52:55 -06:00
imSp4rky 9466db37a6 style: tighten comments added on this branch 2026-08-07 11:43:13 -06:00
imSp4rky 431aa8fa54 fix(downloader): keep speed-limited batches in a single process
Spawned children re-import the module and start with no TokenBucket, so
--download-processes silently dropped a configured --speed-limit /
global_speed_limit for any 24+ segment batch. Fall back to the in-process
path while a limit is set (same gate as the ranged-parallel paths), log the
fallback, and document the interaction.
2026-08-07 11:36:17 -06:00
imSp4rky cfae8f8882 Merge branch 'dev' into feat/downloader-speed
Catch the branch up with dev (45 commits) so testers can run the perf work
against current dev.

Conflict resolutions:
- downloaders/requests.py: keep the branch's racing-gated read1, adaptive
  controller speed reporting and byte-range slice guards; take dev's
  content-encoding checks (raw reads are skipped on an encoded body) and its
  rolling SpeedWindow for the no-controller speed fallback. The single-URL
  ranged-parallel gate now requires no speed limit and no caller Range.
- service.py / session.py: keep the branch's TimeoutHTTPAdapter and retry
  config, raise POOL_MAX_SIZE to 64 so dev's pool-starvation fix survives
  (pool_block=True stalls threads on a pool smaller than the in-flight count).
- docs: dev split configuration.md into per-area pages, so the branch's rnet
  network option rows moved to reference/configuration/network.md.
2026-08-07 10:45:47 -06:00
imSp4rky 041ba5293b feat(titles): add optional Episode.part, selected with -w s1e1.2
Opt-in and additive: an episode without a part behaves exactly as before.
2026-08-04 17:38:30 -06:00
imSp4rky c4b25756eb docs: split configuration reference into per-area pages 2026-08-04 10:25:32 -06:00
imSp4rky a6997f6bca feat(tracks): add language_priority to sort preferred languages first
subtitle.language_priority and audio.language_priority prepend to the CLI
languages, so the named languages sort first and nothing is filtered.
--exact-lang now applies to the sorts too, through matching_languages, which
follows the same RFC 4647 rule as Tracks.by_language and ignores invalid tags.
2026-08-03 10:15:05 -06:00
imSp4rky 4e1ccf3230 feat(dl): add -fsl/--forced-s-lang to filter forced subtitles by language 2026-08-03 09:27:40 -06:00
imSp4rky 786309ea5c docs: cover the TVDB provider, episode ordering and provider order
Document tvdb_api_key, tvdb_pin, tvdb_order and metadata_providers, the
--tvdb and --tvdb-order flags with their behaviour when an order does
not cover a whole series, the new REST API parameters, and the Matroska
TVDB2 tag prefixes.
2026-08-02 10:48:39 -06:00
imSp4rky bf85c948b4 fix(dl): honour --no-proxy-download for attachments 2026-07-31 13:22:18 -06:00
imSp4rky f7a38732f1 feat(dl): extend --select-titles to movie listings 2026-07-28 21:22:14 -06:00
imSp4rky 40ae213504 docs: cleanup and correct documentation 2026-07-26 21:21:01 -06:00
imSp4rky 8cf1700684 fix(import): stop losing side-loaded subtitles and blaming the manifest URL
Only fall back to Title.language when the manifest labels nothing, and reject und as firmly as a missing tag. Exported direct-URL subtitles now replace the re-parsed set, and fetch failures stay distinct from parse failures.
2026-07-25 11:32:17 -06:00
imSp4rky 5de6fe9014 Merge branch 'dev' into feat/downloader-speed
# Conflicts:
#	unshackle/core/downloaders/requests.py
#	unshackle/core/manifests/dash.py
2026-07-16 12:47:59 -06:00
imSp4rky f0e0c6d39e feat(downloader): fan large segment batches across worker processes
One CPython interpreter caps sustained segment throughput at ~160 MB/s;
spawned children each run the existing thread-pool path on a contiguous
chunk of the batch, reaching NIC line rate on fast CDNs. Opt-in via
--download-processes (default 1), engaged only for 24+ segment batches.
Sessions are rebuilt per child from a picklable spec; RnetSession falls
back to a single process when it has no impersonate preset.
2026-07-16 10:26:44 -06:00