Supplied IDs resolve directly via a shared resolve_by_ids and never fall back to title search; metadata_providers gains per-kind ordering and a disable_metadata switch stops all automatic lookups while explicit IDs keep working.
BREAKING: --tmdb/--imdb/--tvdb are now mutually exclusive and require a usable provider; --enrich also overwrites original language; the imdbapi provider and imdb_api_enabled are replaced by a keyless imdb provider.
--enrich now sets Title.language when the service left it unset, so video tracks stop falling back to und. Adds original_language to MetadataResult (TMDB, TVDB, OMDb, IMDxAPI) and falls back to OMDb for --imdb, which previously did nothing while imdb_api_enabled defaults to false.
Each dl run works in temp/task_<id>/, removed on every exit path. Orphans from hard-killed processes are swept via lock liveness, and env clear temp skips directories whose task is still running.
TVDB covers TV far better than the existing sources, so add it as a
provider and let users decide where it sits in the lookup chain.
- new TVDB v4 provider: search, get_by_id, get_external_ids and
find_by_imdb_id, with a process-level token cache and 401 re-auth
- metadata_providers config both orders and filters the providers used,
so TVDB can be primary rather than a TMDB fallback. Cross-validation
keeps its own fixed trust ranking, unchanged by search order
- --tvdb-order renumbers episodes into a TVDB season order. A service
does not always use aired order, so the source order is detected
rather than assumed, and the renumber is refused when an order that
omits episodes would give two of them the same filename
- --tvdb sets the series ID directly, skipping the lookup
- the Matroska TVDB2 tag now populates on every lookup path, including
--imdb, which previously produced no ID tags when IMDxAPI was off
Search matches on year as well as title so an identically named remake
cannot win on title alone.
Only merge back tracks from another manifest, so a drifted id no longer duplicates a rung or wipes the main manifest's own subtitles. track_is_encrypted now recognises an ISM ProtectionHeader.
Detect the dot/space style from the full folder template instead of per segment, so `{artist}/{album} ({year})` no longer yields `Some.Artist/Some Album (2025)`. Also drops empty `[]` groups like the existing `()` handling and removes redundant cleanup passes.
UNITY_MATRIX emitted 11 int32s instead of 9, oversizing mvhd/tkhd so pymp4 desynced and fTTML subtitles yielded no cues. Also rewrites PIFF senc to CENC so shaka can decrypt Smooth, labels HEV1 as hev1, and raises when a fragment survives decryption still carrying a sample-encryption box.
Smooth fragments set tfhd sample_description_index=2 against a server init carrying a second, identical sample entry. A lone entry left that dangling: mp4decrypt skipped every such fragment silently and demuxers stopped after the first, yielding a file that looked fine but was almost entirely encrypted.
Only fall back to Title.language when the manifest labels nothing, and reject und as firmly as a missing tag. Exported direct-URL subtitles now replace the re-parsed set, and fetch failures stay distinct from parse failures.
Optional Episode.air_date names daily/sports content by date instead of SxxExx (no template change). Presentation-only; `{year}` dropped only when it duplicates the date's year. Round-trips via export/import.
Allow directories.services entries to be git repo specs (URL/owner-repo/ @branch) alongside local paths: clone + TTL-pull as cache, list-order priority with a duplicate-ignored summary, a refresh-services command that force-syncs and reports the per-service diff, and a dirty-clone guard that protects local edits on the automated path. Adds binaries.Git and a redact_paths config toggle.
cdm lookups used the raw service key while decryption upper-cases its keys, so a case-mismatched cdm entry silently used the default. Add a shared ci_get helper, route cdm/decryption through resolvers, drop a redundant decryption mutation and dead downloader writes, and add a regression bench.
* Add native Music core workflow
* feat(music): add shared music core helpers, album folder template, and UX cleanup
Consolidate duplicated music-service logic into core and extend the native music workflow.
- add core/music/extract.py: shared stateless helpers (first_text, first_number, year/duration/name formatting, classify_release_kind, dedupe_track_options, build_music_from_songs) so services stop carrying their own copies
- add core/music/display.py: shared rich rendering (render_track_panel, render_album_header, render_artwork_preview) with TrackRow / MusicHeaderInfo data holders
- export the new helpers from core/music/__init__.py
- add dedicated `albums` folder template kind (output_template.folder.albums) resolving albums -> songs -> "{artist} - {album} ({year})"; whitelist music template variables (album_artist, track_total, disc_total, release_type, genre, explicit, isrc, upc, label)
- fix song filename crash: config.get_output_template(...) did not exist; use config.output_template.get("songs") with a sane default
- strip emojis from music output (renderer, dl.py music branch) to match unshackle UX; remove dead MusicSongPlan import and music_icon logic
- document the albums folder key in unshackle-example.yaml
- add tests for extract, display, and the folder template
---------
Co-authored-by: MrMovies-Dev <MrMovies-Dev@users.noreply.github.com>
Smooth Streaming manifests carry no range attributes, so every ISM video track was labelled SDR even when the stream is HDR10/HLG/DV, breaking range-based selection (-r HDR10 / -r DV) for ISM services.
- ism_init: walk the full HEVC SPS (incl. scaling-list and st_ref_pic_set skippers) to read the VUI colour triple (colour_primaries, transfer_characteristics, matrix_coeffs); expose parse_codec_private_data_colour() keyed by FourCC. No unshackle imports added.
- ism: new ISM.get_video_range() maps the CICP triple via Video.Range.from_cicp (PQ -> HDR10, HLG -> HLG, BT.709/absent -> SDR); DVHE/DVH1 FourCCs map straight to DV since DV bitstreams signal Unspecified (2,2,2) in the VUI. to_tracks() now sets range_ on every video track. Soft-fails to SDR on malformed data.
- ism: accept RnetSession in download_track() so TLS-impersonated sessions pass the type check.
- tests: real PQ/HLG/BT.709 (x265-minted) and Dolby Vision (live-manifest, DoViProfile=stn, out-of-order SPS,PPS,VPS NALs) CodecPrivateData samples; byte-level VUI assertions in test_ism_init and manifest->Range characterization in new test_ism_range.
ISM (Smooth Streaming) tracks raw-concatenate moof+mdat fragments with no ftyp/moov, so shaka-packager/mp4decrypt fail with PARSER_FAILURE (exit 2) on decrypt. The init box was previously built by n_m3u8dl_re, removed in the downloader consolidation.
Add ism_init.py, a dependency-free byte-level MP4 init-segment synthesizer that rebuilds ftyp+moov from the manifest CodecPrivateData, ported from yt-dlp's write_piff_header and N_m3u8DL-RE's MSSMoovProcessor with full codec parity:
- AVC (H264/AVC1/DAVC), with SPS/PPS picked by NAL type rather than position and NALUnitLengthField honored
- HEVC (HVC1/HEV1) with chroma format and bit depths parsed from the de-emulated SPS via exp-Golomb so 10-bit/HDR signals correctly, and profile/tier/level lifted from the SPS PTL
- Dolby Vision (DVHE/DVH1) as hvcC with a dvh1 sample entry
- AAC (AACL/AACH) with the AudioSpecificConfig synthesized from SamplingRate/Channels when the manifest omits CodecPrivateData
- EC-3 with a real dec3 box extracted from the WAVEFORMATEXTENSIBLE CodecPrivateData (Dolby GUID located by search, not fixed offset)
- TTML subtitles as stpp/sthd/subt, wired for fragmented-TTML tracks
CENC wrapping (encv/enca + sinf/tenc with default_KID) covers encrypted tracks: the per-sample IV size is derived from the fragment senc/saiz (PIFF override flag, payload arithmetic, saiz fallback) instead of assuming 8, and the constant-IV tenc form is supported. Read the track_ID from the first fragment's tfhd so the moov matches and the muxer does not drop samples.
Wire ISM.download_track to prepend the synthesized init before merging; unsupported codecs soft-fail to raw concatenation with a warning. Harden against real-world inputs: 2-letter/uppercase manifest language tags normalize to ISO-639-2 (und fallback), >65535 Hz sample rates no longer overflow the 16.16 field, truncated tfhd returns None, struct.error joins the soft-fail handler, and the emulation-prevention scan no longer over-strips consecutive escapes.
Add regression tests (37) covering box structure, every supported FourCC, 10-bit SPS parsing, ASC synthesis, dec3 extraction, IV-size derivation and the crash fixes. Validated structurally per codec with ffmpeg-minted fragments: shaka-packager parses synth-init+fragments with exit 0 and ffprobe reports the expected codec, including a live run against a public Smooth Streaming server.
unshackle's DASH parser only recognised Widevine and PlayReady ContentProtection, so services using W3C EME ClearKey had to fake a Widevine object and monkey-patch get_content_keys. Add a first-class ClearKeyCENC DRM type so services just implement a license callback.
- ClearKeyCENC (core/drm/clearkey_cenc.py): KID-based, no CDM/PSSH; builds the W3C JSON license request (unpadded base64url), parses the JWK Set response (dict/str/bytes), falls back to POSTing the manifest Laurl when the service returns None, decrypts via the same shaka/ mp4decrypt CENC path as Widevine
- DASH.get_drm emits ClearKeyCENC for scheme e2719d58-...; KID from own or sibling mp4protection cenc:default_KID, Laurl across dashif/legacy/ bare namespaces
- track.download dispatches prepare_drm for ClearKeyCENC; dl.prepare_drm gains a clearkey branch (cache/vault lookup, license-failure tolerated when content_keys pre-populated, vault push, export)
- Service.get_clearkey_license base callback (default None -> Laurl); drm_from_dict reconstructs ClearKeyCENC for export/import round-trip
- EXAMPLE service + config demo the callback
- Tests: tests/core/test_clearkey_cenc.py and an export round-trip case
- Docs: DRM_CONFIG.md ClearKey section
Instrument the full download pipeline in the structured JSONL debug logger and make adding logging to new features a one-liner.
Coverage:
- DRM license request/response, content keys (incl. remote-CDM seam) and decrypt timing across Widevine/PlayReady/ClearKey
- DASH/HLS/ISM manifest fetch + parse milestones (HLS.to_tracks also covers the m3u8_parser path used by iTunes/ATV-style services)
- Per-backend vault get/add via the Vaults manager, track selection, subtitle conversion, repackage, normalize_vui, and full mkvmerge mux (command, duration, output size, warnings)
- All external tooling (ffmpeg, ffprobe, mkvmerge, mkvpropedit, dovi_tool, SubtitleEdit, ccextractor) via a unified `tool_run` op, centralised in run_step/ffprobe + log_tool_run
DX:
- Add log_event() / timed_operation() primitives (no-op when disabled); retrofit ~91 guard/timing blocks onto them
- Fix message= collision in log_drm_operation/log_vault_query/log_service_call that raised TypeError on the live decrypt path
Redaction (redact_all = redact_text -> redact_url -> redact_path):
- Collapse content/CDN/api URLs to `redacted[.ext]`
- Strip local path prefixes (install root -> <unshackle>, venv -> <venv>, home -> ~)
- Apply to every logged string so shared logs leak no URLs, paths or usernames
- Drop per-request service_call logging (manifest parse is the request seam)
services.<TAG>.dl values only applied when the key was also set in the global dl: section (equality check against config.dl missed Click's declared defaults). Gate on Click's ParameterSource instead: CLI/env > service dl > global dl > defaults, converted via each option's own type.
- record parameter sources on serve's hand-built context so client values are never clobbered
- accept range/list as natural keys for range_/list_
- harden QualityList (YAML int) and SlowDelayRange (YAML bool) converts
ast.Num/node.n deprecated since Python 3.8; NodeVisitor falls back to visit_Num with a DeprecationWarning per dispatch. Under -W error this surfaced as a misleading fps ValueError in Video.init. Replace with visit_Constant, reject non-numeric constants, and pin parse results for int/fraction/float inputs in a regression test.
imSp4rky