Commit Graph
1231 Commits
Author SHA1 Message Date
imSp4rky 1a0beebf70 merge: TheTVDB v4 metadata provider 2026-08-02 10:50:19 -06:00
imSp4rky 786309ea5c docs: cover the TVDB provider, episode ordering and provider order
Document tvdb_api_key, tvdb_pin, tvdb_order and metadata_providers, the
--tvdb and --tvdb-order flags with their behaviour when an order does
not cover a whole series, the new REST API parameters, and the Matroska
TVDB2 tag prefixes.
2026-08-02 10:48:39 -06:00
imSp4rky 4ac34069ec feat(providers): add TheTVDB v4 as a metadata provider
TVDB covers TV far better than the existing sources, so add it as a
provider and let users decide where it sits in the lookup chain.

- new TVDB v4 provider: search, get_by_id, get_external_ids and
  find_by_imdb_id, with a process-level token cache and 401 re-auth
- metadata_providers config both orders and filters the providers used,
  so TVDB can be primary rather than a TMDB fallback. Cross-validation
  keeps its own fixed trust ranking, unchanged by search order
- --tvdb-order renumbers episodes into a TVDB season order. A service
  does not always use aired order, so the source order is detected
  rather than assumed, and the renumber is refused when an order that
  omits episodes would give two of them the same filename
- --tvdb sets the series ID directly, skipping the lookup
- the Matroska TVDB2 tag now populates on every lookup path, including
  --imdb, which previously produced no ID tags when IMDxAPI was off

Search matches on year as well as title so an identically named remake
cannot win on title alone.
2026-08-02 10:48:32 -06:00
imSp4rky 8baf064750 feat(tracks): add subtitle.group_by to keep a language with its variants
sort_subtitles applied the language sort before the type sort. Python sorts
are stable, so the last pass wins and type became the major key: every forced
track, then every normal, then every SDH, each block ordered by language. In a
title with many languages this put a track far from its own SDH version, for
example Finnish at track 5 and Finnish SDH at track 42.

Add a group_by parameter that selects the major key. "type" is the default and
keeps the current order byte for byte. "language" swaps the two passes so each
language stays next to its own variants. Read it from the new subtitle.group_by
config key.

Also correct the docs: type_priority was described as sorting the variants
within each language, which was never true. It ranks the types across all
languages.
2026-08-02 10:45:28 -06:00
imSp4rky bf85c948b4 fix(dl): honour --no-proxy-download for attachments 2026-07-31 13:22:18 -06:00
imSp4rky 734df87bad feat(dl): add --speed-limit and serve global_speed_limit
Shared token bucket across all download threads; decimal suffixes (5M = 5.0 MB/s) or plain bytes/sec
2026-07-31 12:48:46 -06:00
imSp4rky e9a40a3af3 perf: speed up the code hash with sha1 and a plain file reader 2026-07-30 17:11:18 -06:00
imSp4rky c14277e6dc feat: identify the running code by source hash 2026-07-30 16:38:11 -06:00
imSp4rky 72b8ab7027 feat: show git commit hash alongside the version 2026-07-30 13:44:07 -06:00
imSp4rky 42d7fd8303 fix(import): scope the merge-back and detect ISM protection
Only merge back tracks from another manifest, so a drifted id no longer duplicates a rung or wipes the main manifest's own subtitles. track_is_encrypted now recognises an ISM ProtectionHeader.
2026-07-29 10:48:34 -06:00
sp4rk.y cebfd7bc80 Merge pull request #145 from ApeDevOne/dev
fix(import): restore missing DASH manifest data and HEVC tracks
2026-07-29 10:13:09 -06:00
imSp4rky bcf0815cf4 fix(tracks): size the no-proxy download session's connection pool 2026-07-29 08:44:27 -06:00
RocketBoy bece8fb746 Implement manifest data resolution for DASH/ISM tracks along with Import Fix for json exports with no "manifest_url"
Added a new function to resolve import manifest data for DASH/ISM tracks, enhancing the handling of track data population from export dictionaries.
2026-07-29 18:23:02 +05:30
imSp4rky f7a38732f1 feat(dl): extend --select-titles to movie listings 2026-07-28 21:22:14 -06:00
imSp4rky a24c9b92da fix(tracks): ignore leftover PIFF uuid when verifying decryption
A decrypter detaches only the atom it consumed, so content shipping both boxes keeps the uuid through a good decrypt and failed on mp4decrypt.
2026-07-27 13:11:13 -06:00
imSp4rky 3c0fef75a7 docs(core): drop service tags from framework comments 2026-07-27 12:15:14 -06:00
imSp4rky 40ae213504 docs: cleanup and correct documentation 2026-07-26 21:21:01 -06:00
imSp4rky 8bacb247ee fix(lang): surface orig in errors and flag the original audio track over the API
Error messages named the resolved language for a user who only ever defaulted to orig. Audio tracks now carry is_original, resolved through Tracks.by_language so it agrees with the downloader.
2026-07-26 18:22:46 -06:00
imSp4rky b74b64825e fix(templates): use one separator style across a whole folder path
Detect the dot/space style from the full folder template instead of per segment, so `{artist}/{album} ({year})` no longer yields `Some.Artist/Some Album (2025)`. Also drops empty `[]` groups like the existing `()` handling and removes redundant cleanup passes.
2026-07-26 09:48:58 -06:00
sp4rk.y 4c54c9ec1b Merge pull request #144 from CodeName393/fix-template-formatting
fix(core): strip trailing spaces and dots in template formatting to prevent WinError 3
2026-07-26 09:28:56 -06:00
CodeName393 41106b0d24 fix(core): strip trailing spaces and dots in template formatting to prevent WinError 3
- Clean up redundant spaces and separators when conditional tags (e.g. {year?}) are omitted.
- Strip leading and trailing spaces and dots in sanitize_filename and TemplateFormatter.
- Prevent FileNotFoundError [WinError 3] caused by trailing spaces in Windows directory paths.
2026-07-26 18:16:36 +09:00
imSp4rky d60d8abf53 fix(ism): repair synthesized init and verify fragments decrypted
UNITY_MATRIX emitted 11 int32s instead of 9, oversizing mvhd/tkhd so pymp4 desynced and fTTML subtitles yielded no cues. Also rewrites PIFF senc to CENC so shaka can decrypt Smooth, labels HEV1 as hev1, and raises when a fragment survives decryption still carrying a sample-encryption box.
2026-07-25 15:55:31 -06:00
imSp4rky b7029c343e fix(ism): emit two stsd entries so sample_description_index 2 resolves
Smooth fragments set tfhd sample_description_index=2 against a server init carrying a second, identical sample entry. A lone entry left that dangling: mp4decrypt skipped every such fragment silently and demuxers stopped after the first, yielding a file that looked fine but was almost entirely encrypted.
2026-07-25 12:59:55 -06:00
imSp4rky 8cf1700684 fix(import): stop losing side-loaded subtitles and blaming the manifest URL
Only fall back to Title.language when the manifest labels nothing, and reject und as firmly as a missing tag. Exported direct-URL subtitles now replace the re-parsed set, and fetch failures stay distinct from parse failures.
2026-07-25 11:32:17 -06:00
imSp4rky 7c767bfd31 perf(pipeline): fix pool starvation, unlock vault writes, prep mux
Stop remounting adapters on shared sessions (restores dropped 429/5xx retries, ends 16-conn pool starvation, pool sized to 64); defer DRM vault write-backs off the global lock; add mux output_path groundwork for future parallelism.
2026-07-23 19:46:24 -06:00
imSp4rky ae9ffc7731 feat(drm): auto-repack tracks with a residual encrypted sample entry
Detect an encv/enca sample entry surviving decryption (moov-scoped 4CC scan) and force a repack so mkvmerge isn't first-FourCC-locked to a generic encrypted codec; warn if one persists after repacking.

Closes #142
2026-07-23 09:33:14 -06:00
imSp4rky c2f7fd87a3 feat(drm): detect revoked PlayReady devices from license responses
Add is_revocation()/REVOCATION_CODES and centrally raise DeviceRevoked when a license fault carries a device/cert revocation HRESULT, so any service surfaces it without its own decoding.
2026-07-23 08:48:51 -06:00
imSp4rky 51750a03b0 feat(drm): add PlayReady HRESULT decoder
Maps DRM_E_/DRM_S_ result codes to symbol + description so services can turn raw license HRESULTs (e.g. 0x8004C065) into readable errors.
2026-07-22 09:18:14 -06:00
imSp4rky 95a754fa6c fix: harden ClearKey/CENC decryption correctness 2026-07-19 10:49:45 -06:00
imSp4rky 446367f58d fix: skip decrypt on mislabeled-clear segments 2026-07-19 10:21:47 -06:00
imSp4rky 100857d9bb perf(dash): speed up segment collection and URL templating 2026-07-18 23:48:10 -06:00
imSp4rky b54aadea7a fix(downloader): honor Retry-After with cancel-safe backoff, add fault-injection bench and tests 2026-07-18 20:21:30 -06:00
imSp4rky 2c8a98f337 feat: original-language aware DUAL/MULTi tagging and new DUBBED variable
DUAL now requires the original language plus one dub (strict default);
dual_multi_mode: count restores the old pure-count behavior.
2026-07-18 11:02:02 -06:00
imSp4rky 2c8b6774af perf(downloader): unblock hedge-loser reads at teardown 2026-07-16 23:22:15 -06:00
imSp4rky a7f80ed476 refactor(downloader): extract shared byte-range split helper 2026-07-16 23:22:14 -06:00
imSp4rky a856d14d25 fix(api): reject backslash traversal in safe_cache_key 2026-07-16 23:22:14 -06:00
imSp4rky cd0b9e1709 fix(webvtt): don't shorten cues when splicing boundary duplicates 2026-07-16 23:22:14 -06:00
imSp4rky 09fe2e88b2 perf(api): append job history instead of rewriting the file 2026-07-16 23:22:13 -06:00
imSp4rky 036368b3c3 fix(api): keep sub_format "original" working through the API 2026-07-16 23:22:13 -06:00
imSp4rky 15a368f7e9 perf(api): skip progress re-parse when the file is unchanged 2026-07-16 23:22:12 -06:00
imSp4rky 0b93de94a2 fix(subtitle): convert ASS to SRT in the SDH-strip fallback 2026-07-16 23:22:12 -06:00
imSp4rky e50c005106 fix(dash): license ClearKeyCENC before decrypt 2026-07-16 23:22:11 -06:00
imSp4rky 2086be5ea6 chore(scripts): add attempt-1-only tail-slow fault to downloader bench 2026-07-16 23:22:04 -06:00
imSp4rky 8fecb49605 chore(scripts): harden bench suite report capture
Speedtest failures print an exit-code note instead of an empty section,
the report file writes progressively again (utf8 rewrite at the end),
and pytest-asyncio's py3.14 deprecation warning is filtered.
2026-07-16 18:07:55 -06:00
imSp4rky b2363e11c7 chore(scripts): expand bench suite for cross-machine runs
Adds machine info to the report header, multiprocess and strided-split
runs, hedge/slow-tail cases, the rnet path, and the leak regression
tests; ps1 reports save as utf8 and expected failures no longer render
as NativeCommandError blocks.
2026-07-16 18:01:13 -06:00
imSp4rky a11fbcd328 perf(downloader): stride multiprocess chunk assignment across children
Positionally clustered slow segments no longer pile into one child:
16.4s -> 8.4s at procs=4 with a 4-segment slow tail, uniform load
unchanged.
2026-07-16 17:44:07 -06:00
imSp4rky 7408162805 test(downloader): leak regression tests and bench sweep-safety hardening
Cross-platform WinError 32 regression tests; the bench now drains
leaked workers after a failed run and relays --fast-timeouts to
--procs children.
2026-07-16 17:44:06 -06:00
imSp4rky b841388028 fix(downloader): stop leaked racers holding segment handles after a failed batch
A leaked racer could retry and reopen its .!dev once dl.py cleared the
global cancel flag for the next track (WinError 32 at cleanup). A
batch-local abort event set at teardown makes its exit unconditional.
Also join terminated mp children and detect one that dies silently.
2026-07-16 17:43:55 -06:00
imSp4rky bdef22f335 test(scripts): size-verify senv-brand MP4s that ffprobe can't parse 2026-07-16 12:59:27 -06:00
imSp4rky 5de6fe9014 Merge branch 'dev' into feat/downloader-speed
# Conflicts:
#	unshackle/core/downloaders/requests.py
#	unshackle/core/manifests/dash.py
2026-07-16 12:47:59 -06:00