Commit Graph
927 Commits
Author SHA1 Message Date
imSp4rky bf85c948b4 fix(dl): honour --no-proxy-download for attachments 2026-07-31 13:22:18 -06:00
imSp4rky 734df87bad feat(dl): add --speed-limit and serve global_speed_limit
Shared token bucket across all download threads; decimal suffixes (5M = 5.0 MB/s) or plain bytes/sec
2026-07-31 12:48:46 -06:00
imSp4rky e9a40a3af3 perf: speed up the code hash with sha1 and a plain file reader 2026-07-30 17:11:18 -06:00
imSp4rky c14277e6dc feat: identify the running code by source hash 2026-07-30 16:38:11 -06:00
imSp4rky 72b8ab7027 feat: show git commit hash alongside the version 2026-07-30 13:44:07 -06:00
imSp4rky 42d7fd8303 fix(import): scope the merge-back and detect ISM protection
Only merge back tracks from another manifest, so a drifted id no longer duplicates a rung or wipes the main manifest's own subtitles. track_is_encrypted now recognises an ISM ProtectionHeader.
2026-07-29 10:48:34 -06:00
sp4rk.y cebfd7bc80 Merge pull request #145 from ApeDevOne/dev
fix(import): restore missing DASH manifest data and HEVC tracks
2026-07-29 10:13:09 -06:00
imSp4rky bcf0815cf4 fix(tracks): size the no-proxy download session's connection pool 2026-07-29 08:44:27 -06:00
RocketBoy bece8fb746 Implement manifest data resolution for DASH/ISM tracks along with Import Fix for json exports with no "manifest_url"
Added a new function to resolve import manifest data for DASH/ISM tracks, enhancing the handling of track data population from export dictionaries.
2026-07-29 18:23:02 +05:30
imSp4rky f7a38732f1 feat(dl): extend --select-titles to movie listings 2026-07-28 21:22:14 -06:00
imSp4rky a24c9b92da fix(tracks): ignore leftover PIFF uuid when verifying decryption
A decrypter detaches only the atom it consumed, so content shipping both boxes keeps the uuid through a good decrypt and failed on mp4decrypt.
2026-07-27 13:11:13 -06:00
imSp4rky 3c0fef75a7 docs(core): drop service tags from framework comments 2026-07-27 12:15:14 -06:00
imSp4rky 40ae213504 docs: cleanup and correct documentation 2026-07-26 21:21:01 -06:00
imSp4rky 8bacb247ee fix(lang): surface orig in errors and flag the original audio track over the API
Error messages named the resolved language for a user who only ever defaulted to orig. Audio tracks now carry is_original, resolved through Tracks.by_language so it agrees with the downloader.
2026-07-26 18:22:46 -06:00
imSp4rky b74b64825e fix(templates): use one separator style across a whole folder path
Detect the dot/space style from the full folder template instead of per segment, so `{artist}/{album} ({year})` no longer yields `Some.Artist/Some Album (2025)`. Also drops empty `[]` groups like the existing `()` handling and removes redundant cleanup passes.
2026-07-26 09:48:58 -06:00
sp4rk.y 4c54c9ec1b Merge pull request #144 from CodeName393/fix-template-formatting
fix(core): strip trailing spaces and dots in template formatting to prevent WinError 3
2026-07-26 09:28:56 -06:00
CodeName393 41106b0d24 fix(core): strip trailing spaces and dots in template formatting to prevent WinError 3
- Clean up redundant spaces and separators when conditional tags (e.g. {year?}) are omitted.
- Strip leading and trailing spaces and dots in sanitize_filename and TemplateFormatter.
- Prevent FileNotFoundError [WinError 3] caused by trailing spaces in Windows directory paths.
2026-07-26 18:16:36 +09:00
imSp4rky d60d8abf53 fix(ism): repair synthesized init and verify fragments decrypted
UNITY_MATRIX emitted 11 int32s instead of 9, oversizing mvhd/tkhd so pymp4 desynced and fTTML subtitles yielded no cues. Also rewrites PIFF senc to CENC so shaka can decrypt Smooth, labels HEV1 as hev1, and raises when a fragment survives decryption still carrying a sample-encryption box.
2026-07-25 15:55:31 -06:00
imSp4rky b7029c343e fix(ism): emit two stsd entries so sample_description_index 2 resolves
Smooth fragments set tfhd sample_description_index=2 against a server init carrying a second, identical sample entry. A lone entry left that dangling: mp4decrypt skipped every such fragment silently and demuxers stopped after the first, yielding a file that looked fine but was almost entirely encrypted.
2026-07-25 12:59:55 -06:00
imSp4rky 8cf1700684 fix(import): stop losing side-loaded subtitles and blaming the manifest URL
Only fall back to Title.language when the manifest labels nothing, and reject und as firmly as a missing tag. Exported direct-URL subtitles now replace the re-parsed set, and fetch failures stay distinct from parse failures.
2026-07-25 11:32:17 -06:00
imSp4rky 7c767bfd31 perf(pipeline): fix pool starvation, unlock vault writes, prep mux
Stop remounting adapters on shared sessions (restores dropped 429/5xx retries, ends 16-conn pool starvation, pool sized to 64); defer DRM vault write-backs off the global lock; add mux output_path groundwork for future parallelism.
2026-07-23 19:46:24 -06:00
imSp4rky ae9ffc7731 feat(drm): auto-repack tracks with a residual encrypted sample entry
Detect an encv/enca sample entry surviving decryption (moov-scoped 4CC scan) and force a repack so mkvmerge isn't first-FourCC-locked to a generic encrypted codec; warn if one persists after repacking.

Closes #142
2026-07-23 09:33:14 -06:00
imSp4rky c2f7fd87a3 feat(drm): detect revoked PlayReady devices from license responses
Add is_revocation()/REVOCATION_CODES and centrally raise DeviceRevoked when a license fault carries a device/cert revocation HRESULT, so any service surfaces it without its own decoding.
2026-07-23 08:48:51 -06:00
imSp4rky 51750a03b0 feat(drm): add PlayReady HRESULT decoder
Maps DRM_E_/DRM_S_ result codes to symbol + description so services can turn raw license HRESULTs (e.g. 0x8004C065) into readable errors.
2026-07-22 09:18:14 -06:00
imSp4rky 95a754fa6c fix: harden ClearKey/CENC decryption correctness 2026-07-19 10:49:45 -06:00
imSp4rky 446367f58d fix: skip decrypt on mislabeled-clear segments 2026-07-19 10:21:47 -06:00
imSp4rky 100857d9bb perf(dash): speed up segment collection and URL templating 2026-07-18 23:48:10 -06:00
imSp4rky b54aadea7a fix(downloader): honor Retry-After with cancel-safe backoff, add fault-injection bench and tests 2026-07-18 20:21:30 -06:00
imSp4rky 2c8a98f337 feat: original-language aware DUAL/MULTi tagging and new DUBBED variable
DUAL now requires the original language plus one dub (strict default);
dual_multi_mode: count restores the old pure-count behavior.
2026-07-18 11:02:02 -06:00
imSp4rky 2c8b6774af perf(downloader): unblock hedge-loser reads at teardown 2026-07-16 23:22:15 -06:00
imSp4rky a7f80ed476 refactor(downloader): extract shared byte-range split helper 2026-07-16 23:22:14 -06:00
imSp4rky a856d14d25 fix(api): reject backslash traversal in safe_cache_key 2026-07-16 23:22:14 -06:00
imSp4rky cd0b9e1709 fix(webvtt): don't shorten cues when splicing boundary duplicates 2026-07-16 23:22:14 -06:00
imSp4rky 09fe2e88b2 perf(api): append job history instead of rewriting the file 2026-07-16 23:22:13 -06:00
imSp4rky 036368b3c3 fix(api): keep sub_format "original" working through the API 2026-07-16 23:22:13 -06:00
imSp4rky 15a368f7e9 perf(api): skip progress re-parse when the file is unchanged 2026-07-16 23:22:12 -06:00
imSp4rky 0b93de94a2 fix(subtitle): convert ASS to SRT in the SDH-strip fallback 2026-07-16 23:22:12 -06:00
imSp4rky e50c005106 fix(dash): license ClearKeyCENC before decrypt 2026-07-16 23:22:11 -06:00
imSp4rky 2086be5ea6 chore(scripts): add attempt-1-only tail-slow fault to downloader bench 2026-07-16 23:22:04 -06:00
imSp4rky 8fecb49605 chore(scripts): harden bench suite report capture
Speedtest failures print an exit-code note instead of an empty section,
the report file writes progressively again (utf8 rewrite at the end),
and pytest-asyncio's py3.14 deprecation warning is filtered.
2026-07-16 18:07:55 -06:00
imSp4rky b2363e11c7 chore(scripts): expand bench suite for cross-machine runs
Adds machine info to the report header, multiprocess and strided-split
runs, hedge/slow-tail cases, the rnet path, and the leak regression
tests; ps1 reports save as utf8 and expected failures no longer render
as NativeCommandError blocks.
2026-07-16 18:01:13 -06:00
imSp4rky a11fbcd328 perf(downloader): stride multiprocess chunk assignment across children
Positionally clustered slow segments no longer pile into one child:
16.4s -> 8.4s at procs=4 with a 4-segment slow tail, uniform load
unchanged.
2026-07-16 17:44:07 -06:00
imSp4rky 7408162805 test(downloader): leak regression tests and bench sweep-safety hardening
Cross-platform WinError 32 regression tests; the bench now drains
leaked workers after a failed run and relays --fast-timeouts to
--procs children.
2026-07-16 17:44:06 -06:00
imSp4rky b841388028 fix(downloader): stop leaked racers holding segment handles after a failed batch
A leaked racer could retry and reopen its .!dev once dl.py cleared the
global cancel flag for the next track (WinError 32 at cleanup). A
batch-local abort event set at teardown makes its exit unconditional.
Also join terminated mp children and detect one that dies silently.
2026-07-16 17:43:55 -06:00
imSp4rky bdef22f335 test(scripts): size-verify senv-brand MP4s that ffprobe can't parse 2026-07-16 12:59:27 -06:00
imSp4rky 5de6fe9014 Merge branch 'dev' into feat/downloader-speed
# Conflicts:
#	unshackle/core/downloaders/requests.py
#	unshackle/core/manifests/dash.py
2026-07-16 12:47:59 -06:00
imSp4rky 7ca7a6066e chore: restore memory/ to .gitignore from dev 2026-07-16 12:44:26 -06:00
imSp4rky 7eee34014e perf(dash): collapse single-URL byte-range segments into one direct download
Also throttle hedge-median recompute to 0.5s, cache replace_fields format
patterns and short-circuit when no $ token, and stream HLS merge with
copyfileobj. Adds test_dash_collapse and live A/B tooling.
2026-07-16 12:42:16 -06:00
imSp4rky 094c598ef5 perf: little improvements here and there 2026-07-16 11:02:34 -06:00
imSp4rky 78d80eb68a test(scripts): add downloader benchmark and live manifest test tooling
bench_downloader.py sweeps the unified downloader over synthetic
loopback segments (latency/rate shaping, fault injection, hedge and
sweep-safety checks) or real URLs, with --procs and --no-read1 A/B
flags. live_manifest_test.py drives the full DASH/HLS/ISM pipeline
against 11 public streams on 8 CDNs; bench-urls/ holds reproducible
real-CDN URL lists.
2026-07-16 10:27:14 -06:00