Commit Graph
957 Commits
Author SHA1 Message Date
imSp4rky 7c767bfd31 perf(pipeline): fix pool starvation, unlock vault writes, prep mux
Stop remounting adapters on shared sessions (restores dropped 429/5xx retries, ends 16-conn pool starvation, pool sized to 64); defer DRM vault write-backs off the global lock; add mux output_path groundwork for future parallelism.
2026-07-23 19:46:24 -06:00
imSp4rky ae9ffc7731 feat(drm): auto-repack tracks with a residual encrypted sample entry
Detect an encv/enca sample entry surviving decryption (moov-scoped 4CC scan) and force a repack so mkvmerge isn't first-FourCC-locked to a generic encrypted codec; warn if one persists after repacking.

Closes #142
2026-07-23 09:33:14 -06:00
imSp4rky c2f7fd87a3 feat(drm): detect revoked PlayReady devices from license responses
Add is_revocation()/REVOCATION_CODES and centrally raise DeviceRevoked when a license fault carries a device/cert revocation HRESULT, so any service surfaces it without its own decoding.
2026-07-23 08:48:51 -06:00
imSp4rky 51750a03b0 feat(drm): add PlayReady HRESULT decoder
Maps DRM_E_/DRM_S_ result codes to symbol + description so services can turn raw license HRESULTs (e.g. 0x8004C065) into readable errors.
2026-07-22 09:18:14 -06:00
imSp4rky 95a754fa6c fix: harden ClearKey/CENC decryption correctness 2026-07-19 10:49:45 -06:00
imSp4rky 446367f58d fix: skip decrypt on mislabeled-clear segments 2026-07-19 10:21:47 -06:00
imSp4rky 100857d9bb perf(dash): speed up segment collection and URL templating 2026-07-18 23:48:10 -06:00
imSp4rky b54aadea7a fix(downloader): honor Retry-After with cancel-safe backoff, add fault-injection bench and tests 2026-07-18 20:21:30 -06:00
imSp4rky 2c8a98f337 feat: original-language aware DUAL/MULTi tagging and new DUBBED variable
DUAL now requires the original language plus one dub (strict default);
dual_multi_mode: count restores the old pure-count behavior.
2026-07-18 11:02:02 -06:00
imSp4rky 2c8b6774af perf(downloader): unblock hedge-loser reads at teardown 2026-07-16 23:22:15 -06:00
imSp4rky a7f80ed476 refactor(downloader): extract shared byte-range split helper 2026-07-16 23:22:14 -06:00
imSp4rky a856d14d25 fix(api): reject backslash traversal in safe_cache_key 2026-07-16 23:22:14 -06:00
imSp4rky cd0b9e1709 fix(webvtt): don't shorten cues when splicing boundary duplicates 2026-07-16 23:22:14 -06:00
imSp4rky 09fe2e88b2 perf(api): append job history instead of rewriting the file 2026-07-16 23:22:13 -06:00
imSp4rky 036368b3c3 fix(api): keep sub_format "original" working through the API 2026-07-16 23:22:13 -06:00
imSp4rky 15a368f7e9 perf(api): skip progress re-parse when the file is unchanged 2026-07-16 23:22:12 -06:00
imSp4rky 0b93de94a2 fix(subtitle): convert ASS to SRT in the SDH-strip fallback 2026-07-16 23:22:12 -06:00
imSp4rky e50c005106 fix(dash): license ClearKeyCENC before decrypt 2026-07-16 23:22:11 -06:00
imSp4rky 2086be5ea6 chore(scripts): add attempt-1-only tail-slow fault to downloader bench 2026-07-16 23:22:04 -06:00
imSp4rky 8fecb49605 chore(scripts): harden bench suite report capture
Speedtest failures print an exit-code note instead of an empty section,
the report file writes progressively again (utf8 rewrite at the end),
and pytest-asyncio's py3.14 deprecation warning is filtered.
2026-07-16 18:07:55 -06:00
imSp4rky b2363e11c7 chore(scripts): expand bench suite for cross-machine runs
Adds machine info to the report header, multiprocess and strided-split
runs, hedge/slow-tail cases, the rnet path, and the leak regression
tests; ps1 reports save as utf8 and expected failures no longer render
as NativeCommandError blocks.
2026-07-16 18:01:13 -06:00
imSp4rky a11fbcd328 perf(downloader): stride multiprocess chunk assignment across children
Positionally clustered slow segments no longer pile into one child:
16.4s -> 8.4s at procs=4 with a 4-segment slow tail, uniform load
unchanged.
2026-07-16 17:44:07 -06:00
imSp4rky 7408162805 test(downloader): leak regression tests and bench sweep-safety hardening
Cross-platform WinError 32 regression tests; the bench now drains
leaked workers after a failed run and relays --fast-timeouts to
--procs children.
2026-07-16 17:44:06 -06:00
imSp4rky b841388028 fix(downloader): stop leaked racers holding segment handles after a failed batch
A leaked racer could retry and reopen its .!dev once dl.py cleared the
global cancel flag for the next track (WinError 32 at cleanup). A
batch-local abort event set at teardown makes its exit unconditional.
Also join terminated mp children and detect one that dies silently.
2026-07-16 17:43:55 -06:00
imSp4rky bdef22f335 test(scripts): size-verify senv-brand MP4s that ffprobe can't parse 2026-07-16 12:59:27 -06:00
imSp4rky 5de6fe9014 Merge branch 'dev' into feat/downloader-speed
# Conflicts:
#	unshackle/core/downloaders/requests.py
#	unshackle/core/manifests/dash.py
2026-07-16 12:47:59 -06:00
imSp4rky 7ca7a6066e chore: restore memory/ to .gitignore from dev 2026-07-16 12:44:26 -06:00
imSp4rky 7eee34014e perf(dash): collapse single-URL byte-range segments into one direct download
Also throttle hedge-median recompute to 0.5s, cache replace_fields format
patterns and short-circuit when no $ token, and stream HLS merge with
copyfileobj. Adds test_dash_collapse and live A/B tooling.
2026-07-16 12:42:16 -06:00
imSp4rky 094c598ef5 perf: little improvements here and there 2026-07-16 11:02:34 -06:00
imSp4rky 78d80eb68a test(scripts): add downloader benchmark and live manifest test tooling
bench_downloader.py sweeps the unified downloader over synthetic
loopback segments (latency/rate shaping, fault injection, hedge and
sweep-safety checks) or real URLs, with --procs and --no-read1 A/B
flags. live_manifest_test.py drives the full DASH/HLS/ISM pipeline
against 11 public streams on 8 CDNs; bench-urls/ holds reproducible
real-CDN URL lists.
2026-07-16 10:27:14 -06:00
imSp4rky e06cbe3dd8 perf(downloader): use per-arrival reads only while a segment is hedge-raced
Unconditional read1 for every segmented download paid a per-TLS-record
Python iteration on all segments and capped single-process throughput
at ~160 MB/s (253 MB/s with blocking full-chunk reads, measured on a
fast CDN). A segment can only be claimed once a hedge racer exists for
it, so fine-grained claimed() checks are needed only during an actual
race: download() now takes a racing predicate and workers pass the
hedged-set membership check.
2026-07-16 10:27:14 -06:00
imSp4rky f0e0c6d39e feat(downloader): fan large segment batches across worker processes
One CPython interpreter caps sustained segment throughput at ~160 MB/s;
spawned children each run the existing thread-pool path on a contiguous
chunk of the batch, reaching NIC line rate on fast CDNs. Opt-in via
--download-processes (default 1), engaged only for 24+ segment batches.
Sessions are rebuilt per child from a picklable spec; RnetSession falls
back to a single process when it has no impersonate preset.
2026-07-16 10:26:44 -06:00
imSp4rky 2adf8bad87 fix(downloader): validate 206 on byte-range segments and force identity encoding
Segments carrying their own Range (DASH SegmentBase, HLS EXT-X-BYTERANGE) now
fail the attempt when a server ignores the Range and returns 200, instead of
writing the whole parent resource as the segment and corrupting the merge. The
resume path no longer clobbers such a slice's Range, and retries rewrite the
slice whole. All segment GETs request Accept-Encoding: identity and the rnet
path mirrors the gzip guard, so transparent CDN (de)compression can't break the
Content-Length truncation check. Adds tests pinning these plus the adaptive
start-at-cap wiring and the small-segment tail-probe gate.
2026-07-16 00:00:26 -06:00
imSp4rky 4cf7a651a0 style: apply ruff format to branch files predating the format sweep 2026-07-16 00:00:19 -06:00
imSp4rky 686d996346 ci: give the build job its own uv cache key
stops the 'unable to reserve cache' annotation when lint and build
race to save the same key
2026-07-15 15:00:49 -06:00
imSp4rky 46cf72062d ci: pin setup-uv and dependency-review to full versions
neither publishes a moving major tag anymore, so v8/v5 don't resolve
2026-07-15 14:56:37 -06:00
imSp4rky 0a6f1cbe07 ci: bump actions to node 24 versions and serialize docs deploys
checkout v7, setup-uv v8, setup-python v6, dependency-review v5; a
concurrency group queues docs deploys so parallel runs can't race the push.
2026-07-15 14:54:05 -06:00
imSp4rky e31e60a418 style: run ruff format over the codebase 2026-07-15 14:49:27 -06:00
imSp4rky d853ef9600 ci: add multi-os test, lint, and wheel build workflows
Also fixes the sdist leaving wheels empty, pins subby to its git
source in wheel metadata, and fixes docstrings that broke mkdocs --strict.
2026-07-15 14:49:19 -06:00
imSp4rky 59a7c76b2f feat(ism): harden Smooth parsing and derive fps/codec from spec fields
Regex-based UrlPattern substitution (handles {Bitrate}/{start_time} spellings), AudioTag/FourCC codec fallback, Width/Height fallbacks, integer fragment times, live-manifest and missing-language guards. Extract fps from the HEVC SPS VUI (HEVC-only; AVC timing untrusted).
2026-07-14 21:24:14 -06:00
imSp4rky 2ccae1a0ee feat(cdm): stamp remote CDM DRM type at load for reliable detection 2026-07-14 17:32:00 -06:00
imSp4rky c6ecdb605c fix(services): register modules in sys.modules so repo-cloned services can self-import and remove old benchmark that should have been deleted 2026-07-14 13:46:29 -06:00
imSp4rky b8ec9e2498 fix(tracks): keep regional language variants distinct in exact selection
by_language now applies RFC 4647 Lookup in exact mode, preferring the string-equal tag so --exact-lang separates base/regional pairs (ar/ar-EG, en/en-US) instead of collapsing them, and dedupes by id to prevent the duplicate-track-id crash. Adds a language-selection regression test.
2026-07-14 09:17:22 -06:00
imSp4rky 1cf9e5b487 docs: correct DRM, muxing, and proxy references (+ case-insensitive proxy prefix fix) 2026-07-13 09:09:46 -06:00
imSp4rky 2e14e30f2f build(mypy): enable explicit_package_bases 2026-07-12 20:26:05 -06:00
imSp4rky b32b301dcc fix(proxies): show ExpressVPN country/location counts at startup 2026-07-12 20:25:56 -06:00
imSp4rky a450184ab8 test(remote): isolate job history writes from the real api_history.jsonl 2026-07-12 10:30:30 -06:00
imSp4rky bd679d2571 docs(changelog): regenerate with git-cliff 2026-07-12 09:52:35 -06:00
imSp4rky d7a2bb6b8e fix(subs): stop segmented WebVTT merge from stripping formatting
Merge cues at the string level instead of round-tripping through
pycaption, which dropped inline tags and cue settings. STYLE and
REGION blocks are kept, duplicate boundary cues splice correctly,
and a bad timing line now skips the cue instead of crashing.

Also rank pysubs2 above SubtitleEdit for ASS to TTML/VTT since
SubtitleEdit flattens inline styling there. Documented the measured
backend fidelity table and added regression tests for the merge.
2026-07-12 09:39:39 -06:00
imSp4rky 6ef24661c8 style(dl): restore isort imports and drop dead pass 2026-07-12 08:55:01 -06:00