Request 4 MiB sub-ranges once a host cuts a body, skip the session's own urllib3 Retry inside download(), fail the ranged path at once on a 4xx, and cap resumes at 50 of 1 MiB each. Adds a deny fault profile.
RFC 9110 clamps a range end past the resource to its last byte; the guard
rejected that valid response. The descriptive-audio filter also had no
emptiness check, so a title with only described audio failed with a
misleading missing-language error.
Submitting every segment up front convoyed on ThreadPoolExecutor's process-global lock, starving the event drain so per-track progress bars only advanced once the download had finished.
Renames 314 defs and 41 module/class-level constants. Keeps 11 names underscored where the stripped form collides in scope, plus 5.5.0 shims for the three symbols external service repos call.
RFC 9110 permits a server to ignore Range and answer 200 with the whole
resource. When the item's slice starts at byte 0 and spans exactly the
response body, that 200 is byte-identical to the 206, so keep it instead of
burning all five attempts and failing the track. Every other 200 (nonzero
start, span mismatch, open-ended, encoded) still fails the attempt to
protect the merge.
The rnet branch zeroed content_length only for gzip/deflate/br, so an
unknown coding (zstd, future tokens) kept a bogus length and, worse, the
drop-the-partial-instead-of-Range-resuming guard could never fire on the
rnet path because content_encoded stayed False. Use the same fail-closed
_is_content_encoded check the requests branch uses.
Any batch failure sets the process-global DOWNLOAD_CANCELLED, which
TokenBucket.consume waits on, so a test elsewhere could void every timing
assertion here; add the autouse clear fixture the other downloader test
modules already use. Also capture the debt sleep via a stub instead of
timing a real Event.wait, whose occasional spurious early wake tripped the
1.0s lower bound.
Spawned children re-import the module and start with no TokenBucket, so
--download-processes silently dropped a configured --speed-limit /
global_speed_limit for any 24+ segment batch. Fall back to the in-process
path while a limit is set (same gate as the ranged-parallel paths), log the
fallback, and document the interaction.
Catch the branch up with dev (45 commits) so testers can run the perf work
against current dev.
Conflict resolutions:
- downloaders/requests.py: keep the branch's racing-gated read1, adaptive
controller speed reporting and byte-range slice guards; take dev's
content-encoding checks (raw reads are skipped on an encoded body) and its
rolling SpeedWindow for the no-controller speed fallback. The single-URL
ranged-parallel gate now requires no speed limit and no caller Range.
- service.py / session.py: keep the branch's TimeoutHTTPAdapter and retry
config, raise POOL_MAX_SIZE to 64 so dev's pool-starvation fix survives
(pool_block=True stalls threads on a pool smaller than the in-flight count).
- docs: dev split configuration.md into per-area pages, so the branch's rnet
network option rows moved to reference/configuration/network.md.
Reading .raw skips urllib3's decoding, so brotli-served subtitles landed compressed and converted to empty files. Also drops a partial rather than Range-resuming an encoded body, since a Range indexes the encoded bytes.
Segments carrying their own Range (DASH SegmentBase, HLS EXT-X-BYTERANGE) now
fail the attempt when a server ignores the Range and returns 200, instead of
writing the whole parent resource as the segment and corrupting the merge. The
resume path no longer clobbers such a slice's Range, and retries rewrite the
slice whole. All segment GETs request Accept-Encoding: identity and the rnet
path mirrors the gzip guard, so transparent CDN (de)compression can't break the
Content-Length truncation check. Adds tests pinning these plus the adaptive
start-at-cap wiring and the small-segment tail-probe gate.
imSp4rky