test(remote): isolate job history writes from the real api_history.jsonl

Merge cues at the string level instead of round-tripping through
pycaption, which dropped inline tags and cue settings. STYLE and
REGION blocks are kept, duplicate boundary cues splice correctly,
and a bad timing line now skips the cue instead of crashing.

Also rank pysubs2 above SubtitleEdit for ASS to TTML/VTT since
SubtitleEdit flattens inline styling there. Documented the measured
backend fidelity table and added regression tests for the merge.

.gitignore

@@ -37,6 +37,7 @@ logs/
 services/
 /.[^/]*/
 !/.github/
+memory/
 
 # Byte-compiled / optimized / DLL files
 __pycache__/

CHANGELOG.md

@@ -6,7 +6,55 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 This changelog is automatically generated using [git-cliff](https://git-cliff.org).
 
-## [5.3.0] - 2026-06-30
+## [Unreleased]
+
+### Features
+
+- *api*: Report per-episode and per-track progress on download jobs
+- *api*: Add job retry/priority/clear, history, config and maintenance endpoints
+- *api*: Expose choice values and multiple flag in service cli_params
+- *core*: Add debug logging for rnet requests
+- *tracks*: Add subtitle.type_priority config for variant ordering
+- *proxies*: [**breaking**] Rewrite ExpressVPN provider to Android device login
+- *dl*: Animate merge/decrypt/mux stages with gradient pulse bars
+- *tags*: Add imdb_api_enabled toggle, off by default
+- *tags*: Add OMDb metadata provider with omdb_api_key config
+
+### Bug Fixes
+
+- *api*: Use PBKDF2 instead of SHA256 for cache-namespace hashing
+- *api*: Pass track-selection params to service context
+- *remote*: Add missing get_clearkey_license to RemoteService
+- *remote*: Harden serve API access control and input handling
+- *core*: Decode subprocess output as UTF-8 with replacement
+- *tracks*: Treat ccextractor signal-crash as non-fatal
+- *downloader*: Release hedge loser file handles before merge sweep
+- *dl*: Restore real progress bars where progression is trackable
+- *dl*: Select only best descriptive audio track per language
+- *dl*: [**breaking**] Make audio/video track selection consistent and crash-safe
+- *tags*: Point IMDbApi provider at api.tiffara.com (IMDxAPI)
+- *gitignore*: Add memory directory to .gitignore
+- *dl*: Prevent missing tracks error when excluding track types
+- *subs*: Stop segmented WebVTT merge from stripping formatting
+
+### Documentation
+
+- Restructure documentation into MkDocs Material site
+- Expand config reference and genericize service names
+
+### Performance Improvements
+
+- *downloader*: Cut per-segment overhead, bound stalled connections
+
+### Changes
+
+- *core*: [**breaking**] Bundle download_track args into DownloadContext
+
+### Builds
+
+- *deps*: Cap pycaption below 2.2.27
+
+## [5.3.0] - 2026-07-01
 
 ### Features
 
@@ -770,6 +818,8 @@ This changelog is automatically generated using [git-cliff](https://git-cliff.or
 - Reorganize Planned Features section in README for clarity
 - Improve track selection logic in dl.py
 
+[unreleased]: https://github.com/unshackle-dl/unshackle/compare/5.3.0..HEAD
+[5.3.0]: https://github.com/unshackle-dl/unshackle/compare/5.2.0..5.3.0
 [5.2.0]: https://github.com/unshackle-dl/unshackle/compare/5.1.0..5.2.0
 [5.1.0]: https://github.com/unshackle-dl/unshackle/compare/5.0.0..5.1.0
 [5.0.0]: https://github.com/unshackle-dl/unshackle/compare/4.0.0..5.0.0

docs/guide/subtitles.md

@@ -226,6 +226,47 @@ With `conversion_method: auto`, unshackle ranks these automatically per conversi
 Setting it to a specific value pins that backend as the first choice, falling back to
 others only if the pin cannot handle the pair.
 
+#### What each backend actually preserves
+
+The table below comes from round-tripping a subtitle that carries italics, bold,
+underline, positioning, and colour through every backend. Each cell lists the
+styling that survives; `—` means the backend cannot handle that pair.
+
+| Conversion | `subtitleedit` | `pysubs2` | `subby` | `pycaption` |
+| --- | --- | --- | --- | --- |
+| WebVTT → SRT | italic, bold, underline, position | italic, underline | italic, position | *(strips all)* |
+| WebVTT → ASS | italic, bold, underline, position | italic, bold, underline | — | — |
+| WebVTT → TTML | italic, bold, underline, position | italic, bold, underline | position | *(strips all)* |
+| ASS/SSA → SRT | all (+ colour) | italic, underline | — | — |
+| ASS/SSA → TTML | position only | italic, bold, underline | — | — |
+| ASS/SSA → VTT | position only | italic, underline | — | — |
+
+Reading the table:
+
+- **Keep the original**: leaving `--sub-format` unset (or set to `original`) never
+  round-trips the file, so every style survives. Only convert when your player cannot
+  read the source format.
+- **`subtitleedit`** (SubtitleEdit / `seconv`): the best choice for anything → SRT. It
+  is the only backend that carries colour, and it embeds `{\an8}` tags to preserve
+  positioning. When writing TTML or WebVTT from ASS it flattens inline styling to plain
+  text and keeps only positioning, so avoid it there.
+- **`pysubs2`**: keeps inline italic/underline on every pair, and bold except when
+  writing SRT or WebVTT. It never carries positioning or colour. SSA/ASS is its native
+  model, which makes it the best pick for SSA↔ASS.
+- **`subby`**: reads only WebVTT/fVTT/SAMI (never ASS) and is tuned for → SRT, where it
+  uniquely converts WebVTT cue settings into `{\an8}` positioning. Its
+  `CommonIssuesFixer` may also drop near-duplicate cues.
+- **`pycaption`**: strips all styling; last-resort fallback only.
+
+!!! tip "What `auto` picks"
+    `auto` prefers `subtitleedit` first when it is installed, then falls back to `subby`
+    for → SRT and `pysubs2` otherwise. The one exception comes from the table above:
+    for ASS/SSA → TTML or WebVTT it picks `pysubs2` even when SubtitleEdit is
+    installed, since SubtitleEdit flattens inline styling on those pairs. Most installs
+    do not ship SubtitleEdit, so in practice `auto` means `subby` (→ SRT) or `pysubs2`.
+    Install [SubtitleEdit / `seconv`](#subtitle-configuration) if you need colour or the
+    highest → SRT fidelity.
+
 !!! note "Config wins over a service's preference"
     A service may set a `preferred_conversion_method` on its own tracks (for example when
     it ships subtitles that a particular backend handles best). An explicit

docs/reference/configuration.md

@@ -748,8 +748,8 @@ All default to an empty string and enable optional metadata/geolocation features
 
 | Key | Type | Default | Description |
 |-----|------|---------|-------------|
-| `imdb_api_enabled` | bool | `false` | Use the free imdbapi.dev metadata provider, which needs no API key. Off by default since the site has been unreliable. |
-| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a more reliable alternative to imdbapi.dev. Free keys are available on the OMDb site. |
+| `imdb_api_enabled` | bool | `false` | Use the free IMDxAPI (api.tiffara.com) metadata provider, which needs no API key. Off by default since the site has been unreliable. |
+| `omdb_api_key` | str | `""` | [OMDb](https://www.omdbapi.com/) API key for IMDb metadata lookups; a more reliable alternative to IMDxAPI. Free keys are available on the OMDb site. |
 | `tmdb_api_key` | str | `""` | TMDB API key for metadata enrichment and external-ID tags. |
 | `simkl_client_id` | str | `""` | SIMKL client ID for metadata lookups; an alternative/fallback source to TMDB. |
 | `decrypt_labs_api_key` | str | `""` | Global Decrypt Labs API key (used by remote CDM / vault). |

tests/remote/conftest.py

@@ -86,6 +86,14 @@ def _wait_for_health(url: str, timeout: float = 60.0) -> bool:
     return False
 
 
+@pytest.fixture(autouse=True)
+def _isolated_job_history(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+    """Redirect job history writes to tmp_path so tests don't append to the server's api_history.jsonl."""
+    from unshackle.core.api import download_manager
+
+    monkeypatch.setattr(download_manager, "_history_path", lambda: tmp_path / "api_history.jsonl")
+
+
 @pytest.fixture(scope="session")
 def secret_key(request: pytest.FixtureRequest) -> str:
     return str(request.config.getoption("--secret-key"))

tests/tracks/test_subtitle_convert.py

@@ -87,6 +87,18 @@ def test_subtitleedit_ranks_first_when_available(monkeypatch):
     assert chain[0] == "subtitleedit"
 
 
+def test_styled_ass_to_ttml_vtt_prefers_pysubs2_over_subtitleedit(monkeypatch):
+    # Measured: SE flattens ASS inline styling to TTML/VTT; pysubs2 keeps i/b/u.
+    monkeypatch.setattr(binaries, "SubtitleEdit", "/usr/bin/seconv")
+    for target in (Codec.TimedTextMarkupLang, Codec.WebVTT):
+        chain = [b.name for b in sc.resolve_backends(Codec.SubStationAlphav4, target)]
+        assert chain[0] == "pysubs2", f"expected pysubs2 first for ASS->{target.name}, got {chain}"
+        assert "subtitleedit" in chain  # still a fallback
+    # ASS->SRT keeps SE first (only backend that keeps colour there).
+    chain = [b.name for b in sc.resolve_backends(Codec.SubStationAlphav4, Codec.SubRip)]
+    assert chain[0] == "subtitleedit"
+
+
 def test_pin_then_fallback_orders_pin_first():
     chain = [b.name for b in sc.resolve_backends(Codec.WebVTT, Codec.SubRip, pin="pysubs2")]
     assert chain[0] == "pysubs2"

tests/tracks/test_webvtt_merge.py

@@ -0,0 +1,113 @@
+"""Regression tests for segmented-WebVTT merging (``core/utils/webvtt.py``).
+
+The merge used to round-trip cues through pycaption's reader/writer (and a pysubs2
+pre-normalization step), both of which silently dropped inline formatting despite
+``preserve_formatting=True``. It is now a pure string-level splice: cue
+timing lines, settings, and payload are kept verbatim, so span tags
+(``<i>``/``<u>``/``<b>``), ASS-style overrides (``{\\an8}``), and entities all
+survive, while duplicate headers collapse and boundary-duplicate cues dedupe.
+"""
+
+from __future__ import annotations
+
+from unshackle.core.utils.webvtt import merge_segmented_webvtt
+
+# Two segments, each a full WebVTT document with its own X-TIMESTAMP-MAP header,
+# as they arrive concatenated from a DASH/HLS subtitle stream. The first cue carries
+# span tags, an {\an8} override, a cue setting, and a literal &lt; entity.
+SEGMENTED_VTT = (
+    "WEBVTT\n"
+    "X-TIMESTAMP-MAP=MPEGTS:900000,LOCAL:00:00:00.000\n\n"
+    "00:00:01.000 --> 00:00:03.000 line:10% align:center\n"
+    "<i>Hello</i> {\\an8}world <u>und</u> <b>bold</b> 5 &lt; 6\n\n"
+    "WEBVTT\n"
+    "X-TIMESTAMP-MAP=MPEGTS:1800000,LOCAL:00:00:00.000\n\n"
+    "00:00:04.000 --> 00:00:06.000\n"
+    "<i>Second</i> cue\n"
+)
+
+
+def test_merge_preserves_inline_formatting():
+    out = merge_segmented_webvtt(SEGMENTED_VTT, segment_durations=[900000, 900000], timescale=1)
+    for tag in ("<i>", "</i>", "<u>", "</u>", "<b>", "</b>", "{\\an8}"):
+        assert tag in out, f"{tag!r} was stripped from merged output:\n{out}"
+
+
+def test_merge_preserves_cue_settings_and_entities():
+    out = merge_segmented_webvtt(SEGMENTED_VTT, segment_durations=[900000, 900000], timescale=1)
+    assert "line:10% align:center" in out  # cue positioning survives
+    assert "&lt;" in out  # a literal entity is not turned into a stray tag
+
+
+def test_merge_collapses_headers_into_single_document():
+    out = merge_segmented_webvtt(SEGMENTED_VTT, segment_durations=[900000, 900000], timescale=1)
+    assert out.count("WEBVTT") == 1
+    assert "X-TIMESTAMP-MAP" not in out
+
+
+def test_merge_dedupes_boundary_duplicate_cues():
+    # A cue that spans a segment boundary is emitted in both segments; the merge
+    # should collapse the adjacent identical pair into one.
+    dup = SEGMENTED_VTT + (
+        "\nWEBVTT\n"
+        "X-TIMESTAMP-MAP=MPEGTS:2700000,LOCAL:00:00:00.000\n\n"
+        "00:00:04.000 --> 00:00:06.000\n"
+        "<i>Second</i> cue\n"
+    )
+    out = merge_segmented_webvtt(dup, segment_durations=[900000, 900000, 900000], timescale=1)
+    assert out.count("<i>Second</i> cue") == 1, f"boundary duplicate not deduped:\n{out}"
+
+
+def test_merge_keeps_timing_lines_verbatim():
+    # String-level merge must not re-render timestamps (pycaption shortened
+    # 00:00:01.000 -> 00:01.000) and must keep the settings tail untouched.
+    out = merge_segmented_webvtt(SEGMENTED_VTT, segment_durations=[900000, 900000], timescale=1)
+    assert "00:00:01.000 --> 00:00:03.000 line:10% align:center" in out
+    assert "00:00:04.000 --> 00:00:06.000" in out
+
+
+def test_merge_splices_triple_duplicate_into_one():
+    # A cue spanning two boundaries appears in three consecutive segments; the
+    # kept cue must be extended each time (latent bug: extending the dropped one).
+    vtt = (
+        "WEBVTT\n\n00:00:01.000 --> 00:00:02.000\nSame cue\n\n"
+        "WEBVTT\n\n00:00:02.000 --> 00:00:03.000\nSame cue\n\n"
+        "WEBVTT\n\n00:00:03.000 --> 00:00:04.000\nSame cue\n"
+    )
+    out = merge_segmented_webvtt(vtt)
+    assert out.count("Same cue") == 1
+    assert "00:00:01.000 --> 00:00:04.000" in out
+
+
+def test_merge_drops_cue_ids_and_notes_keeps_style():
+    vtt = (
+        "WEBVTT\n\n"
+        "STYLE\n::cue { color: red }\n\n"
+        "NOTE a comment\n\n"
+        "some-cue-id-42\n00:00:01.000 --> 00:00:02.000\nHello\n"
+    )
+    out = merge_segmented_webvtt(vtt)
+    assert "Hello" in out
+    assert "some-cue-id-42" not in out
+    assert "NOTE" not in out
+    assert "STYLE\n::cue { color: red }" in out  # kept; players without STYLE support ignore it
+
+
+def test_merge_dedupes_repeated_style_blocks_before_cues():
+    # Every segment repeats the same STYLE block; keep exactly one, before the first cue.
+    style = "WEBVTT\n\nSTYLE\n::cue { color: red }\n\n"
+    vtt = style + "00:00:01.000 --> 00:00:02.000\nOne\n\n" + style + "00:00:03.000 --> 00:00:04.000\nTwo\n"
+    out = merge_segmented_webvtt(vtt)
+    assert out.count("STYLE") == 1
+    assert out.index("STYLE") < out.index("-->")
+
+
+def test_merge_skips_malformed_timing_line():
+    vtt = (
+        "WEBVTT\n\n"
+        "garbage --> not-a-time\nBroken\n\n"
+        "00:00:01.000 --> 00:00:02.000\nGood\n"
+    )
+    out = merge_segmented_webvtt(vtt)
+    assert "Good" in out
+    assert "Broken" not in out

unshackle/commands/dl.py

@@ -842,25 +842,41 @@ class dl:
                             if name == "shaka_packager":
                                 r = subprocess.run(
                                     [str(binary), "--version"],
-                                    capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=5,
+                                    capture_output=True,
+                                    text=True,
+                                    encoding="utf-8",
+                                    errors="replace",
+                                    timeout=5,
                                 )
                                 version = (r.stdout or r.stderr or "").strip()
                             elif name in ("ffmpeg", "ffprobe"):
                                 r = subprocess.run(
                                     [str(binary), "-version"],
-                                    capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=5,
+                                    capture_output=True,
+                                    text=True,
+                                    encoding="utf-8",
+                                    errors="replace",
+                                    timeout=5,
                                 )
                                 version = (r.stdout or "").split("\n")[0].strip()
                             elif name == "mkvmerge":
                                 r = subprocess.run(
                                     [str(binary), "--version"],
-                                    capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=5,
+                                    capture_output=True,
+                                    text=True,
+                                    encoding="utf-8",
+                                    errors="replace",
+                                    timeout=5,
                                 )
                                 version = (r.stdout or "").strip()
                             elif name == "mp4decrypt":
                                 r = subprocess.run(
                                     [str(binary)],
-                                    capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=5,
+                                    capture_output=True,
+                                    text=True,
+                                    encoding="utf-8",
+                                    errors="replace",
+                                    timeout=5,
                                 )
                                 output = (r.stdout or "") + (r.stderr or "")
                                 lines = [line.strip() for line in output.split("\n") if line.strip()]
@@ -1303,7 +1319,9 @@ class dl:
         else:
             vaults_only = not cdm_only
 
-        config.decryption = resolve_decryption(config.decryption_map, config.decryption, service.__class__.__name__.upper())
+        config.decryption = resolve_decryption(
+            config.decryption_map, config.decryption, service.__class__.__name__.upper()
+        )
 
         log_event(
             "drm_mode_config",
@@ -2134,33 +2152,6 @@ class dl:
                 events.subscribe(events.Types.TRACK_REPACKED, service.on_track_repacked)
                 events.subscribe(events.Types.TRACK_MULTIPLEX, service.on_track_multiplex)
 
-            if hasattr(service, "NO_SUBTITLES") and service.NO_SUBTITLES:
-                console.log("Skipping subtitles - service does not support subtitle downloads")
-                no_subs = True
-                s_lang = None
-                title.tracks.subtitles = []
-            elif no_subs:
-                console.log("Skipped subtitles as --no-subs was used...")
-                s_lang = None
-                title.tracks.subtitles = []
-
-            if no_video:
-                console.log("Skipped video as --no-video was used...")
-                v_lang = None
-                title.tracks.videos = []
-
-            if no_audio:
-                console.log("Skipped audio as --no-audio was used...")
-                a_lang = None
-                title.tracks.audio = []
-
-            if no_chapters:
-                console.log("Skipped chapters as --no-chapters was used...")
-                title.tracks.chapters = []
-
-            if no_proxy_download and any(service.session.proxies.values()):
-                console.log("Bypassing proxy for downloads as --no-proxy-download was used...")
-
             tracks_label = "Getting Remote Tracks..." if self.is_remote else "Getting Tracks..."
             with console.status(tracks_label, spinner="dots"):
                 try:
@@ -2292,10 +2283,58 @@ class dl:
                 console.print(Padding(Panel(available_tracks, title="Available Tracks"), (0, 5)))
                 continue
 
+            # Determine which tracks to keep
+            keep_videos = True
+            keep_audio = True
+            keep_subtitles = True
+            keep_chapters = True
+
+            if video_only or audio_only or subs_only or chapters_only:
+                keep_videos = video_only
+                keep_audio = audio_only
+                keep_subtitles = subs_only
+                keep_chapters = chapters_only
+
+            if hasattr(service, "NO_SUBTITLES") and service.NO_SUBTITLES:
+                console.log("Skipping subtitles - service does not support subtitle downloads")
+                keep_subtitles = False
+                s_lang = None
+            elif no_subs:
+                console.log("Skipped subtitles as --no-subs was used...")
+                keep_subtitles = False
+                s_lang = None
+
+            if no_video:
+                console.log("Skipped video as --no-video was used...")
+                keep_videos = False
+                v_lang = None
+
+            if no_audio:
+                console.log("Skipped audio as --no-audio was used...")
+                keep_audio = False
+                a_lang = None
+
+            if no_chapters:
+                console.log("Skipped chapters as --no-chapters was used...")
+                keep_chapters = False
+
+            if no_proxy_download and any(service.session.proxies.values()):
+                console.log("Bypassing proxy for downloads as --no-proxy-download was used...")
+
+            # Clear unwanted tracks
+            if not keep_videos:
+                title.tracks.videos = []
+            if not keep_audio:
+                title.tracks.audio = []
+            if not keep_subtitles:
+                title.tracks.subtitles = []
+            if not keep_chapters:
+                title.tracks.chapters = []
+
             with console.status("Selecting tracks...", spinner="dots"):
                 if isinstance(title, (Movie, Episode)):
                     # filter video tracks
-                    if vcodec:
+                    if keep_videos and vcodec:
                         title.tracks.select_video(lambda x: x.codec in vcodec)
                         missing_codecs = [c for c in vcodec if not any(x.codec == c for x in title.tracks.videos)]
                         for codec in missing_codecs:
@@ -2304,7 +2343,7 @@ class dl:
                             self.log.error(f"There's no {', '.join(c.name for c in vcodec)} Video Track...")
                             sys.exit(1)
 
-                    if range_:
+                    if keep_videos and range_:
                         # Special handling for HYBRID - don't filter, keep all HDR10 and DV tracks
                         if Video.Range.HYBRID not in range_:
                             title.tracks.select_video(lambda x: x.range in range_)
@@ -2315,7 +2354,7 @@ class dl:
                                 self.log.error(f"There's no {', '.join(r.name for r in range_)} Video Track...")
                                 sys.exit(1)
 
-                    if vbitrate:
+                    if keep_videos and vbitrate:
                         if any(r == Video.Range.HYBRID for r in range_):
                             # In HYBRID mode, only apply bitrate filter to non-DV tracks
                             # DV tracks are kept regardless since they're only used for RPU metadata
@@ -2331,7 +2370,7 @@ class dl:
                                 self.log.error(f"There's no {vbitrate}kbps Video Track...")
                                 sys.exit(1)
 
-                    if vbitrate_min is not None and vbitrate_max is not None:
+                    if keep_videos and vbitrate_min is not None and vbitrate_max is not None:
                         title.tracks.select_video(
                             lambda x: x.bitrate and vbitrate_min <= x.bitrate // 1000 <= vbitrate_max
                         )
@@ -2344,7 +2383,7 @@ class dl:
                     video_multi_lang = (
                         "best" in effective_video_lang or "all" in effective_video_lang or len(video_languages) > 1
                     )
-                    if video_languages and "all" not in video_languages:
+                    if keep_videos and video_languages and "all" not in video_languages:
                         processed_video_lang = []
                         for language in video_languages:
                             if language == "orig":
@@ -2367,9 +2406,7 @@ class dl:
                         if missing_v_langs:
                             missing_str = ", ".join(missing_v_langs)
                             if best_available:
-                                remaining_v_langs = [
-                                    tok for tok in processed_video_lang if tok not in missing_v_langs
-                                ]
+                                remaining_v_langs = [tok for tok in processed_video_lang if tok not in missing_v_langs]
                                 if remaining_v_langs:
                                     self.log.warning(
                                         f"{missing_str} not found in video tracks, "
@@ -2377,9 +2414,7 @@ class dl:
                                     )
                                     processed_video_lang = remaining_v_langs
                                 else:
-                                    self.log.error(
-                                        f"{missing_str} not found in video tracks and no fallback available"
-                                    )
+                                    self.log.error(f"{missing_str} not found in video tracks and no fallback available")
                                     sys.exit(1)
                             else:
                                 self.log.error(missing_str + " not found in video tracks")
@@ -2394,7 +2429,7 @@ class dl:
                     has_hybrid = any(r == Video.Range.HYBRID for r in range_)
                     non_hybrid_ranges = [r for r in range_ if r != Video.Range.HYBRID]
                     effective_quality = quality
-                    if quality:
+                    if keep_videos and quality:
                         missing_resolutions = []
                         if has_hybrid:
                             hybrid_candidate_tracks, non_hybrid_tracks = Tracks.partition_hybrid_videos(
@@ -2468,7 +2503,7 @@ class dl:
 
                     # choose best track by range and quality
                     pre_hybrid_videos: list[Video] = list(title.tracks.videos) if has_hybrid else []
-                    if has_hybrid:
+                    if keep_videos and has_hybrid:
                         # Apply hybrid selection for HYBRID tracks
                         hybrid_candidate_tracks, non_hybrid_tracks = Tracks.partition_hybrid_videos(
                             title.tracks.videos, non_hybrid_ranges
@@ -2522,7 +2557,7 @@ class dl:
                         # the lowest DV) so the standalone mux loop skips them. Tracks also
                         # picked as explicit deliverables stay unflagged.
                         Tracks.flag_hybrid_ingredients(hybrid_selected, non_hybrid_selected)
-                    else:
+                    elif keep_videos:
                         selected_videos: list[Video] = []
                         if video_multi_lang:
                             unique_video_langs = list(dict.fromkeys(str(v.language) for v in title.tracks.videos))
@@ -2549,7 +2584,7 @@ class dl:
                         title.tracks.videos = selected_videos
 
                     # validate hybrid mode requirements
-                    if any(r == Video.Range.HYBRID for r in range_):
+                    if keep_videos and any(r == Video.Range.HYBRID for r in range_):
                         base_tracks = [
                             v for v in title.tracks.videos if v.range in (Video.Range.HDR10, Video.Range.HDR10P)
                         ]
@@ -2613,7 +2648,7 @@ class dl:
                                 sys.exit(1)
 
                     # filter subtitle tracks
-                    if require_subs:
+                    if keep_subtitles and require_subs:
                         missing_langs = [
                             lang
                             for lang in require_subs
@@ -2627,7 +2662,7 @@ class dl:
                         self.log.info(
                             f"Required languages found ({', '.join(require_subs)}), downloading all available subtitles"
                         )
-                    elif s_lang and "all" not in s_lang:
+                    elif keep_subtitles and s_lang and "all" not in s_lang:
                         from unshackle.core.utilities import is_exact_match
 
                         match_func = is_exact_match if exact_lang else is_close_match
@@ -2661,12 +2696,12 @@ class dl:
                                 self.log.error(f"There's no {s_lang} Subtitle Track...")
                                 sys.exit(1)
 
-                    if not forced_subs:
+                    if keep_subtitles and not forced_subs:
                         title.tracks.select_subtitles(lambda x: not x.forced)
 
                 # filter audio tracks
                 # might have no audio tracks if part of the video, e.g. transport stream hls
-                if len(title.tracks.audio) > 0:
+                if keep_audio and len(title.tracks.audio) > 0:
                     if not audio_description:
                         title.tracks.select_audio(lambda x: not x.descriptive)  # exclude descriptive audio
                     if acodec:
@@ -2747,57 +2782,19 @@ class dl:
                             self.log.error(f"There's no {processed_lang} Audio Track, cannot continue...")
                             sys.exit(1)
 
-                if (
-                    video_only
-                    or audio_only
-                    or subs_only
-                    or chapters_only
-                    or no_subs
-                    or no_audio
-                    or no_chapters
-                    or no_video
-                ):
-                    keep_videos = False
-                    keep_audio = False
-                    keep_subtitles = False
-                    keep_chapters = False
+                # Reconstruct track set to only include kept tracks
+                kept_tracks = []
+                if keep_videos:
+                    kept_tracks.extend(title.tracks.videos)
+                if keep_audio:
+                    kept_tracks.extend(title.tracks.audio)
+                if keep_subtitles:
+                    kept_tracks.extend(title.tracks.subtitles)
+                if keep_chapters:
+                    kept_tracks.extend(title.tracks.chapters)
+                kept_tracks.extend(title.tracks.attachments)
 
-                    if video_only or audio_only or subs_only or chapters_only:
-                        if video_only:
-                            keep_videos = True
-                        if audio_only:
-                            keep_audio = True
-                        if subs_only:
-                            keep_subtitles = True
-                        if chapters_only:
-                            keep_chapters = True
-                    else:
-                        keep_videos = True
-                        keep_audio = True
-                        keep_subtitles = True
-                        keep_chapters = True
-
-                    if no_subs:
-                        keep_subtitles = False
-                    if no_audio:
-                        keep_audio = False
-                    if no_chapters:
-                        keep_chapters = False
-                    if no_video:
-                        keep_videos = False
-
-                    kept_tracks = []
-                    if keep_videos:
-                        kept_tracks.extend(title.tracks.videos)
-                    if keep_audio:
-                        kept_tracks.extend(title.tracks.audio)
-                    if keep_subtitles:
-                        kept_tracks.extend(title.tracks.subtitles)
-                    if keep_chapters:
-                        kept_tracks.extend(title.tracks.chapters)
-                    kept_tracks.extend(title.tracks.attachments)
-
-                    title.tracks = Tracks(kept_tracks, manifest_url=title.tracks.manifest_url)
+                title.tracks = Tracks(kept_tracks, manifest_url=title.tracks.manifest_url)
 
             selected_tracks, tracks_progress_callables = title.tracks.tree(add_progress=True)
 

unshackle/core/providers/imdbapi.py

@@ -8,7 +8,7 @@ import requests
 from unshackle.core.config import config
 from unshackle.core.providers._base import ExternalIds, MetadataProvider, MetadataResult, _clean, fuzzy_match
 
-# Mapping from our kind ("movie"/"tv") to imdbapi.dev title types
+# Mapping from our kind ("movie"/"tv") to IMDxAPI title types
 KIND_TO_TYPES: dict[str, list[str]] = {
     "movie": ["movie"],
     "tv": ["tvSeries", "tvMiniSeries"],
@@ -16,11 +16,11 @@ KIND_TO_TYPES: dict[str, list[str]] = {
 
 
 class IMDBApiProvider(MetadataProvider):
-    """IMDb metadata provider using imdbapi.dev (free, no API key)."""
+    """IMDb metadata provider using IMDxAPI (api.tiffara.com, free, no API key)."""
 
     NAME = "imdbapi"
     REQUIRES_KEY = False
-    BASE_URL = "https://api.imdbapi.dev"
+    BASE_URL = "https://api.tiffara.com"
 
     def is_available(self) -> bool:
         return config.imdb_api_enabled

unshackle/core/tracks/subtitle_convert.py

@@ -154,6 +154,11 @@ class SubtitleEditBackend:
         return source in self.reads and target in self.writes
 
     def rank(self, source: Codec, target: Codec) -> int:
+        if source in (Codec.SubStationAlpha, Codec.SubStationAlphav4) and target in (
+            Codec.TimedTextMarkupLang,
+            Codec.WebVTT,
+        ):
+            return 3
         return 0
 
     def convert(self, source: Codec, src: Path, target: Codec, out: Path) -> None:

unshackle/core/utils/webvtt.py

@@ -1,225 +1,74 @@
 import re
-import sys
-import typing
 from typing import Optional
 
-import pysubs2
-from pycaption import Caption, CaptionList, CaptionNode, CaptionReadError, WebVTTReader, WebVTTWriter
-
-from unshackle.core.config import config
+# Timing line: optional hours, verbatim settings tail.
+_TIMING = re.compile(r"^((?:\d+:)?\d{2}:\d{2}\.\d{3})[ \t]+-->[ \t]+((?:\d+:)?\d{2}:\d{2}\.\d{3})(.*)$")
 
 
-class CaptionListExt(CaptionList):
-    @typing.no_type_check
-    def __init__(self, iterable=None, layout_info=None):
-        self.first_segment_mpegts = 0
-        super().__init__(iterable, layout_info)
-
-
-class CaptionExt(Caption):
-    @typing.no_type_check
-    def __init__(self, start, end, nodes, style=None, layout_info=None, segment_index=0, mpegts=0, cue_time=0.0):
-        style = style or {}
-        self.segment_index: int = segment_index
-        self.mpegts: float = mpegts
-        self.cue_time: float = cue_time
-        super().__init__(start, end, nodes, style, layout_info)
-
-
-class WebVTTReaderExt(WebVTTReader):
-    # HLS extension support <https://datatracker.ietf.org/doc/html/rfc8216#section-3.5>
-    RE_TIMESTAMP_MAP = re.compile(r"X-TIMESTAMP-MAP.*")
-    RE_MPEGTS = re.compile(r"MPEGTS:(\d+)")
-    RE_LOCAL = re.compile(r"LOCAL:((?:(\d{1,}):)?(\d{2}):(\d{2})\.(\d{3}))")
-
-    def _parse(self, lines: list[str]) -> CaptionList:
-        captions = CaptionListExt()
-        start = None
-        end = None
-        nodes: list[CaptionNode] = []
-        layout_info = None
-        found_timing = False
-        segment_index = -1
-        mpegts = 0
-        cue_time = 0.0
-
-        # The first segment MPEGTS is needed to calculate the rest. It is possible that
-        # the first segment contains no cue and is ignored by pycaption, this acts as a fallback.
-        captions.first_segment_mpegts = 0
-
-        for i, line in enumerate(lines):
-            if "-->" in line:
-                found_timing = True
-                timing_line = i
-                last_start_time = captions[-1].start if captions else 0
-                try:
-                    start, end, layout_info = self._parse_timing_line(line, last_start_time)
-                except CaptionReadError as e:
-                    new_msg = f"{e.args[0]} (line {timing_line})"
-                    tb = sys.exc_info()[2]
-                    raise type(e)(new_msg).with_traceback(tb) from None
-
-            elif "" == line:
-                if found_timing and nodes:
-                    found_timing = False
-                    caption = CaptionExt(
-                        start,
-                        end,
-                        nodes,
-                        layout_info=layout_info,
-                        segment_index=segment_index,
-                        mpegts=mpegts,
-                        cue_time=cue_time,
-                    )
-                    captions.append(caption)
-                    nodes = []
-
-            elif "WEBVTT" in line:
-                # Merged segmented VTT doesn't have index information, track manually.
-                segment_index += 1
-                mpegts = 0
-                cue_time = 0.0
-            elif m := self.RE_TIMESTAMP_MAP.match(line):
-                if r := self.RE_MPEGTS.search(m.group()):
-                    mpegts = int(r.group(1))
-
-                cue_time = self._parse_local(m.group())
-
-                # Early assignment in case the first segment contains no cue.
-                if segment_index == 0:
-                    captions.first_segment_mpegts = mpegts
-
-            else:
-                if found_timing:
-                    if nodes:
-                        nodes.append(CaptionNode.create_break())
-                    nodes.append(CaptionNode.create_text(self._decode(line)))
-                else:
-                    # it's a comment or some metadata; ignore it
-                    pass
-
-        # Add a last caption if there are remaining nodes
-        if nodes:
-            caption = CaptionExt(start, end, nodes, layout_info=layout_info, segment_index=segment_index, mpegts=mpegts)
-            captions.append(caption)
-
-        return captions
-
-    @staticmethod
-    def _parse_local(string: str) -> float:
-        """
-        Parse WebVTT LOCAL time and convert it to seconds.
-        """
-        m = WebVTTReaderExt.RE_LOCAL.search(string)
-        if not m:
-            return 0
-
-        parsed = m.groups()
-        if not parsed:
-            return 0
-        hours = int(parsed[1])
-        minutes = int(parsed[2])
-        seconds = int(parsed[3])
-        milliseconds = int(parsed[4])
-        return (milliseconds / 1000) + seconds + (minutes * 60) + (hours * 3600)
+def _timestamp_ms(ts: str) -> int:
+    hms, ms = ts.rsplit(".", 1)
+    parts = [int(p) for p in hms.split(":")]
+    while len(parts) < 3:
+        parts.insert(0, 0)
+    hours, minutes, seconds = parts
+    return ((hours * 60 + minutes) * 60 + seconds) * 1000 + int(ms)
 
 
 def merge_segmented_webvtt(vtt_raw: str, segment_durations: Optional[list[int]] = None, timescale: int = 1) -> str:
     """
-    Merge Segmented WebVTT data.
+    Merge concatenated Segmented WebVTT documents into a single document.
 
-    Parameters:
-        vtt_raw: The concatenated WebVTT files to merge. All WebVTT headers must be
-            appropriately spaced apart, or it may produce unwanted effects like
-            considering headers as captions, timestamp lines, etc.
-        segment_durations: A list of each segment's duration. If not provided it will try
-            to get it from the X-TIMESTAMP-MAP headers, specifically the MPEGTS number.
-        timescale: The number of time units per second.
+    Strips the per-segment WEBVTT and X-TIMESTAMP-MAP headers and splices cues
+    repeated across a segment boundary (identical payload, <=1ms gap or overlap).
+    Timing lines, cue settings and payload are kept verbatim, so inline formatting
+    (span tags, entities, ASS-style overrides) survives untouched.
 
-    This parses the X-TIMESTAMP-MAP data to compute new absolute timestamps, replacing
-    the old start and end timestamp values. All X-TIMESTAMP-MAP header information will
-    be removed from the output as they are no longer of concern. Consider this function
-    the opposite of a WebVTT Segmenter, a WebVTT Joiner of sorts.
+    No timestamp offset is applied: segments are expected to carry absolute cue
+    times already (same guard as shaka-player and N_m3u8DL-RE; re-offsetting
+    double-shifts such streams). segment_durations/timescale are accepted for
+    call-site compatibility and only matter if offsetting is ever reintroduced.
 
-    Algorithm borrowed from N_m3u8DL-RE and shaka-player.
+    Unique STYLE/REGION blocks are kept (players that don't support them ignore
+    them); cue identifiers and NOTE blocks are dropped; cues with a malformed
+    timing line are skipped (downstream sanitizers catch worse).
     """
-    MPEG_TIMESCALE = 90_000
+    # each kept cue: [start, end, settings, payload, normalized payload for dedupe]
+    cues: list[list[str]] = []
+    headers: list[str] = []  # unique STYLE/REGION blocks, first-seen order
+    prev: Optional[list[str]] = None
+    prev_end_ms = 0
 
-    # Check config for conversion method preference
-    conversion_method = config.subtitle.get("conversion_method", "auto")
-    use_pysubs2 = conversion_method in ("pysubs2", "auto")
+    for block in re.split(r"\n[ \t]*\n", vtt_raw.replace("\r\n", "\n").replace("\r", "\n")):
+        lines = [
+            line
+            for line in block.split("\n")
+            if not (line.startswith("WEBVTT") or line.lstrip().startswith("X-TIMESTAMP-MAP"))
+        ]
+        timing_i = next((i for i, line in enumerate(lines) if "-->" in line), None)
+        if timing_i is None:  # NOTE or other metadata block
+            first = next((line.strip() for line in lines if line.strip()), "")
+            if first.startswith(("STYLE", "REGION")):
+                header = "\n".join(lines).strip("\n")
+                if header not in headers:  # segments repeat the same block
+                    headers.append(header)
+            continue
+        timing = _TIMING.match(lines[timing_i].strip())
+        if not timing:
+            continue
+        start, end, settings = timing.groups()
+        payload = [line for line in lines[timing_i + 1 :] if line.strip()]
+        if not payload:
+            continue
 
-    if use_pysubs2:
-        # Try using pysubs2 first for more lenient parsing
-        try:
-            # Use pysubs2 to parse and normalize the VTT
-            subs = pysubs2.SSAFile.from_string(vtt_raw)
-            # Convert back to WebVTT string for pycaption processing
-            normalized_vtt = subs.to_string("vtt")
-            vtt = WebVTTReaderExt().read(normalized_vtt)
-        except Exception:
-            # Fall back to direct pycaption parsing
-            vtt = WebVTTReaderExt().read(vtt_raw)
-    else:
-        # Use pycaption directly
-        vtt = WebVTTReaderExt().read(vtt_raw)
-    for lang in vtt.get_languages():
-        prev_caption = None
-        duplicate_index: list[int] = []
-        captions = vtt.get_captions(lang)
+        normalized = "\n".join(line.strip() for line in payload)
+        if prev is not None and _timestamp_ms(start) - prev_end_ms <= 1 and normalized == prev[4]:
+            prev[1] = end  # splice: extend the kept cue, drop the duplicate
+            prev_end_ms = _timestamp_ms(end)
+            continue
 
-        # Some providers can produce "segment_index" values that are
-        # outside the provided segment_durations list after normalization/merge.
-        # This used to crash with IndexError and abort the entire download.
-        if segment_durations and captions:
-            max_idx = max(getattr(c, "segment_index", 0) for c in captions)
-            if max_idx >= len(segment_durations):
-                # Pad with the last known duration (or 0 if empty) so indexing is safe.
-                pad_val = segment_durations[-1] if segment_durations else 0
-                segment_durations = segment_durations + [pad_val] * (max_idx - len(segment_durations) + 1)
+        prev = [start, end, settings, "\n".join(payload), normalized]
+        prev_end_ms = _timestamp_ms(end)
+        cues.append(prev)
 
-        if captions[0].segment_index == 0:
-            first_segment_mpegts = captions[0].mpegts
-        else:
-            first_segment_mpegts = segment_durations[0] if segment_durations else captions.first_segment_mpegts
-
-        caption: CaptionExt
-        for i, caption in enumerate(captions):
-            # DASH WebVTT doesn't have MPEGTS timestamp like HLS. Instead,
-            # calculate the timestamp from SegmentTemplate/SegmentList duration.
-            likely_dash = first_segment_mpegts == 0 and caption.mpegts == 0
-            if likely_dash and segment_durations:
-                # Defensive: segment_index can still be out of range if captions are malformed.
-                if caption.segment_index < 0 or caption.segment_index >= len(segment_durations):
-                    continue
-                duration = segment_durations[caption.segment_index]
-                caption.mpegts = MPEG_TIMESCALE * (duration / timescale)
-
-            if caption.mpegts == 0:
-                continue
-
-            # Commeted to fix DSNP subs being out of sync and mistimed.
-            # seconds = (caption.mpegts - first_segment_mpegts) / MPEG_TIMESCALE - caption.cue_time
-            # offset = seconds * 1_000_000  # pycaption use microseconds
-
-            # if caption.start < offset:
-            #    caption.start += offset
-            #    caption.end += offset
-
-            # If the difference between current and previous captions is <=1ms
-            # and the payload is equal then splice.
-            if (
-                prev_caption
-                and not caption.is_empty()
-                and (caption.start - prev_caption.end) <= 1000  # 1ms in microseconds
-                and caption.get_text() == prev_caption.get_text()
-            ):
-                prev_caption.end = caption.end
-                duplicate_index.append(i)
-
-            prev_caption = caption
-
-        # Remove duplicate
-        captions[:] = [c for c_index, c in enumerate(captions) if c_index not in set(duplicate_index)]
-
-    return WebVTTWriter().write(vtt)
+    blocks = headers + [f"{c[0]} --> {c[1]}{c[2]}\n{c[3]}" for c in cues]
+    return "WEBVTT\n\n" + "\n\n".join(blocks) + "\n"