diff --git a/docs/dev/rest-api/endpoints.md b/docs/dev/rest-api/endpoints.md index a3756b1c..c8036152 100644 --- a/docs/dev/rest-api/endpoints.md +++ b/docs/dev/rest-api/endpoints.md @@ -339,7 +339,7 @@ When you give both `season` and `episode`, the server combines them into a `"{se "bitrate": 8000, "bitrate_bps": 8000000, "width": 3840, "height": 2160, "resolution": "2160p", "fps": "24", "range": "HDR10", "range_display": "HDR", "scan_type": null, "closed_captions": [], - "dv_compatible_bitstream": false, "hdr10plus": false, "hdr10_base": false, "language": "en", + "dv_compatible_bitstream": false, "hdr10plus": false, "hdr10_base": false, "stereo": false, "language": "en", "is_original_lang": true, "name": null, "track_name": "English", "needs_repack": false, "edition": [], "note": null, "data": null, "drm": [], "descriptor": "DASH" diff --git a/docs/dev/service-architecture.md b/docs/dev/service-architecture.md index 8f670844..da900363 100644 --- a/docs/dev/service-architecture.md +++ b/docs/dev/service-architecture.md @@ -454,6 +454,7 @@ Two DV mechanisms sit on top of the track model. **DV fixup** re-exposes DV sign HLS and DASH streams that carry a valid DV RPU but advertise plain HEVC, with DV only in the supplemental codecs (`dv_compatible_bitstream`). `--list` shows such a track as `HDR10+DV` or `HDR10P+DV`. A service sets `hdr10_base` on a DV track that has an HDR10 base layer (DV profile 8.1), and `--list` shows it as `DV+HDR10`. +The HLS and DASH parsers set `stereo` on a stereoscopic 3D (MV-HEVC) track: an HLS `REQ-VIDEO-LAYOUT` that names `CH-STEREO`, or a DV profile 20 codec. `--list` shows it as `3D`. **Hybrid** processing (`Video.Range.HYBRID`) selects an HDR10(+) base and a DV donor, extracts the RPU with `dovi_tool`, and injects it onto the base layer, to make a single track that carries both HDR10 metadata and the DV RPU. Both are automatic once the diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 7ab8d3ac..29b3a825 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -268,7 +268,7 @@ download. For example, `dovi_tool` only matters if you download Dolby Vision vid | Tool | Binary looked up | Why it is needed | | --- | --- | --- | | Bento4 (`mp4decrypt`) | `mp4decrypt` | Alternative DRM decryptor to shaka-packager | -| MP4Box (GPAC) | `MP4Box` | Muxing DTS:X Profile 2 (DTS-UHD) and MPEG-H audio | +| MP4Box (GPAC) | `MP4Box` | Muxing DTS:X Profile 2 (DTS-UHD) and MPEG-H audio, and 3D video | | dovi_tool | `dovi_tool` | Dolby Vision metadata handling | | HDR10Plus_tool | `hdr10plus_tool` | HDR10+ metadata handling | | SubtitleEdit | `seconv`, `SubtitleEdit` | Subtitle conversion (the `SeConv` CLI from 5.x) | @@ -280,9 +280,9 @@ download. For example, `dovi_tool` only matters if you download Dolby Vision vid | Docker | `docker` | Gluetun VPN proxy support | | Git | `git` | Fetching and updating remote service repositories | -!!! info "MP4Box is only used for DTS:X Profile 2 and MPEG-H" - Titles with DTS:X Profile 2 (DTS-UHD) or MPEG-H audio mux to MP4; everything else muxes to MKV - with `mkvmerge`. Install with `apt install gpac`. +!!! info "MP4Box is only used for DTS:X Profile 2, MPEG-H and 3D video" + Titles with DTS:X Profile 2 (DTS-UHD) or MPEG-H audio, or a stereoscopic 3D video track, mux to + MP4; everything else muxes to MKV with `mkvmerge`. Install with `apt install gpac`. !!! tip "SubtitleEdit specifics" unshackle looks for `seconv` first, the batch CLI shipped with diff --git a/docs/guide/downloading.md b/docs/guide/downloading.md index 366b69c5..177c8ab1 100644 --- a/docs/guide/downloading.md +++ b/docs/guide/downloading.md @@ -164,6 +164,10 @@ unshackle dl -q 2160 -r HDR10,DV EXAMPLE '...' The output file of a track with a Dolby Vision profile 8 RPU plays as Dolby Vision on a DV screen. +`--list` adds `3D` to a stereoscopic 3D (MV-HEVC) track, for example `[H.265, DV, 3D]`. When a title +has a 2D and a 3D track at the same resolution, unshackle selects the 2D track. A 3D title muxes to +MP4, see [3D output](output-and-naming.md#3d-output). + !!! warning "HYBRID requires dovi_tool" `-r HYBRID` makes a single hybrid track by injecting the Dolby Vision RPU metadata onto an HDR10/HDR10+ base layer with **dovi_tool**. diff --git a/docs/guide/output-and-naming.md b/docs/guide/output-and-naming.md index 7a2aa40d..15b4d66c 100644 --- a/docs/guide/output-and-naming.md +++ b/docs/guide/output-and-naming.md @@ -15,7 +15,7 @@ When a download finishes, unshackle builds the final path from three pieces: - **Output directory**: `directories.downloads` from your config, or whatever you pass to `-o/--output` on the command line for a single run. - **Folder template**: an optional per-title subfolder. unshackle **always** puts TV episodes and songs in a folder. Movies only get one if you define a `movies` folder template, or a single-string `folder` template that applies to all title kinds (see [Folder templates](#folder-templates)). - **Filename template**: the `output_template` for the title kind (movie, series, or song). -- **Extension**: chosen by the muxer for a movie or an episode: `.mkv` for video, `.mka` for audio-only, `.mks` for subtitle-only, `.mp4` for [DTS:X Profile 2 and MPEG-H audio](#dtsx-profile-2-and-mpeg-h-output). unshackle does not mux a song. It remuxes a song into the container of its codec, such as `.flac`, `.opus`, or `.m4a`. Read [Music output files](#music-output-files). +- **Extension**: chosen by the muxer for a movie or an episode: `.mkv` for video, `.mka` for audio-only, `.mks` for subtitle-only, `.mp4` for [DTS:X Profile 2 and MPEG-H audio](#dtsx-profile-2-and-mpeg-h-output) and [3D video](#3d-output). unshackle does not mux a song. It remuxes a song into the container of its codec, such as `.flac`, `.opus`, or `.m4a`. Read [Music output files](#music-output-files). !!! example "A typical episode path" ```text @@ -110,6 +110,7 @@ Every variable below is valid in both output and folder templates. unshackle tak | `video` | Video codec | `H.264`, `H.265` | | `hdr` | Dynamic-range label | `HDR`, `HDR10P`, `DV`, `DV.HDR`, `DV.HDR10P`, `HLG` | | `hfr` | High frame rate marker (frame rate > 30) | `HFR` | +| `3d` | Stereoscopic 3D marker (MV-HEVC video) | `3D` | | `edition` | Edition label from the track, if any | `Directors Cut` | | `tag` | Release-group tag (from `config.tag` or `--tag`) | `TAG` | | `repack` | `REPACK` when the `--repack` flag is used | `REPACK` | @@ -429,6 +430,23 @@ An MP4 title loses: - SubStation Alpha (`.ass`) subtitles - container tags (`mkvpropedit`) +### 3D output + +A stereoscopic 3D (MV-HEVC) video track also muxes to MP4 with MP4Box, because Matroska has no +mapping for its second view and `mkvmerge` drops it. There is no 3D Matroska output. unshackle +finds a 3D track from the manifest, or from a second video layer in the downloaded file. The MP4 +title loses the same items as above. For a 3D title, unshackle also: + +- decrypts the track with mp4decrypt, whatever [`decryption`](../reference/configuration/drm.md#decryption) selects +- writes QuickTime chapters +- keeps the whole downloaded segments of a `--sample` run, because the MP4Box cut removes the 3D signalling +- does not repack the track or rewrite its colour metadata with FFmpeg +- writes no `--include-sample` sample, because `mkvmerge` cuts only Matroska files +- never uses the track in a `HYBRID` merge + +A Dolby Vision profile 20 track has no HDR10 base layer, so a player without Dolby Vision shows +wrong colours. + You configure muxing (the combination of video, audio, subtitle, chapter and attachment tracks into a single Matroska file with `mkvmerge`) under the `muxing` config key. unshackle muxes only movies and episodes. Read [Music output files](#music-output-files) for what a music download does instead. ```yaml title="unshackle.yaml" @@ -475,6 +493,7 @@ The `--no-mux` flag skips muxing entirely and writes the individual track files, [`--sample`](cli-reference.md#output-muxing-files) and `--include-sample` write a sample into a `Sample` folder inside the output folder of the title. The sample keeps the filename of the title, with a `SAMPLE` tag at the end. The tag uses the separator of the output template. +`--include-sample` cuts the sample with `mkvmerge`, so a title that muxes to MP4 gets no sample. ```text title="A movie with --include-sample" Some.Movie.2026.1080p.EXAMPLE.WEB-DL.H.264-TAG/ diff --git a/docs/reference/configuration/drm.md b/docs/reference/configuration/drm.md index 4dd1f0f0..9e0b0168 100644 --- a/docs/reference/configuration/drm.md +++ b/docs/reference/configuration/drm.md @@ -187,6 +187,10 @@ service tag case-insensitively and uses the `default` entry as the fallback. unshackle decrypts HLS AES-128 ClearKey in-process and ignores this setting. +unshackle decrypts a stereoscopic 3D (MV-HEVC) track with mp4decrypt whatever this setting says. +shaka-packager refuses its Dolby Vision configuration (version 3), and before v3.5.0 it also drops +the boxes of the second view. + mp4decrypt finds a content key by the KID in the first `sinf` box of the track. When that box has no CENC `tenc` (for example, a PIFF `sinf` before the CENC one), unshackle also gives mp4decrypt the content key of that KID by track ID. When mp4decrypt copies a track through diff --git a/docs/reference/configuration/output.md b/docs/reference/configuration/output.md index 63ae2750..818468a1 100644 --- a/docs/reference/configuration/output.md +++ b/docs/reference/configuration/output.md @@ -40,7 +40,7 @@ folder-kind keys. `absolute`, `date`, `quality`, `resolution`, `source`, `tag`, `track_number`, `artist`, `album_artist`, `album`, `disc`, `track_total`, `disc_total`, `release_type`, `genre`, `explicit`, `isrc`, `upc`, `label`, `audio`, `audio_channels`, `audio_full`, `atmos`, - `dual`, `multi`, `dubbed`, `video`, `hdr`, `hfr`, `edition`, `repack`, `lang_tag`, + `dual`, `multi`, `dubbed`, `video`, `hdr`, `hfr`, `3d`, `edition`, `repack`, `lang_tag`, `title_type`. !!! warning "`{part?}` is already inside `{season_episode}`" @@ -129,7 +129,7 @@ Rule semantics: - A value can be a scalar or a list. A list matches if **any** entry matches. - Comparison is case-insensitive string equality. - The keys of `when` are the filename template variables for the release itself: - `title_type`, `quality`, `resolution`, `source`, `video`, `hdr`, `hfr`, `audio`, + `title_type`, `quality`, `resolution`, `source`, `video`, `hdr`, `hfr`, `3d`, `audio`, `audio_channels`, `audio_full`, `atmos`, `dual`, `multi`, `dubbed`, `edition`, `repack`, and `lang_tag`. Only these are available. unshackle adds the title's own fields (`title`, `year`, `season`, `episode`, and so on) after it evaluates the rules, so a condition cannot diff --git a/docs/reference/configuration/post-scripts.md b/docs/reference/configuration/post-scripts.md index 0f3a8de8..e987d933 100644 --- a/docs/reference/configuration/post-scripts.md +++ b/docs/reference/configuration/post-scripts.md @@ -113,7 +113,7 @@ values. | `{year}` `{season}` `{episode}` `{episode_name}` | Season and episode are plain numbers (`1`, `5`) | | `{part}` | Part number of a split episode as a plain number (`1` for `Part.1`). Empty when the episode has no part | | `{season_episode}` `{absolute}` `{date}` | As in the filename template: `{season_episode}` keeps the part suffix (`S01E05.Part.1`) while `{episode}` drops it | -| `{quality}` `{resolution}` `{hdr}` `{hfr}` `{vcodec}` `{acodec}` `{edition}` `{tag}` `{lang_tag}` | As in the filename template, for this file | +| `{quality}` `{resolution}` `{hdr}` `{hfr}` `{3d}` `{vcodec}` `{acodec}` `{edition}` `{tag}` `{lang_tag}` | As in the filename template, for this file | | `{tmdb}` `{imdb}` `{tvdb}` | The tagging IDs in use, otherwise empty. See [Tagging IDs](#tagging-ids) | | `{error}` | Failure message. Empty on success | @@ -132,7 +132,7 @@ Every variable from your `output_template` is available here as well, including music has none of them. - **The naming context renders it empty on purpose.** `{hdr}` is empty for an SDR file, because SDR is the absence of an HDR tag in a filename rather than a tag that - reads `SDR`. `{edition}`, `{atmos}`, `{multi}`, `{dual}` and `{hfr}` behave the same + reads `SDR`. `{edition}`, `{atmos}`, `{multi}`, `{dual}`, `{hfr}` and `{3d}` behave the same way. If your script needs the word `SDR`, derive it from an empty `{hdr}` yourself. - The post-script is a `failure` one. A failed download has no output file to read metadata from, so everything that comes from the output file is empty. See diff --git a/tests/core/test_export.py b/tests/core/test_export.py index 3a5dc784..a59b67d1 100644 --- a/tests/core/test_export.py +++ b/tests/core/test_export.py @@ -476,3 +476,16 @@ def test_import_keeps_every_period_when_no_exported_period_is_left( assert video.data["dash"]["filtered_period_ids"] == [] assert "keeping every period" in caplog.text + + +def test_a_3d_video_stays_3d_through_export_and_import(tmp_path: Path) -> None: + export = tmp_path / "export.json" + title = make_title() + title.tracks.videos[0].stereo = True + runner = make_dl() + for track in [*title.tracks.videos, *title.tracks.audio]: + runner.write_export(export, title, track) + assert [row.get("stereo") for row in entry(export)["tracks"]][0] is True + svc = ImportService(import_ctx(), "EXAMPLE", "movie-1", str(export)) + movie = next(iter(svc.get_titles())) + assert svc.get_tracks(movie).videos[0].stereo diff --git a/tests/core/test_hybrid_hdr10plus.py b/tests/core/test_hybrid_hdr10plus.py index 916df041..9c4dbb30 100644 --- a/tests/core/test_hybrid_hdr10plus.py +++ b/tests/core/test_hybrid_hdr10plus.py @@ -856,7 +856,7 @@ def video( codec: Optional[Video.Codec] = Video.Codec.HEVC, native_hybrid: bool = False, ) -> Any: - return SimpleNamespace(id=id_, height=height, range=range_, codec=codec, native_hybrid=native_hybrid) + return SimpleNamespace(id=id_, height=height, range=range_, codec=codec, native_hybrid=native_hybrid, stereo=False) def test_hybrid_fails_with_neither_a_base_nor_dv() -> None: diff --git a/tests/core/test_import_tracks.py b/tests/core/test_import_tracks.py index 794ebc52..804531f3 100644 --- a/tests/core/test_import_tracks.py +++ b/tests/core/test_import_tracks.py @@ -297,3 +297,8 @@ def test_a_file_row_can_name_its_codec_and_range_by_name(tmp_path: Path) -> None service = file_export(tmp_path, [row]) video = service.get_tracks(next(iter(service.get_titles()))).videos[0] assert (video.codec, video.range) == (Video.Codec.HEVC, Video.Range.HDR10P) + + +def test_a_file_row_marked_stereo_imports_as_a_3d_video(tmp_path: Path) -> None: + service = file_export(tmp_path, [{**FILE_ROWS[0], "stereo": True}]) + assert service.get_tracks(next(iter(service.get_titles()))).videos[0].stereo diff --git a/tests/core/test_stereo_video.py b/tests/core/test_stereo_video.py new file mode 100644 index 00000000..bc5ef88e --- /dev/null +++ b/tests/core/test_stereo_video.py @@ -0,0 +1,275 @@ +from __future__ import annotations + +import io +import itertools +import logging +import subprocess +from pathlib import Path +from types import SimpleNamespace +from typing import Any + +import pytest +from langcodes import Language + +from unshackle.commands.dl import dl, hybrid_plan, pick_video +from unshackle.core import binaries +from unshackle.core.config import config +from unshackle.core.drm.verify import decrypter_for +from unshackle.core.manifests import DASH, HLS +from unshackle.core.titles.movie import Movie +from unshackle.core.tracks import Audio, Chapter, Chapters, Track, Tracks, Video +from unshackle.core.tracks import tracks as tracks_module + +VIDEO_IDS = itertools.count() + +MASTER = """#EXTM3U +#EXT-X-VERSION:12 +#EXT-X-STREAM-INF:BANDWIDTH=6786883,CODECS="dvh1.20.02",RESOLUTION=960x540,VIDEO-RANGE=PQ,REQ-VIDEO-LAYOUT="CH-MONO,CH-STEREO" +stereo.m3u8 +#EXT-X-STREAM-INF:BANDWIDTH=5000000,CODECS="hvc1.2.4.L150.90",RESOLUTION=1920x1080,REQ-VIDEO-LAYOUT="CH-STEREO" +mvhevc.m3u8 +#EXT-X-STREAM-INF:BANDWIDTH=4000000,CODECS="dvh1.08.06",RESOLUTION=1920x1080,VIDEO-RANGE=PQ +flat.m3u8 +""" + + +@pytest.mark.parametrize( + ("codecs", "layout", "expected"), + [ + ("dvh1.20.06,ec-3", None, True), + ("dvhe.20.09", None, True), + ("hvc1.2.4.L150.90", '"CH-MONO,CH-STEREO"', True), + ("dvh1.08.06", '"CH-MONO"', False), + ("dvh1.05.06", None, False), + ("dvh1.200.06", None, False), + (None, None, False), + ], +) +def test_is_stereo(codecs: str | None, layout: str | None, expected: bool) -> None: + assert Video.is_stereo(codecs, video_layout=layout) is expected + + +def test_hls_marks_stereo_tracks_from_the_layout_or_the_codec() -> None: + tracks = HLS.from_text(MASTER, url="https://cdn.example/master.m3u8").to_tracks(Language.get("en")) + stereo = {str(v.url).rsplit("/", 1)[-1]: v.stereo for v in tracks.videos} + assert stereo == {"stereo.m3u8": True, "mvhevc.m3u8": True, "flat.m3u8": False} + + +def test_a_stereo_track_lists_as_3d_and_keeps_the_flag_through_to_dict() -> None: + video = Video(url="https://x/v.mp4", language="en", codec=Video.Codec.HEVC, range_=Video.Range.DV, stereo=True) + assert str(video).startswith("VID | [H.265, DV, 3D] |") + assert Video.from_dict(video.to_dict()).stereo + + +def stereo_video(tmp_path: Path, stereo: bool = True, **kwargs: Any) -> Video: + n = next(VIDEO_IDS) + path = tmp_path / f"v{n}.mp4" + path.write_bytes(b"\x00" * 16) + video = Video(url=f"https://x/v{n}.mp4", language="en", codec=Video.Codec.HEVC, stereo=stereo, **kwargs) + video.path = path + return video + + +def test_dash_marks_a_dv_profile_20_representation_stereo() -> None: + mpd = """ + + + + + + + + +""" + videos = DASH.from_text(mpd, url="https://cdn.example/manifest.mpd").to_tracks(language="en").videos + assert {v.data["dash"]["representation"].get("id"): v.stereo for v in videos} == {"v3d": True, "v2d": False} + + +def box(kind: bytes, payload: bytes = b"") -> bytes: + return (8 + len(payload)).to_bytes(4, "big") + kind + payload + + +def test_a_track_with_a_second_video_layer_decrypts_with_mp4decrypt( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + (layered := tmp_path / "3d.mp4").write_bytes(box(b"moov", box(b"lhvC"))) + (flat := tmp_path / "2d.mp4").write_bytes(box(b"moov", box(b"hvcC"))) + monkeypatch.setattr(config, "decryption", "Shaka") + assert decrypter_for(layered) == "mp4decrypt" + assert decrypter_for(flat) == "shaka" + monkeypatch.setattr(config, "decryption", "MP4DECRYPT") + assert decrypter_for(flat) == "mp4decrypt" + + +def test_a_3d_title_muxes_with_mp4box(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + calls: list[str] = [] + monkeypatch.setattr(Tracks, "_mux_mp4", lambda self, **kwargs: calls.append(kwargs["title"]) or (Path("x"), 0, [])) + assert Tracks(stereo_video(tmp_path)).mux("Film")[1] == 0 + assert calls == ["Film"] + + +@pytest.fixture +def mp4box_commands(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> list[list[str]]: + commands: list[list[str]] = [] + + class FakePopen: + def __init__(self, command: list[str], **kwargs: Any) -> None: + commands.append([str(c) for c in command]) + Path(command[-1]).write_bytes(b"mp4") + self.stdout = io.StringIO("") + + def wait(self) -> int: + return 0 + + monkeypatch.setattr(binaries, "MP4Box", Path("/bin/MP4Box"), raising=False) + monkeypatch.setattr(tracks_module.subprocess, "Popen", FakePopen) + monkeypatch.setattr(tracks_module, "run_step", lambda command, **kwargs: commands.append([str(c) for c in command])) + monkeypatch.setattr(config, "sample", None) + monkeypatch.setattr(config.directories, "temp", tmp_path) + return commands + + +def test_a_3d_mp4_mux_keeps_the_sample_whole( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, mp4box_commands: list[list[str]] +) -> None: + monkeypatch.setattr(config, "sample", (120.0, 180.0)) + Tracks(stereo_video(tmp_path))._mux_mp4(title="Film", output_path=tmp_path / "out.mkv") + assert len(mp4box_commands) == 1 + assert "-splitx" not in mp4box_commands[0] + + +def test_a_3d_mp4_mux_writes_quicktime_chapters(tmp_path: Path, mp4box_commands: list[list[str]]) -> None: + for stereo, flag in ((True, "-chapqt"), (False, "-chap")): + tracks = Tracks(stereo_video(tmp_path, stereo=stereo)) + tracks.chapters = Chapters([Chapter(0.0, "Story")]) + tracks._mux_mp4(title="Film", output_path=tmp_path / f"out{stereo}.mkv") + assert flag in mp4box_commands[-1] and ({"-chap", "-chapqt"} - {flag}).isdisjoint(mp4box_commands[-1]) + + +def test_a_3d_track_is_never_a_hybrid_ingredient(tmp_path: Path) -> None: + flat, layered = (stereo_video(tmp_path, stereo=s, range_=Video.Range.DV) for s in (False, True)) + candidates, standalone = Tracks.partition_hybrid_videos([flat, layered], []) + assert candidates == [flat] and standalone == [] + + +def test_2d_ranks_over_3d_at_one_resolution_and_the_signatures_differ(tmp_path: Path) -> None: + flat = stereo_video(tmp_path, stereo=False, height=1080, bitrate=5_000_000) + layered = stereo_video(tmp_path, stereo=True, height=1080, bitrate=9_000_000) + uhd = stereo_video(tmp_path, stereo=True, height=2160, bitrate=40_000_000, width=3840) + tracks = Tracks(layered, flat, uhd) + tracks.sort_videos() + assert tracks.videos == [uhd, flat, layered] + assert Tracks.signature(flat) != Tracks.signature( + stereo_video(tmp_path, stereo=True, height=1080, bitrate=5_000_000) + ) + + +def test_a_3d_track_gets_no_ffmpeg_vui_rewrite_or_repack(tmp_path: Path) -> None: + track = stereo_video(tmp_path, range_=Video.Range.HDR10) + assert track.vui_bsf() is None + assert track.repackage() is False + assert track.path.exists() + + +def test_the_filename_context_marks_a_3d_title(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(config, "tag_rules", []) + monkeypatch.setattr(config, "language_tags", {}) + + class Svc: + pass + + def context(stereo: bool) -> str: + movie = Movie(id_="movie-0001", service=Svc, name="The Film", year=2024, language="en") + movie.tracks = Tracks(Video(url="https://x/v.mp4", language="en", codec=Video.Codec.HEVC, stereo=stereo)) + video = SimpleNamespace( + width=3840, + height=2160, + format="HEVC", + scan_type="Progressive", + hdr_format_commercial="", + hdr_format="", + hdr_format_profile="", + transfer_characteristics="PQ", + transfer_characteristics_original=None, + frame_rate="47.952", + ) + return str(movie.build_base_template_context(SimpleNamespace(video_tracks=[video], audio_tracks=[]))["3d"]) + + assert (context(True), context(False)) == ("3D", "") + + +def test_worst_prefers_the_2d_track_at_one_height(tmp_path: Path) -> None: + videos = [ + stereo_video(tmp_path, stereo=s, height=2160, bitrate=b) + for s, b in ((False, 9_000_000), (False, 5_000_000), (True, 40_000_000), (True, 30_000_000)) + ] + ranked = Tracks(videos) + ranked.sort_videos() + assert pick_video(ranked.videos, worst=True) is videos[1] + assert pick_video(ranked.videos, worst=False) is videos[0] + assert pick_video([videos[3], videos[2]], worst=True) is videos[2] + assert pick_video([], worst=True) is None + + +def test_hybrid_never_pairs_a_3d_dv_track(tmp_path: Path) -> None: + base = stereo_video(tmp_path, stereo=False, range_=Video.Range.HDR10, height=2160) + layered = stereo_video(tmp_path, stereo=True, range_=Video.Range.DV, height=2160) + assert hybrid_plan([base, layered], generate_dv=False) == ([], None) + + +def test_mp4_names_skip_the_extra_tracks_of_a_3d_video_file(tmp_path: Path, mp4box_commands: list[list[str]]) -> None: + video = stereo_video(tmp_path) + video.path.write_bytes(box(b"moov", box(b"trak") + box(b"trak"))) + audio = Audio(url="https://x/a.mp4", language="en", codec=Audio.Codec.EC3, name="English") + audio.path = tmp_path / "a.mp4" + audio.path.write_bytes(b"\x00" * 16) + Tracks([video, audio])._mux_mp4(title="Film", output_path=tmp_path / "out.mkv") + command = mp4box_commands[0] + assert command[command.index("-name") + 1] == "3=English" + + +def test_a_3d_mux_that_loses_its_second_view_warns( + tmp_path: Path, mp4box_commands: list[list[str]], caplog: pytest.LogCaptureFixture +) -> None: + Tracks(stereo_video(tmp_path))._mux_mp4(title="Film", output_path=tmp_path / "out.mkv") + assert "lost its second view" in caplog.text + + +def test_a_downloaded_file_with_a_second_view_marks_the_video_3d( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + video = stereo_video(tmp_path, stereo=False) + monkeypatch.setattr(Track, "download", lambda self, *a, **k: self.path.write_bytes(box(b"moov", box(b"lhvC")))) + video.download(None, None) + assert video.stereo + + +def test_dash_reads_dv_profile_20_from_supplemental_codecs() -> None: + mpd = """ + + + + v.mp4 + + +""" + assert DASH.from_text(mpd, url="https://cdn.example/m.mpd").to_tracks(language="en").videos[0].stereo + + +def test_hls_reads_each_variant_layout_even_when_two_share_a_playlist() -> None: + master = """#EXTM3U +#EXT-X-STREAM-INF:BANDWIDTH=2,CODECS="hvc1.2.4.L150.90",RESOLUTION=1920x1080,REQ-VIDEO-LAYOUT="CH-STEREO" +shared.m3u8 +#EXT-X-STREAM-INF:BANDWIDTH=1,CODECS="hvc1.2.4.L150.90",RESOLUTION=1920x1080,AUDIO="a" +shared.m3u8 +""" + tracks = HLS.from_text(master, url="https://cdn.example/master.m3u8").to_tracks(Language.get("en")) + assert any(v.stereo for v in tracks.videos) + + +def test_a_sample_is_cut_only_from_a_matroska_file(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + runner = dl.__new__(dl) + runner.log = logging.getLogger("download") + monkeypatch.setattr(subprocess, "run", lambda *a, **k: pytest.fail("mkvmerge ran on an MP4")) + runner.write_sample(tmp_path / "Film.mp4", SimpleNamespace(), SimpleNamespace()) + assert not (tmp_path / "Sample").exists() diff --git a/tests/remote/unit/test_remote_service_helpers.py b/tests/remote/unit/test_remote_service_helpers.py index 45ccdc29..8c984a02 100644 --- a/tests/remote/unit/test_remote_service_helpers.py +++ b/tests/remote/unit/test_remote_service_helpers.py @@ -58,6 +58,11 @@ def test_a_dv_track_keeps_its_hdr10_base_label_over_remote() -> None: assert deserialize_video(serialize_video_track(v)).range_label == "DV+HDR10" +def test_a_stereo_track_stays_3d_over_remote() -> None: + v = Video(url="https://x/v.mp4", language="en", codec=Video.Codec.HEVC, range_=Video.Range.DV, stereo=True) + assert deserialize_video(serialize_video_track(v)).stereo + + def test_a_track_note_shows_over_remote_and_export_and_is_not_an_edition() -> None: from unshackle.core.tracks.track import Track diff --git a/unshackle/commands/dl.py b/unshackle/commands/dl.py index 61eea03c..e1941873 100644 --- a/unshackle/commands/dl.py +++ b/unshackle/commands/dl.py @@ -336,18 +336,26 @@ def hybrid_gate_failure(videos: Sequence[Video], generate_dv: bool) -> Optional[ def hybrid_plan(videos: Sequence[Video], generate_dv: bool) -> tuple[list[Video], Optional[Video]]: - """The hybrid bases and the DV track to pair with them. + """The hybrid bases and the 2D DV track to pair with them. With no DV track, a base qualifies only with ``--generate-dv``. A resolution with a finished hybrid (``native_hybrid``) gets no base: that track is its hybrid. """ - dv_tracks = [v for v in videos if v.range == Video.Range.DV] + dv_tracks = [v for v in videos if v.range == Video.Range.DV and not v.stereo] native_heights = {v.height for v in videos if v.native_hybrid} base_pool = [v for v in videos if v.height not in native_heights and (dv_tracks or generate_dv)] matching_dv = min(dv_tracks, key=lambda v: v.height or 0) if dv_tracks else None return select_hybrid_bases(base_pool), matching_dv +def pick_video(candidates: list[Video], worst: bool) -> Optional[Video]: + """The best candidate, or with ``worst`` the last one, preferring 2D: a 3D track sorts after 2D at one height.""" + if worst: + pool = [v for v in candidates if not v.stereo] or candidates + return pool[-1] if pool else None + return next(iter(candidates), None) + + def hybrid_source_tracks(base: Video, dv: Optional[Video], base_container: Optional[Path] = None) -> list[Video]: """Copies of the base and DV tracks for the hybrid step, with the base on its container file. @@ -2874,7 +2882,7 @@ class dl: and (not codec or t.codec == codec) and (vlang is None or str(t.language) == vlang) ] - match = candidates[-1] if worst and candidates else next(iter(candidates), None) + match = pick_video(candidates, worst) if match and match not in non_hybrid_selected: non_hybrid_selected.append(match) @@ -2903,7 +2911,7 @@ class dl: and (not codec or t.codec == codec) and (vlang is None or str(t.language) == vlang) ] - match = candidates[-1] if worst and candidates else next(iter(candidates), None) + match = pick_video(candidates, worst) if match and match not in selected_videos: selected_videos.append(match) title.tracks.videos = selected_videos @@ -2941,7 +2949,7 @@ class dl: and (not codec or t.codec == codec) and (vlang is None or str(t.language) == vlang) ] - match = candidates[-1] if worst and candidates else next(iter(candidates), None) + match = pick_video(candidates, worst) if match and match not in fallback_selected: fallback_selected.append(match) title.tracks.videos = fallback_selected @@ -3448,13 +3456,9 @@ class dl: self.suggest_missing_fonts(missing_fonts) # Handle DRM decryption BEFORE repacking (must decrypt first!) - service_name = service.__class__.__name__.upper() - decryption_method = resolve_decryption(config.decryption_map, config.decryption, service_name) - decrypt_tool = "mp4decrypt" if decryption_method.lower() == "mp4decrypt" else "Shaka Packager" - drm_tracks = [track for track in title.tracks if track.drm] if drm_tracks: - with console.status(f"Decrypting tracks with {decrypt_tool}..."): + with console.status("Decrypting tracks..."): has_decrypted = False for track in drm_tracks: drm = track.get_drm_for_cdm(self.cdm) @@ -3470,7 +3474,7 @@ class dl: f"No matching DRM found for track {track} with CDM type {type(self.cdm).__name__}" ) if has_decrypted: - self.log.info(f"Decrypted tracks with {decrypt_tool}") + self.log.info("Decrypted tracks") if ( not no_subs @@ -4050,7 +4054,11 @@ class dl: """Cut a 60 second sample from a finished file into a ``Sample`` folder next to it. The sample starts 2 minutes in. A file shorter than 3 minutes gives its middle 60 seconds. + mkvmerge cuts only Matroska: an MP4 (3D video, or audio Matroska cannot carry) gets no sample. """ + if source.suffix.lower() != ".mkv": + self.log.warning(f"No sample for {source.name}: mkvmerge cuts only Matroska files") + return duration = float(media_info.general_tracks[0].duration or 0) / 1000 start: float start, end = DEFAULT_SAMPLE @@ -4518,6 +4526,8 @@ class dl: } if isinstance(t, Video): row.update(width=t.width, height=t.height, range=t.range.name.lower() if t.range else "") + if t.stereo: + row["stereo"] = True elif isinstance(t, Audio): row.update(channels=str(t.channels) if t.channels else "", atmos=bool(t.joc), descriptive=t.descriptive) elif isinstance(t, Subtitle): diff --git a/unshackle/commands/env.py b/unshackle/commands/env.py index c02e6006..8405b90f 100644 --- a/unshackle/commands/env.py +++ b/unshackle/commands/env.py @@ -48,6 +48,13 @@ def get_dependencies() -> list[dict]: "desc": "MKV metadata", "cat": "Core", }, + { + "name": "MP4Box", + "binary": binaries.MP4Box, + "required": False, + "desc": "MP4 muxing", + "cat": "Core", + }, { "name": "shaka-packager", "binary": binaries.ShakaPackager, diff --git a/unshackle/core/api/handlers.py b/unshackle/core/api/handlers.py index 2450edea..fd0034a0 100644 --- a/unshackle/core/api/handlers.py +++ b/unshackle/core/api/handlers.py @@ -1744,6 +1744,7 @@ def serialize_video_track(track: Video, include_url: bool = False) -> Dict[str, "dv_compatible_bitstream": bool(getattr(track, "dv_compatible_bitstream", False)), "hdr10plus": bool(getattr(track, "hdr10plus", False)), "hdr10_base": bool(getattr(track, "hdr10_base", False)), + "stereo": bool(getattr(track, "stereo", False)), } result.update(serialize_track_common(track)) if include_url and hasattr(track, "url") and track.url: diff --git a/unshackle/core/binaries.py b/unshackle/core/binaries.py index 137e4bd2..59a2562f 100644 --- a/unshackle/core/binaries.py +++ b/unshackle/core/binaries.py @@ -79,6 +79,7 @@ __binaries = { "DoviTool": ("dovi_tool",), "HDR10PlusTool": ("hdr10plus_tool", "HDR10Plus_tool"), "Mp4decrypt": ("mp4decrypt",), + "MP4Box": ("MP4Box", "mp4box"), "Docker": ("docker",), "Git": ("git",), } @@ -226,6 +227,7 @@ __all__ = ( "DoviTool", "HDR10PlusTool", "Mp4decrypt", + "MP4Box", "Docker", "Git", "find", diff --git a/unshackle/core/config.py b/unshackle/core/config.py index 6afe6629..b1a74e8f 100644 --- a/unshackle/core/config.py +++ b/unshackle/core/config.py @@ -279,6 +279,7 @@ class Config: "video", "hdr", "hfr", + "3d", "edition", "repack", "lang_tag", diff --git a/unshackle/core/drm/clearkey_cenc.py b/unshackle/core/drm/clearkey_cenc.py index 77aaa6c0..954e142f 100644 --- a/unshackle/core/drm/clearkey_cenc.py +++ b/unshackle/core/drm/clearkey_cenc.py @@ -17,7 +17,7 @@ from unshackle.core import binaries from unshackle.core.config import config from unshackle.core.console import console from unshackle.core.drm import key_args as drm_key_args -from unshackle.core.drm.verify import kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box +from unshackle.core.drm.verify import decrypter_for, kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box from unshackle.core.utilities import log_event from unshackle.core.utils import run_proxy @@ -196,7 +196,7 @@ class ClearKeyCENC: if not path or not path.exists(): raise ValueError("Tried to decrypt a file that does not exist.") - decrypter = str(getattr(config, "decryption", "")).lower() + decrypter = decrypter_for(path) tool = "mp4decrypt" if decrypter == "mp4decrypt" else "shaka-packager" log_event( diff --git a/unshackle/core/drm/playready.py b/unshackle/core/drm/playready.py index 20c880ab..759697d1 100644 --- a/unshackle/core/drm/playready.py +++ b/unshackle/core/drm/playready.py @@ -25,7 +25,7 @@ from unshackle.core.config import config from unshackle.core.console import console from unshackle.core.constants import AnyTrack from unshackle.core.drm import key_args as drm_key_args -from unshackle.core.drm.verify import kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box +from unshackle.core.drm.verify import decrypter_for, kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box from unshackle.core.session import AnySession, utf8_text from unshackle.core.utilities import call_licence, get_boxes, log_event from unshackle.core.utils import run_proxy @@ -557,7 +557,7 @@ class PlayReady: if not path or not path.exists(): raise ValueError("Tried to decrypt a file that does not exist.") - decrypter = str(getattr(config, "decryption", "")).lower() + decrypter = decrypter_for(path) tool = "mp4decrypt" if decrypter == "mp4decrypt" else "shaka-packager" log_event( diff --git a/unshackle/core/drm/verify.py b/unshackle/core/drm/verify.py index 9dfb3ab7..2706d39c 100644 --- a/unshackle/core/drm/verify.py +++ b/unshackle/core/drm/verify.py @@ -5,6 +5,8 @@ from pathlib import Path from typing import Any, BinaryIO, Callable, Iterator, NamedTuple, Optional from uuid import UUID +from unshackle.core.config import config + log = logging.getLogger(__name__) DECRYPT_HOOK: Optional[Callable[..., None]] = None @@ -221,6 +223,37 @@ def kidless_track_kids(data: bytes) -> dict[int, UUID]: return kids +def find_child_box(buf: bytes, box_type: bytes) -> Optional[bytes]: + """Return the first box of the given type nested anywhere in buf, header included.""" + i = buf.find(box_type, 4) + while i != -1: + size = int.from_bytes(buf[i - 4 : i], "big") + if 8 <= size <= len(buf) - (i - 4): + return buf[i - 4 : i - 4 + size] + i = buf.find(box_type, i + 1) + return None + + +def has_second_view(path: Path) -> bool: + """True if the MP4 holds a second video layer (an ``lhvC`` box, as stereoscopic MV-HEVC has).""" + try: + return find_child_box(read_top_level_box(path, b"moov") or b"", b"lhvC") is not None + except OSError: + return False + + +def decrypter_for(path: Path) -> str: + """The configured decrypter, or mp4decrypt for a multi-layer (MV-HEVC) track. + + shaka-packager refuses a version 3 dvvC (Dolby Vision profile 20) and, before v3.5.0, drops lhvC and vexu. + """ + configured = str(getattr(config, "decryption", "")).lower() + if configured != "mp4decrypt" and has_second_view(path): + log.debug(f"Decrypting {path.name} with mp4decrypt: it holds a second video layer (lhvC)") + return "mp4decrypt" + return configured + + def mp4decrypt_or_shaka(drm: Any, path: Path) -> None: """Decrypt ``path`` with mp4decrypt, and again with shaka-packager when mp4decrypt copied it through. diff --git a/unshackle/core/drm/widevine.py b/unshackle/core/drm/widevine.py index deea9cb6..f4127a02 100644 --- a/unshackle/core/drm/widevine.py +++ b/unshackle/core/drm/widevine.py @@ -21,7 +21,7 @@ from unshackle.core.config import config from unshackle.core.console import console from unshackle.core.constants import AnyTrack from unshackle.core.drm import key_args as drm_key_args -from unshackle.core.drm.verify import kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box +from unshackle.core.drm.verify import decrypter_for, kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box from unshackle.core.session import AnySession, utf8_text from unshackle.core.utilities import call_licence, get_boxes, log_event from unshackle.core.utils import run_proxy @@ -307,7 +307,7 @@ class Widevine: if not path or not path.exists(): raise ValueError("Tried to decrypt a file that does not exist.") - decrypter = str(getattr(config, "decryption", "")).lower() + decrypter = decrypter_for(path) tool = "mp4decrypt" if decrypter == "mp4decrypt" else "shaka-packager" log_event( diff --git a/unshackle/core/import_service.py b/unshackle/core/import_service.py index c4587ead..3921eefd 100644 --- a/unshackle/core/import_service.py +++ b/unshackle/core/import_service.py @@ -562,6 +562,7 @@ class ImportService: width=int_or_none(row.get("width")), height=int_or_none(row.get("height")), fps=row.get("fps") or None, + stereo=bool(row.get("stereo")), ) else: track = Audio( diff --git a/unshackle/core/manifests/dash.py b/unshackle/core/manifests/dash.py index 02dc425b..4d4e3f74 100644 --- a/unshackle/core/manifests/dash.py +++ b/unshackle/core/manifests/dash.py @@ -333,6 +333,7 @@ class DASH: video_args["dv_compatible_bitstream"] = Video.is_dv_composite( track_codec, codecs, get("supplementalCodecs") ) + video_args["stereo"] = Video.is_stereo(codecs, get("supplementalCodecs")) elif content_type == "audio": track_type = Audio track_codec = Audio.Codec.from_codecs(codecs or "") diff --git a/unshackle/core/manifests/hls.py b/unshackle/core/manifests/hls.py index ccb535ab..20c0306c 100644 --- a/unshackle/core/manifests/hls.py +++ b/unshackle/core/manifests/hls.py @@ -222,7 +222,11 @@ class HLS: supplemental_codecs = self.supplemental_codecs_by_uri() - for playlist in self.manifest.playlists: + video_layouts = [ + (raw.get("stream_info") or {}).get("req_video_layout") + for raw in (getattr(self.manifest, "data", None) or {}).get("playlists", []) + ] + for playlist, video_layout in zip(self.manifest.playlists, video_layouts): video_codec: Optional[Video.Codec] = None primary_track_type: type[Audio] | type[Video] try: @@ -272,6 +276,9 @@ class HLS: (playlist.stream_info.closed_captions or "").strip('"'), [] ), dv_compatible_bitstream=dv_compatible_bitstream, + stereo=Video.is_stereo( + playlist.stream_info.codecs, supplemental_codecs.get(playlist.uri), video_layout + ), ) if primary_track_type is Video else {} diff --git a/unshackle/core/remote_service.py b/unshackle/core/remote_service.py index 84b43a30..1fd977d9 100644 --- a/unshackle/core/remote_service.py +++ b/unshackle/core/remote_service.py @@ -119,7 +119,7 @@ class RemoteError(SystemExit): """A remote call that failed: logged, then exits a CLI run like ``SystemExit(1)``. A serve relaying for a REST client catches it instead and answers with ``message``; ``kind`` is the - remote server's error code, or CONNECT / TIMEOUT / AUTH when the session could not be reached or opened. + remote server's error code, or CONNECT / TIMEOUT / AUTH when the remote session could not be reached or opened. (Not ``code``: SystemExit's ``code`` is the exit status.) """ @@ -132,7 +132,7 @@ class RemoteError(SystemExit): @property def session_broke(self) -> bool: - """Whether the remote session itself is gone, not just this one call.""" + """Whether the remote session itself is gone, not only this one call.""" return self.kind in SESSION_LOST @@ -348,6 +348,7 @@ def deserialize_video(data: Dict[str, Any]) -> Video: dv_compatible_bitstream=bool(data.get("dv_compatible_bitstream", False)), hdr10plus=bool(data.get("hdr10plus", False)), hdr10_base=bool(data.get("hdr10_base", False)), + stereo=bool(data.get("stereo", False)), ) diff --git a/unshackle/core/titles/title.py b/unshackle/core/titles/title.py index 3ad1006f..2fbf913b 100644 --- a/unshackle/core/titles/title.py +++ b/unshackle/core/titles/title.py @@ -106,6 +106,7 @@ class Title: "video": "", "hdr": "", "hfr": "", + "3d": "", "edition": "", "lang_tag": "", "title_type": {"Movie": "movie", "Episode": "series", "Song": "music"}.get( @@ -163,6 +164,7 @@ class Title: frame_rate = float(primary_video_track.frame_rate) if primary_video_track.frame_rate else 0.0 context["hfr"] = "HFR" if frame_rate > 30 else "" + context["3d"] = "3D" if any(v.stereo for v in self.tracks.videos[:1]) else "" if primary_audio_track: codec = primary_audio_track.format diff --git a/unshackle/core/tracks/track.py b/unshackle/core/tracks/track.py index 9968c034..50faf223 100644 --- a/unshackle/core/tracks/track.py +++ b/unshackle/core/tracks/track.py @@ -30,6 +30,7 @@ from unshackle.core.downloaders.requests import requests from unshackle.core.drm import DRM_T, BlowfishStripe, ClearKeyCENC, PlayReady, Widevine from unshackle.core.drm.verify import ( decrypt_track, + find_child_box, has_encrypted_sample_entry, read_top_level_box, senc_protects_samples, @@ -114,17 +115,6 @@ def iter_top_level_boxes(path: Path) -> Iterable[tuple[bytes, int, int]]: offset += size -def find_child_box(buf: bytes, box_type: bytes) -> Optional[bytes]: - """Return the first box of the given type nested anywhere in buf, header included.""" - i = buf.find(box_type, 4) - while i != -1: - size = int.from_bytes(buf[i - 4 : i], "big") - if 8 <= size <= len(buf) - (i - 4): - return buf[i - 4 : i - 4 + size] - i = buf.find(box_type, i + 1) - return None - - def strip_duplicate_init_boxes(src: Path, dst: Path) -> int: """Write src to dst without its repeated ftyp/moov pairs, returning how many it dropped. diff --git a/unshackle/core/tracks/tracks.py b/unshackle/core/tracks/tracks.py index 77b127fe..f04749eb 100644 --- a/unshackle/core/tracks/tracks.py +++ b/unshackle/core/tracks/tracks.py @@ -21,6 +21,7 @@ from unshackle.core import binaries from unshackle.core.config import config from unshackle.core.console import GradientPulseBarColumn, console from unshackle.core.constants import AnyTrack, TrackT +from unshackle.core.drm.verify import children, has_second_view, read_top_level_box from unshackle.core.events import events from unshackle.core.tracks.attachment import Attachment from unshackle.core.tracks.audio import Audio @@ -68,6 +69,12 @@ def file_track_ids(path: Path) -> dict[str, int]: return ids or {"video": 0, "audio": 1} +def mp4_track_count(path: Path) -> int: + """How many tracks ``MP4Box -add`` takes from a file: each ``trak`` of an MP4, or 1 for any other file.""" + moov = read_top_level_box(path, b"moov") or b"" + return sum(1 for kind, _, _ in children(moov, 8) if kind == b"trak") or 1 + + class Tracks: """ Video, Audio, Subtitle, Chapter, and Attachment Track Store. @@ -287,7 +294,17 @@ class Tracks: def signature(track: AnyTrack) -> tuple: """The attributes that listing and selection read. Two tracks with one signature look and select the same.""" if isinstance(track, Video): - return Video, track.codec, track.range, track.width, track.height, track.fps, track.bitrate, track.name + return ( + Video, + track.codec, + track.range, + track.stereo, + track.width, + track.height, + track.fps, + track.bitrate, + track.name, + ) if isinstance(track, Audio): return ( Audio, @@ -333,11 +350,13 @@ class Tracks: def sort_videos( self, by_language: Optional[Sequence[Union[str, Language]]] = None, exact_match: bool = False ) -> None: - """Sort video tracks by resolution then bitrate, and optionally language.""" + """Sort video tracks by resolution, 2D before 3D, then bitrate, and optionally language. + + A 3D track carries two views, so its higher bitrate would otherwise rank it over the 2D one. + """ if not self.videos: return - # resolution first, then bitrate (unknown-bitrate tracks still rank by resolution) - self.videos.sort(key=lambda x: (x.height or 0, float(x.bitrate or 0.0)), reverse=True) + self.videos.sort(key=lambda x: (x.height or 0, not x.stereo, float(x.bitrate or 0.0)), reverse=True) for language in reversed(by_language or []): if str(language) in ("all", "best"): language = self._original_language(self.videos, exact_match) @@ -486,10 +505,10 @@ class Tracks: HDR10/HDR10+/DV tracks are hybrid ingredients. They only enter the standalone pool when the user explicitly requested their range alongside HYBRID, so for example `-r HYBRID` muxes only the hybrid while `-r HYBRID,HDR10P` also delivers - HDR10+. + HDR10+. A 3D track is never an ingredient: dovi_tool does not handle a second view. """ ingredient_ranges = (Video.Range.HDR10, Video.Range.HDR10P, Video.Range.DV) - hybrid_candidates = [v for v in videos if v.range in ingredient_ranges] + hybrid_candidates = [v for v in videos if v.range in ingredient_ranges and not v.stereo] non_hybrid = [v for v in videos if v.range not in ingredient_ranges or v.range in non_hybrid_ranges] return hybrid_candidates, non_hybrid @@ -654,19 +673,21 @@ class Tracks: """ Multiplex the Tracks into an MP4 Container with MP4Box. - Only used for a codec Matroska cannot carry, DTS-UHD or MPEG-H, because MP4 carries - far less: it drops the forced, hearing-impaired and original flags, holds no - attachments at all, and leaves out a subtitle it has no format for rather than - mangling it. Losing those beats shipping audio no player can name. + Only used for what Matroska cannot carry: DTS-UHD or MPEG-H audio, and stereoscopic 3D + (MV-HEVC) video, whose second view mkvmerge drops. MP4 carries far less: it drops the + forced, hearing-impaired and original flags, holds no attachments at all, and leaves + out a subtitle it has no format for rather than mangling it. Losing those beats + shipping audio no player can name or a 3D film as 2D. Returns the same (path, returncode, errors) shape as the Matroska mux. """ mp4box = getattr(binaries, "MP4Box", None) if not mp4box: - raise RuntimeError("MP4Box (GPAC) not found, muxing DTS-UHD or MPEG-H audio needs it") + raise RuntimeError("MP4Box (GPAC) not found, muxing DTS-UHD or MPEG-H audio or 3D video needs it") progress = progress or partial(lambda **kwargs: None) + stereo = any(v.stereo for v in self.videos) cl = [str(mp4box)] sample_base = self.sample_base names: list[str] = [] @@ -685,7 +706,6 @@ class Tracks: ) continue events.emit(events.Types.TRACK_MULTIPLEX, track=track) - track_id += 1 source = track.path candidate = config.directories.temp / f"mp4mux_{track.id}_{os.getpid()}_{threading.get_ident()}.mp4" @@ -706,7 +726,8 @@ class Tracks: cl.extend(["-add", f"{source}:lang={track.mux_language}" + (f":delay={delay}" if delay else "")]) name = "" if isinstance(track, Video) else (track.get_track_name() or "") if name: - names.extend(["-name", f"{track_id}={name}"]) + names.extend(["-name", f"{track_id + 1}={name}"]) + track_id += mp4_track_count(source) if not track_id: raise ValueError("No tracks provided, at least one track must be provided.") @@ -719,7 +740,7 @@ class Tracks: title=sanitize_filename(title), random=f"{self.chapters.id}_{os.getpid()}_{threading.get_ident()}" ) self.chapters.dump(chapters_path, fallback_name=config.chapter_fallback_name) - cl.extend(["-chap", str(chapters_path)]) + cl.extend(["-chapqt" if stereo else "-chap", str(chapters_path)]) if self.attachments: log_event( @@ -749,7 +770,9 @@ class Tracks: "muxer": str(mp4box), "command": full_command, "container": "mp4", - "reason": "DTS-UHD or MPEG-H audio has no Matroska CodecID", + "reason": "3D video has no Matroska mapping" + if stereo + else "DTS-UHD or MPEG-H audio has no Matroska CodecID", }, ) @@ -778,7 +801,7 @@ class Tracks: returncode = p.wait() if returncode < 2 and any(MP4BOX_OPEN_FAILED.search(line) for line in errors): returncode = 2 - if config.sample and returncode == 0: + if config.sample and returncode == 0 and not stereo: start, end = (seconds - sample_base for seconds in config.sample) cut_path = output_path.with_suffix(".cut.mp4") try: @@ -791,6 +814,8 @@ class Tracks: except RuntimeError as e: returncode = 2 errors.append(f"MP4Box could not cut the sample window: {e}") + if stereo and returncode == 0 and not has_second_view(output_path): + logging.getLogger("Tracks").warning(f"The 3D mux of {output_path.name} lost its second view") mux_duration_ms = round((time.monotonic() - mux_start_time) * 1000, 1) output_size = output_path.stat().st_size if output_path.exists() else 0 @@ -868,7 +893,9 @@ class Tracks: video.needs_repack = True video.data["audio_language"] = lang_code - if any(at.path and at.path.exists() and has_mp4_only_audio_entry(at.path) for at in self.audio): + if any(v.stereo for v in self.videos) or any( + at.path and at.path.exists() and has_mp4_only_audio_entry(at.path) for at in self.audio + ): return self._mux_mp4( title=title, delete=delete, diff --git a/unshackle/core/tracks/video.py b/unshackle/core/tracks/video.py index dcc76b9d..be2221db 100644 --- a/unshackle/core/tracks/video.py +++ b/unshackle/core/tracks/video.py @@ -15,6 +15,7 @@ from langcodes import Language from unshackle.core import binaries from unshackle.core.config import config +from unshackle.core.drm.verify import has_second_view from unshackle.core.tracks.subtitle import Subtitle from unshackle.core.tracks.track import Track from unshackle.core.utilities import FPS, get_boxes, log_event @@ -228,6 +229,7 @@ class Video(Track): dv_compatible_bitstream: bool = False, hdr10plus: bool = False, hdr10_base: bool = False, + stereo: bool = False, **kwargs: Any, ) -> None: """ @@ -248,6 +250,7 @@ class Video(Track): All strings will be cast to either a number or float. hdr10plus: The DV track also carries HDR10+ metadata. The HDR10+ probe sets it. hdr10_base: The DV track has an HDR10 base layer, as DV profile 8.1 does. + stereo: The track is stereoscopic 3D (MV-HEVC), with a second view for the other eye. Note: If codec, bitrate, width, height, or fps is not specified, unshackle can skip some checks or assume a value. Give as much information as possible. @@ -298,6 +301,7 @@ class Video(Track): self.dv_compatible_bitstream = dv_compatible_bitstream self.hdr10plus = hdr10plus self.hdr10_base = hdr10_base + self.stereo = stereo self.hybrid_base_only = False self.native_hybrid = False @@ -315,6 +319,7 @@ class Video(Track): "dv_compatible_bitstream": self.dv_compatible_bitstream, "hdr10plus": self.hdr10plus, "hdr10_base": self.hdr10_base, + "stereo": self.stereo, } ) return data @@ -334,6 +339,7 @@ class Video(Track): dv_compatible_bitstream=data.get("dv_compatible_bitstream", False), hdr10plus=data.get("hdr10plus", False), hdr10_base=data.get("hdr10_base", False), + stereo=data.get("stereo", False), ) @staticmethod @@ -347,6 +353,16 @@ class Video(Track): """Whether a codecs list names a Dolby Vision codec.""" return bool(Video.dv_fourccs(codecs)) + @staticmethod + def is_stereo( + codecs: Optional[str], supplemental_codecs: Optional[str] = None, video_layout: Optional[str] = None + ) -> bool: + """Whether a track is stereoscopic 3D: the HLS REQ-VIDEO-LAYOUT names CH-STEREO, or a codec is DV profile 20.""" + tokens = re.split(r"[\s,]+", f"{codecs or ''},{supplemental_codecs or ''}".lower()) + return "CH-STEREO" in (video_layout or "").upper() or any( + re.match(r"dvh[1e]\.20(\.|$)", token) for token in tokens + ) + @staticmethod def is_dv_composite( codec: Optional[Video.Codec], codecs: Optional[str], supplemental_codecs: Optional[str] @@ -377,12 +393,15 @@ class Video(Track): return f"{label}+DV" if self.dv_compatible_bitstream and self.range != Video.Range.DV else label def __str__(self) -> str: + labels = filter( + None, [self.codec.value if self.codec else None, self.range_label, "3D" if self.stereo else None] + ) return " | ".join( filter( None, [ "VID", - "[" + (", ".join(filter(None, [self.codec.value if self.codec else None, self.range_label]))) + "]", + f"[{', '.join(labels)}]", str(self.language), ", ".join( filter( @@ -459,16 +478,34 @@ class Video(Track): self.path = output_path original_path.unlink() + def download(self, *args: Any, **kwargs: Any) -> None: + """Download the track and mark it 3D when the file holds a second view; a 3D track is never repacked.""" + super().download(*args, **kwargs) + if not self.stereo and self.path and has_second_view(self.path): + self.stereo = True + if self.stereo: + self.needs_repack = False + + def repackage(self, bsf_v: Optional[str] = None) -> bool: + """Remux with FFmpeg, except a 3D track: an FFmpeg remux rewrites its stereo and DV boxes.""" + if self.stereo: + logging.getLogger("Video").warning( + f"Skipped the FFmpeg repack of 3D track {self.id}: FFmpeg rewrites its stereo boxes" + ) + return False + return super().repackage(bsf_v) + def vui_bsf(self) -> Optional[str]: """Return the ``-bsf:v`` value that rewrites SPS VUI colour metadata to match ``self.range``. - Returns None when a rewrite is not necessary: SDR/DV/HYBRID ranges, non-AVC/HEVC codecs, a - missing file, or when the bitstream already ships the correct colour metadata. Otherwise + Returns None when a rewrite is not necessary or not safe: SDR/DV/HYBRID ranges, non-AVC/HEVC + codecs, a 3D track (an FFmpeg remux rewrites its stereo and DV boxes), a missing file, or when the + bitstream already ships the correct colour metadata. Otherwise returns a ``{h264,hevc}_metadata=...`` string suitable for FFmpeg ``-bsf:v``. """ if not self.path or not self.path.exists(): return None - if self.codec not in (Video.Codec.AVC, Video.Codec.HEVC): + if self.codec not in (Video.Codec.AVC, Video.Codec.HEVC) or self.stereo: return None if self.range in (Video.Range.SDR, Video.Range.DV, Video.Range.HYBRID): return None diff --git a/unshackle/unshackle-example.yaml b/unshackle/unshackle-example.yaml index b8c9a1b6..a3553515 100644 --- a/unshackle/unshackle-example.yaml +++ b/unshackle/unshackle-example.yaml @@ -22,12 +22,12 @@ tag: "NOGRP" # ? suffix, e.g. {year?}. output_template: # Scene-style naming (dot-separated) - movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" - series: "{title}.{year?}.{season_episode}.{episode_name?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" + movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" + series: "{title}.{year?}.{season_episode}.{episode_name?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" songs: "{track_number}.{title}.{repack?}.{edition?}.{lang_tag?}.{source?}.WEB-DL.{audio_full}.{atmos?}-{tag}" folder: - movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" - series: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" + movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" + series: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}" albums: "{album_artist} - {album} ({year?})" #