mirror of
https://github.com/unshackle-dl/unshackle.git
synced 2026-11-12 12:05:07 +00:00
feat(tracks): download and mux stereoscopic 3D (MV-HEVC) video to MP4
Mark 3D from REQ-VIDEO-LAYOUT, DV profile 20 codecs or an lhvC box; mux with MP4Box, decrypt with mp4decrypt, skip FFmpeg repack, sample cut and hybrid; add {3d} and env MP4Box.
This commit is contained in:
imSp4rky
@@ -339,7 +339,7 @@ When you give both `season` and `episode`, the server combines them into a `"{se
|
||||
"bitrate": 8000, "bitrate_bps": 8000000, "width": 3840, "height": 2160,
|
||||
"resolution": "2160p", "fps": "24", "range": "HDR10",
|
||||
"range_display": "HDR", "scan_type": null, "closed_captions": [],
|
||||
"dv_compatible_bitstream": false, "hdr10plus": false, "hdr10_base": false, "language": "en",
|
||||
"dv_compatible_bitstream": false, "hdr10plus": false, "hdr10_base": false, "stereo": false, "language": "en",
|
||||
"is_original_lang": true,
|
||||
"name": null, "track_name": "English", "needs_repack": false, "edition": [], "note": null, "data": null,
|
||||
"drm": [], "descriptor": "DASH"
|
||||
|
||||
@@ -454,6 +454,7 @@ Two DV mechanisms sit on top of the track model. **DV fixup** re-exposes DV sign
|
||||
HLS and DASH streams that carry a valid DV RPU but advertise plain HEVC, with DV only in the
|
||||
supplemental codecs (`dv_compatible_bitstream`). `--list` shows such a track as `HDR10+DV` or `HDR10P+DV`.
|
||||
A service sets `hdr10_base` on a DV track that has an HDR10 base layer (DV profile 8.1), and `--list` shows it as `DV+HDR10`.
|
||||
The HLS and DASH parsers set `stereo` on a stereoscopic 3D (MV-HEVC) track: an HLS `REQ-VIDEO-LAYOUT` that names `CH-STEREO`, or a DV profile 20 codec. `--list` shows it as `3D`.
|
||||
**Hybrid** processing (`Video.Range.HYBRID`) selects an HDR10(+) base and a DV donor,
|
||||
extracts the RPU with `dovi_tool`, and injects it onto the base layer, to make a single
|
||||
track that carries both HDR10 metadata and the DV RPU. Both are automatic once the
|
||||
|
||||
@@ -268,7 +268,7 @@ download. For example, `dovi_tool` only matters if you download Dolby Vision vid
|
||||
| Tool | Binary looked up | Why it is needed |
|
||||
| --- | --- | --- |
|
||||
| Bento4 (`mp4decrypt`) | `mp4decrypt` | Alternative DRM decryptor to shaka-packager |
|
||||
| MP4Box (GPAC) | `MP4Box` | Muxing DTS:X Profile 2 (DTS-UHD) and MPEG-H audio |
|
||||
| MP4Box (GPAC) | `MP4Box` | Muxing DTS:X Profile 2 (DTS-UHD) and MPEG-H audio, and 3D video |
|
||||
| dovi_tool | `dovi_tool` | Dolby Vision metadata handling |
|
||||
| HDR10Plus_tool | `hdr10plus_tool` | HDR10+ metadata handling |
|
||||
| SubtitleEdit | `seconv`, `SubtitleEdit` | Subtitle conversion (the `SeConv` CLI from 5.x) |
|
||||
@@ -280,9 +280,9 @@ download. For example, `dovi_tool` only matters if you download Dolby Vision vid
|
||||
| Docker | `docker` | Gluetun VPN proxy support |
|
||||
| Git | `git` | Fetching and updating remote service repositories |
|
||||
|
||||
!!! info "MP4Box is only used for DTS:X Profile 2 and MPEG-H"
|
||||
Titles with DTS:X Profile 2 (DTS-UHD) or MPEG-H audio mux to MP4; everything else muxes to MKV
|
||||
with `mkvmerge`. Install with `apt install gpac`.
|
||||
!!! info "MP4Box is only used for DTS:X Profile 2, MPEG-H and 3D video"
|
||||
Titles with DTS:X Profile 2 (DTS-UHD) or MPEG-H audio, or a stereoscopic 3D video track, mux to
|
||||
MP4; everything else muxes to MKV with `mkvmerge`. Install with `apt install gpac`.
|
||||
|
||||
!!! tip "SubtitleEdit specifics"
|
||||
unshackle looks for `seconv` first, the batch CLI shipped with
|
||||
|
||||
@@ -164,6 +164,10 @@ unshackle dl -q 2160 -r HDR10,DV EXAMPLE '...'
|
||||
|
||||
The output file of a track with a Dolby Vision profile 8 RPU plays as Dolby Vision on a DV screen.
|
||||
|
||||
`--list` adds `3D` to a stereoscopic 3D (MV-HEVC) track, for example `[H.265, DV, 3D]`. When a title
|
||||
has a 2D and a 3D track at the same resolution, unshackle selects the 2D track. A 3D title muxes to
|
||||
MP4, see [3D output](output-and-naming.md#3d-output).
|
||||
|
||||
!!! warning "HYBRID requires dovi_tool"
|
||||
`-r HYBRID` makes a single hybrid track by injecting the Dolby Vision RPU
|
||||
metadata onto an HDR10/HDR10+ base layer with **dovi_tool**.
|
||||
|
||||
@@ -15,7 +15,7 @@ When a download finishes, unshackle builds the final path from three pieces:
|
||||
- **Output directory**: `directories.downloads` from your config, or whatever you pass to `-o/--output` on the command line for a single run.
|
||||
- **Folder template**: an optional per-title subfolder. unshackle **always** puts TV episodes and songs in a folder. Movies only get one if you define a `movies` folder template, or a single-string `folder` template that applies to all title kinds (see [Folder templates](#folder-templates)).
|
||||
- **Filename template**: the `output_template` for the title kind (movie, series, or song).
|
||||
- **Extension**: chosen by the muxer for a movie or an episode: `.mkv` for video, `.mka` for audio-only, `.mks` for subtitle-only, `.mp4` for [DTS:X Profile 2 and MPEG-H audio](#dtsx-profile-2-and-mpeg-h-output). unshackle does not mux a song. It remuxes a song into the container of its codec, such as `.flac`, `.opus`, or `.m4a`. Read [Music output files](#music-output-files).
|
||||
- **Extension**: chosen by the muxer for a movie or an episode: `.mkv` for video, `.mka` for audio-only, `.mks` for subtitle-only, `.mp4` for [DTS:X Profile 2 and MPEG-H audio](#dtsx-profile-2-and-mpeg-h-output) and [3D video](#3d-output). unshackle does not mux a song. It remuxes a song into the container of its codec, such as `.flac`, `.opus`, or `.m4a`. Read [Music output files](#music-output-files).
|
||||
|
||||
!!! example "A typical episode path"
|
||||
```text
|
||||
@@ -110,6 +110,7 @@ Every variable below is valid in both output and folder templates. unshackle tak
|
||||
| `video` | Video codec | `H.264`, `H.265` |
|
||||
| `hdr` | Dynamic-range label | `HDR`, `HDR10P`, `DV`, `DV.HDR`, `DV.HDR10P`, `HLG` |
|
||||
| `hfr` | High frame rate marker (frame rate > 30) | `HFR` |
|
||||
| `3d` | Stereoscopic 3D marker (MV-HEVC video) | `3D` |
|
||||
| `edition` | Edition label from the track, if any | `Directors Cut` |
|
||||
| `tag` | Release-group tag (from `config.tag` or `--tag`) | `TAG` |
|
||||
| `repack` | `REPACK` when the `--repack` flag is used | `REPACK` |
|
||||
@@ -429,6 +430,23 @@ An MP4 title loses:
|
||||
- SubStation Alpha (`.ass`) subtitles
|
||||
- container tags (`mkvpropedit`)
|
||||
|
||||
### 3D output
|
||||
|
||||
A stereoscopic 3D (MV-HEVC) video track also muxes to MP4 with MP4Box, because Matroska has no
|
||||
mapping for its second view and `mkvmerge` drops it. There is no 3D Matroska output. unshackle
|
||||
finds a 3D track from the manifest, or from a second video layer in the downloaded file. The MP4
|
||||
title loses the same items as above. For a 3D title, unshackle also:
|
||||
|
||||
- decrypts the track with mp4decrypt, whatever [`decryption`](../reference/configuration/drm.md#decryption) selects
|
||||
- writes QuickTime chapters
|
||||
- keeps the whole downloaded segments of a `--sample` run, because the MP4Box cut removes the 3D signalling
|
||||
- does not repack the track or rewrite its colour metadata with FFmpeg
|
||||
- writes no `--include-sample` sample, because `mkvmerge` cuts only Matroska files
|
||||
- never uses the track in a `HYBRID` merge
|
||||
|
||||
A Dolby Vision profile 20 track has no HDR10 base layer, so a player without Dolby Vision shows
|
||||
wrong colours.
|
||||
|
||||
You configure muxing (the combination of video, audio, subtitle, chapter and attachment tracks into a single Matroska file with `mkvmerge`) under the `muxing` config key. unshackle muxes only movies and episodes. Read [Music output files](#music-output-files) for what a music download does instead.
|
||||
|
||||
```yaml title="unshackle.yaml"
|
||||
@@ -475,6 +493,7 @@ The `--no-mux` flag skips muxing entirely and writes the individual track files,
|
||||
[`--sample`](cli-reference.md#output-muxing-files) and `--include-sample` write a sample into a
|
||||
`Sample` folder inside the output folder of the title. The sample keeps the filename of the
|
||||
title, with a `SAMPLE` tag at the end. The tag uses the separator of the output template.
|
||||
`--include-sample` cuts the sample with `mkvmerge`, so a title that muxes to MP4 gets no sample.
|
||||
|
||||
```text title="A movie with --include-sample"
|
||||
Some.Movie.2026.1080p.EXAMPLE.WEB-DL.H.264-TAG/
|
||||
|
||||
@@ -187,6 +187,10 @@ service tag case-insensitively and uses the `default` entry as the fallback.
|
||||
|
||||
unshackle decrypts HLS AES-128 ClearKey in-process and ignores this setting.
|
||||
|
||||
unshackle decrypts a stereoscopic 3D (MV-HEVC) track with mp4decrypt whatever this setting says.
|
||||
shaka-packager refuses its Dolby Vision configuration (version 3), and before v3.5.0 it also drops
|
||||
the boxes of the second view.
|
||||
|
||||
mp4decrypt finds a content key by the KID in the first `sinf` box of the track. When that box
|
||||
has no CENC `tenc` (for example, a PIFF `sinf` before the CENC one), unshackle also gives
|
||||
mp4decrypt the content key of that KID by track ID. When mp4decrypt copies a track through
|
||||
|
||||
@@ -40,7 +40,7 @@ folder-kind keys.
|
||||
`absolute`, `date`, `quality`, `resolution`, `source`, `tag`, `track_number`, `artist`,
|
||||
`album_artist`, `album`, `disc`, `track_total`, `disc_total`, `release_type`, `genre`,
|
||||
`explicit`, `isrc`, `upc`, `label`, `audio`, `audio_channels`, `audio_full`, `atmos`,
|
||||
`dual`, `multi`, `dubbed`, `video`, `hdr`, `hfr`, `edition`, `repack`, `lang_tag`,
|
||||
`dual`, `multi`, `dubbed`, `video`, `hdr`, `hfr`, `3d`, `edition`, `repack`, `lang_tag`,
|
||||
`title_type`.
|
||||
|
||||
!!! warning "`{part?}` is already inside `{season_episode}`"
|
||||
@@ -129,7 +129,7 @@ Rule semantics:
|
||||
- A value can be a scalar or a list. A list matches if **any** entry matches.
|
||||
- Comparison is case-insensitive string equality.
|
||||
- The keys of `when` are the filename template variables for the release itself:
|
||||
`title_type`, `quality`, `resolution`, `source`, `video`, `hdr`, `hfr`, `audio`,
|
||||
`title_type`, `quality`, `resolution`, `source`, `video`, `hdr`, `hfr`, `3d`, `audio`,
|
||||
`audio_channels`, `audio_full`, `atmos`, `dual`, `multi`, `dubbed`, `edition`, `repack`,
|
||||
and `lang_tag`. Only these are available. unshackle adds the title's own fields (`title`,
|
||||
`year`, `season`, `episode`, and so on) after it evaluates the rules, so a condition cannot
|
||||
|
||||
@@ -113,7 +113,7 @@ values.
|
||||
| `{year}` `{season}` `{episode}` `{episode_name}` | Season and episode are plain numbers (`1`, `5`) |
|
||||
| `{part}` | Part number of a split episode as a plain number (`1` for `Part.1`). Empty when the episode has no part |
|
||||
| `{season_episode}` `{absolute}` `{date}` | As in the filename template: `{season_episode}` keeps the part suffix (`S01E05.Part.1`) while `{episode}` drops it |
|
||||
| `{quality}` `{resolution}` `{hdr}` `{hfr}` `{vcodec}` `{acodec}` `{edition}` `{tag}` `{lang_tag}` | As in the filename template, for this file |
|
||||
| `{quality}` `{resolution}` `{hdr}` `{hfr}` `{3d}` `{vcodec}` `{acodec}` `{edition}` `{tag}` `{lang_tag}` | As in the filename template, for this file |
|
||||
| `{tmdb}` `{imdb}` `{tvdb}` | The tagging IDs in use, otherwise empty. See [Tagging IDs](#tagging-ids) |
|
||||
| `{error}` | Failure message. Empty on success |
|
||||
|
||||
@@ -132,7 +132,7 @@ Every variable from your `output_template` is available here as well, including
|
||||
music has none of them.
|
||||
- **The naming context renders it empty on purpose.** `{hdr}` is empty for an SDR
|
||||
file, because SDR is the absence of an HDR tag in a filename rather than a tag that
|
||||
reads `SDR`. `{edition}`, `{atmos}`, `{multi}`, `{dual}` and `{hfr}` behave the same
|
||||
reads `SDR`. `{edition}`, `{atmos}`, `{multi}`, `{dual}`, `{hfr}` and `{3d}` behave the same
|
||||
way. If your script needs the word `SDR`, derive it from an empty `{hdr}` yourself.
|
||||
- The post-script is a `failure` one. A failed download has no output file to read metadata
|
||||
from, so everything that comes from the output file is empty. See
|
||||
|
||||
@@ -476,3 +476,16 @@ def test_import_keeps_every_period_when_no_exported_period_is_left(
|
||||
|
||||
assert video.data["dash"]["filtered_period_ids"] == []
|
||||
assert "keeping every period" in caplog.text
|
||||
|
||||
|
||||
def test_a_3d_video_stays_3d_through_export_and_import(tmp_path: Path) -> None:
|
||||
export = tmp_path / "export.json"
|
||||
title = make_title()
|
||||
title.tracks.videos[0].stereo = True
|
||||
runner = make_dl()
|
||||
for track in [*title.tracks.videos, *title.tracks.audio]:
|
||||
runner.write_export(export, title, track)
|
||||
assert [row.get("stereo") for row in entry(export)["tracks"]][0] is True
|
||||
svc = ImportService(import_ctx(), "EXAMPLE", "movie-1", str(export))
|
||||
movie = next(iter(svc.get_titles()))
|
||||
assert svc.get_tracks(movie).videos[0].stereo
|
||||
|
||||
@@ -856,7 +856,7 @@ def video(
|
||||
codec: Optional[Video.Codec] = Video.Codec.HEVC,
|
||||
native_hybrid: bool = False,
|
||||
) -> Any:
|
||||
return SimpleNamespace(id=id_, height=height, range=range_, codec=codec, native_hybrid=native_hybrid)
|
||||
return SimpleNamespace(id=id_, height=height, range=range_, codec=codec, native_hybrid=native_hybrid, stereo=False)
|
||||
|
||||
|
||||
def test_hybrid_fails_with_neither_a_base_nor_dv() -> None:
|
||||
|
||||
@@ -297,3 +297,8 @@ def test_a_file_row_can_name_its_codec_and_range_by_name(tmp_path: Path) -> None
|
||||
service = file_export(tmp_path, [row])
|
||||
video = service.get_tracks(next(iter(service.get_titles()))).videos[0]
|
||||
assert (video.codec, video.range) == (Video.Codec.HEVC, Video.Range.HDR10P)
|
||||
|
||||
|
||||
def test_a_file_row_marked_stereo_imports_as_a_3d_video(tmp_path: Path) -> None:
|
||||
service = file_export(tmp_path, [{**FILE_ROWS[0], "stereo": True}])
|
||||
assert service.get_tracks(next(iter(service.get_titles()))).videos[0].stereo
|
||||
|
||||
@@ -0,0 +1,275 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import itertools
|
||||
import logging
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from langcodes import Language
|
||||
|
||||
from unshackle.commands.dl import dl, hybrid_plan, pick_video
|
||||
from unshackle.core import binaries
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.drm.verify import decrypter_for
|
||||
from unshackle.core.manifests import DASH, HLS
|
||||
from unshackle.core.titles.movie import Movie
|
||||
from unshackle.core.tracks import Audio, Chapter, Chapters, Track, Tracks, Video
|
||||
from unshackle.core.tracks import tracks as tracks_module
|
||||
|
||||
VIDEO_IDS = itertools.count()
|
||||
|
||||
MASTER = """#EXTM3U
|
||||
#EXT-X-VERSION:12
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=6786883,CODECS="dvh1.20.02",RESOLUTION=960x540,VIDEO-RANGE=PQ,REQ-VIDEO-LAYOUT="CH-MONO,CH-STEREO"
|
||||
stereo.m3u8
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=5000000,CODECS="hvc1.2.4.L150.90",RESOLUTION=1920x1080,REQ-VIDEO-LAYOUT="CH-STEREO"
|
||||
mvhevc.m3u8
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=4000000,CODECS="dvh1.08.06",RESOLUTION=1920x1080,VIDEO-RANGE=PQ
|
||||
flat.m3u8
|
||||
"""
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("codecs", "layout", "expected"),
|
||||
[
|
||||
("dvh1.20.06,ec-3", None, True),
|
||||
("dvhe.20.09", None, True),
|
||||
("hvc1.2.4.L150.90", '"CH-MONO,CH-STEREO"', True),
|
||||
("dvh1.08.06", '"CH-MONO"', False),
|
||||
("dvh1.05.06", None, False),
|
||||
("dvh1.200.06", None, False),
|
||||
(None, None, False),
|
||||
],
|
||||
)
|
||||
def test_is_stereo(codecs: str | None, layout: str | None, expected: bool) -> None:
|
||||
assert Video.is_stereo(codecs, video_layout=layout) is expected
|
||||
|
||||
|
||||
def test_hls_marks_stereo_tracks_from_the_layout_or_the_codec() -> None:
|
||||
tracks = HLS.from_text(MASTER, url="https://cdn.example/master.m3u8").to_tracks(Language.get("en"))
|
||||
stereo = {str(v.url).rsplit("/", 1)[-1]: v.stereo for v in tracks.videos}
|
||||
assert stereo == {"stereo.m3u8": True, "mvhevc.m3u8": True, "flat.m3u8": False}
|
||||
|
||||
|
||||
def test_a_stereo_track_lists_as_3d_and_keeps_the_flag_through_to_dict() -> None:
|
||||
video = Video(url="https://x/v.mp4", language="en", codec=Video.Codec.HEVC, range_=Video.Range.DV, stereo=True)
|
||||
assert str(video).startswith("VID | [H.265, DV, 3D] |")
|
||||
assert Video.from_dict(video.to_dict()).stereo
|
||||
|
||||
|
||||
def stereo_video(tmp_path: Path, stereo: bool = True, **kwargs: Any) -> Video:
|
||||
n = next(VIDEO_IDS)
|
||||
path = tmp_path / f"v{n}.mp4"
|
||||
path.write_bytes(b"\x00" * 16)
|
||||
video = Video(url=f"https://x/v{n}.mp4", language="en", codec=Video.Codec.HEVC, stereo=stereo, **kwargs)
|
||||
video.path = path
|
||||
return video
|
||||
|
||||
|
||||
def test_dash_marks_a_dv_profile_20_representation_stereo() -> None:
|
||||
mpd = """<?xml version="1.0"?>
|
||||
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" type="static" mediaPresentationDuration="PT10S">
|
||||
<Period>
|
||||
<AdaptationSet contentType="video" mimeType="video/mp4">
|
||||
<SegmentTemplate media="$RepresentationID$/$Number$.m4s" initialization="$RepresentationID$/init.mp4" duration="2" />
|
||||
<Representation id="v3d" codecs="dvh1.20.06" bandwidth="9000000" width="1920" height="1080" />
|
||||
<Representation id="v2d" codecs="dvh1.08.06" bandwidth="5000000" width="1920" height="1080" />
|
||||
</AdaptationSet>
|
||||
</Period>
|
||||
</MPD>"""
|
||||
videos = DASH.from_text(mpd, url="https://cdn.example/manifest.mpd").to_tracks(language="en").videos
|
||||
assert {v.data["dash"]["representation"].get("id"): v.stereo for v in videos} == {"v3d": True, "v2d": False}
|
||||
|
||||
|
||||
def box(kind: bytes, payload: bytes = b"") -> bytes:
|
||||
return (8 + len(payload)).to_bytes(4, "big") + kind + payload
|
||||
|
||||
|
||||
def test_a_track_with_a_second_video_layer_decrypts_with_mp4decrypt(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
(layered := tmp_path / "3d.mp4").write_bytes(box(b"moov", box(b"lhvC")))
|
||||
(flat := tmp_path / "2d.mp4").write_bytes(box(b"moov", box(b"hvcC")))
|
||||
monkeypatch.setattr(config, "decryption", "Shaka")
|
||||
assert decrypter_for(layered) == "mp4decrypt"
|
||||
assert decrypter_for(flat) == "shaka"
|
||||
monkeypatch.setattr(config, "decryption", "MP4DECRYPT")
|
||||
assert decrypter_for(flat) == "mp4decrypt"
|
||||
|
||||
|
||||
def test_a_3d_title_muxes_with_mp4box(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
calls: list[str] = []
|
||||
monkeypatch.setattr(Tracks, "_mux_mp4", lambda self, **kwargs: calls.append(kwargs["title"]) or (Path("x"), 0, []))
|
||||
assert Tracks(stereo_video(tmp_path)).mux("Film")[1] == 0
|
||||
assert calls == ["Film"]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mp4box_commands(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> list[list[str]]:
|
||||
commands: list[list[str]] = []
|
||||
|
||||
class FakePopen:
|
||||
def __init__(self, command: list[str], **kwargs: Any) -> None:
|
||||
commands.append([str(c) for c in command])
|
||||
Path(command[-1]).write_bytes(b"mp4")
|
||||
self.stdout = io.StringIO("")
|
||||
|
||||
def wait(self) -> int:
|
||||
return 0
|
||||
|
||||
monkeypatch.setattr(binaries, "MP4Box", Path("/bin/MP4Box"), raising=False)
|
||||
monkeypatch.setattr(tracks_module.subprocess, "Popen", FakePopen)
|
||||
monkeypatch.setattr(tracks_module, "run_step", lambda command, **kwargs: commands.append([str(c) for c in command]))
|
||||
monkeypatch.setattr(config, "sample", None)
|
||||
monkeypatch.setattr(config.directories, "temp", tmp_path)
|
||||
return commands
|
||||
|
||||
|
||||
def test_a_3d_mp4_mux_keeps_the_sample_whole(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, mp4box_commands: list[list[str]]
|
||||
) -> None:
|
||||
monkeypatch.setattr(config, "sample", (120.0, 180.0))
|
||||
Tracks(stereo_video(tmp_path))._mux_mp4(title="Film", output_path=tmp_path / "out.mkv")
|
||||
assert len(mp4box_commands) == 1
|
||||
assert "-splitx" not in mp4box_commands[0]
|
||||
|
||||
|
||||
def test_a_3d_mp4_mux_writes_quicktime_chapters(tmp_path: Path, mp4box_commands: list[list[str]]) -> None:
|
||||
for stereo, flag in ((True, "-chapqt"), (False, "-chap")):
|
||||
tracks = Tracks(stereo_video(tmp_path, stereo=stereo))
|
||||
tracks.chapters = Chapters([Chapter(0.0, "Story")])
|
||||
tracks._mux_mp4(title="Film", output_path=tmp_path / f"out{stereo}.mkv")
|
||||
assert flag in mp4box_commands[-1] and ({"-chap", "-chapqt"} - {flag}).isdisjoint(mp4box_commands[-1])
|
||||
|
||||
|
||||
def test_a_3d_track_is_never_a_hybrid_ingredient(tmp_path: Path) -> None:
|
||||
flat, layered = (stereo_video(tmp_path, stereo=s, range_=Video.Range.DV) for s in (False, True))
|
||||
candidates, standalone = Tracks.partition_hybrid_videos([flat, layered], [])
|
||||
assert candidates == [flat] and standalone == []
|
||||
|
||||
|
||||
def test_2d_ranks_over_3d_at_one_resolution_and_the_signatures_differ(tmp_path: Path) -> None:
|
||||
flat = stereo_video(tmp_path, stereo=False, height=1080, bitrate=5_000_000)
|
||||
layered = stereo_video(tmp_path, stereo=True, height=1080, bitrate=9_000_000)
|
||||
uhd = stereo_video(tmp_path, stereo=True, height=2160, bitrate=40_000_000, width=3840)
|
||||
tracks = Tracks(layered, flat, uhd)
|
||||
tracks.sort_videos()
|
||||
assert tracks.videos == [uhd, flat, layered]
|
||||
assert Tracks.signature(flat) != Tracks.signature(
|
||||
stereo_video(tmp_path, stereo=True, height=1080, bitrate=5_000_000)
|
||||
)
|
||||
|
||||
|
||||
def test_a_3d_track_gets_no_ffmpeg_vui_rewrite_or_repack(tmp_path: Path) -> None:
|
||||
track = stereo_video(tmp_path, range_=Video.Range.HDR10)
|
||||
assert track.vui_bsf() is None
|
||||
assert track.repackage() is False
|
||||
assert track.path.exists()
|
||||
|
||||
|
||||
def test_the_filename_context_marks_a_3d_title(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr(config, "tag_rules", [])
|
||||
monkeypatch.setattr(config, "language_tags", {})
|
||||
|
||||
class Svc:
|
||||
pass
|
||||
|
||||
def context(stereo: bool) -> str:
|
||||
movie = Movie(id_="movie-0001", service=Svc, name="The Film", year=2024, language="en")
|
||||
movie.tracks = Tracks(Video(url="https://x/v.mp4", language="en", codec=Video.Codec.HEVC, stereo=stereo))
|
||||
video = SimpleNamespace(
|
||||
width=3840,
|
||||
height=2160,
|
||||
format="HEVC",
|
||||
scan_type="Progressive",
|
||||
hdr_format_commercial="",
|
||||
hdr_format="",
|
||||
hdr_format_profile="",
|
||||
transfer_characteristics="PQ",
|
||||
transfer_characteristics_original=None,
|
||||
frame_rate="47.952",
|
||||
)
|
||||
return str(movie.build_base_template_context(SimpleNamespace(video_tracks=[video], audio_tracks=[]))["3d"])
|
||||
|
||||
assert (context(True), context(False)) == ("3D", "")
|
||||
|
||||
|
||||
def test_worst_prefers_the_2d_track_at_one_height(tmp_path: Path) -> None:
|
||||
videos = [
|
||||
stereo_video(tmp_path, stereo=s, height=2160, bitrate=b)
|
||||
for s, b in ((False, 9_000_000), (False, 5_000_000), (True, 40_000_000), (True, 30_000_000))
|
||||
]
|
||||
ranked = Tracks(videos)
|
||||
ranked.sort_videos()
|
||||
assert pick_video(ranked.videos, worst=True) is videos[1]
|
||||
assert pick_video(ranked.videos, worst=False) is videos[0]
|
||||
assert pick_video([videos[3], videos[2]], worst=True) is videos[2]
|
||||
assert pick_video([], worst=True) is None
|
||||
|
||||
|
||||
def test_hybrid_never_pairs_a_3d_dv_track(tmp_path: Path) -> None:
|
||||
base = stereo_video(tmp_path, stereo=False, range_=Video.Range.HDR10, height=2160)
|
||||
layered = stereo_video(tmp_path, stereo=True, range_=Video.Range.DV, height=2160)
|
||||
assert hybrid_plan([base, layered], generate_dv=False) == ([], None)
|
||||
|
||||
|
||||
def test_mp4_names_skip_the_extra_tracks_of_a_3d_video_file(tmp_path: Path, mp4box_commands: list[list[str]]) -> None:
|
||||
video = stereo_video(tmp_path)
|
||||
video.path.write_bytes(box(b"moov", box(b"trak") + box(b"trak")))
|
||||
audio = Audio(url="https://x/a.mp4", language="en", codec=Audio.Codec.EC3, name="English")
|
||||
audio.path = tmp_path / "a.mp4"
|
||||
audio.path.write_bytes(b"\x00" * 16)
|
||||
Tracks([video, audio])._mux_mp4(title="Film", output_path=tmp_path / "out.mkv")
|
||||
command = mp4box_commands[0]
|
||||
assert command[command.index("-name") + 1] == "3=English"
|
||||
|
||||
|
||||
def test_a_3d_mux_that_loses_its_second_view_warns(
|
||||
tmp_path: Path, mp4box_commands: list[list[str]], caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
Tracks(stereo_video(tmp_path))._mux_mp4(title="Film", output_path=tmp_path / "out.mkv")
|
||||
assert "lost its second view" in caplog.text
|
||||
|
||||
|
||||
def test_a_downloaded_file_with_a_second_view_marks_the_video_3d(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
video = stereo_video(tmp_path, stereo=False)
|
||||
monkeypatch.setattr(Track, "download", lambda self, *a, **k: self.path.write_bytes(box(b"moov", box(b"lhvC"))))
|
||||
video.download(None, None)
|
||||
assert video.stereo
|
||||
|
||||
|
||||
def test_dash_reads_dv_profile_20_from_supplemental_codecs() -> None:
|
||||
mpd = """<?xml version="1.0"?>
|
||||
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" type="static" mediaPresentationDuration="PT10M">
|
||||
<Period id="1"><AdaptationSet contentType="video" mimeType="video/mp4" lang="en">
|
||||
<Representation id="v" codecs="hvc1.2.4.L153" supplementalCodecs="dvh1.20.09" bandwidth="1" width="3840" height="2160">
|
||||
<BaseURL>v.mp4</BaseURL><SegmentBase indexRange="0-1"/>
|
||||
</Representation>
|
||||
</AdaptationSet></Period>
|
||||
</MPD>"""
|
||||
assert DASH.from_text(mpd, url="https://cdn.example/m.mpd").to_tracks(language="en").videos[0].stereo
|
||||
|
||||
|
||||
def test_hls_reads_each_variant_layout_even_when_two_share_a_playlist() -> None:
|
||||
master = """#EXTM3U
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=2,CODECS="hvc1.2.4.L150.90",RESOLUTION=1920x1080,REQ-VIDEO-LAYOUT="CH-STEREO"
|
||||
shared.m3u8
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=1,CODECS="hvc1.2.4.L150.90",RESOLUTION=1920x1080,AUDIO="a"
|
||||
shared.m3u8
|
||||
"""
|
||||
tracks = HLS.from_text(master, url="https://cdn.example/master.m3u8").to_tracks(Language.get("en"))
|
||||
assert any(v.stereo for v in tracks.videos)
|
||||
|
||||
|
||||
def test_a_sample_is_cut_only_from_a_matroska_file(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
runner = dl.__new__(dl)
|
||||
runner.log = logging.getLogger("download")
|
||||
monkeypatch.setattr(subprocess, "run", lambda *a, **k: pytest.fail("mkvmerge ran on an MP4"))
|
||||
runner.write_sample(tmp_path / "Film.mp4", SimpleNamespace(), SimpleNamespace())
|
||||
assert not (tmp_path / "Sample").exists()
|
||||
@@ -58,6 +58,11 @@ def test_a_dv_track_keeps_its_hdr10_base_label_over_remote() -> None:
|
||||
assert deserialize_video(serialize_video_track(v)).range_label == "DV+HDR10"
|
||||
|
||||
|
||||
def test_a_stereo_track_stays_3d_over_remote() -> None:
|
||||
v = Video(url="https://x/v.mp4", language="en", codec=Video.Codec.HEVC, range_=Video.Range.DV, stereo=True)
|
||||
assert deserialize_video(serialize_video_track(v)).stereo
|
||||
|
||||
|
||||
def test_a_track_note_shows_over_remote_and_export_and_is_not_an_edition() -> None:
|
||||
from unshackle.core.tracks.track import Track
|
||||
|
||||
|
||||
+21
-11
@@ -336,18 +336,26 @@ def hybrid_gate_failure(videos: Sequence[Video], generate_dv: bool) -> Optional[
|
||||
|
||||
|
||||
def hybrid_plan(videos: Sequence[Video], generate_dv: bool) -> tuple[list[Video], Optional[Video]]:
|
||||
"""The hybrid bases and the DV track to pair with them.
|
||||
"""The hybrid bases and the 2D DV track to pair with them.
|
||||
|
||||
With no DV track, a base qualifies only with ``--generate-dv``. A resolution with a
|
||||
finished hybrid (``native_hybrid``) gets no base: that track is its hybrid.
|
||||
"""
|
||||
dv_tracks = [v for v in videos if v.range == Video.Range.DV]
|
||||
dv_tracks = [v for v in videos if v.range == Video.Range.DV and not v.stereo]
|
||||
native_heights = {v.height for v in videos if v.native_hybrid}
|
||||
base_pool = [v for v in videos if v.height not in native_heights and (dv_tracks or generate_dv)]
|
||||
matching_dv = min(dv_tracks, key=lambda v: v.height or 0) if dv_tracks else None
|
||||
return select_hybrid_bases(base_pool), matching_dv
|
||||
|
||||
|
||||
def pick_video(candidates: list[Video], worst: bool) -> Optional[Video]:
|
||||
"""The best candidate, or with ``worst`` the last one, preferring 2D: a 3D track sorts after 2D at one height."""
|
||||
if worst:
|
||||
pool = [v for v in candidates if not v.stereo] or candidates
|
||||
return pool[-1] if pool else None
|
||||
return next(iter(candidates), None)
|
||||
|
||||
|
||||
def hybrid_source_tracks(base: Video, dv: Optional[Video], base_container: Optional[Path] = None) -> list[Video]:
|
||||
"""Copies of the base and DV tracks for the hybrid step, with the base on its container file.
|
||||
|
||||
@@ -2874,7 +2882,7 @@ class dl:
|
||||
and (not codec or t.codec == codec)
|
||||
and (vlang is None or str(t.language) == vlang)
|
||||
]
|
||||
match = candidates[-1] if worst and candidates else next(iter(candidates), None)
|
||||
match = pick_video(candidates, worst)
|
||||
if match and match not in non_hybrid_selected:
|
||||
non_hybrid_selected.append(match)
|
||||
|
||||
@@ -2903,7 +2911,7 @@ class dl:
|
||||
and (not codec or t.codec == codec)
|
||||
and (vlang is None or str(t.language) == vlang)
|
||||
]
|
||||
match = candidates[-1] if worst and candidates else next(iter(candidates), None)
|
||||
match = pick_video(candidates, worst)
|
||||
if match and match not in selected_videos:
|
||||
selected_videos.append(match)
|
||||
title.tracks.videos = selected_videos
|
||||
@@ -2941,7 +2949,7 @@ class dl:
|
||||
and (not codec or t.codec == codec)
|
||||
and (vlang is None or str(t.language) == vlang)
|
||||
]
|
||||
match = candidates[-1] if worst and candidates else next(iter(candidates), None)
|
||||
match = pick_video(candidates, worst)
|
||||
if match and match not in fallback_selected:
|
||||
fallback_selected.append(match)
|
||||
title.tracks.videos = fallback_selected
|
||||
@@ -3448,13 +3456,9 @@ class dl:
|
||||
self.suggest_missing_fonts(missing_fonts)
|
||||
|
||||
# Handle DRM decryption BEFORE repacking (must decrypt first!)
|
||||
service_name = service.__class__.__name__.upper()
|
||||
decryption_method = resolve_decryption(config.decryption_map, config.decryption, service_name)
|
||||
decrypt_tool = "mp4decrypt" if decryption_method.lower() == "mp4decrypt" else "Shaka Packager"
|
||||
|
||||
drm_tracks = [track for track in title.tracks if track.drm]
|
||||
if drm_tracks:
|
||||
with console.status(f"Decrypting tracks with {decrypt_tool}..."):
|
||||
with console.status("Decrypting tracks..."):
|
||||
has_decrypted = False
|
||||
for track in drm_tracks:
|
||||
drm = track.get_drm_for_cdm(self.cdm)
|
||||
@@ -3470,7 +3474,7 @@ class dl:
|
||||
f"No matching DRM found for track {track} with CDM type {type(self.cdm).__name__}"
|
||||
)
|
||||
if has_decrypted:
|
||||
self.log.info(f"Decrypted tracks with {decrypt_tool}")
|
||||
self.log.info("Decrypted tracks")
|
||||
|
||||
if (
|
||||
not no_subs
|
||||
@@ -4050,7 +4054,11 @@ class dl:
|
||||
"""Cut a 60 second sample from a finished file into a ``Sample`` folder next to it.
|
||||
|
||||
The sample starts 2 minutes in. A file shorter than 3 minutes gives its middle 60 seconds.
|
||||
mkvmerge cuts only Matroska: an MP4 (3D video, or audio Matroska cannot carry) gets no sample.
|
||||
"""
|
||||
if source.suffix.lower() != ".mkv":
|
||||
self.log.warning(f"No sample for {source.name}: mkvmerge cuts only Matroska files")
|
||||
return
|
||||
duration = float(media_info.general_tracks[0].duration or 0) / 1000
|
||||
start: float
|
||||
start, end = DEFAULT_SAMPLE
|
||||
@@ -4518,6 +4526,8 @@ class dl:
|
||||
}
|
||||
if isinstance(t, Video):
|
||||
row.update(width=t.width, height=t.height, range=t.range.name.lower() if t.range else "")
|
||||
if t.stereo:
|
||||
row["stereo"] = True
|
||||
elif isinstance(t, Audio):
|
||||
row.update(channels=str(t.channels) if t.channels else "", atmos=bool(t.joc), descriptive=t.descriptive)
|
||||
elif isinstance(t, Subtitle):
|
||||
|
||||
@@ -48,6 +48,13 @@ def get_dependencies() -> list[dict]:
|
||||
"desc": "MKV metadata",
|
||||
"cat": "Core",
|
||||
},
|
||||
{
|
||||
"name": "MP4Box",
|
||||
"binary": binaries.MP4Box,
|
||||
"required": False,
|
||||
"desc": "MP4 muxing",
|
||||
"cat": "Core",
|
||||
},
|
||||
{
|
||||
"name": "shaka-packager",
|
||||
"binary": binaries.ShakaPackager,
|
||||
|
||||
@@ -1744,6 +1744,7 @@ def serialize_video_track(track: Video, include_url: bool = False) -> Dict[str,
|
||||
"dv_compatible_bitstream": bool(getattr(track, "dv_compatible_bitstream", False)),
|
||||
"hdr10plus": bool(getattr(track, "hdr10plus", False)),
|
||||
"hdr10_base": bool(getattr(track, "hdr10_base", False)),
|
||||
"stereo": bool(getattr(track, "stereo", False)),
|
||||
}
|
||||
result.update(serialize_track_common(track))
|
||||
if include_url and hasattr(track, "url") and track.url:
|
||||
|
||||
@@ -79,6 +79,7 @@ __binaries = {
|
||||
"DoviTool": ("dovi_tool",),
|
||||
"HDR10PlusTool": ("hdr10plus_tool", "HDR10Plus_tool"),
|
||||
"Mp4decrypt": ("mp4decrypt",),
|
||||
"MP4Box": ("MP4Box", "mp4box"),
|
||||
"Docker": ("docker",),
|
||||
"Git": ("git",),
|
||||
}
|
||||
@@ -226,6 +227,7 @@ __all__ = (
|
||||
"DoviTool",
|
||||
"HDR10PlusTool",
|
||||
"Mp4decrypt",
|
||||
"MP4Box",
|
||||
"Docker",
|
||||
"Git",
|
||||
"find",
|
||||
|
||||
@@ -279,6 +279,7 @@ class Config:
|
||||
"video",
|
||||
"hdr",
|
||||
"hfr",
|
||||
"3d",
|
||||
"edition",
|
||||
"repack",
|
||||
"lang_tag",
|
||||
|
||||
@@ -17,7 +17,7 @@ from unshackle.core import binaries
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.console import console
|
||||
from unshackle.core.drm import key_args as drm_key_args
|
||||
from unshackle.core.drm.verify import kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box
|
||||
from unshackle.core.drm.verify import decrypter_for, kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box
|
||||
from unshackle.core.utilities import log_event
|
||||
from unshackle.core.utils import run_proxy
|
||||
|
||||
@@ -196,7 +196,7 @@ class ClearKeyCENC:
|
||||
if not path or not path.exists():
|
||||
raise ValueError("Tried to decrypt a file that does not exist.")
|
||||
|
||||
decrypter = str(getattr(config, "decryption", "")).lower()
|
||||
decrypter = decrypter_for(path)
|
||||
tool = "mp4decrypt" if decrypter == "mp4decrypt" else "shaka-packager"
|
||||
|
||||
log_event(
|
||||
|
||||
@@ -25,7 +25,7 @@ from unshackle.core.config import config
|
||||
from unshackle.core.console import console
|
||||
from unshackle.core.constants import AnyTrack
|
||||
from unshackle.core.drm import key_args as drm_key_args
|
||||
from unshackle.core.drm.verify import kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box
|
||||
from unshackle.core.drm.verify import decrypter_for, kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box
|
||||
from unshackle.core.session import AnySession, utf8_text
|
||||
from unshackle.core.utilities import call_licence, get_boxes, log_event
|
||||
from unshackle.core.utils import run_proxy
|
||||
@@ -557,7 +557,7 @@ class PlayReady:
|
||||
if not path or not path.exists():
|
||||
raise ValueError("Tried to decrypt a file that does not exist.")
|
||||
|
||||
decrypter = str(getattr(config, "decryption", "")).lower()
|
||||
decrypter = decrypter_for(path)
|
||||
tool = "mp4decrypt" if decrypter == "mp4decrypt" else "shaka-packager"
|
||||
|
||||
log_event(
|
||||
|
||||
@@ -5,6 +5,8 @@ from pathlib import Path
|
||||
from typing import Any, BinaryIO, Callable, Iterator, NamedTuple, Optional
|
||||
from uuid import UUID
|
||||
|
||||
from unshackle.core.config import config
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
DECRYPT_HOOK: Optional[Callable[..., None]] = None
|
||||
@@ -221,6 +223,37 @@ def kidless_track_kids(data: bytes) -> dict[int, UUID]:
|
||||
return kids
|
||||
|
||||
|
||||
def find_child_box(buf: bytes, box_type: bytes) -> Optional[bytes]:
|
||||
"""Return the first box of the given type nested anywhere in buf, header included."""
|
||||
i = buf.find(box_type, 4)
|
||||
while i != -1:
|
||||
size = int.from_bytes(buf[i - 4 : i], "big")
|
||||
if 8 <= size <= len(buf) - (i - 4):
|
||||
return buf[i - 4 : i - 4 + size]
|
||||
i = buf.find(box_type, i + 1)
|
||||
return None
|
||||
|
||||
|
||||
def has_second_view(path: Path) -> bool:
|
||||
"""True if the MP4 holds a second video layer (an ``lhvC`` box, as stereoscopic MV-HEVC has)."""
|
||||
try:
|
||||
return find_child_box(read_top_level_box(path, b"moov") or b"", b"lhvC") is not None
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def decrypter_for(path: Path) -> str:
|
||||
"""The configured decrypter, or mp4decrypt for a multi-layer (MV-HEVC) track.
|
||||
|
||||
shaka-packager refuses a version 3 dvvC (Dolby Vision profile 20) and, before v3.5.0, drops lhvC and vexu.
|
||||
"""
|
||||
configured = str(getattr(config, "decryption", "")).lower()
|
||||
if configured != "mp4decrypt" and has_second_view(path):
|
||||
log.debug(f"Decrypting {path.name} with mp4decrypt: it holds a second video layer (lhvC)")
|
||||
return "mp4decrypt"
|
||||
return configured
|
||||
|
||||
|
||||
def mp4decrypt_or_shaka(drm: Any, path: Path) -> None:
|
||||
"""Decrypt ``path`` with mp4decrypt, and again with shaka-packager when mp4decrypt copied it through.
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ from unshackle.core.config import config
|
||||
from unshackle.core.console import console
|
||||
from unshackle.core.constants import AnyTrack
|
||||
from unshackle.core.drm import key_args as drm_key_args
|
||||
from unshackle.core.drm.verify import kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box
|
||||
from unshackle.core.drm.verify import decrypter_for, kidless_track_kids, mp4decrypt_or_shaka, read_top_level_box
|
||||
from unshackle.core.session import AnySession, utf8_text
|
||||
from unshackle.core.utilities import call_licence, get_boxes, log_event
|
||||
from unshackle.core.utils import run_proxy
|
||||
@@ -307,7 +307,7 @@ class Widevine:
|
||||
if not path or not path.exists():
|
||||
raise ValueError("Tried to decrypt a file that does not exist.")
|
||||
|
||||
decrypter = str(getattr(config, "decryption", "")).lower()
|
||||
decrypter = decrypter_for(path)
|
||||
tool = "mp4decrypt" if decrypter == "mp4decrypt" else "shaka-packager"
|
||||
|
||||
log_event(
|
||||
|
||||
@@ -562,6 +562,7 @@ class ImportService:
|
||||
width=int_or_none(row.get("width")),
|
||||
height=int_or_none(row.get("height")),
|
||||
fps=row.get("fps") or None,
|
||||
stereo=bool(row.get("stereo")),
|
||||
)
|
||||
else:
|
||||
track = Audio(
|
||||
|
||||
@@ -333,6 +333,7 @@ class DASH:
|
||||
video_args["dv_compatible_bitstream"] = Video.is_dv_composite(
|
||||
track_codec, codecs, get("supplementalCodecs")
|
||||
)
|
||||
video_args["stereo"] = Video.is_stereo(codecs, get("supplementalCodecs"))
|
||||
elif content_type == "audio":
|
||||
track_type = Audio
|
||||
track_codec = Audio.Codec.from_codecs(codecs or "")
|
||||
|
||||
@@ -222,7 +222,11 @@ class HLS:
|
||||
|
||||
supplemental_codecs = self.supplemental_codecs_by_uri()
|
||||
|
||||
for playlist in self.manifest.playlists:
|
||||
video_layouts = [
|
||||
(raw.get("stream_info") or {}).get("req_video_layout")
|
||||
for raw in (getattr(self.manifest, "data", None) or {}).get("playlists", [])
|
||||
]
|
||||
for playlist, video_layout in zip(self.manifest.playlists, video_layouts):
|
||||
video_codec: Optional[Video.Codec] = None
|
||||
primary_track_type: type[Audio] | type[Video]
|
||||
try:
|
||||
@@ -272,6 +276,9 @@ class HLS:
|
||||
(playlist.stream_info.closed_captions or "").strip('"'), []
|
||||
),
|
||||
dv_compatible_bitstream=dv_compatible_bitstream,
|
||||
stereo=Video.is_stereo(
|
||||
playlist.stream_info.codecs, supplemental_codecs.get(playlist.uri), video_layout
|
||||
),
|
||||
)
|
||||
if primary_track_type is Video
|
||||
else {}
|
||||
|
||||
@@ -119,7 +119,7 @@ class RemoteError(SystemExit):
|
||||
"""A remote call that failed: logged, then exits a CLI run like ``SystemExit(1)``.
|
||||
|
||||
A serve relaying for a REST client catches it instead and answers with ``message``; ``kind`` is the
|
||||
remote server's error code, or CONNECT / TIMEOUT / AUTH when the session could not be reached or opened.
|
||||
remote server's error code, or CONNECT / TIMEOUT / AUTH when the remote session could not be reached or opened.
|
||||
(Not ``code``: SystemExit's ``code`` is the exit status.)
|
||||
"""
|
||||
|
||||
@@ -132,7 +132,7 @@ class RemoteError(SystemExit):
|
||||
|
||||
@property
|
||||
def session_broke(self) -> bool:
|
||||
"""Whether the remote session itself is gone, not just this one call."""
|
||||
"""Whether the remote session itself is gone, not only this one call."""
|
||||
return self.kind in SESSION_LOST
|
||||
|
||||
|
||||
@@ -348,6 +348,7 @@ def deserialize_video(data: Dict[str, Any]) -> Video:
|
||||
dv_compatible_bitstream=bool(data.get("dv_compatible_bitstream", False)),
|
||||
hdr10plus=bool(data.get("hdr10plus", False)),
|
||||
hdr10_base=bool(data.get("hdr10_base", False)),
|
||||
stereo=bool(data.get("stereo", False)),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -106,6 +106,7 @@ class Title:
|
||||
"video": "",
|
||||
"hdr": "",
|
||||
"hfr": "",
|
||||
"3d": "",
|
||||
"edition": "",
|
||||
"lang_tag": "",
|
||||
"title_type": {"Movie": "movie", "Episode": "series", "Song": "music"}.get(
|
||||
@@ -163,6 +164,7 @@ class Title:
|
||||
|
||||
frame_rate = float(primary_video_track.frame_rate) if primary_video_track.frame_rate else 0.0
|
||||
context["hfr"] = "HFR" if frame_rate > 30 else ""
|
||||
context["3d"] = "3D" if any(v.stereo for v in self.tracks.videos[:1]) else ""
|
||||
|
||||
if primary_audio_track:
|
||||
codec = primary_audio_track.format
|
||||
|
||||
@@ -30,6 +30,7 @@ from unshackle.core.downloaders.requests import requests
|
||||
from unshackle.core.drm import DRM_T, BlowfishStripe, ClearKeyCENC, PlayReady, Widevine
|
||||
from unshackle.core.drm.verify import (
|
||||
decrypt_track,
|
||||
find_child_box,
|
||||
has_encrypted_sample_entry,
|
||||
read_top_level_box,
|
||||
senc_protects_samples,
|
||||
@@ -114,17 +115,6 @@ def iter_top_level_boxes(path: Path) -> Iterable[tuple[bytes, int, int]]:
|
||||
offset += size
|
||||
|
||||
|
||||
def find_child_box(buf: bytes, box_type: bytes) -> Optional[bytes]:
|
||||
"""Return the first box of the given type nested anywhere in buf, header included."""
|
||||
i = buf.find(box_type, 4)
|
||||
while i != -1:
|
||||
size = int.from_bytes(buf[i - 4 : i], "big")
|
||||
if 8 <= size <= len(buf) - (i - 4):
|
||||
return buf[i - 4 : i - 4 + size]
|
||||
i = buf.find(box_type, i + 1)
|
||||
return None
|
||||
|
||||
|
||||
def strip_duplicate_init_boxes(src: Path, dst: Path) -> int:
|
||||
"""Write src to dst without its repeated ftyp/moov pairs, returning how many it dropped.
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ from unshackle.core import binaries
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.console import GradientPulseBarColumn, console
|
||||
from unshackle.core.constants import AnyTrack, TrackT
|
||||
from unshackle.core.drm.verify import children, has_second_view, read_top_level_box
|
||||
from unshackle.core.events import events
|
||||
from unshackle.core.tracks.attachment import Attachment
|
||||
from unshackle.core.tracks.audio import Audio
|
||||
@@ -68,6 +69,12 @@ def file_track_ids(path: Path) -> dict[str, int]:
|
||||
return ids or {"video": 0, "audio": 1}
|
||||
|
||||
|
||||
def mp4_track_count(path: Path) -> int:
|
||||
"""How many tracks ``MP4Box -add`` takes from a file: each ``trak`` of an MP4, or 1 for any other file."""
|
||||
moov = read_top_level_box(path, b"moov") or b""
|
||||
return sum(1 for kind, _, _ in children(moov, 8) if kind == b"trak") or 1
|
||||
|
||||
|
||||
class Tracks:
|
||||
"""
|
||||
Video, Audio, Subtitle, Chapter, and Attachment Track Store.
|
||||
@@ -287,7 +294,17 @@ class Tracks:
|
||||
def signature(track: AnyTrack) -> tuple:
|
||||
"""The attributes that listing and selection read. Two tracks with one signature look and select the same."""
|
||||
if isinstance(track, Video):
|
||||
return Video, track.codec, track.range, track.width, track.height, track.fps, track.bitrate, track.name
|
||||
return (
|
||||
Video,
|
||||
track.codec,
|
||||
track.range,
|
||||
track.stereo,
|
||||
track.width,
|
||||
track.height,
|
||||
track.fps,
|
||||
track.bitrate,
|
||||
track.name,
|
||||
)
|
||||
if isinstance(track, Audio):
|
||||
return (
|
||||
Audio,
|
||||
@@ -333,11 +350,13 @@ class Tracks:
|
||||
def sort_videos(
|
||||
self, by_language: Optional[Sequence[Union[str, Language]]] = None, exact_match: bool = False
|
||||
) -> None:
|
||||
"""Sort video tracks by resolution then bitrate, and optionally language."""
|
||||
"""Sort video tracks by resolution, 2D before 3D, then bitrate, and optionally language.
|
||||
|
||||
A 3D track carries two views, so its higher bitrate would otherwise rank it over the 2D one.
|
||||
"""
|
||||
if not self.videos:
|
||||
return
|
||||
# resolution first, then bitrate (unknown-bitrate tracks still rank by resolution)
|
||||
self.videos.sort(key=lambda x: (x.height or 0, float(x.bitrate or 0.0)), reverse=True)
|
||||
self.videos.sort(key=lambda x: (x.height or 0, not x.stereo, float(x.bitrate or 0.0)), reverse=True)
|
||||
for language in reversed(by_language or []):
|
||||
if str(language) in ("all", "best"):
|
||||
language = self._original_language(self.videos, exact_match)
|
||||
@@ -486,10 +505,10 @@ class Tracks:
|
||||
HDR10/HDR10+/DV tracks are hybrid ingredients. They only enter the standalone
|
||||
pool when the user explicitly requested their range alongside HYBRID, so for
|
||||
example `-r HYBRID` muxes only the hybrid while `-r HYBRID,HDR10P` also delivers
|
||||
HDR10+.
|
||||
HDR10+. A 3D track is never an ingredient: dovi_tool does not handle a second view.
|
||||
"""
|
||||
ingredient_ranges = (Video.Range.HDR10, Video.Range.HDR10P, Video.Range.DV)
|
||||
hybrid_candidates = [v for v in videos if v.range in ingredient_ranges]
|
||||
hybrid_candidates = [v for v in videos if v.range in ingredient_ranges and not v.stereo]
|
||||
non_hybrid = [v for v in videos if v.range not in ingredient_ranges or v.range in non_hybrid_ranges]
|
||||
return hybrid_candidates, non_hybrid
|
||||
|
||||
@@ -654,19 +673,21 @@ class Tracks:
|
||||
"""
|
||||
Multiplex the Tracks into an MP4 Container with MP4Box.
|
||||
|
||||
Only used for a codec Matroska cannot carry, DTS-UHD or MPEG-H, because MP4 carries
|
||||
far less: it drops the forced, hearing-impaired and original flags, holds no
|
||||
attachments at all, and leaves out a subtitle it has no format for rather than
|
||||
mangling it. Losing those beats shipping audio no player can name.
|
||||
Only used for what Matroska cannot carry: DTS-UHD or MPEG-H audio, and stereoscopic 3D
|
||||
(MV-HEVC) video, whose second view mkvmerge drops. MP4 carries far less: it drops the
|
||||
forced, hearing-impaired and original flags, holds no attachments at all, and leaves
|
||||
out a subtitle it has no format for rather than mangling it. Losing those beats
|
||||
shipping audio no player can name or a 3D film as 2D.
|
||||
|
||||
Returns the same (path, returncode, errors) shape as the Matroska mux.
|
||||
"""
|
||||
mp4box = getattr(binaries, "MP4Box", None)
|
||||
if not mp4box:
|
||||
raise RuntimeError("MP4Box (GPAC) not found, muxing DTS-UHD or MPEG-H audio needs it")
|
||||
raise RuntimeError("MP4Box (GPAC) not found, muxing DTS-UHD or MPEG-H audio or 3D video needs it")
|
||||
|
||||
progress = progress or partial(lambda **kwargs: None)
|
||||
|
||||
stereo = any(v.stereo for v in self.videos)
|
||||
cl = [str(mp4box)]
|
||||
sample_base = self.sample_base
|
||||
names: list[str] = []
|
||||
@@ -685,7 +706,6 @@ class Tracks:
|
||||
)
|
||||
continue
|
||||
events.emit(events.Types.TRACK_MULTIPLEX, track=track)
|
||||
track_id += 1
|
||||
|
||||
source = track.path
|
||||
candidate = config.directories.temp / f"mp4mux_{track.id}_{os.getpid()}_{threading.get_ident()}.mp4"
|
||||
@@ -706,7 +726,8 @@ class Tracks:
|
||||
cl.extend(["-add", f"{source}:lang={track.mux_language}" + (f":delay={delay}" if delay else "")])
|
||||
name = "" if isinstance(track, Video) else (track.get_track_name() or "")
|
||||
if name:
|
||||
names.extend(["-name", f"{track_id}={name}"])
|
||||
names.extend(["-name", f"{track_id + 1}={name}"])
|
||||
track_id += mp4_track_count(source)
|
||||
|
||||
if not track_id:
|
||||
raise ValueError("No tracks provided, at least one track must be provided.")
|
||||
@@ -719,7 +740,7 @@ class Tracks:
|
||||
title=sanitize_filename(title), random=f"{self.chapters.id}_{os.getpid()}_{threading.get_ident()}"
|
||||
)
|
||||
self.chapters.dump(chapters_path, fallback_name=config.chapter_fallback_name)
|
||||
cl.extend(["-chap", str(chapters_path)])
|
||||
cl.extend(["-chapqt" if stereo else "-chap", str(chapters_path)])
|
||||
|
||||
if self.attachments:
|
||||
log_event(
|
||||
@@ -749,7 +770,9 @@ class Tracks:
|
||||
"muxer": str(mp4box),
|
||||
"command": full_command,
|
||||
"container": "mp4",
|
||||
"reason": "DTS-UHD or MPEG-H audio has no Matroska CodecID",
|
||||
"reason": "3D video has no Matroska mapping"
|
||||
if stereo
|
||||
else "DTS-UHD or MPEG-H audio has no Matroska CodecID",
|
||||
},
|
||||
)
|
||||
|
||||
@@ -778,7 +801,7 @@ class Tracks:
|
||||
returncode = p.wait()
|
||||
if returncode < 2 and any(MP4BOX_OPEN_FAILED.search(line) for line in errors):
|
||||
returncode = 2
|
||||
if config.sample and returncode == 0:
|
||||
if config.sample and returncode == 0 and not stereo:
|
||||
start, end = (seconds - sample_base for seconds in config.sample)
|
||||
cut_path = output_path.with_suffix(".cut.mp4")
|
||||
try:
|
||||
@@ -791,6 +814,8 @@ class Tracks:
|
||||
except RuntimeError as e:
|
||||
returncode = 2
|
||||
errors.append(f"MP4Box could not cut the sample window: {e}")
|
||||
if stereo and returncode == 0 and not has_second_view(output_path):
|
||||
logging.getLogger("Tracks").warning(f"The 3D mux of {output_path.name} lost its second view")
|
||||
mux_duration_ms = round((time.monotonic() - mux_start_time) * 1000, 1)
|
||||
output_size = output_path.stat().st_size if output_path.exists() else 0
|
||||
|
||||
@@ -868,7 +893,9 @@ class Tracks:
|
||||
video.needs_repack = True
|
||||
video.data["audio_language"] = lang_code
|
||||
|
||||
if any(at.path and at.path.exists() and has_mp4_only_audio_entry(at.path) for at in self.audio):
|
||||
if any(v.stereo for v in self.videos) or any(
|
||||
at.path and at.path.exists() and has_mp4_only_audio_entry(at.path) for at in self.audio
|
||||
):
|
||||
return self._mux_mp4(
|
||||
title=title,
|
||||
delete=delete,
|
||||
|
||||
@@ -15,6 +15,7 @@ from langcodes import Language
|
||||
|
||||
from unshackle.core import binaries
|
||||
from unshackle.core.config import config
|
||||
from unshackle.core.drm.verify import has_second_view
|
||||
from unshackle.core.tracks.subtitle import Subtitle
|
||||
from unshackle.core.tracks.track import Track
|
||||
from unshackle.core.utilities import FPS, get_boxes, log_event
|
||||
@@ -228,6 +229,7 @@ class Video(Track):
|
||||
dv_compatible_bitstream: bool = False,
|
||||
hdr10plus: bool = False,
|
||||
hdr10_base: bool = False,
|
||||
stereo: bool = False,
|
||||
**kwargs: Any,
|
||||
) -> None:
|
||||
"""
|
||||
@@ -248,6 +250,7 @@ class Video(Track):
|
||||
All strings will be cast to either a number or float.
|
||||
hdr10plus: The DV track also carries HDR10+ metadata. The HDR10+ probe sets it.
|
||||
hdr10_base: The DV track has an HDR10 base layer, as DV profile 8.1 does.
|
||||
stereo: The track is stereoscopic 3D (MV-HEVC), with a second view for the other eye.
|
||||
|
||||
Note: If codec, bitrate, width, height, or fps is not specified, unshackle can
|
||||
skip some checks or assume a value. Give as much information as possible.
|
||||
@@ -298,6 +301,7 @@ class Video(Track):
|
||||
self.dv_compatible_bitstream = dv_compatible_bitstream
|
||||
self.hdr10plus = hdr10plus
|
||||
self.hdr10_base = hdr10_base
|
||||
self.stereo = stereo
|
||||
self.hybrid_base_only = False
|
||||
self.native_hybrid = False
|
||||
|
||||
@@ -315,6 +319,7 @@ class Video(Track):
|
||||
"dv_compatible_bitstream": self.dv_compatible_bitstream,
|
||||
"hdr10plus": self.hdr10plus,
|
||||
"hdr10_base": self.hdr10_base,
|
||||
"stereo": self.stereo,
|
||||
}
|
||||
)
|
||||
return data
|
||||
@@ -334,6 +339,7 @@ class Video(Track):
|
||||
dv_compatible_bitstream=data.get("dv_compatible_bitstream", False),
|
||||
hdr10plus=data.get("hdr10plus", False),
|
||||
hdr10_base=data.get("hdr10_base", False),
|
||||
stereo=data.get("stereo", False),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
@@ -347,6 +353,16 @@ class Video(Track):
|
||||
"""Whether a codecs list names a Dolby Vision codec."""
|
||||
return bool(Video.dv_fourccs(codecs))
|
||||
|
||||
@staticmethod
|
||||
def is_stereo(
|
||||
codecs: Optional[str], supplemental_codecs: Optional[str] = None, video_layout: Optional[str] = None
|
||||
) -> bool:
|
||||
"""Whether a track is stereoscopic 3D: the HLS REQ-VIDEO-LAYOUT names CH-STEREO, or a codec is DV profile 20."""
|
||||
tokens = re.split(r"[\s,]+", f"{codecs or ''},{supplemental_codecs or ''}".lower())
|
||||
return "CH-STEREO" in (video_layout or "").upper() or any(
|
||||
re.match(r"dvh[1e]\.20(\.|$)", token) for token in tokens
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def is_dv_composite(
|
||||
codec: Optional[Video.Codec], codecs: Optional[str], supplemental_codecs: Optional[str]
|
||||
@@ -377,12 +393,15 @@ class Video(Track):
|
||||
return f"{label}+DV" if self.dv_compatible_bitstream and self.range != Video.Range.DV else label
|
||||
|
||||
def __str__(self) -> str:
|
||||
labels = filter(
|
||||
None, [self.codec.value if self.codec else None, self.range_label, "3D" if self.stereo else None]
|
||||
)
|
||||
return " | ".join(
|
||||
filter(
|
||||
None,
|
||||
[
|
||||
"VID",
|
||||
"[" + (", ".join(filter(None, [self.codec.value if self.codec else None, self.range_label]))) + "]",
|
||||
f"[{', '.join(labels)}]",
|
||||
str(self.language),
|
||||
", ".join(
|
||||
filter(
|
||||
@@ -459,16 +478,34 @@ class Video(Track):
|
||||
self.path = output_path
|
||||
original_path.unlink()
|
||||
|
||||
def download(self, *args: Any, **kwargs: Any) -> None:
|
||||
"""Download the track and mark it 3D when the file holds a second view; a 3D track is never repacked."""
|
||||
super().download(*args, **kwargs)
|
||||
if not self.stereo and self.path and has_second_view(self.path):
|
||||
self.stereo = True
|
||||
if self.stereo:
|
||||
self.needs_repack = False
|
||||
|
||||
def repackage(self, bsf_v: Optional[str] = None) -> bool:
|
||||
"""Remux with FFmpeg, except a 3D track: an FFmpeg remux rewrites its stereo and DV boxes."""
|
||||
if self.stereo:
|
||||
logging.getLogger("Video").warning(
|
||||
f"Skipped the FFmpeg repack of 3D track {self.id}: FFmpeg rewrites its stereo boxes"
|
||||
)
|
||||
return False
|
||||
return super().repackage(bsf_v)
|
||||
|
||||
def vui_bsf(self) -> Optional[str]:
|
||||
"""Return the ``-bsf:v`` value that rewrites SPS VUI colour metadata to match ``self.range``.
|
||||
|
||||
Returns None when a rewrite is not necessary: SDR/DV/HYBRID ranges, non-AVC/HEVC codecs, a
|
||||
missing file, or when the bitstream already ships the correct colour metadata. Otherwise
|
||||
Returns None when a rewrite is not necessary or not safe: SDR/DV/HYBRID ranges, non-AVC/HEVC
|
||||
codecs, a 3D track (an FFmpeg remux rewrites its stereo and DV boxes), a missing file, or when the
|
||||
bitstream already ships the correct colour metadata. Otherwise
|
||||
returns a ``{h264,hevc}_metadata=...`` string suitable for FFmpeg ``-bsf:v``.
|
||||
"""
|
||||
if not self.path or not self.path.exists():
|
||||
return None
|
||||
if self.codec not in (Video.Codec.AVC, Video.Codec.HEVC):
|
||||
if self.codec not in (Video.Codec.AVC, Video.Codec.HEVC) or self.stereo:
|
||||
return None
|
||||
if self.range in (Video.Range.SDR, Video.Range.DV, Video.Range.HYBRID):
|
||||
return None
|
||||
|
||||
@@ -22,12 +22,12 @@ tag: "NOGRP"
|
||||
# ? suffix, e.g. {year?}.
|
||||
output_template:
|
||||
# Scene-style naming (dot-separated)
|
||||
movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
series: "{title}.{year?}.{season_episode}.{episode_name?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
series: "{title}.{year?}.{season_episode}.{episode_name?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
songs: "{track_number}.{title}.{repack?}.{edition?}.{lang_tag?}.{source?}.WEB-DL.{audio_full}.{atmos?}-{tag}"
|
||||
folder:
|
||||
movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
series: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
movies: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
series: "{title}.{year?}.{repack?}.{edition?}.{lang_tag?}.{3d?}.{quality}.{source}.WEB-DL.{dual?}.{multi?}.{dubbed?}.{audio_full}.{atmos?}.{hdr?}.{hfr?}.{video}-{tag}"
|
||||
albums: "{album_artist} - {album} ({year?})"
|
||||
|
||||
#
|
||||
|
||||
Reference in New Issue
Block a user