49 Commits

Author SHA1 Message Date
3e45f3efe7 fix(netflix): harden ESN cache checks and Widevine type test
Handle Netflix ESN cache values more defensively to avoid key/type errors and
stale reuse by validating cache shape, cache expiry, and device type before
reusing values. Also log the final ESN safely when cache data is not a dict.

Alias `pywidevine.Cdm` to `WidevineCDM` and use it in DRM system detection so
Widevine instances are identified correctly.

Also include related config updates: add ESN map entry for system ID `12063`,
ignore `binaries/`, and refresh local runtime defaults in `unshackle.yaml`.fix(netflix): harden ESN cache checks and Widevine type test

Handle Netflix ESN cache values more defensively to avoid key/type errors and
stale reuse by validating cache shape, cache expiry, and device type before
reusing values. Also log the final ESN safely when cache data is not a dict.

Alias `pywidevine.Cdm` to `WidevineCDM` and use it in DRM system detection so
Widevine instances are identified correctly.

Also include related config updates: add ESN map entry for system ID `12063`,
ignore `binaries/`, and refresh local runtime defaults in `unshackle.yaml`.
2026-03-02 17:29:32 +07:00
fb14f412d4 update .gitignore 2026-03-02 02:59:45 +07:00
27048d56ee fix(netflix): scope 720p QC filter to explicit 720 requests
Refine QC manifest profile selection so `l40` profiles are filtered out
only when the user requests **only** 720p, instead of whenever 720 is
included. This prevents unintended profile narrowing for mixed-quality
requests and default quality runs.

Also bump the Netflix client `platform` version from `138.0.0.0` to
`145.0.0.0` to keep manifest requests aligned with current expectations.fix(netflix): scope 720p QC filter to explicit 720 requests

Refine QC manifest profile selection so `l40` profiles are filtered out
only when the user requests **only** 720p, instead of whenever 720 is
included. This prevents unintended profile narrowing for mixed-quality
requests and default quality runs.

Also bump the Netflix client `platform` version from `138.0.0.0` to
`145.0.0.0` to keep manifest requests aligned with current expectations.
2026-03-02 02:59:33 +07:00
66ba78a928 Update payload challenge 2026-03-02 02:58:56 +07:00
7c84cf67e6 Merge remote-tracking branch 'origin/update-unshackle' 2026-02-28 22:08:39 +07:00
Andy
cee7d9a75f fix(n_m3u8dl_re): pass all content keys for DualKey DRM decryption 2026-02-15 13:37:49 -07:00
Andy
bf9087a1ce chore(release): bump version to 3.0.0
BREAKING CHANGE: PlayReady users without explicit playready_devices no longer get access to all devices by default.

Key changes:
- feat(drm): add MonaLisa DRM support to core infrastructure
- feat(cdm): add remote PlayReady CDM support via pyplayready RemoteCdm
- feat(serve): add PlayReady CDM support alongside Widevine
- feat(gluetun): Gluetun VPN integration and Windscribe support
- feat(audio): codec lists and split muxing
- feat(tracks): prioritize Atmos audio tracks over higher bitrate non-Atmos
- feat(video): detect interlaced scan type from MPD manifests
- feat(cdm): normalize CDM detection for local and remote implementations
- fix(serve)!: make PlayReady users config consistently a mapping
- 50+ additional bug fixes across HLS/DASH, proxies, subtitles, and more
2026-02-15 13:04:42 -07:00
Andy
23cc351f77 feat(tracks): prioritize Atmos audio tracks over higher bitrate non-Atmos 2026-02-15 12:08:27 -07:00
Andy
132d3549f9 fix(main): update copyright year dynamically in version display 2026-02-11 16:01:33 -07:00
Andy
3ee554401a feat(HLS): improve audio codec handling with error handling for codec extraction 2026-02-10 08:34:54 -07:00
0c370c7738 fix: preserve spaces in sanitized filenames
Remove space from structural chars regex so spaces are kept as-is
rather than being replaced with the spacer character.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-20 00:17:20 +07:00
8cefca84e9 Merge branch 'update-unshackle' 2025-12-19 22:29:30 +07:00
f9aac210c5 Set api_mode for Decrypt Labs vault 2025-11-24 18:11:38 +07:00
ad7a276305 Update config 2025-11-24 15:48:10 +07:00
5bed770471 Set default episode name to Episode 01 2025-10-25 16:16:58 +07:00
fe7a3f019f Use correct name of image attachment 2025-10-25 16:09:22 +07:00
45f18b046f Update credentials Surfshark VPN 2025-10-25 16:08:12 +07:00
7080ee2379 Bump to 1.4.8 and add DecryptLabs CDM 2025-10-21 12:22:58 +07:00
7757ef7eb9 Update .gitignore 2025-10-21 01:40:00 +07:00
6a15cd0a5d refactor(netflix): unify DRM handling and improve track hydration logic
- Added DRM system detection method to distinguish Widevine and PlayReady CDMs
- Implemented create_drm method to instantiate appropriate DRM objects based on system
- Updated track hydration to add all tracks on first profile/range, only videos otherwise
- Changed get_playready_license to reuse Widevine license retrieval method
- Replaced direct Widevine instantiations with create_drm calls for DRM object creation
- Added conditional debug logs for unavailable audio and subtitle tracks when hydrating
- Cleaned up DRM imports and type annotations for drm_system attribute in Netflix class
2025-09-09 18:03:14 +07:00
5d4b71b388 fix(MSL): raise exception on error in MSL response message
- Replace silent log of error with raising an exception
- Ensure errors in MSL response messages do not go unnoticed
- Prevent continuation on critical MSL response errors
- Improve error handling robustness in MSL class
2025-09-09 18:01:50 +07:00
7fe4be4542 refactor(netflix): support multiple video ranges and improve profile handling
- Add support for processing multiple video ranges in parallel
- Handle HYBRID range by fetching HDR10 and DV profiles separately
- Introduce get_profiles_for_range method to retrieve profiles for given range
- Refactor profile fetching logic with detailed logging and error handling
- Validate all requested video ranges are supported by the current codec
- Allow H.264 codec only with SDR range and enforce checks for multiple ranges
- Improve track hydration logic to avoid duplicates across ranges and profiles
- Add logging for multi-range processing and profile fetching details
2025-09-08 20:42:33 +07:00
aae9fb1927 fix(core): increase timeout for IP info request to 3 seconds
- Updated HTTP request timeout from 1 to 3 seconds in get_ip_info function
- Improved reliability of external IP info retrieval by allowing longer response time
2025-09-08 20:41:51 +07:00
ab59cfbf93 fix(core): add retry and timeout to get_ip_info requests
- Import HTTPAdapter from requests.adapters
- Configure session adapters for http and https with max_retries=3
- Add timeout of 1 second to the get request in get_ip_info
- Ensure retries and timeout improve request reliability and responsiveness
2025-09-08 16:09:08 +07:00
326320651b merge upstream 2025-09-07 12:02:38 +00:00
73700f3228 merge upstream 2025-09-03 10:27:04 +00:00
b4cefa6597 fix(dl): increase upper tolerance for video bitrate selection
- Changed max_bitrate from vbitrate + 100 to vbitrate + 200
- Updated video track selection to use the new max_bitrate range
- Ensured better matching of video tracks within adjusted bitrate tolerance
- Improved error logging message for bitrate selection range
2025-09-02 13:47:51 +07:00
cea302afae Merge pull request 'Update Unshackle' (#1) from update-unshackle into main
Reviewed-on: http://mac:3002/unshackle-dl/unshackle/pulls/1
2025-09-02 06:07:53 +00:00
59a1691ac4 Sync Update from origin repo 2025-09-02 13:05:28 +07:00
93ef794412 refactor(Netflix): extract track hydration logic into separate method
- Moved hydration of unavailable audio and subtitle tracks into hydrate_all_tracks method
- Replaced large inline hydration code with a single call to hydrate_all_tracks
- Improved clarity by encapsulating hydration steps including logging and error handling
- Maintained original behavior with detailed debug and warning messages
- Added comprehensive parameters to hydrate_all_tracks for audio, subtitle, primary tracks, and language context
- Ensured hydration method returns complete Tracks object for easier track management and addition
2025-09-02 12:30:54 +07:00
0cf2367781 refactor(Netflix): improve audio and subtitle track hydration logic
- Add primary audio track storage for subtitle-only hydration cases
- Introduce helper methods for track validation, empty track tuple, and logging hydration attempts
- Enhance hydration loop to reuse last successful or primary audio tracks for context
- Log detailed hydration attempt information including track IDs and request types
- Use None in API calls instead of 'N/A' for missing tracks to prevent errors
- Comment out debug log for video profiles to reduce noise
- Simplify handling of mismatched audio and subtitle hydration lengths with improved track fallback logic
2025-09-02 04:11:53 +07:00
ae3f896348 refactor(titles): comment out config.tag suffix in episode name generation
- Disabled appending of config.tag to episode name by commenting out related lines
- Preserved existing logic for codec and HFR suffix additions
- Ensured filename sanitization remains unchanged
2025-08-30 14:10:31 +07:00
0d2237d09a fix(Netflix): correct video range and codec validation logic
- Adjust condition order in video range and codec compatibility check
- Change range membership check from object to name string in profile loop
- Remove redundant error logging and sys.exit call in profile handling
- Improve consistency of video profile retrieval based on codec and range
2025-08-30 11:37:31 +07:00
33ceed0016 fix(proxy_providers): update SurfsharkVPN service credentials
- Changed username for SurfsharkVPN in unshackle.yaml
- Changed password for SurfsharkVPN in unshackle.yaml
2025-08-30 11:37:16 +07:00
bb85ac2767 fix(dl): add bitrate tolerance for video track selection
- Implement bitrate selection with a tolerance of +100 kbps and -800 kbps
- Ensure minimum bitrate does not go below 0 when calculating lower bound
- Update error message to reflect bitrate range instead of exact value
- Enhance logging for cases with no matching video tracks within tolerance range
2025-08-30 01:14:34 +07:00
c7be94c0fc chore(config): add Netflix credentials and comment out wvds directory
- Added Netflix service credentials with a default user and password
- Commented out the wvds directory in the configuration file
- Retained example credentials for reference in comments
2025-08-29 20:54:34 +07:00
c60035cb1d feat(netflix): add hybrid HDR10 and DV profile support and Android CDM improvements
- Introduce new descriptive subtitle option in CLI and internal logic
- Support hybrid video range by separately fetching HDR10 and DV profiles
- Add detailed error handling and logging for hybrid mode processing
- Extend ESN handling to support different device types (Chrome and Android)
- Implement Android CDM login using email and password credentials
- Update ESN caching logic with type-aware expiration handling
- Adjust manifest parsing to handle optional hydrate_tracks parameter
- Enhance subtitle filtering to optionally skip descriptive subtitles
- Expand ALIASES to include lowercase variants "netflix" and "nf"
- Add new ESN mapping entry for Android device in config.yaml
2025-08-29 20:53:52 +07:00
3c24d83293 refactor(msl): improve key exchange handling and code cleanup
- Replace commented Widevine key exchange code with active parsing and key extraction
- Add checks for CDM session and CDM availability before license parsing
- Update key permission strings to lowercase and align with key extraction logic
- Handle AsymmetricWrapped scheme separately with RSA decryption of keys
- Change EntityAuthentication to Unauthenticated for certain challenge cases
- Remove redundant jsonpickle encoding/decoding for cached keys, store raw data instead
- Add detailed logging for key UUIDs and extracted Widevine keys
- Fix key comparison by converting kid bytes to UUID on comparison
- Raise Exception instead of using logger exit on MSL response error
- Update imports to consolidate pywidevine package classes used
2025-08-29 20:52:30 +07:00
fcd1ebcf83 fix(netflix): improve audio and subtitle track hydration logic
- Update joc value for atmos content profile from 6 to 16
- Add informative log message summarizing total audio and subtitle tracks to hydrate
- Refactor hydration loop to handle mismatched lengths of audio and subtitle tracks more clearly
- Skip hydration if no audio tracks are available for the current index
- Ensure valid subtitle track ID is used in manifest request to avoid API errors
- Add detailed debug logs for processing hydrated audio and subtitle streams
- Handle exceptions gracefully for each stream and track hydration step with warnings
- Log when no tracks need hydration to improve observability
2025-08-28 11:25:42 +07:00
e1f69eb307 feat(netflix): add Widevine CDM integration with MSL handshake and ESN mapping
- Extend MSL handshake method to support Widevine key exchange scheme using CDM instance
- Implement CDM session handling with service certificate and license challenge during handshake
- Add exception handling for key exchange errors instead of exiting logger
- Modify Netflix service to include CDM instance and pass it to MSL handshake call
- Update get_esn method to use ESN mapping from config for security level 1 CDM systems
- Add new ESN mapping entry in config.yaml for a specific CDM SystemID
- Remove commented out Widevine key exchange placeholder code and replace with full implementation
- Include CDM initialization logs and tweak manifest params to support DRM challenges
- Ensure fallback to random ESN generation for non-level 1 security or missing cached ESN
2025-08-28 02:26:44 +07:00
d18fbdb542 fix(unshackle): update remote_cdm device configuration
- Replace chrome-2 device with android device in remote_cdm list
- Change device_name to "andorid" and device_type to ANDROID
- Update system_id to 8131 and security_level to 1
- Add type field with value "decrypt_labs"
- Update host and secret fields to match decrypt_labs credentials
2025-08-28 02:21:04 +07:00
d5cbc4e088 fix(cdm): adjust scheme value based on security level
- Change scheme to "widevine" if security level is 3, otherwise use "L1"
- Apply this logic when setting init_data and license response schemes
- Ensure correct CDM scheme usage according to security level context
2025-08-28 02:20:23 +07:00
831fa10ce5 feat(dl): enhance episode folder structure and rename series folders
- Create nested folder structure for episodes: {title}/Season {season:02}/{filename}
- Keep existing folder naming for songs unchanged
- Modify episode folder naming to only show title for main folder
- Adjust episode file naming format to include separators and sanitization
- Add get_season_folder() method returning 'Season XX' for episodes
- Disable series year inclusion in folder and episode naming by default in config
- Comment out additional service and audio language tags in episode naming code
2025-08-26 21:35:22 +07:00
2a414720e7 feat(netflix): implement initial Netflix service with MSL DRM support
- Add MSL core implementation for handling Netflix message security layer
- Create MSL keys and message encryption/signature utilities
- Implement handshake to establish encrypted session keys with Netflix endpoint
- Add entity and user authentication scheme support for MSL
- Provide methods for message creation, sending, decryption, and parsing
- Implement Netflix service class with CLI integration via Click
- Support title metadata retrieval and parse movie or series accordingly
- Implement track extraction with profile and codec handling logic
- Add chapter extraction from Netflix metadata with error handling
- Implement Widevine license request using MSL messaging
- Add utility to split profiles based on video codec types
- Define schemes for key exchange, user and entity authentication with MSL
- Enable caching and loading of MSL keys with expiration checks
- Include gzip compression and base64 key decoding helpers within MSL class
2025-08-26 17:59:47 +07:00
f377bbfb74 chore(config): add comprehensive unshackle.yaml configuration file
- Define tagging options for filenames including group and metadata tags
- Configure terminal background color and file naming conventions
- Set caching parameters for title metadata with expiration controls
- Add muxing and default directories settings
- Provide flexible credentials management with profile and default support
- Configure CDM devices and remote CDMs for Widevine and PlayReady
- Define local and HTTP key vaults with options to disable pushing keys
- Set downloader preferences and specific settings for aria2c, n_m3u8dl_re, and curl_impersonate
- Introduce default parameters for download commands and subtitle conversion methods
- Add service-specific API keys, profiles, and device configurations
- Include proxy provider credentials for external VPN services
2025-08-26 17:59:21 +07:00
fb58e9f52a chore(git): update .gitignore file
- Remove unshackle.yaml from ignore list
- Delete 'services/' directory from ignore list
- Add 'Cache' to ignore list at the end of the file
2025-08-26 17:58:56 +07:00
4d2e84a45a fix(movie): adjust naming format and comment out audio and service tags
- Append " -" suffix to the movie name after replacing "$" with "S"
- Comment out adding service name to the title
- Disable appending "WEB-DL" tag to the title
- Comment out adding "DUAL" tag for two audio languages
- Comment out adding "MULTi" tag for more than two audio languages
- Comment out appending config tag suffix to video codec in name
2025-08-26 17:58:44 +07:00
f85ddce6f2 feat(downloaders): improve aria2c download progress reporting
- Added RPC calls to get detailed global and active download statistics
- Calculated total downloaded size, content size, and active download speed from active downloads
- Included stopped downloads in totals and handle error states with logged messages
- Yielded enhanced progress updates with combined downloaded sizes and speeds
- Added more granular progress dictionary keys for richer status reporting
- Added sleep delay in main aria2c function to reduce CPU usage during monitoring loop
- Updated docstring examples to reflect new progress data format and keys
2025-08-26 17:58:23 +07:00
354ba6c2e3 fix(core): correct filename sanitization regex and cleanup
- Adjust regex to replace semicolon only with spacer
- Remove colon from characters replaced by spacer, handle as removal instead
- Comment out removal of extra neighbouring spacers to prevent unintended collapses
- Refine unsafe characters removal pattern to avoid filename issues
2025-08-26 17:57:53 +07:00
25 changed files with 2488 additions and 98 deletions

5
.gitignore vendored
View File

@@ -1,5 +1,4 @@
# unshackle
unshackle.yaml
unshackle.yml
update_check.json
*.mkv
@@ -26,7 +25,8 @@ unshackle/WVDs/
unshackle/PRDs/
temp/
logs/
services/
Temp/
binaries/
# Byte-compiled / optimized / DLL files
__pycache__/
@@ -237,3 +237,4 @@ CLAUDE.md
marimo/_static/
marimo/_lsp/
__marimo__/
Cache

View File

@@ -6,7 +6,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
This changelog is automatically generated using [git-cliff](https://git-cliff.org).
## [Unreleased]
## [3.0.0] - 2026-02-15
### Features
@@ -21,6 +21,9 @@ This changelog is automatically generated using [git-cliff](https://git-cliff.or
- *drm*: Add MonaLisa DRM support to core infrastructure
- *audio*: Codec lists and split muxing
- *proxy*: Add specific server selection for WindscribeVPN
- *cdm*: Normalize CDM detection for local and remote implementations
- *HLS*: Improve audio codec handling with error handling for codec extraction
- *tracks*: Prioritize Atmos audio tracks over higher bitrate non-Atmos
### Bug Fixes
@@ -53,11 +56,39 @@ This changelog is automatically generated using [git-cliff](https://git-cliff.or
- *dl*: Always clean up hybrid temp hevc outputs
- *hls*: Finalize n_m3u8dl_re outputs
- *downloader*: Restore requests progress for single-url downloads
- *dl*: Invert audio codec suffixing when splitting
- *dl*: Support snake_case keys for RemoteCdm
- *aria2c*: Warn on config mismatch and wait for RPC ready
- *serve*: [**breaking**] Make PlayReady users config consistently a mapping
- *dl*: Preserve proxy_query selector (not resolved URI)
- *gluetun*: Stop leaking proxy/vpn secrets to process list
- *monalisa*: Avoid leaking secrets and add worker safety
- *dl*: Avoid selecting all variants when multiple audio codecs requested
- *hls*: Keep range offset numeric and align MonaLisa licensing
- *titles*: Remove trailing space from HDR dynamic range label
- *config*: Normalize playready_remote remote_cdm keys
- *titles*: Avoid None/double spaces in HDR tokens
- *naming*: Keep technical tokens with scene_naming off
- *api*: Log PSSH extraction failures
- *proxies*: Harden surfshark and windscribe selection
- *service*: Redact proxy credentials in logs
- *monalisa*: Harden wasm calls and license handling
- *hls*: Remove no-op encryption_data reassignment
- *serve*: Default PlayReady access to none
- *tracks*: Close temp session and improve path type error
- *main*: Update copyright year dynamically in version display
### Reverts
- *monalisa*: Pass key via argv again
### Documentation
- Add configuration documentation WIP
- *changelog*: Add 2.4.0 release notes
- *changelog*: Update cliff config and regenerate changelog
- *changelog*: Complete 2.4.0 notes
- *config*: Clarify sdh_method uses subtitle-filter
### Performance Improvements
@@ -451,7 +482,7 @@ This changelog is automatically generated using [git-cliff](https://git-cliff.or
- Reorganize Planned Features section in README for clarity
- Improve track selection logic in dl.py
[unreleased]: https://github.com/unshackle-dl/unshackle/compare/2.3.0..HEAD
[3.0.0]: https://github.com/unshackle-dl/unshackle/compare/2.3.0..3.0.0
[2.3.0]: https://github.com/unshackle-dl/unshackle/compare/2.2.0..2.3.0
[2.2.0]: https://github.com/unshackle-dl/unshackle/compare/2.1.0..2.2.0
[2.1.0]: https://github.com/unshackle-dl/unshackle/compare/2.0.0..2.1.0

View File

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "unshackle"
version = "2.4.0"
version = "3.0.0"
description = "Modular Movie, TV, and Music Archival Software."
authors = [{ name = "unshackle team" }]
requires-python = ">=3.10,<3.13"
@@ -57,7 +57,7 @@ dependencies = [
"chardet>=5.2.0,<6",
"curl-cffi>=0.7.0b4,<0.14",
"pyplayready>=0.6.3,<0.7",
"httpx[http2]>=0.28.1,<0.29",
"httpx>=0.28.1,<0.29",
"cryptography>=45.0.0,<47",
"subby",
"aiohttp>=3.13.3,<4",

View File

@@ -14,7 +14,7 @@ from unshackle.core.constants import context_settings
short_help="Serve your Local Widevine/PlayReady Devices and REST API for Remote Access.",
context_settings=context_settings,
)
@click.option("-h", "--host", type=str, default="127.0.0.1", help="Host to serve from.")
@click.option("-h", "--host", type=str, default="0.0.0.0", help="Host to serve from.")
@click.option("-p", "--port", type=int, default=8786, help="Port to serve from.")
@click.option("--caddy", is_flag=True, default=False, help="Also serve with Caddy.")
@click.option(

View File

@@ -1 +1 @@
__version__ = "2.4.0"
__version__ = "3.0.0"

View File

@@ -1,5 +1,6 @@
import atexit
import logging
from datetime import datetime
import click
import urllib3
@@ -58,7 +59,7 @@ def main(version: bool, debug: bool) -> None:
r" ▀▀▀ ▀▀ █▪ ▀▀▀▀ ▀▀▀ · ▀ ▀ ·▀▀▀ ·▀ ▀.▀▀▀ ▀▀▀ ",
style="ascii.art",
),
f"v [repr.number]{__version__}[/] - © 2025 - github.com/unshackle-dl/unshackle",
f"v [repr.number]{__version__}[/] - © 2025-{datetime.now().year} - github.com/unshackle-dl/unshackle",
),
(1, 11, 1, 10),
expand=True,

View File

@@ -192,8 +192,10 @@ def build_download_args(
if ad_keyword:
args["--ad-keyword"] = ad_keyword
key_args = []
if content_keys:
args["--key"] = next((f"{kid.hex}:{key.lower()}" for kid, key in content_keys.items()), None)
for kid, key in content_keys.items():
key_args.extend(["--key", f"{kid.hex}:{key.lower()}"])
decryption_config = config.decryption.lower()
engine_name = DECRYPTION_ENGINE.get(decryption_config) or "SHAKA_PACKAGER"
@@ -221,6 +223,9 @@ def build_download_args(
elif value is not False and value is not None:
command.extend([flag, str(value)])
# Append all content keys (multiple --key flags supported by N_m3u8DL-RE)
command.extend(key_args)
if headers:
for key, value in headers.items():
if key.lower() not in ("accept-encoding", "cookie"):

View File

@@ -11,7 +11,6 @@ from Cryptodome.Util.Padding import unpad
from curl_cffi.requests import Session as CurlSession
from m3u8.model import Key
from requests import Session
import httpx
class ClearKey:
@@ -71,8 +70,8 @@ class ClearKey:
"""
if not isinstance(m3u_key, Key):
raise ValueError(f"Provided M3U Key is in an unexpected type {m3u_key!r}")
if not isinstance(session, (Session, CurlSession, httpx.Client, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not a {type(session)}")
if not isinstance(session, (Session, CurlSession, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not a {type(session)}")
if not m3u_key.method.startswith("AES"):
raise ValueError(f"Provided M3U Key is not an AES Clear Key, {m3u_key.method}")

View File

@@ -15,7 +15,6 @@ from urllib.parse import urljoin, urlparse
from uuid import UUID
from zlib import crc32
import httpx
import requests
from curl_cffi.requests import Session as CurlSession
from langcodes import Language, tag_is_valid
@@ -51,7 +50,7 @@ class DASH:
self.url = url
@classmethod
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession, httpx.Client]] = None, **args: Any) -> DASH:
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession]] = None, **args: Any) -> DASH:
if not url:
raise requests.URLRequired("DASH manifest URL must be provided for relative path computations.")
if not isinstance(url, str):
@@ -59,15 +58,16 @@ class DASH:
if not session:
session = Session()
elif not isinstance(session, (Session, CurlSession, httpx.Client)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not {session!r}")
try:
res = session.get(url, **args)
res.raise_for_status()
except Exception as e:
raise RuntimeError("Failed to request the MPD document.") from e
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
res = session.get(url, **args)
if res.url != url:
url = res.url
if not res.ok:
raise requests.ConnectionError("Failed to request the MPD document.", response=res)
return DASH.from_text(res.text, url)
@classmethod
@@ -261,8 +261,8 @@ class DASH:
):
if not session:
session = Session()
elif not isinstance(session, (Session, CurlSession, httpx.Client)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not {session!r}")
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
if proxy:
session.proxies.update({"all": proxy})

View File

@@ -15,7 +15,6 @@ from urllib.parse import urljoin
from uuid import UUID
from zlib import crc32
import httpx
import m3u8
import requests
from curl_cffi.requests import Response as CurlResponse
@@ -39,7 +38,7 @@ from unshackle.core.utilities import get_debug_logger, get_extension, is_close_m
class HLS:
def __init__(self, manifest: M3U8, session: Optional[Union[Session, CurlSession, httpx.Client]] = None):
def __init__(self, manifest: M3U8, session: Optional[Union[Session, CurlSession]] = None):
if not manifest:
raise ValueError("HLS manifest must be provided.")
if not isinstance(manifest, M3U8):
@@ -51,7 +50,7 @@ class HLS:
self.session = session or Session()
@classmethod
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession, httpx.Client]] = None, **args: Any) -> HLS:
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession]] = None, **args: Any) -> HLS:
if not url:
raise requests.URLRequired("HLS manifest URL must be provided.")
if not isinstance(url, str):
@@ -59,15 +58,23 @@ class HLS:
if not session:
session = Session()
elif not isinstance(session, (Session, CurlSession, httpx.Client)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not {session!r}")
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
res = session.get(url, **args)
# Handle requests and curl_cffi response objects
if isinstance(res, requests.Response):
if not res.ok:
raise requests.ConnectionError("Failed to request the M3U(8) document.", response=res)
content = res.text
elif isinstance(res, CurlResponse):
if not res.ok:
raise requests.ConnectionError("Failed to request the M3U(8) document.", response=res)
content = res.text
else:
raise TypeError(f"Expected response to be a requests.Response or curl_cffi.Response, not {type(res)}")
try:
res = session.get(url, **args)
res.raise_for_status()
except Exception as e:
raise RuntimeError("Failed to request the M3U(8) document.") from e
content = res.text
master = m3u8.loads(content, uri=url)
return cls(master, session)
@@ -109,9 +116,14 @@ class HLS:
for playlist in self.manifest.playlists:
audio_group = playlist.stream_info.audio
if audio_group:
audio_codec = Audio.Codec.from_codecs(playlist.stream_info.codecs)
audio_codecs_by_group_id[audio_group] = audio_codec
audio_codec: Optional[Audio.Codec] = None
if audio_group and playlist.stream_info.codecs:
try:
audio_codec = Audio.Codec.from_codecs(playlist.stream_info.codecs)
except ValueError:
audio_codec = None
if audio_codec:
audio_codecs_by_group_id[audio_group] = audio_codec
try:
# TODO: Any better way to figure out the primary track type?
@@ -258,7 +270,7 @@ class HLS:
save_path: Path,
save_dir: Path,
progress: partial,
session: Optional[Union[Session, CurlSession, httpx.Client]] = None,
session: Optional[Union[Session, CurlSession]] = None,
proxy: Optional[str] = None,
max_workers: Optional[int] = None,
license_widevine: Optional[Callable] = None,
@@ -267,8 +279,8 @@ class HLS:
) -> None:
if not session:
session = Session()
elif not isinstance(session, (Session, CurlSession, httpx.Client)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not {session!r}")
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
if proxy:
# Handle proxies differently based on session type
@@ -841,7 +853,7 @@ class HLS:
@staticmethod
def parse_session_data_keys(
manifest: M3U8, session: Optional[Union[Session, CurlSession, httpx.Client]] = None
manifest: M3U8, session: Optional[Union[Session, CurlSession]] = None
) -> list[m3u8.model.Key]:
"""Parse `com.apple.hls.keys` session data and return Key objects."""
keys: list[m3u8.model.Key] = []
@@ -916,7 +928,7 @@ class HLS:
def get_track_kid_from_init(
master: M3U8,
track: AnyTrack,
session: Union[Session, CurlSession, httpx.Client],
session: Union[Session, CurlSession],
) -> Optional[UUID]:
"""
Extract the track's Key ID from its init segment (EXT-X-MAP).
@@ -983,7 +995,7 @@ class HLS:
@staticmethod
def get_drm(
key: Union[m3u8.model.SessionKey, m3u8.model.Key],
session: Optional[Union[Session, CurlSession, httpx.Client]] = None,
session: Optional[Union[Session, CurlSession]] = None,
) -> DRM_T:
"""
Convert HLS EXT-X-KEY data to an initialized DRM object.
@@ -995,8 +1007,8 @@ class HLS:
Raises a NotImplementedError if the key system is not supported.
"""
if not isinstance(session, (Session, CurlSession, httpx.Client, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not {type(session)}")
if not isinstance(session, (Session, CurlSession, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {type(session)}")
if not session:
session = Session()

View File

@@ -9,7 +9,6 @@ from functools import partial
from pathlib import Path
from typing import Any, Callable, Optional, Union
import httpx
import requests
from curl_cffi.requests import Session as CurlSession
from langcodes import Language, tag_is_valid
@@ -36,13 +35,13 @@ class ISM:
self.url = url
@classmethod
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession, httpx.Client]] = None, **kwargs: Any) -> "ISM":
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession]] = None, **kwargs: Any) -> "ISM":
if not url:
raise requests.URLRequired("ISM manifest URL must be provided")
if not session:
session = Session()
elif not isinstance(session, (Session, CurlSession, httpx.Client)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not {session!r}")
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
res = session.get(url, **kwargs)
if res.url != url:
url = res.url

View File

@@ -4,7 +4,6 @@ from __future__ import annotations
from typing import Optional, Union
import httpx
import m3u8
from curl_cffi.requests import Session as CurlSession
from requests import Session
@@ -17,7 +16,7 @@ def parse(
master: m3u8.M3U8,
language: str,
*,
session: Optional[Union[Session, CurlSession, httpx.Client]] = None,
session: Optional[Union[Session, CurlSession]] = None,
) -> Tracks:
"""Parse a variant playlist to ``Tracks`` with basic information, defer DRM loading."""
tracks = HLS(master, session=session).to_tracks(language)

View File

@@ -13,7 +13,6 @@ from typing import Any, Callable, Iterable, Optional, Union
from uuid import UUID
from zlib import crc32
import httpx
from curl_cffi.requests import Session as CurlSession
from langcodes import Language
from requests import Session
@@ -614,8 +613,8 @@ class Track:
raise TypeError(f"Expected url to be a {str}, not {type(url)}")
if not isinstance(byte_range, (str, type(None))):
raise TypeError(f"Expected byte_range to be a {str}, not {type(byte_range)}")
if not isinstance(session, (Session, CurlSession, httpx.Client, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession} or {httpx.Client}, not {type(session)}")
if not isinstance(session, (Session, CurlSession, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {type(session)}")
if not url:
if self.descriptor != self.Descriptor.URL:

View File

@@ -221,13 +221,15 @@ class Tracks:
self.videos.sort(key=lambda x: not is_close_match(language, [x.language]))
def sort_audio(self, by_language: Optional[Sequence[Union[str, Language]]] = None) -> None:
"""Sort audio tracks by bitrate, descriptive, and optionally language."""
"""Sort audio tracks by bitrate, Atmos, descriptive, and optionally language."""
if not self.audio:
return
# descriptive
self.audio.sort(key=lambda x: x.descriptive)
# bitrate (within each descriptive group)
# bitrate (highest first)
self.audio.sort(key=lambda x: float(x.bitrate or 0.0), reverse=True)
# Atmos tracks first (prioritize over higher bitrate non-Atmos)
self.audio.sort(key=lambda x: not x.atmos)
# descriptive tracks last
self.audio.sort(key=lambda x: x.descriptive)
# language
for language in reversed(by_language or []):
if str(language) in ("all", "best"):

View File

@@ -0,0 +1,10 @@
from .MSLObject import MSLObject
class MSLKeys(MSLObject):
def __init__(self, encryption=None, sign=None, rsa=None, mastertoken=None, cdm_session=None):
self.encryption = encryption
self.sign = sign
self.rsa = rsa
self.mastertoken = mastertoken
self.cdm_session = cdm_session

View File

@@ -0,0 +1,6 @@
import jsonpickle
class MSLObject:
def __repr__(self):
return "<{} {}>".format(self.__class__.__name__, jsonpickle.encode(self, unpicklable=False))

View File

@@ -0,0 +1,416 @@
import base64
import gzip
import json
import logging
import os
import random
import re
import sys
import time
import zlib
from datetime import datetime
from io import BytesIO
from typing import Optional, Any
import jsonpickle
import requests
from Cryptodome.Cipher import AES, PKCS1_OAEP
from Cryptodome.Hash import HMAC, SHA256
from Cryptodome.PublicKey import RSA
from Cryptodome.Random import get_random_bytes
from Cryptodome.Util import Padding
from unshackle.core.cacher import Cacher
from .MSLKeys import MSLKeys
from .schemes import EntityAuthenticationSchemes # noqa: F401
from .schemes import KeyExchangeSchemes
from .schemes.EntityAuthentication import EntityAuthentication
from .schemes.KeyExchangeRequest import KeyExchangeRequest
from pywidevine import Cdm, PSSH, Key
class MSL:
log = logging.getLogger("MSL")
def __init__(self, session, endpoint, sender, keys, message_id, user_auth=None):
self.session = session
self.endpoint = endpoint
self.sender = sender
self.keys = keys
self.user_auth = user_auth
self.message_id = message_id
@classmethod
def handshake(cls, scheme: KeyExchangeSchemes, session: requests.Session, endpoint: str, sender: str, cache: Cacher, cdm: Optional[Cdm] = None, config: Any = None):
cache = cache.get(sender)
message_id = random.randint(0, pow(2, 52))
msl_keys = MSL.load_cache_data(cache)
if msl_keys is not None:
cls.log.info("Using cached MSL data")
else:
msl_keys = MSLKeys()
if scheme != KeyExchangeSchemes.Widevine:
msl_keys.rsa = RSA.generate(2048)
if scheme == KeyExchangeSchemes.Widevine:
if not cdm:
raise Exception('Key exchange scheme Widevine but CDM instance is None.')
session_id = cdm.open()
msl_keys.cdm_session = session_id
cdm.set_service_certificate(session_id, config["certificate"])
challenge = cdm.get_license_challenge(
session_id=session_id,
pssh=PSSH("AAAANHBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAABQIARIQAAAAAAPSZ0kAAAAAAAAAAA=="),
license_type="OFFLINE",
privacy_mode=True,
)
keyrequestdata = KeyExchangeRequest.Widevine(challenge)
entityauthdata = EntityAuthentication.Unauthenticated(sender)
# entityauthdata = EntityAuthentication.Widevine("TV", base64.b64encode(challenge).decode())
else:
entityauthdata = EntityAuthentication.Unauthenticated(sender)
keyrequestdata = KeyExchangeRequest.AsymmetricWrapped(
keypairid="superKeyPair",
mechanism="JWK_RSA",
publickey=msl_keys.rsa.publickey().exportKey(format="DER")
)
data = jsonpickle.encode({
"entityauthdata": entityauthdata,
"headerdata": base64.b64encode(MSL.generate_msg_header(
message_id=message_id,
sender=sender,
is_handshake=True,
keyrequestdata=keyrequestdata
).encode("utf-8")).decode("utf-8"),
"signature": ""
}, unpicklable=False)
data += json.dumps({
"payload": base64.b64encode(json.dumps({
"messageid": message_id,
"data": "",
"sequencenumber": 1,
"endofmsg": True
}).encode("utf-8")).decode("utf-8"),
"signature": ""
})
try:
r = session.post(
url=endpoint,
data=data
)
except requests.HTTPError as e:
raise Exception(f"- Key exchange failed, response data is unexpected: {e.response.text}")
key_exchange = r.json() # expecting no payloads, so this is fine
if "errordata" in key_exchange:
raise Exception("- Key exchange failed: " + json.loads(base64.b64decode(
key_exchange["errordata"]
).decode())["errormsg"])
# parse the crypto keys
key_response_data = json.JSONDecoder().decode(base64.b64decode(
key_exchange["headerdata"]
).decode("utf-8"))["keyresponsedata"]
if key_response_data["scheme"] != str(scheme):
raise Exception("- Key exchange scheme mismatch occurred")
key_data = key_response_data["keydata"]
if scheme == KeyExchangeSchemes.Widevine:
if not msl_keys.cdm_session:
raise Exception("- No CDM session available")
if not cdm:
raise Exception("- No CDM available")
cdm.parse_license(msl_keys.cdm_session, key_data["cdmkeyresponse"])
keys = cdm.get_keys(msl_keys.cdm_session)
cls.log.info(f"Keys: {keys}")
encryption_key = MSL.get_widevine_key(
kid=base64.b64decode(key_data["encryptionkeyid"]),
keys=keys,
permissions=["allow_encrypt", "allow_decrypt"]
)
msl_keys.encryption = encryption_key
cls.log.info(f"Encryption key: {encryption_key}")
sign = MSL.get_widevine_key(
kid=base64.b64decode(key_data["hmackeyid"]),
keys=keys,
permissions=["allow_sign", "allow_signature_verify"]
)
cls.log.info(f"Sign key: {sign}")
msl_keys.sign = sign
elif scheme == KeyExchangeSchemes.AsymmetricWrapped:
cipher_rsa = PKCS1_OAEP.new(msl_keys.rsa)
msl_keys.encryption = MSL.base64key_decode(
json.JSONDecoder().decode(cipher_rsa.decrypt(
base64.b64decode(key_data["encryptionkey"])
).decode("utf-8"))["k"]
)
msl_keys.sign = MSL.base64key_decode(
json.JSONDecoder().decode(cipher_rsa.decrypt(
base64.b64decode(key_data["hmackey"])
).decode("utf-8"))["k"]
)
msl_keys.mastertoken = key_response_data["mastertoken"]
MSL.cache_keys(msl_keys, cache)
cls.log.info("MSL handshake successful")
return cls(
session=session,
endpoint=endpoint,
sender=sender,
keys=msl_keys,
message_id=message_id
)
@staticmethod
def load_cache_data(cacher: Cacher):
if not cacher or cacher == {}:
return None
# with open(msl_keys_path, encoding="utf-8") as fd:
# msl_keys = jsonpickle.decode(fd.read())
msl_keys = cacher.data
if msl_keys.rsa:
# noinspection PyTypeChecker
# expects RsaKey, but is a string, this is because jsonpickle can't pickle RsaKey object
# so as a workaround it exports to PEM, and then when reading, it imports that PEM back
# to an RsaKey :)
msl_keys.rsa = RSA.importKey(msl_keys.rsa)
# If it's expired or close to, return None as it's unusable
if msl_keys.mastertoken and ((datetime.utcfromtimestamp(int(json.JSONDecoder().decode(
base64.b64decode(msl_keys.mastertoken["tokendata"]).decode("utf-8")
)["expiration"])) - datetime.now()).total_seconds() / 60 / 60) < 10:
return None
return msl_keys
@staticmethod
def cache_keys(msl_keys, cache: Cacher):
# os.makedirs(os.path.dirname(cache), exist_ok=True)
if msl_keys.rsa:
# jsonpickle can't pickle RsaKey objects :(
msl_keys.rsa = msl_keys.rsa.export_key()
# with open(cache, "w", encoding="utf-8") as fd:
# fd.write()
cache.set(msl_keys)
if msl_keys.rsa:
# re-import now
msl_keys.rsa = RSA.importKey(msl_keys.rsa)
@staticmethod
def generate_msg_header(message_id, sender, is_handshake, userauthdata=None, keyrequestdata=None,
compression="GZIP"):
"""
The MSL header carries all MSL data used for entity and user authentication, message encryption
and verification, and service tokens. Portions of the MSL header are encrypted.
https://github.com/Netflix/msl/wiki/Messages#header-data
:param message_id: number against which payload chunks are bound to protect against replay.
:param sender: ESN
:param is_handshake: This flag is set true if the message is a handshake message and will not include any
payload chunks. It will include keyrequestdata.
:param userauthdata: UserAuthData
:param keyrequestdata: KeyRequestData
:param compression: Supported compression algorithms.
:return: The base64 encoded JSON String of the header
"""
header_data = {
"messageid": message_id,
"renewable": True, # MUST be True if is_handshake
"handshake": is_handshake,
"capabilities": {
"compressionalgos": [compression] if compression else [],
"languages": ["en-US"], # bcp-47
"encoderformats": ["JSON"]
},
"timestamp": int(time.time()),
# undocumented or unused:
"sender": sender,
"nonreplayable": False,
"recipient": "Netflix",
}
if userauthdata:
header_data["userauthdata"] = userauthdata
if keyrequestdata:
header_data["keyrequestdata"] = [keyrequestdata]
return jsonpickle.encode(header_data, unpicklable=False)
@classmethod
def get_widevine_key(cls, kid, keys: list[Key], permissions):
cls.log.info(f"KID: {Key.kid_to_uuid(kid)}")
for key in keys:
# cls.log.info(f"KEY: {key.kid_to_uuid}")
if key.kid != Key.kid_to_uuid(kid):
continue
if key.type != "OPERATOR_SESSION":
cls.log.warning(f"Widevine Key Exchange: Wrong key type (not operator session) key {key}")
continue
if not set(permissions) <= set(key.permissions):
cls.log.warning(f"Widevine Key Exchange: Incorrect permissions, key {key}, needed perms {permissions}")
continue
return key.key
return None
def send_message(self, endpoint, params, application_data, userauthdata=None):
message = self.create_message(application_data, userauthdata)
res = self.session.post(url=endpoint, data=message, params=params)
header, payload_data = self.parse_message(res.text)
if "errordata" in header:
raise Exception(
"- MSL response message contains an error: {}".format(
json.loads(base64.b64decode(header["errordata"].encode("utf-8")).decode("utf-8"))
)
)
return header, payload_data
def create_message(self, application_data, userauthdata=None):
self.message_id += 1 # new message must ue a new message id
headerdata = self.encrypt(self.generate_msg_header(
message_id=self.message_id,
sender=self.sender,
is_handshake=False,
userauthdata=userauthdata
))
header = json.dumps({
"headerdata": base64.b64encode(headerdata.encode("utf-8")).decode("utf-8"),
"signature": self.sign(headerdata).decode("utf-8"),
"mastertoken": self.keys.mastertoken
})
payload_chunks = [self.encrypt(json.dumps({
"messageid": self.message_id,
"data": self.gzip_compress(json.dumps(application_data).encode("utf-8")).decode("utf-8"),
"compressionalgo": "GZIP",
"sequencenumber": 1, # todo ; use sequence_number from master token instead?
"endofmsg": True
}))]
message = header
for payload_chunk in payload_chunks:
message += json.dumps({
"payload": base64.b64encode(payload_chunk.encode("utf-8")).decode("utf-8"),
"signature": self.sign(payload_chunk).decode("utf-8")
})
return message
def decrypt_payload_chunks(self, payload_chunks):
"""
Decrypt and extract data from payload chunks
:param payload_chunks: List of payload chunks
:return: json object
"""
raw_data = ""
for payload_chunk in payload_chunks:
# todo ; verify signature of payload_chunk["signature"] against payload_chunk["payload"]
# expecting base64-encoded json string
payload_chunk = json.loads(base64.b64decode(payload_chunk["payload"]).decode("utf-8"))
# decrypt the payload
payload_decrypted = AES.new(
key=self.keys.encryption,
mode=AES.MODE_CBC,
iv=base64.b64decode(payload_chunk["iv"])
).decrypt(base64.b64decode(payload_chunk["ciphertext"]))
payload_decrypted = Padding.unpad(payload_decrypted, 16)
payload_decrypted = json.loads(payload_decrypted.decode("utf-8"))
# decode and uncompress data if compressed
payload_data = base64.b64decode(payload_decrypted["data"])
if payload_decrypted.get("compressionalgo") == "GZIP":
payload_data = zlib.decompress(payload_data, 16 + zlib.MAX_WBITS)
raw_data += payload_data.decode("utf-8")
data = json.loads(raw_data)
if "error" in data:
error = data["error"]
error_display = error.get("display")
error_detail = re.sub(r" \(E3-[^)]+\)", "", error.get("detail", ""))
if error_display:
self.log.critical(f"- {error_display}")
if error_detail:
self.log.critical(f"- {error_detail}")
if not (error_display or error_detail):
self.log.critical(f"- {error}")
raise Exception(f"- MSL response message contains an error: {error}")
# sys.exit(1)
return data["result"]
def parse_message(self, message):
"""
Parse an MSL message into a header and list of payload chunks
:param message: MSL message
:returns: a 2-item tuple containing message and list of payload chunks if available
"""
parsed_message = json.loads("[{}]".format(message.replace("}{", "},{")))
header = parsed_message[0]
encrypted_payload_chunks = parsed_message[1:] if len(parsed_message) > 1 else []
if encrypted_payload_chunks:
payload_chunks = self.decrypt_payload_chunks(encrypted_payload_chunks)
else:
payload_chunks = {}
return header, payload_chunks
@staticmethod
def gzip_compress(data):
out = BytesIO()
with gzip.GzipFile(fileobj=out, mode="w") as fd:
fd.write(data)
return base64.b64encode(out.getvalue())
@staticmethod
def base64key_decode(payload):
length = len(payload) % 4
if length == 2:
payload += "=="
elif length == 3:
payload += "="
elif length != 0:
raise ValueError("Invalid base64 string")
return base64.urlsafe_b64decode(payload.encode("utf-8"))
def encrypt(self, plaintext):
"""
Encrypt the given Plaintext with the encryption key
:param plaintext:
:return: Serialized JSON String of the encryption Envelope
"""
iv = get_random_bytes(16)
return json.dumps({
"ciphertext": base64.b64encode(
AES.new(
self.keys.encryption,
AES.MODE_CBC,
iv
).encrypt(
Padding.pad(plaintext.encode("utf-8"), 16)
)
).decode("utf-8"),
"keyid": "{}_{}".format(self.sender, json.loads(
base64.b64decode(self.keys.mastertoken["tokendata"]).decode("utf-8")
)["sequencenumber"]),
"sha256": "AA==",
"iv": base64.b64encode(iv).decode("utf-8")
})
def sign(self, text):
"""
Calculates the HMAC signature for the given text with the current sign key and SHA256
:param text:
:return: Base64 encoded signature
"""
return base64.b64encode(HMAC.new(self.keys.sign, text.encode("utf-8"), SHA256).digest())

View File

@@ -0,0 +1,59 @@
from .. import EntityAuthenticationSchemes
from ..MSLObject import MSLObject
# noinspection PyPep8Naming
class EntityAuthentication(MSLObject):
def __init__(self, scheme, authdata):
"""
Data used to identify and authenticate the entity associated with a message.
https://github.com/Netflix/msl/wiki/Entity-Authentication-%28Configuration%29
:param scheme: Entity Authentication Scheme identifier
:param authdata: Entity Authentication data
"""
self.scheme = str(scheme)
self.authdata = authdata
@classmethod
def Unauthenticated(cls, identity):
"""
The unauthenticated entity authentication scheme does not provide encryption or authentication and only
identifies the entity. Therefore entity identities can be harvested and spoofed. The benefit of this
authentication scheme is that the entity has control over its identity. This may be useful if the identity is
derived from or related to other data, or if retaining the identity is desired across state resets or in the
event of MSL errors requiring entity re-authentication.
"""
return cls(
scheme=EntityAuthenticationSchemes.Unauthenticated,
authdata={"identity": identity}
)
@classmethod
def Widevine(cls, devtype, keyrequest):
"""
The Widevine entity authentication scheme is used by devices with the Widevine CDM. It does not provide
encryption or authentication and only identifies the entity. Therefore entity identities can be harvested
and spoofed. The entity identity is composed from the provided device type and Widevine key request data. The
Widevine CDM properties can be extracted from the key request data.
When coupled with the Widevine key exchange scheme, the entity identity can be cryptographically validated by
comparing the entity authentication key request data against the key exchange key request data.
Note that the local entity will not know its entity identity when using this scheme.
> Devtype
An arbitrary value identifying the device type the local entity wishes to assume. The data inside the Widevine
key request may be optionally used to validate the claimed device type.
:param devtype: Local entity device type
:param keyrequest: Widevine key request
"""
return cls(
scheme=EntityAuthenticationSchemes.Widevine,
authdata={
"devtype": devtype,
"keyrequest": keyrequest
}
)

View File

@@ -0,0 +1,80 @@
import base64
from .. import KeyExchangeSchemes
from ..MSLObject import MSLObject
# noinspection PyPep8Naming
class KeyExchangeRequest(MSLObject):
def __init__(self, scheme, keydata):
"""
Session key exchange data from a requesting entity.
https://github.com/Netflix/msl/wiki/Key-Exchange-%28Configuration%29
:param scheme: Key Exchange Scheme identifier
:param keydata: Key Request data
"""
self.scheme = str(scheme)
self.keydata = keydata
@classmethod
def AsymmetricWrapped(cls, keypairid, mechanism, publickey):
"""
Asymmetric wrapped key exchange uses a generated ephemeral asymmetric key pair for key exchange. It will
typically be used when there is no other data or keys from which to base secure key exchange.
This mechanism provides perfect forward secrecy but does not guarantee that session keys will only be available
to the requesting entity if the requesting MSL stack has been modified to perform the operation on behalf of a
third party.
> Key Pair ID
The key pair ID is included as a sanity check.
> Mechanism & Public Key
The following mechanisms are associated public key formats are currently supported.
Field Public Key Format Description
RSA SPKI RSA-OAEP encrypt/decrypt
ECC SPKI ECIES encrypt/decrypt
JWEJS_RSA SPKI RSA-OAEP JSON Web Encryption JSON Serialization
JWE_RSA SPKI RSA-OAEP JSON Web Encryption Compact Serialization
JWK_RSA SPKI RSA-OAEP JSON Web Key
JWK_RSAES SPKI RSA PKCS#1 JSON Web Key
:param keypairid: key pair ID
:param mechanism: asymmetric key type
:param publickey: public key
"""
return cls(
scheme=KeyExchangeSchemes.AsymmetricWrapped,
keydata={
"keypairid": keypairid,
"mechanism": mechanism,
"publickey": base64.b64encode(publickey).decode("utf-8")
}
)
@classmethod
def Widevine(cls, keyrequest):
"""
Google Widevine provides a secure key exchange mechanism. When requested the Widevine component will issue a
one-time use key request. The Widevine server library can be used to authenticate the request and return
randomly generated symmetric keys in a protected key response bound to the request and Widevine client library.
The key response also specifies the key identities, types and their permitted usage.
The Widevine key request also contains a model identifier and a unique device identifier with an expectation of
long-term persistence. These values are available from the Widevine client library and can be retrieved from
the key request by the Widevine server library.
The Widevine client library will protect the returned keys from inspection or misuse.
:param keyrequest: Base64-encoded Widevine CDM license challenge (PSSH: b'\x0A\x7A\x00\x6C\x38\x2B')
"""
if not isinstance(keyrequest, str):
keyrequest = base64.b64encode(keyrequest).decode()
return cls(
scheme=KeyExchangeSchemes.Widevine,
keydata={"keyrequest": keyrequest}
)

View File

@@ -0,0 +1,59 @@
from ..MSLObject import MSLObject
from . import UserAuthenticationSchemes
# noinspection PyPep8Naming
class UserAuthentication(MSLObject):
def __init__(self, scheme, authdata):
"""
Data used to identify and authenticate the user associated with a message.
https://github.com/Netflix/msl/wiki/User-Authentication-%28Configuration%29
:param scheme: User Authentication Scheme identifier
:param authdata: User Authentication data
"""
self.scheme = str(scheme)
self.authdata = authdata
@classmethod
def EmailPassword(cls, email, password):
"""
Email and password is a standard user authentication scheme in wide use.
:param email: user email address
:param password: user password
"""
return cls(
scheme=UserAuthenticationSchemes.EmailPassword,
authdata={
"email": email,
"password": password
}
)
@classmethod
def NetflixIDCookies(cls, netflixid, securenetflixid):
"""
Netflix ID HTTP cookies are used when the user has previously logged in to a web site. Possession of the
cookies serves as proof of user identity, in the same manner as they do when communicating with the web site.
The Netflix ID cookie and Secure Netflix ID cookie are HTTP cookies issued by the Netflix web site after
subscriber login. The Netflix ID cookie is encrypted and identifies the subscriber and analogous to a
subscribers username. The Secure Netflix ID cookie is tied to a Netflix ID cookie and only sent over HTTPS
and analogous to a subscribers password.
In some cases the Netflix ID and Secure Netflix ID cookies will be unavailable to the MSL stack or application.
If either or both of the Netflix ID or Secure Netflix ID cookies are absent in the above data structure the
HTTP cookie headers will be queried for it; this is only acceptable when HTTPS is used as the underlying
transport protocol.
:param netflixid: Netflix ID cookie
:param securenetflixid: Secure Netflix ID cookie
"""
return cls(
scheme=UserAuthenticationSchemes.NetflixIDCookies,
authdata={
"netflixid": netflixid,
"securenetflixid": securenetflixid
}
)

View File

@@ -0,0 +1,24 @@
from enum import Enum
class Scheme(Enum):
def __str__(self):
return str(self.value)
class EntityAuthenticationSchemes(Scheme):
"""https://github.com/Netflix/msl/wiki/Entity-Authentication-%28Configuration%29"""
Unauthenticated = "NONE"
Widevine = "WIDEVINE"
class UserAuthenticationSchemes(Scheme):
"""https://github.com/Netflix/msl/wiki/User-Authentication-%28Configuration%29"""
EmailPassword = "EMAIL_PASSWORD"
NetflixIDCookies = "NETFLIXID"
class KeyExchangeSchemes(Scheme):
"""https://github.com/Netflix/msl/wiki/Key-Exchange-%28Configuration%29"""
AsymmetricWrapped = "ASYMMETRIC_WRAPPED"
Widevine = "WIDEVINE"

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

259
unshackle/unshackle.yaml Normal file
View File

@@ -0,0 +1,259 @@
# Group or Username to postfix to the end of all download filenames following a dash
tag: Kenzuya
# Enable/disable tagging with group name (default: true)
tag_group_name: true
# Enable/disable tagging with IMDB/TMDB/TVDB details (default: true)
tag_imdb_tmdb: true
# Set terminal background color (custom option not in CONFIG.md)
set_terminal_bg: false
# Set file naming convention
# true for style - Prime.Suspect.S07E01.The.Final.Act.Part.One.1080p.ITV.WEB-DL.AAC2.0.H.264
# false for style - Prime Suspect S07E01 The Final Act - Part One
scene_naming: true
# Whether to include the year in series names for episodes and folders (default: true)
# true for style - Show Name (2023) S01E01 Episode Name
# false for style - Show Name S01E01 Episode Name
series_year: false
# Check for updates from GitHub repository on startup (default: true)
update_checks: true
# How often to check for updates, in hours (default: 24)
update_check_interval: 24
# Title caching configuration
# Cache title metadata to reduce redundant API calls
title_cache_enabled: true # Enable/disable title caching globally (default: true)
title_cache_time: 1800 # Cache duration in seconds (default: 1800 = 30 minutes)
title_cache_max_retention: 86400 # Maximum cache retention for fallback when API fails (default: 86400 = 24 hours)
# Muxing configuration
muxing:
set_title: true
# Login credentials for each Service
credentials:
# Direct credentials (no profile support)
EXAMPLE: email@example.com:password
# Per-profile credentials with default fallback
SERVICE_NAME:
default: default@email.com:password # Used when no -p/--profile is specified
profile1: user1@email.com:password1
profile2: user2@email.com:password2
# Per-profile credentials without default (requires -p/--profile)
SERVICE_NAME2:
john: john@example.com:johnspassword
jane: jane@example.com:janespassword
# You can also use list format for passwords with special characters
SERVICE_NAME3:
default: ["user@email.com", ":PasswordWith:Colons"]
Netflix:
default: ["ariel-prinsess828@ezweb.ne.jp", "AiNe892186"]
# default: ["pbgarena0838@gmail.com", "Andhika1978"]
# Override default directories used across unshackle
directories:
cache: Cache
# cookies: Cookies
dcsl: DCSL # Device Certificate Status List
downloads: /home/kenzuya/Mounts/ketuakenzuya/Downloads
logs: Logs
temp: Temp
# wvds: WVDs
# Additional directories that can be configured:
# commands: Commands
# services:
# - /path/to/services
# - /other/path/to/services
# vaults: Vaults
# fonts: Fonts
# Pre-define which Widevine or PlayReady device to use for each Service
cdm:
# Global default CDM device (fallback for all services/profiles)
default: chromecdm
# Direct service-specific CDM
DIFFERENT_EXAMPLE: PRD_1
# Per-profile CDM configuration
EXAMPLE:
john_sd: chromecdm_903_l3 # Profile 'john_sd' uses Chrome CDM L3
jane_uhd: nexus_5_l1 # Profile 'jane_uhd' uses Nexus 5 L1
default: generic_android_l3 # Default CDM for this service
# Use pywidevine Serve-compliant Remote CDMs
remote_cdm:
- name: "chromecdm"
device_name: widevine
device_type: CHROME
system_id: 36586
security_level: 3
type: "decrypt_labs"
host: https://keyxtractor.decryptlabs.com
secret: 7547150416_41da0a32d6237d83_KeyXtractor_api_ext
- name: "android"
device_name: andorid
device_type: ANDROID
system_id: 8131
security_level: 1
type: "decrypt_labs"
host: https://keyxtractor.decryptlabs.com
secret: decrypt_labs_special_ultimate
# Key Vaults store your obtained Content Encryption Keys (CEKs)
# Use 'no_push: true' to prevent a vault from receiving pushed keys
# while still allowing it to provide keys when requested
key_vaults:
- type: SQLite
name: Local
path: key_store.db
- type: HTTP
name: "DRMLab Vault"
host: "https://api.drmlab.io/vault/"
username: "unshackle"
password: "gEX75q7I5YVkvgF5SUkcNd41IbGrDtTT"
api_mode: "json"
- type: HTTP
name: "Decrypt Labs - Key Vault"
api_mode: "decrypt_labs"
host: "https://keyvault.decryptlabs.com"
password: "7547150416_41da0a32d6237d83_KeyXtractor_api_ext"
# Additional vault types:
# - type: API
# name: "Remote Vault"
# uri: "https://key-vault.example.com"
# token: "secret_token"
# no_push: true # This vault will only provide keys, not receive them
# - type: MySQL
# name: "MySQL Vault"
# host: "127.0.0.1"
# port: 3306
# database: vault
# username: user
# password: pass
# no_push: false # Default behavior - vault both provides and receives keys
# Choose what software to use to download data
downloader: aria2c
# Options: requests | aria2c | curl_impersonate | n_m3u8dl_re
# Can also be a mapping:
# downloader:
# NF: requests
# AMZN: n_m3u8dl_re
# DSNP: n_m3u8dl_re
# default: requests
# aria2c downloader configuration
aria2c:
max_concurrent_downloads: 4
max_connection_per_server: 3
split: 5
file_allocation: falloc # none | prealloc | falloc | trunc
# N_m3u8DL-RE downloader configuration
n_m3u8dl_re:
thread_count: 16
ad_keyword: "advertisement"
use_proxy: true
# curl_impersonate downloader configuration
curl_impersonate:
browser: chrome120
# Pre-define default options and switches of the dl command
dl:
sub_format: srt
downloads: 4
workers: 16
lang:
- orig
- id
EXAMPLE:
bitrate: CBR
# Chapter Name to use when exporting a Chapter without a Name
chapter_fallback_name: "Chapter {j:02}"
# Case-Insensitive dictionary of headers for all Services
headers:
Accept-Language: "en-US,en;q=0.8"
User-Agent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.75 Safari/537.36"
# Override default filenames used across unshackle
filenames:
log: "unshackle_{name}_{time}.log"
config: "config.yaml"
root_config: "unshackle.yaml"
chapters: "Chapters_{title}_{random}.txt"
subtitle: "Subtitle_{id}_{language}.srt"
# API key for The Movie Database (TMDB)
tmdb_api_key: "8f5c14ef648a0abdd262cf809e11fcd4"
# conversion_method:
# - auto (default): Smart routing - subby for WebVTT/SAMI, standard for others
# - subby: Always use subby with advanced processing
# - pycaption: Use only pycaption library (no SubtitleEdit, no subby)
# - subtitleedit: Prefer SubtitleEdit when available, fall back to pycaption
subtitle:
conversion_method: auto
sdh_method: auto
# Configuration for pywidevine's serve functionality
serve:
users:
secret_key_for_user:
devices:
- generic_nexus_4464_l3
username: user
# devices:
# - '/path/to/device.wvd'
# Configuration data for each Service
services:
# Service-specific configuration goes here
# Profile-specific configurations can be nested under service names
# Example: with profile-specific device configs
EXAMPLE:
# Global service config
api_key: "service_api_key"
# Profile-specific device configurations
profiles:
john_sd:
device:
app_name: "AIV"
device_model: "SHIELD Android TV"
jane_uhd:
device:
app_name: "AIV"
device_model: "Fire TV Stick 4K"
# Example: Service with different regions per profile
SERVICE_NAME:
profiles:
us_account:
region: "US"
api_endpoint: "https://api.us.service.com"
uk_account:
region: "GB"
api_endpoint: "https://api.uk.service.com"
# External proxy provider services
proxy_providers:
basic:
SG:
- "http://127.0.0.1:6004"
surfsharkvpn:
username: SkyCBP7kH8KqxDwy5Qw36mQn # Service credentials from https://my.surfshark.com/vpn/manual-setup/main/openvpn
password: pcmewxKTNPvLENdbKJGh8Cgt # Service credentials (not your login password)

42
uv.lock generated
View File

@@ -608,28 +608,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
]
[[package]]
name = "h2"
version = "4.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "hpack" },
{ name = "hyperframe" },
]
sdist = { url = "https://files.pythonhosted.org/packages/1d/17/afa56379f94ad0fe8defd37d6eb3f89a25404ffc71d4d848893d270325fc/h2-4.3.0.tar.gz", hash = "sha256:6c59efe4323fa18b47a632221a1888bd7fde6249819beda254aeca909f221bf1", size = 2152026, upload-time = "2025-08-23T18:12:19.778Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/69/b2/119f6e6dcbd96f9069ce9a2665e0146588dc9f88f29549711853645e736a/h2-4.3.0-py3-none-any.whl", hash = "sha256:c438f029a25f7945c69e0ccf0fb951dc3f73a5f6412981daee861431b70e2bdd", size = 61779, upload-time = "2025-08-23T18:12:17.779Z" },
]
[[package]]
name = "hpack"
version = "4.1.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/2c/48/71de9ed269fdae9c8057e5a4c0aa7402e8bb16f2c6e90b3aa53327b113f8/hpack-4.1.0.tar.gz", hash = "sha256:ec5eca154f7056aa06f196a557655c5b009b382873ac8d1e66e79e87535f1dca", size = 51276, upload-time = "2025-01-22T21:44:58.347Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/07/c6/80c95b1b2b94682a72cbdbfb85b81ae2daffa4291fbfa1b1464502ede10d/hpack-4.1.0-py3-none-any.whl", hash = "sha256:157ac792668d995c657d93111f46b4535ed114f0c9c8d672271bbec7eae1b496", size = 34357, upload-time = "2025-01-22T21:44:56.92Z" },
]
[[package]]
name = "httpcore"
version = "1.0.9"
@@ -658,20 +636,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
]
[package.optional-dependencies]
http2 = [
{ name = "h2" },
]
[[package]]
name = "hyperframe"
version = "6.1.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/02/e7/94f8232d4a74cc99514c13a9f995811485a6903d48e5d952771ef6322e30/hyperframe-6.1.0.tar.gz", hash = "sha256:f630908a00854a7adeabd6382b43923a4c4cd4b821fcb527e6ab9e15382a3b08", size = 26566, upload-time = "2025-01-22T21:41:49.302Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/48/30/47d0bf6072f7252e6521f3447ccfa40b421b6824517f82854703d0f5a98b/hyperframe-6.1.0-py3-none-any.whl", hash = "sha256:b03380493a519fce58ea5af42e4a42317bf9bd425596f7a0835ffce80f1a42e5", size = 13007, upload-time = "2025-01-22T21:41:47.295Z" },
]
[[package]]
name = "identify"
version = "2.6.16"
@@ -1663,7 +1627,7 @@ wheels = [
[[package]]
name = "unshackle"
version = "2.4.0"
version = "3.0.0"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
@@ -1678,7 +1642,7 @@ dependencies = [
{ name = "curl-cffi" },
{ name = "filelock" },
{ name = "fonttools" },
{ name = "httpx", extra = ["http2"] },
{ name = "httpx" },
{ name = "jsonpickle" },
{ name = "langcodes" },
{ name = "language-data" },
@@ -1736,7 +1700,7 @@ requires-dist = [
{ name = "curl-cffi", specifier = ">=0.7.0b4,<0.14" },
{ name = "filelock", specifier = ">=3.20.3,<4" },
{ name = "fonttools", specifier = ">=4.60.2,<5" },
{ name = "httpx", extras = ["http2"], specifier = ">=0.28.1,<0.29" },
{ name = "httpx", specifier = ">=0.28.1,<0.29" },
{ name = "jsonpickle", specifier = ">=3.0.4,<5" },
{ name = "langcodes", specifier = ">=3.4.0,<4" },
{ name = "language-data", specifier = ">=1.4.0" },