feat(remote): add server-CDM mode, manifest transfer, and region-aware proxy

Server-side:
- Add server_cdm mode: server handles full CDM licensing using its own devices, returns KID:KEY pairs instead of raw license bytes
- Support batch license resolution for multiple tracks in one request
- Extract DRM from manifest ContentProtection when track.drm is empty
- Serialize DASH/ISM manifest XML as base64 in /tracks response
- Include session cookies/headers and server_cdm_type in /tracks response
- Detect server CDM type from actual track DRM + configured devices
- Check server region against client_region to skip unnecessary proxy
- Support decrypt_labs and custom_api remote CDMs for both WV and PR

Client-side:
- Re-parse DASH/ISM manifests locally from base64 to populate track.data
- Match remote tracks to re-parsed tracks by ID with attribute fallback
- Copy DRM objects from re-parsed manifests to remote tracks
- Pre-fetch keys via resolve_server_keys() before downloads start
- Fallback per-track licensing via _proxy_license during download
- Apply session cookies/headers from server for CDN access
- Apply downloader/decryption config directly for remote services
- Preserve pre-injected content_keys during DASH DRM override
- Skip redundant CDM calls when all KIDs already have keys

Docs:
- Add comprehensive remote-services-flow.md with Mermaid diagrams covering proxy mode, server-CDM mode, manifest transfer, and config
This commit is contained in:
Andy
2026-03-18 19:30:59 -06:00
parent 7fafdf024c
commit b1a5f8f37f
5 changed files with 1196 additions and 135 deletions
+283
View File
@@ -0,0 +1,283 @@
# Remote Services — Client ↔ Server Architecture
The `--remote` flag on the `dl` command connects to a remote unshackle server
(`unshackle serve`) that holds service plugins. The client never has service
code — it sends credentials/cookies, the server authenticates and fetches
titles/tracks, and the client handles downloading, decryption, and muxing locally.
## How It Works
The `RemoteService` adapter in `remote_service.py` implements the same interface
as a local `Service`. From dl.py's perspective, it's just another service — the
entire download pipeline runs unchanged.
```
unshackle dl --remote [-s server_name] SERVICE_TAG TITLE_ID [options]
```
## Session Lifecycle
```mermaid
sequenceDiagram
participant Client as Client (dl --remote)
participant Server as Server (serve API)
participant Service as Service Plugin
Note over Client: Load credentials/cookies locally
Note over Client: Detect client region via IP check
Client->>Server: POST /api/session/create
Note right of Client: {service, title_id, credentials,<br/>cookies, client_region, profile}
Note over Server: Check if server region matches client<br/>Skip proxy if same region
Server->>Service: authenticate(cookies, credential)
Service-->>Server: Authenticated session
Server-->>Client: {session_id, service}
Client->>Server: GET /api/session/{id}/titles
Server->>Service: get_titles(title_id)
Server-->>Client: Serialized titles (episodes/movies)
Note over Client: dl.py filters by --wanted
Client->>Server: POST /api/session/{id}/tracks
Server->>Service: get_tracks(title)
Server->>Service: get_chapters(title)
Note over Server: Extract manifest XML as base64<br/>Extract service session cookies/headers<br/>Detect server CDM type from track DRM + config
Server-->>Client: Tracks + manifests + chapters +<br/>session cookies + server_cdm_type
Note over Client: Re-parse manifest locally (DASH/ISM)<br/>HLS tracks download from URL directly
Note over Client: Match tracks by ID to populate track.data
Note over Client: dl.py runs full track selection pipeline
rect rgb(40, 40, 60)
Note over Client: Licensing (depends on mode)
end
rect rgb(40, 60, 40)
Note over Client: Download + Post-Processing (all local)
Note over Client: Concurrent downloads via ThreadPoolExecutor
Note over Client: Decrypt (mp4decrypt / Shaka Packager)
Note over Client: FFMPEG repack
Note over Client: Subtitle conversion
Note over Client: Hybrid HDR10+DV injection (dovi_tool)
Note over Client: Per-resolution muxing (mkvmerge)
Note over Client: Output naming from template
Note over Client: Tagging (TMDB/IMDB/TVDB)
end
Client->>Server: DELETE /api/session/{id}
Server-->>Client: Session cleaned up
```
---
## Licensing Modes
### Proxy Mode (`server_cdm: false` — default)
The client has its own CDM (WVD/PRD file). The server only proxies the license
request through the authenticated service session.
```mermaid
sequenceDiagram
participant Client as Client
participant Server as Server
participant Service as Service Plugin
participant CDM as Client CDM (local WVD/PRD)
Note over Client: track.download() discovers DRM<br/>(PSSH from manifest/init data)
Client->>CDM: Generate challenge from PSSH
CDM-->>Client: Challenge bytes
Client->>Server: POST /api/session/{id}/license
Note right of Client: {track_id, challenge (base64),<br/>drm_type, pssh}
Server->>Service: get_widevine_license(challenge, title, track)<br/>or get_playready_license(challenge, title, track)
Service-->>Server: Raw license response bytes
Server-->>Client: {license: base64_encoded_bytes}
Client->>CDM: Parse license response
CDM-->>Client: Content keys (KID:KEY)
Note over Client: Keys stored in local vaults
Note over Client: Decrypt track with keys
```
**Key points:**
- Client must have a local CDM device file (`.wvd` or `.prd`)
- Server never sees decryption keys — only forwards encrypted license blob
- Client parses the license locally to extract keys
- Keys are cached in local vaults for future use
### Server-CDM Mode (`server_cdm: true`)
The server handles all CDM operations using its own devices. The client does
not need a local CDM. There are two paths depending on when DRM is discovered:
#### Path A: Pre-fetch (DASH services with manifest DRM)
For services where DRM info is in the manifest (ContentProtection elements),
the server resolves keys before downloads start.
```mermaid
sequenceDiagram
participant Client as Client
participant Server as Server
participant Service as Service Plugin
participant CDM as Server CDM (WVD/PRD)
Note over Client: dl.py calls service.resolve_server_keys()<br/>before downloads start
Client->>Server: POST /api/session/{id}/license
Note right of Client: {track_ids: [...], mode: "server_cdm",<br/>drm_type from server_cdm_type}
loop Per track with DRM
Note over Server: Extract PSSH from track manifest<br/>ContentProtection elements
Server->>CDM: Load device from serve.users config
Server->>CDM: Generate challenge from PSSH
Server->>Service: get_license(challenge, title, track)
Service-->>Server: License response
Server->>CDM: Parse license → extract keys
end
Server-->>Client: {keys: {track_id: {kid: key, ...}}}
Note over Client: Inject keys into track.drm.content_keys
Note over Client: prepare_drm finds keys → skips CDM call
Note over Client: Keys stored in local vaults
```
#### Path B: On-demand (HLS services / late DRM discovery)
For services like ATV where DRM is only discovered during download (from init
segments or EXT-X-KEY tags), keys are fetched per-track during download.
```mermaid
sequenceDiagram
participant Client as Client
participant Server as Server
participant Service as Service Plugin
participant CDM as Server CDM (WVD/PRD)
Note over Client: track.download() discovers DRM<br/>(PSSH from init data / EXT-X-KEY)
Note over Client: prepare_drm calls licence callback
Client->>Server: POST /api/session/{id}/license
Note right of Client: {track_id, pssh, drm_type,<br/>mode: "server_cdm"}
Server->>CDM: Load device (Widevine or PlayReady)
Server->>CDM: Generate challenge from PSSH
Server->>Service: get_license(challenge, title, track)
Service-->>Server: License response
Server->>CDM: Parse license → extract keys
CDM-->>Server: Content keys
Server-->>Client: {keys: {kid: key, ...}}
Note over Client: Inject keys into track.drm.content_keys
Note over Client: prepare_drm sees keys → skips re-raise
Note over Client: Keys stored in local vaults
```
**Key points:**
- Client does NOT need a local CDM device file
- Server uses devices from `serve.users.{api_key}.devices` (Widevine) and
`serve.users.{api_key}.playready_devices` (PlayReady)
- Server detects DRM type from actual track DRM objects and available devices
- Keys are returned as `{kid_hex: key_hex}` pairs
- Keys are still cached in client's local vaults (unless `--cdm-only` is used)
- `prepare_drm` skips local CDM if all KIDs already have keys
---
## Configuration
### Client (`unshackle.yaml`)
```yaml
remote_services:
my_server:
url: "https://server:8786"
api_key: "your-secret-key"
server_cdm: true # server handles licensing (optional, default: false)
services: # per-service overrides (optional)
EXAMPLE:
downloader: n_m3u8dl_re
decryption: mp4decrypt
EXAMPLE2:
downloader: n_m3u8dl_re
```
### Server (`unshackle.yaml`)
```yaml
serve:
api_secret: "your-secret-key"
users:
"your-secret-key":
username: api_user
devices: # Widevine CDMs
- xiaomi_mi_a1_15.0.0_l3
playready_devices: # PlayReady CDMs
- qingdao_haier_tv_sl3000
```
---
## Manifest Data Transfer
Track manifests (DASH XML, ISM XML) cannot be JSON-serialized directly (they
contain lxml Element objects). The server serializes them as base64 strings in
the `/tracks` response. The client decodes and re-parses them locally.
| Manifest | Serialization | Client Re-parse | Notes |
| -------- | --------------------------- | ---------------------------------------------- | --------------------------------- |
| **DASH** | `etree.tostring()` → base64 | `DASH(etree.fromstring(xml), url).to_tracks()` | Match by track ID (crc32 hash) |
| **HLS** | Not needed | Downloads playlist from `track.url` directly | `HLS.download_track()` re-fetches |
| **ISM** | `etree.tostring()` → base64 | `ISM(etree.fromstring(xml), url).to_tracks()` | Match by track ID |
The `/tracks` response also includes:
- `session_headers` / `session_cookies` — for CDN authentication
- `server_cdm_type` — "widevine" or "playready" (detected from track DRM + config)
---
## Region & Proxy Handling
1. Client detects its own country via `get_cached_ip_info()`
2. Sends `client_region` in session create (no IP sent, just country code)
3. Server checks its own region — if it matches, no proxy needed
4. If regions differ, server resolves a proxy from its own providers
5. Client can also send explicit `--proxy` which takes precedence
---
## Comparison: Proxy vs Server-CDM
| | Proxy Mode (default) | Server-CDM Mode |
| ----------------------- | ----------------------------- | -------------------------------- |
| **Client needs CDM** | Yes (WVD/PRD file) | No |
| **License request** | Client sends challenge bytes | Client sends PSSH (or track IDs) |
| **License response** | Raw license bytes | KID:KEY pairs |
| **Key extraction** | Client CDM parses license | Server CDM parses license |
| **What leaves server** | Encrypted license blob | Decryption keys |
| **Vault caching** | Client caches keys | Client caches keys |
| **`--cdm-only` effect** | Skip vaults, CDM only | Skip vaults, server CDM only |
| **Config** | `server_cdm: false` (default) | `server_cdm: true` |
## API Endpoints
| Endpoint | Method | Purpose |
| --------------------------- | ------ | --------------------------------- |
| `/api/session/create` | POST | Authenticate, create session |
| `/api/session/{id}/titles` | GET | Get titles for session |
| `/api/session/{id}/tracks` | POST | Get tracks + manifests + chapters |
| `/api/session/{id}/license` | POST | License proxy or server-CDM keys |
| `/api/session/{id}` | GET | Check session validity |
| `/api/session/{id}` | DELETE | Cleanup session |
+47 -32
View File
@@ -1956,6 +1956,9 @@ class dl:
) )
self.cdm = quality_based_cdm self.cdm = quality_based_cdm
if hasattr(service, "resolve_server_keys"):
service.resolve_server_keys(title)
dl_start_time = time.time() dl_start_time = time.time()
try: try:
@@ -2632,6 +2635,9 @@ class dl:
if kid not in drm.content_keys and cdm_only: if kid not in drm.content_keys and cdm_only:
need_license = True need_license = True
if need_license and all(kid in drm.content_keys for kid in all_kids):
need_license = False
if need_license and not vaults_only: if need_license and not vaults_only:
from_vaults = drm.content_keys.copy() from_vaults = drm.content_keys.copy()
@@ -2653,21 +2659,24 @@ class dl:
else: else:
drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate) drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
except Exception as e: except Exception as e:
if isinstance(e, (Widevine.Exceptions.EmptyLicense, Widevine.Exceptions.CEKNotFound)): if drm.content_keys:
msg = str(e) self.log.debug(f"License call failed but keys already in content_keys: {e}")
else: else:
msg = f"An exception occurred in the Service's license function: {e}" if isinstance(e, (Widevine.Exceptions.EmptyLicense, Widevine.Exceptions.CEKNotFound)):
cek_tree.add(f"[logging.level.error]{msg}") msg = str(e)
if not pre_existing_tree: else:
table.add_row(cek_tree) msg = f"An exception occurred in the Service's license function: {e}"
if self.debug_logger: cek_tree.add(f"[logging.level.error]{msg}")
self.debug_logger.log_error( if not pre_existing_tree:
"get_license", table.add_row(cek_tree)
e, if self.debug_logger:
service=self.service, self.debug_logger.log_error(
context={"track": str(track), "exception_type": type(e).__name__}, "get_license",
) e,
raise e service=self.service,
context={"track": str(track), "exception_type": type(e).__name__},
)
raise e
if self.debug_logger: if self.debug_logger:
self.debug_logger.log( self.debug_logger.log(
@@ -2817,31 +2826,37 @@ class dl:
if kid not in drm.content_keys and cdm_only: if kid not in drm.content_keys and cdm_only:
need_license = True need_license = True
if need_license and all(kid in drm.content_keys for kid in all_kids):
need_license = False
if need_license and not vaults_only: if need_license and not vaults_only:
from_vaults = drm.content_keys.copy() from_vaults = drm.content_keys.copy()
try: try:
drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate) drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
except Exception as e: except Exception as e:
if isinstance(e, (PlayReady.Exceptions.EmptyLicense, PlayReady.Exceptions.CEKNotFound)): if drm.content_keys:
msg = str(e) self.log.debug(f"License call failed but keys already in content_keys: {e}")
else: else:
msg = f"An exception occurred in the Service's license function: {e}" if isinstance(e, (PlayReady.Exceptions.EmptyLicense, PlayReady.Exceptions.CEKNotFound)):
cek_tree.add(f"[logging.level.error]{msg}") msg = str(e)
if not pre_existing_tree: else:
table.add_row(cek_tree) msg = f"An exception occurred in the Service's license function: {e}"
if self.debug_logger: cek_tree.add(f"[logging.level.error]{msg}")
self.debug_logger.log_error( if not pre_existing_tree:
"get_license_playready", table.add_row(cek_tree)
e, if self.debug_logger:
service=self.service, self.debug_logger.log_error(
context={ "get_license_playready",
"track": str(track), e,
"exception_type": type(e).__name__, service=self.service,
"drm_type": "PlayReady", context={
}, "track": str(track),
) "exception_type": type(e).__name__,
raise e "drm_type": "PlayReady",
},
)
raise e
for kid_, key in drm.content_keys.items(): for kid_, key in drm.content_keys.items():
is_track_kid_marker = ["", "*"][kid_ == track_kid] is_track_kid_marker = ["", "*"][kid_ == track_kid]
+531 -68
View File
@@ -204,6 +204,61 @@ def serialize_title(title: Title_T) -> Dict[str, Any]:
return result return result
def _extract_manifests(tracks) -> List[Dict[str, Any]]:
"""Extract manifest data from tracks for client-side re-parsing.
Serializes DASH and ISM manifest XML as base64 strings so the client
can reconstruct track.data locally. HLS tracks download directly from
their URL so no manifest serialization is needed.
"""
import base64
from lxml import etree
seen: set[str] = set()
manifests: List[Dict[str, Any]] = []
for track in list(tracks.videos) + list(tracks.audio) + list(tracks.subtitles):
manifest_url = str(track.url) if track.url else None
if not manifest_url or manifest_url in seen:
continue
if track.data.get("dash") and track.data["dash"].get("manifest"):
seen.add(manifest_url)
xml_bytes = etree.tostring(track.data["dash"]["manifest"], xml_declaration=True, encoding="UTF-8")
manifests.append(
{
"type": "dash",
"url": manifest_url,
"data": base64.b64encode(xml_bytes).decode("ascii"),
}
)
elif track.data.get("hls"):
seen.add(manifest_url)
# m3u8 master playlist — serialize as text
hls_obj = track.data["hls"].get("playlist") or track.data["hls"].get("media")
if hls_obj and hasattr(hls_obj, "uri"):
manifests.append(
{
"type": "hls",
"url": manifest_url,
"data": base64.b64encode(manifest_url.encode()).decode("ascii"),
}
)
elif track.data.get("ism") and track.data["ism"].get("manifest"):
seen.add(manifest_url)
xml_bytes = etree.tostring(track.data["ism"]["manifest"], xml_declaration=True, encoding="UTF-8")
manifests.append(
{
"type": "ism",
"url": manifest_url,
"data": base64.b64encode(xml_bytes).decode("ascii"),
}
)
return manifests
def serialize_drm(drm_list) -> Optional[List[Dict[str, Any]]]: def serialize_drm(drm_list) -> Optional[List[Dict[str, Any]]]:
"""Serialize DRM objects to JSON-serializable list.""" """Serialize DRM objects to JSON-serializable list."""
if not drm_list: if not drm_list:
@@ -230,6 +285,7 @@ def serialize_drm(drm_list) -> Optional[List[Dict[str, Any]]]:
elif hasattr(pssh_obj, "__bytes__"): elif hasattr(pssh_obj, "__bytes__"):
# Convert to base64 # Convert to base64
import base64 import base64
drm_info["pssh"] = base64.b64encode(bytes(pssh_obj)).decode() drm_info["pssh"] = base64.b64encode(bytes(pssh_obj)).decode()
elif hasattr(pssh_obj, "to_base64"): elif hasattr(pssh_obj, "to_base64"):
drm_info["pssh"] = pssh_obj.to_base64() drm_info["pssh"] = pssh_obj.to_base64()
@@ -468,13 +524,15 @@ async def search_handler(data: Dict[str, Any], request: Optional[web.Request] =
results = [] results = []
try: try:
for result in service_instance.search(): for result in service_instance.search():
results.append({ results.append(
"id": result.id, {
"title": result.title, "id": result.id,
"description": result.description, "title": result.title,
"label": result.label, "description": result.description,
"url": result.url, "label": result.label,
}) "url": result.url,
}
)
except NotImplementedError: except NotImplementedError:
raise APIError( raise APIError(
APIErrorCode.SERVICE_ERROR, APIErrorCode.SERVICE_ERROR,
@@ -583,7 +641,11 @@ async def list_titles_handler(data: Dict[str, Any], request: Optional[web.Reques
for param in service_module.cli.params: for param in service_module.cli.params:
if hasattr(param, "name") and param.name not in service_kwargs: if hasattr(param, "name") and param.name not in service_kwargs:
# Add default value if parameter is not already provided # Add default value if parameter is not already provided
if hasattr(param, "default") and param.default is not None and not isinstance(param.default, enum.Enum): if (
hasattr(param, "default")
and param.default is not None
and not isinstance(param.default, enum.Enum)
):
service_kwargs[param.name] = param.default service_kwargs[param.name] = param.default
# Handle required parameters that don't have click defaults # Handle required parameters that don't have click defaults
@@ -731,7 +793,11 @@ async def list_tracks_handler(data: Dict[str, Any], request: Optional[web.Reques
for param in service_module.cli.params: for param in service_module.cli.params:
if hasattr(param, "name") and param.name not in service_kwargs: if hasattr(param, "name") and param.name not in service_kwargs:
# Add default value if parameter is not already provided # Add default value if parameter is not already provided
if hasattr(param, "default") and param.default is not None and not isinstance(param.default, enum.Enum): if (
hasattr(param, "default")
and param.default is not None
and not isinstance(param.default, enum.Enum)
):
service_kwargs[param.name] = param.default service_kwargs[param.name] = param.default
# Handle required parameters that don't have click defaults # Handle required parameters that don't have click defaults
@@ -920,7 +986,21 @@ def validate_download_parameters(data: Dict[str, Any]) -> Optional[str]:
return f"Invalid vcodec: {', '.join(invalid)}. Must be one of: {', '.join(valid_vcodecs)}" return f"Invalid vcodec: {', '.join(invalid)}. Must be one of: {', '.join(valid_vcodecs)}"
if "acodec" in data and data["acodec"]: if "acodec" in data and data["acodec"]:
valid_acodecs = ["AAC", "AC3", "EC3", "EAC3", "DD", "DD+", "AC4", "OPUS", "FLAC", "ALAC", "VORBIS", "OGG", "DTS"] valid_acodecs = [
"AAC",
"AC3",
"EC3",
"EAC3",
"DD",
"DD+",
"AC4",
"OPUS",
"FLAC",
"ALAC",
"VORBIS",
"OGG",
"DTS",
]
if isinstance(data["acodec"], str): if isinstance(data["acodec"], str):
acodec_values = [v.strip() for v in data["acodec"].split(",") if v.strip()] acodec_values = [v.strip() for v in data["acodec"].split(",") if v.strip()]
elif isinstance(data["acodec"], list): elif isinstance(data["acodec"], list):
@@ -1035,7 +1115,12 @@ async def download_handler(data: Dict[str, Any], request: Optional[web.Request]
# Extract default values from the service's click command # Extract default values from the service's click command
if hasattr(service_module, "cli") and hasattr(service_module.cli, "params"): if hasattr(service_module, "cli") and hasattr(service_module.cli, "params"):
for param in service_module.cli.params: for param in service_module.cli.params:
if hasattr(param, "name") and hasattr(param, "default") and param.default is not None and not isinstance(param.default, enum.Enum): if (
hasattr(param, "name")
and hasattr(param, "default")
and param.default is not None
and not isinstance(param.default, enum.Enum)
):
# Store service-specific defaults (e.g., drm_system, hydrate_track, profile for NF) # Store service-specific defaults (e.g., drm_system, hydrate_track, profile for NF)
service_specific_defaults[param.name] = param.default service_specific_defaults[param.name] = param.default
@@ -1257,8 +1342,16 @@ def _create_service_instance(
for key, value in data.items(): for key, value in data.items():
if key not in [ if key not in [
"service", "title_id", "profile", "season", "episode", "wanted", "service",
"proxy", "no_proxy", "credentials", "cookies", "title_id",
"profile",
"season",
"episode",
"wanted",
"proxy",
"no_proxy",
"credentials",
"cookies",
]: ]:
service_kwargs[key] = value service_kwargs[key] = value
@@ -1308,6 +1401,7 @@ def _create_service_instance(
cookies.load(ignore_discard=True, ignore_expires=True) cookies.load(ignore_discard=True, ignore_expires=True)
finally: finally:
import os import os
os.unlink(tmp_path) os.unlink(tmp_path)
else: else:
cookies = dl.get_cookie_jar(normalized_service, profile) cookies = dl.get_cookie_jar(normalized_service, profile)
@@ -1360,6 +1454,25 @@ async def session_create_handler(data: Dict[str, Any], request: Optional[web.Req
details={"proxy": data.get("proxy"), "service": normalized_service}, details={"proxy": data.get("proxy"), "service": normalized_service},
) )
client_region = data.get("client_region")
if not proxy_param and not no_proxy and client_region and proxy_providers:
try:
from unshackle.core.utilities import get_cached_ip_info
server_ip_info = get_cached_ip_info(None)
server_region = server_ip_info.get("country", "").lower() if server_ip_info else None
except Exception:
server_region = None
if server_region and server_region == client_region.lower():
log.info(f"Server already in client region '{client_region}', no proxy needed")
else:
try:
proxy_param = resolve_proxy(client_region, proxy_providers)
log.info(f"Using server proxy for client region '{client_region}'")
except ValueError:
log.debug(f"No server proxy available for client region '{client_region}'")
import hashlib import hashlib
import uuid as uuid_mod import uuid as uuid_mod
@@ -1372,7 +1485,12 @@ async def session_create_handler(data: Dict[str, Any], request: Optional[web.Req
session_cache_tag = f"_sessions/{api_key_hash}/{session_id}/{normalized_service}" session_cache_tag = f"_sessions/{api_key_hash}/{session_id}/{normalized_service}"
service_instance, cookies, credential = _create_service_instance( service_instance, cookies, credential = _create_service_instance(
normalized_service, title_id, data, proxy_param, proxy_providers, profile, normalized_service,
title_id,
data,
proxy_param,
proxy_providers,
profile,
) )
service_instance.cache = Cacher(session_cache_tag) service_instance.cache = Cacher(session_cache_tag)
@@ -1388,14 +1506,18 @@ async def session_create_handler(data: Dict[str, Any], request: Optional[web.Req
store = get_session_store() store = get_session_store()
session = await store.create( session = await store.create(
normalized_service, service_instance, session_id=session_id, normalized_service,
service_instance,
session_id=session_id,
) )
session.cache_tag = session_cache_tag session.cache_tag = session_cache_tag
return web.json_response({ return web.json_response(
"session_id": session.session_id, {
"service": normalized_service, "session_id": session.session_id,
}) "service": normalized_service,
}
)
except APIError: except APIError:
raise raise
@@ -1409,8 +1531,7 @@ async def session_create_handler(data: Dict[str, Any], request: Optional[web.Req
) )
async def session_titles_handler(session_id: str, async def session_titles_handler(session_id: str, request: Optional[web.Request] = None) -> web.Response:
request: Optional[web.Request] = None) -> web.Response:
"""Get titles for the authenticated session. """Get titles for the authenticated session.
Called after session/create. This is separate from auth so that Called after session/create. This is separate from auth so that
@@ -1444,10 +1565,12 @@ async def session_titles_handler(session_id: str,
session.title_map[tid] = t session.title_map[tid] = t
serialized_titles.append(serialize_title(t)) serialized_titles.append(serialize_title(t))
return web.json_response({ return web.json_response(
"session_id": session_id, {
"titles": serialized_titles, "session_id": session_id,
}) "titles": serialized_titles,
}
)
except Exception as e: except Exception as e:
log.exception("Error getting titles") log.exception("Error getting titles")
@@ -1459,8 +1582,9 @@ async def session_titles_handler(session_id: str,
) )
async def session_tracks_handler(data: Dict[str, Any], session_id: str, async def session_tracks_handler(
request: Optional[web.Request] = None) -> web.Response: data: Dict[str, Any], session_id: str, request: Optional[web.Request] = None
) -> web.Response:
"""Get tracks and chapters for a specific title in the session. """Get tracks and chapters for a specific title in the session.
Called per-title by the client after session/create returns titles. Called per-title by the client after session/create returns titles.
@@ -1514,21 +1638,61 @@ async def session_tracks_handler(data: Dict[str, Any], session_id: str,
video_tracks = sorted(tracks.videos, key=lambda t: t.bitrate or 0, reverse=True) video_tracks = sorted(tracks.videos, key=lambda t: t.bitrate or 0, reverse=True)
audio_tracks = sorted(tracks.audio, key=lambda t: t.bitrate or 0, reverse=True) audio_tracks = sorted(tracks.audio, key=lambda t: t.bitrate or 0, reverse=True)
return web.json_response({ manifests = _extract_manifests(tracks)
"title": serialize_title(title),
"video": [serialize_video_track(t, include_url=True) for t in video_tracks], svc_session = session.service_instance.session
"audio": [serialize_audio_track(t, include_url=True) for t in audio_tracks], session_headers = dict(svc_session.headers) if hasattr(svc_session, "headers") else {}
"subtitles": [serialize_subtitle_track(t, include_url=True) for t in tracks.subtitles], session_cookies = {}
"chapters": [ if hasattr(svc_session, "cookies"):
{"timestamp": ch.timestamp, "name": ch.name} for cookie in svc_session.cookies:
for ch in session.chapters_by_title.get(str(title_id), []) if hasattr(cookie, "name") and hasattr(cookie, "value"):
], session_cookies[cookie.name] = cookie.value
"attachments": [
{"url": a.url, "name": a.name, "mime_type": a.mime_type, "description": a.description} from unshackle.core.config import config as app_config
for a in tracks.attachments
if hasattr(a, "url") and a.url api_key = request.headers.get("X-Secret-Key", "anonymous") if request else "anonymous"
], user_cfg = app_config.serve.get("users", {}).get(api_key, {})
}) has_wv = bool(user_cfg.get("devices"))
has_pr = bool(user_cfg.get("playready_devices"))
track_has_wv = any(
d.__class__.__name__ == "Widevine" for t in list(tracks.videos) + list(tracks.audio) if t.drm for d in t.drm
)
track_has_pr = any(
d.__class__.__name__ == "PlayReady"
for t in list(tracks.videos) + list(tracks.audio)
if t.drm
for d in t.drm
)
if track_has_pr and has_pr:
server_cdm_type = "playready"
elif track_has_wv and has_wv:
server_cdm_type = "widevine"
elif has_wv:
server_cdm_type = "widevine"
else:
server_cdm_type = "playready"
return web.json_response(
{
"title": serialize_title(title),
"video": [serialize_video_track(t, include_url=True) for t in video_tracks],
"audio": [serialize_audio_track(t, include_url=True) for t in audio_tracks],
"subtitles": [serialize_subtitle_track(t, include_url=True) for t in tracks.subtitles],
"chapters": [
{"timestamp": ch.timestamp, "name": ch.name}
for ch in session.chapters_by_title.get(str(title_id), [])
],
"attachments": [
{"url": a.url, "name": a.name, "mime_type": a.mime_type, "description": a.description}
for a in tracks.attachments
if hasattr(a, "url") and a.url
],
"manifests": manifests,
"session_headers": session_headers,
"session_cookies": session_cookies,
"server_cdm_type": server_cdm_type,
}
)
except Exception as e: except Exception as e:
log.exception(f"Error getting tracks for title {title_id}") log.exception(f"Error getting tracks for title {title_id}")
@@ -1540,8 +1704,9 @@ async def session_tracks_handler(data: Dict[str, Any], session_id: str,
) )
async def session_segments_handler(data: Dict[str, Any], session_id: str, async def session_segments_handler(
request: Optional[web.Request] = None) -> web.Response: data: Dict[str, Any], session_id: str, request: Optional[web.Request] = None
) -> web.Response:
"""Resolve segment URLs for selected tracks. """Resolve segment URLs for selected tracks.
The client calls this after selecting which tracks to download. The client calls this after selecting which tracks to download.
@@ -1613,6 +1778,7 @@ async def session_segments_handler(data: Dict[str, Any], session_id: str,
for k, v in val.items(): for k, v in val.items():
try: try:
import json import json
json.dumps(v) json.dumps(v)
serializable[k] = v serializable[k] = v
except (TypeError, ValueError): except (TypeError, ValueError):
@@ -1621,6 +1787,7 @@ async def session_segments_handler(data: Dict[str, Any], session_id: str,
else: else:
try: try:
import json import json
json.dumps(val) json.dumps(val)
track_data[key] = val track_data[key] = val
except (TypeError, ValueError): except (TypeError, ValueError):
@@ -1645,14 +1812,17 @@ async def session_segments_handler(data: Dict[str, Any], session_id: str,
) )
async def session_license_handler(data: Dict[str, Any], session_id: str, async def session_license_handler(
request: Optional[web.Request] = None) -> web.Response: data: Dict[str, Any], session_id: str, request: Optional[web.Request] = None
"""Proxy a DRM license challenge through the authenticated service. ) -> web.Response:
"""Handle DRM licensing in proxy or server_cdm mode.
The client generates a CDM challenge locally, sends it here, and the server Proxy mode (default): forwards client CDM challenge to the service's
calls the service's get_widevine_license/get_playready_license method using license endpoint, returns raw license bytes for client-side parsing.
the authenticated session. Returns the raw license response for the client
to process with their local CDM. Server-CDM mode (mode="server_cdm"): server uses its own CDM to generate
the challenge, obtain the license, and extract KID:KEY pairs. Supports
batch (track_ids list) and single-track requests.
""" """
import base64 import base64
@@ -1668,13 +1838,98 @@ async def session_license_handler(data: Dict[str, Any], session_id: str,
) )
track_id = data.get("track_id") track_id = data.get("track_id")
track_ids = data.get("track_ids")
challenge_b64 = data.get("challenge") challenge_b64 = data.get("challenge")
drm_type = data.get("drm_type", "widevine") drm_type = data.get("drm_type", "widevine")
mode = data.get("mode", "proxy")
if mode == "server_cdm" and track_ids:
import base64
from unshackle.core.config import config as app_config
api_key = request.headers.get("X-Secret-Key", "anonymous") if request else "anonymous"
user_config = app_config.serve.get("users", {}).get(api_key, {})
service = session.service_instance
all_keys: Dict[str, Dict[str, str]] = {}
seen_pssh: set[str] = set()
for tid in track_ids:
track = session.tracks.get(tid)
if not track:
continue
# If track has no DRM yet, extract it from manifest ContentProtection
if not track.drm and track.data.get("dash"):
from unshackle.core.manifests import DASH as DASHManifest
rep = track.data["dash"].get("representation")
ada = track.data["dash"].get("adaptation_set")
if rep is not None and ada is not None:
track.drm = DASHManifest.get_drm(
rep.findall("ContentProtection") + ada.findall("ContentProtection")
)
if not track.drm:
continue
title = None
for t_id, tracks_dict in session.tracks_by_title.items():
if tid in tracks_dict:
title = session.title_map.get(t_id)
break
if title is None and session.title_map:
title = next(iter(session.title_map.values()))
has_wv_device = bool(user_config.get("devices"))
has_pr_device = bool(user_config.get("playready_devices"))
pssh_str = None
track_drm_type = None
if has_wv_device:
for drm_obj in track.drm:
if drm_obj.__class__.__name__ == "Widevine" and hasattr(drm_obj, "_pssh") and drm_obj._pssh:
pssh_str = drm_obj._pssh.dumps() if hasattr(drm_obj._pssh, "dumps") else None
track_drm_type = "widevine"
break
if not pssh_str and has_pr_device:
for drm_obj in track.drm:
if drm_obj.__class__.__name__ == "PlayReady" and hasattr(drm_obj, "data"):
pssh_str = drm_obj.data.get("pssh_b64")
track_drm_type = "playready"
break
if not pssh_str or pssh_str in seen_pssh:
if pssh_str in seen_pssh:
for prev_tid, prev_keys in all_keys.items():
if prev_keys:
all_keys[tid] = prev_keys
break
continue
seen_pssh.add(pssh_str)
if not track_drm_type:
continue
try:
single_data = {
"track_id": tid,
"drm_type": track_drm_type,
"mode": "server_cdm",
"pssh": pssh_str,
}
single_resp = await session_license_handler(single_data, session_id, request)
resp_json = __import__("json").loads(single_resp.body)
if resp_json.get("keys"):
all_keys[tid] = resp_json["keys"]
except Exception as e:
log.warning(f"Failed to resolve keys for track {tid[:12]}: {e}")
return web.json_response({"keys": all_keys})
if not track_id: if not track_id:
raise APIError(APIErrorCode.INVALID_INPUT, "Missing required parameter: track_id") raise APIError(APIErrorCode.INVALID_INPUT, "Missing required parameter: track_id")
if not challenge_b64:
raise APIError(APIErrorCode.INVALID_INPUT, "Missing required parameter: challenge")
track = session.tracks.get(track_id) track = session.tracks.get(track_id)
if not track: if not track:
@@ -1685,8 +1940,6 @@ async def session_license_handler(data: Dict[str, Any], session_id: str,
) )
try: try:
challenge_bytes = base64.b64decode(challenge_b64)
title = None title = None
for tid, tracks_dict in session.tracks_by_title.items(): for tid, tracks_dict in session.tracks_by_title.items():
if track_id in tracks_dict: if track_id in tracks_dict:
@@ -1708,6 +1961,7 @@ async def session_license_handler(data: Dict[str, Any], session_id: str,
from pyplayready.system.pssh import PSSH as PlayReadyPSSH from pyplayready.system.pssh import PSSH as PlayReadyPSSH
from unshackle.core.drm import PlayReady from unshackle.core.drm import PlayReady
pr_pssh = PlayReadyPSSH(base64.b64decode(pssh_b64)) pr_pssh = PlayReadyPSSH(base64.b64decode(pssh_b64))
pr_drm = PlayReady(pssh=pr_pssh, pssh_b64=pssh_b64) pr_drm = PlayReady(pssh=pr_pssh, pssh_b64=pssh_b64)
track.drm.append(pr_drm) track.drm.append(pr_drm)
@@ -1715,17 +1969,221 @@ async def session_license_handler(data: Dict[str, Any], session_id: str,
from pywidevine.pssh import PSSH as WidevinePSSH from pywidevine.pssh import PSSH as WidevinePSSH
from unshackle.core.drm import Widevine from unshackle.core.drm import Widevine
wv_pssh = WidevinePSSH(pssh_b64) wv_pssh = WidevinePSSH(pssh_b64)
wv_drm = Widevine(pssh=wv_pssh) wv_drm = Widevine(pssh=wv_pssh)
track.drm.append(wv_drm) track.drm.append(wv_drm)
if mode == "server_cdm":
if not track.drm and track.data.get("dash"):
from unshackle.core.manifests import DASH as DASHManifest
rep = track.data["dash"].get("representation")
ada = track.data["dash"].get("adaptation_set")
if rep is not None and ada is not None:
track.drm = DASHManifest.get_drm(
rep.findall("ContentProtection") + ada.findall("ContentProtection")
)
if not pssh_b64 and track.drm:
for drm_obj in track.drm:
drm_class = drm_obj.__class__.__name__
if drm_class == "Widevine" and hasattr(drm_obj, "_pssh") and drm_obj._pssh:
if hasattr(drm_obj._pssh, "dumps"):
pssh_b64 = drm_obj._pssh.dumps()
if drm_type == "widevine":
break
elif drm_class == "PlayReady":
if hasattr(drm_obj, "data") and drm_obj.data.get("pssh_b64"):
pssh_b64 = drm_obj.data["pssh_b64"]
if drm_type == "playready":
break
if not pssh_b64:
raise APIError(APIErrorCode.INVALID_INPUT, "No PSSH available for server_cdm licensing")
from unshackle.core.config import config as app_config
api_key = request.headers.get("X-Secret-Key", "anonymous") if request else "anonymous"
user_config = app_config.serve.get("users", {}).get(api_key, {})
if drm_type == "playready":
from pyplayready.cdm import Cdm as PlayReadyCdm
from pyplayready.device import Device as PlayReadyDevice
from pyplayready.remote.remotecdm import RemoteCdm as PlayReadyRemoteCdm
device_name = (user_config.get("playready_devices") or [None])[0]
if not device_name:
raise APIError(APIErrorCode.INVALID_INPUT, "No PlayReady device configured for this API key")
cdm_api = next((x.copy() for x in app_config.remote_cdm if x.get("name") == device_name), None)
if cdm_api:
cdm_type_api = cdm_api.get("type")
if cdm_type_api == "decrypt_labs":
from unshackle.core.cdm import DecryptLabsRemoteCDM
del cdm_api["name"]
del cdm_api["type"]
if "secret" not in cdm_api or not cdm_api["secret"]:
if app_config.decrypt_labs_api_key:
cdm_api["secret"] = app_config.decrypt_labs_api_key
cdm = DecryptLabsRemoteCDM(service_name=service.__class__.__name__, **cdm_api)
elif cdm_type_api == "custom_api":
from unshackle.core.cdm import CustomRemoteCDM
del cdm_api["name"]
del cdm_api["type"]
cdm = CustomRemoteCDM(service_name=service.__class__.__name__, **cdm_api)
else:
device_type = cdm_api.get("Device Type", cdm_api.get("device_type", ""))
if str(device_type).upper() == "PLAYREADY":
cdm = PlayReadyRemoteCdm(
security_level=cdm_api.get("Security Level", cdm_api.get("security_level", 3000)),
host=cdm_api.get("Host", cdm_api.get("host")),
secret=cdm_api.get("Secret", cdm_api.get("secret")),
device_name=cdm_api.get("Device Name", cdm_api.get("device_name")),
)
else:
raise APIError(
APIErrorCode.INVALID_INPUT,
f"CDM '{device_name}' is not a PlayReady device",
)
else:
prd_path = app_config.directories.prds / f"{device_name}.prd"
if not prd_path.exists():
prd_path = app_config.directories.wvds / f"{device_name}.prd"
if not prd_path.exists():
raise APIError(
APIErrorCode.INVALID_INPUT,
f"PlayReady device '{device_name}' not found",
)
cdm = PlayReadyCdm.from_device(PlayReadyDevice.load(prd_path))
pr_pssh = PlayReadyPSSH(base64.b64decode(pssh_b64))
wrm_header = pr_pssh.wrm_headers[0]
session_id_cdm = cdm.open()
try:
challenge = cdm.get_license_challenge(session_id_cdm, wrm_header)
license_response = service.get_playready_license(
challenge=challenge,
title=title,
track=track,
)
if isinstance(license_response, bytes):
license_str = license_response.decode(errors="ignore")
else:
license_str = str(license_response)
if "<License>" not in license_str:
try:
license_str = base64.b64decode(license_str + "===").decode()
except Exception:
pass
cdm.parse_license(session_id_cdm, license_str)
keys = {}
for key in cdm.get_keys(session_id_cdm):
kid = getattr(key, "key_id", None) or getattr(key, "kid", None)
key_val = getattr(key, "key", None)
if kid and key_val:
kid_hex = kid.hex if hasattr(kid, "hex") else str(kid).replace("-", "")
key_hex = key_val.hex() if hasattr(key_val, "hex") else str(key_val)
keys[kid_hex] = key_hex
finally:
cdm.close(session_id_cdm)
elif drm_type == "widevine":
from pywidevine.cdm import Cdm as WidevineCdm
from pywidevine.device import Device as WidevineDevice
from pywidevine.pssh import PSSH as WvPSSH
from pywidevine.remotecdm import RemoteCdm as WidevineRemoteCdm
device_name = (user_config.get("devices") or [None])[0]
if not device_name:
raise APIError(APIErrorCode.INVALID_INPUT, "No Widevine device configured for this API key")
cdm_api = next((x.copy() for x in app_config.remote_cdm if x.get("name") == device_name), None)
if cdm_api:
cdm_type_api = cdm_api.get("type")
if cdm_type_api == "decrypt_labs":
from unshackle.core.cdm import DecryptLabsRemoteCDM
del cdm_api["name"]
del cdm_api["type"]
if "secret" not in cdm_api or not cdm_api["secret"]:
if app_config.decrypt_labs_api_key:
cdm_api["secret"] = app_config.decrypt_labs_api_key
cdm = DecryptLabsRemoteCDM(service_name=service.__class__.__name__, **cdm_api)
elif cdm_type_api == "custom_api":
from unshackle.core.cdm import CustomRemoteCDM
del cdm_api["name"]
del cdm_api["type"]
cdm = CustomRemoteCDM(service_name=service.__class__.__name__, **cdm_api)
else:
cdm = WidevineRemoteCdm(
device_type=cdm_api.get("Device Type", cdm_api.get("device_type", "")),
system_id=cdm_api.get("System ID", cdm_api.get("system_id", "")),
security_level=cdm_api.get("Security Level", cdm_api.get("security_level", 3)),
host=cdm_api.get("Host", cdm_api.get("host")),
secret=cdm_api.get("Secret", cdm_api.get("secret")),
device_name=cdm_api.get("Device Name", cdm_api.get("device_name")),
)
else:
wvd_path = app_config.directories.wvds / f"{device_name}.wvd"
if not wvd_path.exists():
raise APIError(
APIErrorCode.INVALID_INPUT,
f"Widevine device '{device_name}' not found",
)
cdm = WidevineCdm.from_device(WidevineDevice.load(wvd_path))
wv_pssh = WvPSSH(pssh_b64)
session_id_cdm = cdm.open()
try:
if hasattr(cdm, "service_certificate_challenge"):
try:
cert = service.get_widevine_service_certificate(
challenge=cdm.service_certificate_challenge,
title=title,
track=track,
)
if cert and hasattr(cdm, "set_service_certificate"):
cdm.set_service_certificate(session_id_cdm, cert)
except Exception:
pass
challenge = cdm.get_license_challenge(session_id_cdm, wv_pssh)
license_response = service.get_widevine_license(
challenge=challenge,
title=title,
track=track,
)
cdm.parse_license(session_id_cdm, license_response)
keys = {key.kid.hex: key.key.hex() for key in cdm.get_keys(session_id_cdm, "CONTENT")}
finally:
cdm.close(session_id_cdm)
else:
raise APIError(
APIErrorCode.INVALID_PARAMETERS,
f"Unsupported DRM type for server_cdm: {drm_type}",
)
if not keys:
raise APIError(APIErrorCode.NO_CONTENT, "Server CDM returned no content keys")
log.info(f"Server CDM resolved {len(keys)} key(s) for track {track_id[:12]}")
return web.json_response({"keys": keys})
if not challenge_b64:
raise APIError(APIErrorCode.INVALID_INPUT, "Missing required parameter: challenge")
challenge_bytes = base64.b64decode(challenge_b64)
if drm_type == "widevine": if drm_type == "widevine":
license_response = service.get_widevine_license( license_response = service.get_widevine_license(
challenge=challenge_bytes, title=title, track=track, challenge=challenge_bytes,
title=title,
track=track,
) )
elif drm_type == "playready": elif drm_type == "playready":
license_response = service.get_playready_license( license_response = service.get_playready_license(
challenge=challenge_bytes, title=title, track=track, challenge=challenge_bytes,
title=title,
track=track,
) )
else: else:
raise APIError( raise APIError(
@@ -1738,9 +2196,11 @@ async def session_license_handler(data: Dict[str, Any], session_id: str,
if isinstance(license_response, str): if isinstance(license_response, str):
license_response = license_response.encode("utf-8") license_response = license_response.encode("utf-8")
return web.json_response({ return web.json_response(
"license": base64.b64encode(license_response).decode("ascii"), {
}) "license": base64.b64encode(license_response).decode("ascii"),
}
)
except APIError: except APIError:
raise raise
@@ -1775,18 +2235,21 @@ async def session_info_handler(session_id: str, request: Optional[web.Request] =
) )
from datetime import datetime from datetime import datetime
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
elapsed = (now - session.last_accessed).total_seconds() elapsed = (now - session.last_accessed).total_seconds()
expires_in = max(0, store._ttl - int(elapsed)) expires_in = max(0, store._ttl - int(elapsed))
return web.json_response({ return web.json_response(
"session_id": session.session_id, {
"service": session.service_tag, "session_id": session.session_id,
"valid": True, "service": session.service_tag,
"expires_in": expires_in, "valid": True,
"track_count": len(session.tracks), "expires_in": expires_in,
"title_count": len(session.title_map), "track_count": len(session.tracks),
}) "title_count": len(session.title_map),
}
)
async def session_delete_handler(session_id: str, request: Optional[web.Request] = None) -> web.Response: async def session_delete_handler(session_id: str, request: Optional[web.Request] = None) -> web.Response:
+13
View File
@@ -305,11 +305,24 @@ class DASH:
or (existing_drm and not any(isinstance(drm, Widevine) for drm in existing_drm)) or (existing_drm and not any(isinstance(drm, Widevine) for drm in existing_drm))
) )
pre_existing_keys = {}
if existing_drm:
for drm_obj in existing_drm:
if hasattr(drm_obj, "content_keys") and drm_obj.content_keys:
pre_existing_keys.update(drm_obj.content_keys)
if should_override_drm: if should_override_drm:
track.drm = manifest_drm track.drm = manifest_drm
else: else:
track.drm = existing_drm track.drm = existing_drm
if pre_existing_keys and track.drm:
for drm_obj in track.drm:
if hasattr(drm_obj, "content_keys"):
for kid, key in pre_existing_keys.items():
if kid not in drm_obj.content_keys:
drm_obj.content_keys[kid] = key
manifest_base_url = manifest.findtext("BaseURL") manifest_base_url = manifest.findtext("BaseURL")
if not manifest_base_url: if not manifest_base_url:
manifest_base_url = track.url manifest_base_url = track.url
+322 -35
View File
@@ -4,6 +4,7 @@ Implements the Service interface by proxying authenticate, get_titles,
get_tracks, get_chapters, and license methods to a remote unshackle server. get_tracks, get_chapters, and license methods to a remote unshackle server.
Everything else (track selection, download, decrypt, mux) runs locally. Everything else (track selection, download, decrypt, mux) runs locally.
""" """
from __future__ import annotations from __future__ import annotations
import base64 import base64
@@ -81,7 +82,7 @@ def _enum_get(enum_cls: type[Enum], name: Optional[str], default: Any = None) ->
def _deserialize_video(data: Dict[str, Any]) -> Video: def _deserialize_video(data: Dict[str, Any]) -> Video:
return Video( v = Video(
url=data.get("url") or "https://placeholder", url=data.get("url") or "https://placeholder",
language=Language.get(data.get("language") or "und"), language=Language.get(data.get("language") or "und"),
descriptor=_enum_get(Track.Descriptor, data.get("descriptor"), Track.Descriptor.URL), descriptor=_enum_get(Track.Descriptor, data.get("descriptor"), Track.Descriptor.URL),
@@ -93,10 +94,12 @@ def _deserialize_video(data: Dict[str, Any]) -> Video:
fps=data.get("fps"), fps=data.get("fps"),
id_=data.get("id"), id_=data.get("id"),
) )
v.data["remote"] = True
return v
def _deserialize_audio(data: Dict[str, Any]) -> Audio: def _deserialize_audio(data: Dict[str, Any]) -> Audio:
return Audio( a = Audio(
url=data.get("url") or "https://placeholder", url=data.get("url") or "https://placeholder",
language=Language.get(data.get("language") or "und"), language=Language.get(data.get("language") or "und"),
descriptor=_enum_get(Track.Descriptor, data.get("descriptor"), Track.Descriptor.URL), descriptor=_enum_get(Track.Descriptor, data.get("descriptor"), Track.Descriptor.URL),
@@ -107,6 +110,8 @@ def _deserialize_audio(data: Dict[str, Any]) -> Audio:
descriptive=data.get("descriptive", False), descriptive=data.get("descriptive", False),
id_=data.get("id"), id_=data.get("id"),
) )
a.data["remote"] = True
return a
def _deserialize_subtitle(data: Dict[str, Any]) -> Subtitle: def _deserialize_subtitle(data: Dict[str, Any]) -> Subtitle:
@@ -122,11 +127,51 @@ def _deserialize_subtitle(data: Dict[str, Any]) -> Subtitle:
) )
def _reconstruct_drm(drm_list: Optional[list]) -> list:
"""Reconstruct DRM objects from serialized API data."""
if not drm_list:
return []
result = []
for drm_info in drm_list:
drm_type = drm_info.get("type", "")
pssh_str = drm_info.get("pssh")
if not pssh_str:
continue
try:
if drm_type == "widevine":
from pywidevine.pssh import PSSH as WidevinePSSH
from unshackle.core.drm import Widevine
wv_pssh = WidevinePSSH(pssh_str)
result.append(Widevine(pssh=wv_pssh))
elif drm_type == "playready":
import base64 as b64
from pyplayready.system.pssh import PSSH as PlayReadyPSSH
from unshackle.core.drm import PlayReady
pr_pssh = PlayReadyPSSH(b64.b64decode(pssh_str))
result.append(PlayReady(pssh=pr_pssh, pssh_b64=pssh_str))
except Exception:
continue
return result
def _build_tracks(data: Dict[str, Any]) -> Tracks: def _build_tracks(data: Dict[str, Any]) -> Tracks:
tracks = Tracks() tracks = Tracks()
tracks.videos = [_deserialize_video(v) for v in data.get("video", [])] tracks.videos = [_deserialize_video(v) for v in data.get("video", [])]
tracks.audio = [_deserialize_audio(a) for a in data.get("audio", [])] tracks.audio = [_deserialize_audio(a) for a in data.get("audio", [])]
tracks.subtitles = [_deserialize_subtitle(s) for s in data.get("subtitles", [])] tracks.subtitles = [_deserialize_subtitle(s) for s in data.get("subtitles", [])]
for track_data, track_obj in [
*zip(data.get("video", []), tracks.videos),
*zip(data.get("audio", []), tracks.audio),
]:
drm_objs = _reconstruct_drm(track_data.get("drm"))
if drm_objs:
track_obj.drm = drm_objs
tracks.attachments = [ tracks.attachments = [
Attachment(url=a["url"], name=a.get("name"), mime_type=a.get("mime_type"), description=a.get("description")) Attachment(url=a["url"], name=a.get("name"), mime_type=a.get("mime_type"), description=a.get("description"))
for a in data.get("attachments", []) for a in data.get("attachments", [])
@@ -134,19 +179,126 @@ def _build_tracks(data: Dict[str, Any]) -> Tracks:
return tracks return tracks
def _resolve_manifest_data(tracks: Tracks, manifests: list, session: Any) -> None:
"""Re-parse serialized manifests and populate track.data for downloading.
The server serializes DASH and ISM manifest XML as base64. We decode and
re-parse locally, then match each remote track to the locally-parsed track
by ID to copy track.data. HLS is skipped as it re-fetches from track.url.
"""
import base64 as b64
if not manifests:
return
log_m = logging.getLogger("remote_service")
all_tracks = list(tracks.videos) + list(tracks.audio) + list(tracks.subtitles)
for manifest_info in manifests:
m_type = manifest_info.get("type")
m_url = manifest_info.get("url")
m_data = manifest_info.get("data")
if not m_data or not m_url:
continue
try:
raw = b64.b64decode(m_data)
if m_type == "dash":
from lxml import etree
from unshackle.core.manifests import DASH
xml_tree = etree.fromstring(raw)
dash = DASH(xml_tree, m_url)
fallback_lang = next(
(t.language for t in all_tracks if t.language and str(t.language) != "und"),
None,
)
local_tracks = dash.to_tracks(language=fallback_lang)
local_all = list(local_tracks.videos) + list(local_tracks.audio) + list(local_tracks.subtitles)
for remote_track in all_tracks:
if remote_track.data.get("dash"):
continue
matched = _match_track(remote_track, local_all)
if matched and matched.data.get("dash"):
remote_track.data.update(matched.data)
remote_track.descriptor = matched.descriptor
if matched.drm and not remote_track.drm:
remote_track.drm = matched.drm
elif m_type == "hls":
pass
elif m_type == "ism":
from lxml import etree
from unshackle.core.manifests import ISM
xml_el = etree.fromstring(raw)
ism = ISM(xml_el, m_url)
local_tracks = ism.to_tracks()
local_all = list(local_tracks.videos) + list(local_tracks.audio) + list(local_tracks.subtitles)
for remote_track in all_tracks:
if remote_track.data.get("ism"):
continue
matched = _match_track(remote_track, local_all)
if matched and matched.data.get("ism"):
remote_track.data.update(matched.data)
remote_track.descriptor = matched.descriptor
if matched.drm and not remote_track.drm:
remote_track.drm = matched.drm
except Exception as e:
log_m.warning("Failed to re-parse %s manifest from %s: %s", m_type, m_url, e)
def _match_track(remote_track: Track, local_tracks: list) -> Optional[Track]:
"""Match a remote track to a locally-parsed track by ID or attributes."""
remote_id = str(remote_track.id)
for lt in local_tracks:
if str(lt.id) == remote_id:
return lt
for lt in local_tracks:
if type(lt).__name__ != type(remote_track).__name__:
continue
if lt.codec != remote_track.codec or str(lt.language) != str(remote_track.language):
continue
if hasattr(lt, "width") and hasattr(remote_track, "width"):
if lt.width == remote_track.width and lt.height == remote_track.height:
return lt
elif hasattr(lt, "channels") and hasattr(remote_track, "channels"):
if lt.bitrate == remote_track.bitrate:
return lt
elif hasattr(lt, "forced"):
if lt.forced == remote_track.forced and lt.sdh == remote_track.sdh:
return lt
return None
def _build_title(info: Dict[str, Any], service_tag: str, fallback_id: str) -> Union[Episode, Movie]: def _build_title(info: Dict[str, Any], service_tag: str, fallback_id: str) -> Union[Episode, Movie]:
svc_class = type(service_tag, (), {}) svc_class = type(service_tag, (), {})
lang = Language.get(info["language"]) if info.get("language") else None lang = Language.get(info["language"]) if info.get("language") else None
if info.get("type") == "episode": if info.get("type") == "episode":
return Episode( return Episode(
id_=info.get("id", fallback_id), service=svc_class, id_=info.get("id", fallback_id),
service=svc_class,
title=info.get("series_title", "Unknown"), title=info.get("series_title", "Unknown"),
season=info.get("season", 0), number=info.get("number", 0), season=info.get("season", 0),
name=info.get("name"), year=info.get("year"), language=lang, number=info.get("number", 0),
name=info.get("name"),
year=info.get("year"),
language=lang,
) )
return Movie( return Movie(
id_=info.get("id", fallback_id), service=svc_class, id_=info.get("id", fallback_id),
name=info.get("name", "Unknown"), year=info.get("year"), language=lang, service=svc_class,
name=info.get("name", "Unknown"),
year=info.get("year"),
language=lang,
) )
@@ -158,8 +310,8 @@ def resolve_server(server_name: Optional[str]) -> tuple[str, str, dict]:
"No remote services configured. Add 'remote_services' to your unshackle.yaml:\n\n" "No remote services configured. Add 'remote_services' to your unshackle.yaml:\n\n"
" remote_services:\n" " remote_services:\n"
" my_server:\n" " my_server:\n"
" url: \"https://server:8080\"\n" ' url: "https://server:8080"\n'
" api_key: \"your-api-key\"" ' api_key: "your-api-key"'
) )
if server_name: if server_name:
@@ -167,12 +319,16 @@ def resolve_server(server_name: Optional[str]) -> tuple[str, str, dict]:
if not svc: if not svc:
available = ", ".join(remote_services.keys()) available = ", ".join(remote_services.keys())
raise click.ClickException(f"Remote service '{server_name}' not found. Available: {available}") raise click.ClickException(f"Remote service '{server_name}' not found. Available: {available}")
return svc["url"], svc.get("api_key", ""), svc.get("services", {}) services = svc.get("services", {})
services["_server_cdm"] = svc.get("server_cdm", False)
return svc["url"], svc.get("api_key", ""), services
if len(remote_services) == 1: if len(remote_services) == 1:
name, svc = next(iter(remote_services.items())) name, svc = next(iter(remote_services.items()))
log.info(f"Using remote service: {name}") log.info(f"Using remote service: {name}")
return svc["url"], svc.get("api_key", ""), svc.get("services", {}) services = svc.get("services", {})
services["_server_cdm"] = svc.get("server_cdm", False)
return svc["url"], svc.get("api_key", ""), services
available = ", ".join(remote_services.keys()) available = ", ".join(remote_services.keys())
raise click.ClickException(f"Multiple remote services configured. Use --server to select one: {available}") raise click.ClickException(f"Multiple remote services configured. Use --server to select one: {available}")
@@ -180,6 +336,7 @@ def resolve_server(server_name: Optional[str]) -> tuple[str, str, dict]:
def _load_credentials_for_transport(service_tag: str, profile: Optional[str]) -> Optional[Dict[str, str]]: def _load_credentials_for_transport(service_tag: str, profile: Optional[str]) -> Optional[Dict[str, str]]:
from unshackle.commands.dl import dl from unshackle.commands.dl import dl
credential = dl.get_credentials(service_tag, profile) credential = dl.get_credentials(service_tag, profile)
if credential: if credential:
result: Dict[str, str] = {"username": credential.username, "password": credential.password} result: Dict[str, str] = {"username": credential.username, "password": credential.password}
@@ -191,6 +348,7 @@ def _load_credentials_for_transport(service_tag: str, profile: Optional[str]) ->
def _load_cookies_for_transport(service_tag: str, profile: Optional[str]) -> Optional[str]: def _load_cookies_for_transport(service_tag: str, profile: Optional[str]) -> Optional[str]:
from unshackle.commands.dl import dl from unshackle.commands.dl import dl
cookie_path = dl.get_cookie_path(service_tag, profile) cookie_path = dl.get_cookie_path(service_tag, profile)
if cookie_path and cookie_path.exists(): if cookie_path and cookie_path.exists():
return cookie_path.read_text(encoding="utf-8") return cookie_path.read_text(encoding="utf-8")
@@ -223,7 +381,8 @@ def _resolve_proxy(proxy_arg: Optional[str]) -> Optional[str]:
if requested_provider: if requested_provider:
provider = next( provider = next(
(x for x in providers if x.__class__.__name__.lower() == requested_provider.lower()), None, (x for x in providers if x.__class__.__name__.lower() == requested_provider.lower()),
None,
) )
if not provider: if not provider:
raise click.ClickException(f"Proxy provider '{requested_provider}' not found.") raise click.ClickException(f"Proxy provider '{requested_provider}' not found.")
@@ -251,8 +410,13 @@ class RemoteService:
NO_SUBTITLES: bool = False NO_SUBTITLES: bool = False
def __init__( def __init__(
self, ctx: click.Context, service_tag: str, title_id: str, self,
server_url: str, api_key: str, services_config: dict, ctx: click.Context,
service_tag: str,
title_id: str,
server_url: str,
api_key: str,
services_config: dict,
) -> None: ) -> None:
self.__class__.__name__ = service_tag self.__class__.__name__ = service_tag
console.print(Padding(Rule(f"[rule.text]Service: {service_tag} (Remote)"), (1, 2))) console.print(Padding(Rule(f"[rule.text]Service: {service_tag} (Remote)"), (1, 2)))
@@ -269,6 +433,7 @@ class RemoteService:
self._tracks_by_title: Dict[str, Tracks] = {} self._tracks_by_title: Dict[str, Tracks] = {}
self._chapters_by_title: Dict[str, list] = {} self._chapters_by_title: Dict[str, list] = {}
self._session_id: Optional[str] = None self._session_id: Optional[str] = None
self._server_cdm_type: str = "widevine"
self._session = requests.Session() self._session = requests.Session()
self._session.headers.update(config.headers) self._session.headers.update(config.headers)
@@ -281,7 +446,9 @@ class RemoteService:
) )
self._session.mount("http://", self._session.adapters["https://"]) self._session.mount("http://", self._session.adapters["https://"])
self._apply_service_config(services_config.get(service_tag, {})) svc_config = services_config.get(service_tag, {})
self._server_cdm = services_config.get("_server_cdm", False)
self._apply_service_config(svc_config)
def _apply_service_config(self, svc_config: dict) -> None: def _apply_service_config(self, svc_config: dict) -> None:
if not svc_config: if not svc_config:
@@ -299,6 +466,11 @@ class RemoteService:
target = getattr(config, attr) target = getattr(config, attr)
target[tag] = svc_config[key] target[tag] = svc_config[key]
if svc_config.get("downloader"):
config.downloader = svc_config["downloader"]
if svc_config.get("decryption"):
config.decryption = svc_config["decryption"]
extra = {k: v for k, v in svc_config.items() if k not in config_maps} extra = {k: v for k, v in svc_config.items() if k not in config_maps}
if extra: if extra:
existing = config.services.get(self.service_tag, {}) existing = config.services.get(self.service_tag, {})
@@ -338,6 +510,16 @@ class RemoteService:
if resolved_proxy: if resolved_proxy:
create_data["proxy"] = resolved_proxy create_data["proxy"] = resolved_proxy
if not no_proxy and not proxy:
try:
from unshackle.core.utilities import get_cached_ip_info
ip_info = get_cached_ip_info(self._session)
if ip_info and ip_info.get("country"):
create_data["client_region"] = ip_info["country"].lower()
except Exception:
pass
if profile: if profile:
create_data["profile"] = profile create_data["profile"] = profile
if no_proxy: if no_proxy:
@@ -355,7 +537,9 @@ class RemoteService:
return self._titles return self._titles
result = self.client.get(f"/api/session/{self._session_id}/titles") result = self.client.get(f"/api/session/{self._session_id}/titles")
titles_list = [_build_title(t, self.service_tag, self.title_id) for t in result.get("titles", [])] titles_list = [_build_title(t, self.service_tag, self.title_id) for t in result.get("titles", [])]
self._titles = Series(titles_list) if titles_list and isinstance(titles_list[0], Episode) else Movies(titles_list) self._titles = (
Series(titles_list) if titles_list and isinstance(titles_list[0], Episode) else Movies(titles_list)
)
return self._titles return self._titles
def get_titles_cached(self, title_id: str = None) -> Titles_T: def get_titles_cached(self, title_id: str = None) -> Titles_T:
@@ -367,10 +551,78 @@ class RemoteService:
return self._tracks_by_title[title_id] return self._tracks_by_title[title_id]
result = self.client.post(f"/api/session/{self._session_id}/tracks", {"title_id": title_id}) result = self.client.post(f"/api/session/{self._session_id}/tracks", {"title_id": title_id})
tracks = _build_tracks(result) tracks = _build_tracks(result)
for k, v in result.get("session_headers", {}).items():
if k.lower() not in ("host", "content-length", "content-type"):
self._session.headers[k] = v
for k, v in result.get("session_cookies", {}).items():
self._session.cookies.set(k, v)
_resolve_manifest_data(tracks, result.get("manifests", []), self._session)
self._server_cdm_type = result.get("server_cdm_type", "widevine")
self._tracks_by_title[title_id] = tracks self._tracks_by_title[title_id] = tracks
self._chapters_by_title[title_id] = result.get("chapters", []) self._chapters_by_title[title_id] = result.get("chapters", [])
return tracks return tracks
def resolve_server_keys(self, title: Title_T) -> None:
"""Resolve DRM keys via server CDM for all tracks on a title.
Called by dl.py between track selection and download. The server
decides which CDM device to use and tells the client via
server_cdm_type. We send track IDs and the server does the full
CDM flow, returning KID:KEY pairs.
"""
if not self._server_cdm:
return
from uuid import UUID
track_ids = [str(t.id) for t in title.tracks.videos + title.tracks.audio]
if not track_ids:
return
drm_type = getattr(self, "_server_cdm_type", "widevine")
try:
resp = self.client.post(
f"/api/session/{self._session_id}/license",
{
"track_ids": track_ids,
"mode": "server_cdm",
"drm_type": drm_type,
},
)
keys_by_track = resp.get("keys", {})
for track in title.tracks:
track_keys = keys_by_track.get(str(track.id), {})
if not track_keys:
continue
if track.drm:
for drm_obj in track.drm:
if hasattr(drm_obj, "content_keys"):
for kid_hex, key_hex in track_keys.items():
drm_obj.content_keys[UUID(hex=kid_hex)] = key_hex
else:
from pywidevine.pssh import PSSH as WvPSSH
from unshackle.core.drm import Widevine
first_kid = next(iter(track_keys))
dummy_pssh = WvPSSH.new(key_ids=[UUID(hex=first_kid)])
drm_obj = Widevine(pssh=dummy_pssh, kid=first_kid)
for kid_hex, key_hex in track_keys.items():
drm_obj.content_keys[UUID(hex=kid_hex)] = key_hex
track.drm = [drm_obj]
key_count = sum(len(v) for v in keys_by_track.values())
if key_count:
self.log.info(f"Server CDM resolved {key_count} key(s) for {len(keys_by_track)} track(s)")
except Exception as e:
self.log.warning("Failed to resolve server CDM keys: %s", e)
def get_chapters(self, title: Title_T) -> Chapters: def get_chapters(self, title: Title_T) -> Chapters:
title_id = str(title.id) title_id = str(title.id)
if title_id not in self._chapters_by_title: if title_id not in self._chapters_by_title:
@@ -381,18 +633,28 @@ class RemoteService:
def get_widevine_license(self, *, challenge: bytes, title: Title_T, track: AnyTrack) -> Optional[Union[bytes, str]]: def get_widevine_license(self, *, challenge: bytes, title: Title_T, track: AnyTrack) -> Optional[Union[bytes, str]]:
return self._proxy_license(challenge, track, "widevine") return self._proxy_license(challenge, track, "widevine")
def get_playready_license(self, *, challenge: bytes, title: Title_T, track: AnyTrack) -> Optional[Union[bytes, str]]: def get_playready_license(
self, *, challenge: bytes, title: Title_T, track: AnyTrack
) -> Optional[Union[bytes, str]]:
return self._proxy_license(challenge, track, "playready") return self._proxy_license(challenge, track, "playready")
def get_widevine_service_certificate( def get_widevine_service_certificate(
self, *, challenge: bytes, title: Title_T, track: AnyTrack, self,
*,
challenge: bytes,
title: Title_T,
track: AnyTrack,
) -> Union[bytes, str]: ) -> Union[bytes, str]:
try: try:
resp = self.client.post(f"/api/session/{self._session_id}/license", { resp = self.client.post(
"track_id": str(track.id), f"/api/session/{self._session_id}/license",
"challenge": base64.b64encode(challenge).decode("ascii"), {
"drm_type": "widevine", "is_certificate": True, "track_id": str(track.id),
}) "challenge": base64.b64encode(challenge).decode("ascii"),
"drm_type": "widevine",
"is_certificate": True,
},
)
return base64.b64decode(resp["license"]) return base64.b64decode(resp["license"])
except Exception: except Exception:
return None return None
@@ -401,22 +663,50 @@ class RemoteService:
if isinstance(challenge, str): if isinstance(challenge, str):
challenge = challenge.encode("utf-8") challenge = challenge.encode("utf-8")
payload: Dict[str, Any] = { pssh_b64 = None
"track_id": str(track.id),
"challenge": base64.b64encode(challenge).decode("ascii"),
"drm_type": drm_type,
}
if track.drm: if track.drm:
for drm_obj in track.drm: for drm_obj in track.drm:
drm_class = drm_obj.__class__.__name__ drm_class = drm_obj.__class__.__name__
if drm_type == "playready" and drm_class == "PlayReady": if drm_type == "playready" and drm_class == "PlayReady":
payload["pssh"] = drm_obj.data["pssh_b64"] pssh_b64 = drm_obj.data["pssh_b64"]
break break
elif drm_type == "widevine" and drm_class == "Widevine": elif drm_type == "widevine" and drm_class == "Widevine":
payload["pssh"] = drm_obj.pssh.dumps() pssh_b64 = drm_obj.pssh.dumps()
break break
if self._server_cdm:
from uuid import UUID
if pssh_b64:
try:
resp = self.client.post(
f"/api/session/{self._session_id}/license",
{
"track_id": str(track.id),
"drm_type": drm_type,
"mode": "server_cdm",
"pssh": pssh_b64,
},
)
keys = resp.get("keys", {})
if keys and track.drm:
for drm_obj in track.drm:
if hasattr(drm_obj, "content_keys"):
for kid_hex, key_hex in keys.items():
drm_obj.content_keys[UUID(hex=kid_hex)] = key_hex
return challenge
except Exception as e:
self.log.warning("server_cdm license failed: %s", e)
return challenge
payload = {
"track_id": str(track.id),
"challenge": base64.b64encode(challenge).decode("ascii"),
"drm_type": drm_type,
}
if pssh_b64:
payload["pssh"] = pssh_b64
resp = self.client.post(f"/api/session/{self._session_id}/license", payload) resp = self.client.post(f"/api/session/{self._session_id}/license", payload)
return base64.b64decode(resp["license"]) return base64.b64decode(resp["license"])
@@ -448,11 +738,8 @@ class RemoteService:
if not cache_dir.is_dir(): if not cache_dir.is_dir():
return {} return {}
return { return {
f.stem: f.read_text(encoding="utf-8") f.stem: f.read_text(encoding="utf-8") for f in cache_dir.glob("*.json") if not f.stem.startswith("titles_")
for f in cache_dir.glob("*.json")
if not f.stem.startswith("titles_")
} }
__all__ = ("RemoteClient", "RemoteService", "resolve_server") __all__ = ("RemoteClient", "RemoteService", "resolve_server")