14 Commits

Author SHA1 Message Date
Andy
e10c760821 feat(release): Bump version to 1.4.2 and update changelog with new features and fixes 2025-08-14 17:56:01 +00:00
Andy
990084ab1f feat(tags): Implement session management for API requests with retry logic 2025-08-14 02:14:46 +00:00
Andy
8e598f7d6a Merge branch 'main' of https://github.com/unshackle-dl/unshackle 2025-08-13 15:00:33 +00:00
Andy
06687b51fb feat(config): Add series_year option to control year inclusion in titles and YAML configuration 2025-08-13 15:00:30 +00:00
Sp5rky
eb1be7e253 Update README.md 2025-08-12 20:51:29 -06:00
Andy
eac2ff4cee feat(hls): Enhance segment retrieval by allowing all file types and clean up empty segment directories. Fixes issues with VTT files from HLS not being found correctly due to new HLS "changes" 2025-08-12 20:25:42 +00:00
Andy
798b5bf3cd feat(hls): Enhance segment merging with recursive file search and fallback to binary concatenation 2025-08-11 03:53:17 +00:00
Andy
725f7be563 fix(dl): Adjust per_language logic to ensure correct audio track selection and not download all tracks for selected language. 2025-08-09 17:39:36 +00:00
Andy
b2686ca2b1 feat(vault): Add no_push option to Vault and its subclasses to control key reception 2025-08-08 23:38:52 +00:00
Andy
abc3b4f1a4 feat(dl): Add audio language option to override language for audio tracks 2025-08-08 21:57:49 +00:00
Andy
9952758b38 feat(changelog): Update changelog with enhanced tagging configuration and improvements 2025-08-08 05:03:57 +00:00
Andy
f56e7c1ec8 chore(release): Bump version to 1.4.1 and update changelog with title caching features 2025-08-08 04:57:32 +00:00
Andy
096b7d70f8 Merge remote-tracking branch 'origin/main' into feature/title-caching 2025-08-08 04:50:46 +00:00
Andy
f0493292af feat: Implement title caching system to reduce API calls
- Add configurable title caching with fallback support
- Cache titles for 30 minutes by default, with 24-hour fallback on API failures
- Add --no-cache and --reset-cache CLI flags for cache control
- Implement region-aware caching to handle geo-restricted content
- Use SHA256 hashing for cache keys to handle complex title IDs
- Add cache configuration variables to config system
- Document new caching options in example config

This caching system significantly reduces redundant API calls when debugging
or modifying CLI parameters, improving both performance and reliability.
2025-08-06 17:08:58 +00:00
19 changed files with 598 additions and 85 deletions

View File

@@ -5,6 +5,83 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.4.2] - 2025-08-14
### Added
- **Session Management for API Requests**: Enhanced API reliability with retry logic
- Implemented session management for tags functionality with automatic retry mechanisms
- Improved API request stability and error handling
- **Series Year Configuration**: New `series_year` option for title naming control
- Added configurable `series_year` option to control year inclusion in series titles
- Enhanced YAML configuration with series year handling options
- **Audio Language Override**: New audio language selection option
- Added `audio_language` option to override default language selection for audio tracks
- Provides more granular control over audio track selection
- **Vault Key Reception Control**: Enhanced vault security options
- Added `no_push` option to Vault and its subclasses to control key reception
- Improved key management security and flexibility
### Changed
- **HLS Segment Processing**: Enhanced segment retrieval and merging capabilities
- Enhanced segment retrieval to allow all file types for better compatibility
- Improved segment merging with recursive file search and fallback to binary concatenation
- Fixed issues with VTT files from HLS not being found correctly due to format changes
- Added cleanup of empty segment directories after processing
- **Documentation**: Updated README.md with latest information
### Fixed
- **Audio Track Selection**: Improved per-language logic for audio tracks
- Adjusted `per_language` logic to ensure correct audio track selection
- Fixed issue where all tracks for selected language were being downloaded instead of just the intended ones
## [1.4.1] - 2025-08-08
### Added
- **Title Caching System**: Intelligent title caching to reduce redundant API calls
- Configurable title caching with 30-minute default cache duration
- 24-hour fallback cache on API failures for improved reliability
- Region-aware caching to handle geo-restricted content properly
- SHA256 hashing for cache keys to handle complex title IDs
- Added `--no-cache` CLI flag to bypass caching when needed
- Added `--reset-cache` CLI flag to clear existing cache data
- New cache configuration variables in config system
- Documented caching options in example configuration file
- Significantly improves performance when debugging or modifying CLI parameters
- **Enhanced Tagging Configuration**: New options for customizing tag behavior
- Added `tag_group_name` config option to control group name inclusion in tags
- Added `tag_imdb_tmdb` config option to control IMDB/TMDB details in tags
- Added Simkl API endpoint support as fallback when no TMDB API key is provided
- Enhanced tag_file function to prioritize provided TMDB ID when `--tmdb` flag is used
- Improved TMDB ID handling with better prioritization logic
### Changed
- **Language Selection Enhancement**: Improved default language handling
- Updated language option default to 'orig' when no `-l` flag is set
- Avoids hardcoded 'en' default and respects original content language
- **Tagging Logic Improvements**: Simplified and enhanced tagging functionality
- Simplified Simkl search logic with soft-fail when no results found
- Enhanced tag_file function with better TMDB ID prioritization
- Improved error handling in tagging operations
### Fixed
- **Subtitle Processing**: Enhanced subtitle filtering for edge cases
- Fixed ValueError in subtitle filtering for multiple colons in time references
- Improved handling of subtitles containing complex time formatting
- Better error handling for malformed subtitle timestamps
### Removed
- **Docker Support**: Removed Docker configuration from repository
- Removed Dockerfile and .dockerignore files
- Cleaned up README.md Docker-related documentation
- Focuses on direct installation methods
## [1.4.0] - 2025-08-05
### Added

View File

@@ -2,6 +2,10 @@
<img width="16" height="16" alt="no_encryption" src="https://github.com/user-attachments/assets/6ff88473-0dd2-4bbc-b1ea-c683d5d7a134" /> unshackle
<br/>
<sup><em>Movie, TV, and Music Archival Software</em></sup>
<br/>
<a href="https://discord.gg/mHYyPaCbFK">
<img src="https://img.shields.io/discord/1395571732001325127?label=&logo=discord&logoColor=ffffff&color=7289DA&labelColor=7289DA" alt="Discord">
</a>
</p>
## What is unshackle?

View File

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "unshackle"
version = "1.4.0"
version = "1.4.2"
description = "Modular Movie, TV, and Music Archival Software."
authors = [{ name = "unshackle team" }]
requires-python = ">=3.10,<3.13"

View File

@@ -153,6 +153,13 @@ class dl:
default=[],
help="Language wanted for Video, you would use this if the video language doesn't match the audio.",
)
@click.option(
"-al",
"--a-lang",
type=LANGUAGE_RANGE,
default=[],
help="Language wanted for Audio, overrides -l/--lang for audio tracks.",
)
@click.option("-sl", "--s-lang", type=LANGUAGE_RANGE, default=["all"], help="Language wanted for Subtitles.")
@click.option("-fs", "--forced-subs", is_flag=True, default=False, help="Include forced subtitle tracks.")
@click.option(
@@ -240,6 +247,8 @@ class dl:
help="Max workers/threads to download with per-track. Default depends on the downloader.",
)
@click.option("--downloads", type=int, default=1, help="Amount of tracks to download concurrently.")
@click.option("--no-cache", "no_cache", is_flag=True, default=False, help="Bypass title cache for this download.")
@click.option("--reset-cache", "reset_cache", is_flag=True, default=False, help="Clear title cache before fetching.")
@click.pass_context
def cli(ctx: click.Context, **kwargs: Any) -> dl:
return dl(ctx, **kwargs)
@@ -411,6 +420,7 @@ class dl:
wanted: list[str],
lang: list[str],
v_lang: list[str],
a_lang: list[str],
s_lang: list[str],
forced_subs: bool,
sub_format: Optional[Subtitle.Codec],
@@ -461,7 +471,7 @@ class dl:
self.log.info("Authenticated with Service")
with console.status("Fetching Title Metadata...", spinner="dots"):
titles = service.get_titles()
titles = service.get_titles_cached()
if not titles:
self.log.error("No titles returned, nothing to download...")
sys.exit(1)
@@ -586,8 +596,9 @@ class dl:
if language not in processed_video_sort_lang:
processed_video_sort_lang.append(language)
audio_sort_lang = a_lang or lang
processed_audio_sort_lang = []
for language in lang:
for language in audio_sort_lang:
if language == "orig":
if title.language:
orig_lang = str(title.language) if hasattr(title.language, "__str__") else title.language
@@ -751,9 +762,10 @@ class dl:
if not title.tracks.audio:
self.log.error(f"There's no {abitrate}kbps Audio Track...")
sys.exit(1)
if lang:
audio_languages = a_lang or lang
if audio_languages:
processed_lang = []
for language in lang:
for language in audio_languages:
if language == "orig":
if title.language:
orig_lang = (
@@ -780,7 +792,7 @@ class dl:
selected_audio.append(highest_quality)
title.tracks.audio = selected_audio
elif "all" not in processed_lang:
per_language = 0 if len(processed_lang) > 1 else 1
per_language = 1
title.tracks.audio = title.tracks.by_language(
title.tracks.audio, processed_lang, per_language=per_language
)

View File

@@ -1 +1 @@
__version__ = "1.4.0"
__version__ = "1.4.2"

View File

@@ -91,6 +91,11 @@ class Config:
self.update_checks: bool = kwargs.get("update_checks", True)
self.update_check_interval: int = kwargs.get("update_check_interval", 24)
self.scene_naming: bool = kwargs.get("scene_naming", True)
self.series_year: bool = kwargs.get("series_year", True)
self.title_cache_time: int = kwargs.get("title_cache_time", 1800) # 30 minutes default
self.title_cache_max_retention: int = kwargs.get("title_cache_max_retention", 86400) # 24 hours default
self.title_cache_enabled: bool = kwargs.get("title_cache_enabled", True)
@classmethod
def from_yaml(cls, path: Path) -> Config:

View File

@@ -4,6 +4,7 @@ import base64
import html
import json
import logging
import os
import shutil
import subprocess
import sys
@@ -584,11 +585,24 @@ class HLS:
if DOWNLOAD_LICENCE_ONLY.is_set():
return
if segment_save_dir.exists():
segment_save_dir.rmdir()
def find_segments_recursively(directory: Path) -> list[Path]:
"""Find all segment files recursively in any directory structure created by downloaders."""
segments = []
# First check direct files in the directory
if directory.exists():
segments.extend([x for x in directory.iterdir() if x.is_file()])
# If no direct files, recursively search subdirectories
if not segments:
for subdir in directory.iterdir():
if subdir.is_dir():
segments.extend(find_segments_recursively(subdir))
return sorted(segments)
# finally merge all the discontinuity save files together to the final path
segments_to_merge = [x for x in sorted(save_dir.iterdir()) if x.is_file()]
segments_to_merge = find_segments_recursively(save_dir)
if len(segments_to_merge) == 1:
shutil.move(segments_to_merge[0], save_path)
else:
@@ -601,9 +615,16 @@ class HLS:
discontinuity_data = discontinuity_file.read_bytes()
f.write(discontinuity_data)
f.flush()
os.fsync(f.fileno())
discontinuity_file.unlink()
save_dir.rmdir()
# Clean up empty segment directory
if save_dir.exists() and save_dir.name.endswith("_segments"):
try:
save_dir.rmdir()
except OSError:
# Directory might not be empty, try removing recursively
shutil.rmtree(save_dir, ignore_errors=True)
progress(downloaded="Downloaded")
@@ -613,40 +634,75 @@ class HLS:
@staticmethod
def merge_segments(segments: list[Path], save_path: Path) -> int:
"""
Concatenate Segments by first demuxing with FFmpeg.
Concatenate Segments using FFmpeg concat with binary fallback.
Returns the file size of the merged file.
"""
if not binaries.FFMPEG:
raise EnvironmentError("FFmpeg executable was not found but is required to merge HLS segments.")
demuxer_file = segments[0].parent / "ffmpeg_concat_demuxer.txt"
demuxer_file.write_text("\n".join([f"file '{segment}'" for segment in segments]))
subprocess.check_call(
[
binaries.FFMPEG,
"-hide_banner",
"-loglevel",
"panic",
"-f",
"concat",
"-safe",
"0",
"-i",
demuxer_file,
"-map",
"0",
"-c",
"copy",
save_path,
]
)
demuxer_file.unlink()
# Track segment directories for cleanup
segment_dirs = set()
for segment in segments:
segment.unlink()
# Track all parent directories that contain segments
current_dir = segment.parent
while current_dir.name and "_segments" in str(current_dir):
segment_dirs.add(current_dir)
current_dir = current_dir.parent
def cleanup_segments_and_dirs():
"""Clean up segments and directories after successful merge."""
for segment in segments:
segment.unlink(missing_ok=True)
for segment_dir in segment_dirs:
if segment_dir.exists():
try:
shutil.rmtree(segment_dir)
except OSError:
pass # Directory cleanup failed, but merge succeeded
# Try FFmpeg concat first (preferred method)
if binaries.FFMPEG:
try:
demuxer_file = save_path.parent / f"ffmpeg_concat_demuxer_{save_path.stem}.txt"
demuxer_file.write_text("\n".join([f"file '{segment.absolute()}'" for segment in segments]))
subprocess.check_call(
[
binaries.FFMPEG,
"-hide_banner",
"-loglevel",
"error",
"-f",
"concat",
"-safe",
"0",
"-i",
demuxer_file,
"-map",
"0",
"-c",
"copy",
save_path,
],
timeout=300, # 5 minute timeout
)
demuxer_file.unlink(missing_ok=True)
cleanup_segments_and_dirs()
return save_path.stat().st_size
except (subprocess.CalledProcessError, subprocess.TimeoutExpired, OSError) as e:
# FFmpeg failed, clean up demuxer file and fall back to binary concat
logging.getLogger("HLS").debug(f"FFmpeg concat failed ({e}), falling back to binary concatenation")
demuxer_file.unlink(missing_ok=True)
# Remove partial output file if it exists
save_path.unlink(missing_ok=True)
# Fallback: Binary concatenation
logging.getLogger("HLS").debug(f"Using binary concatenation for {len(segments)} segments")
with open(save_path, "wb") as output_file:
for segment in segments:
with open(segment, "rb") as segment_file:
output_file.write(segment_file.read())
cleanup_segments_and_dirs()
return save_path.stat().st_size
@staticmethod

View File

@@ -21,6 +21,7 @@ from unshackle.core.constants import AnyTrack
from unshackle.core.credential import Credential
from unshackle.core.drm import DRM_T
from unshackle.core.search_result import SearchResult
from unshackle.core.title_cacher import TitleCacher, get_account_hash, get_region_from_proxy
from unshackle.core.titles import Title_T, Titles_T
from unshackle.core.tracks import Chapters, Tracks
from unshackle.core.utilities import get_ip_info
@@ -42,6 +43,12 @@ class Service(metaclass=ABCMeta):
self.session = self.get_session()
self.cache = Cacher(self.__class__.__name__)
self.title_cache = TitleCacher(self.__class__.__name__)
# Store context for cache control flags and credential
self.ctx = ctx
self.credential = None # Will be set in authenticate()
self.current_region = None # Will be set based on proxy/geolocation
if not ctx.parent or not ctx.parent.params.get("no_proxy"):
if ctx.parent:
@@ -79,6 +86,15 @@ class Service(metaclass=ABCMeta):
).decode()
}
)
# Store region from proxy
self.current_region = get_region_from_proxy(proxy)
else:
# No proxy, try to get current region
try:
ip_info = get_ip_info(self.session)
self.current_region = ip_info.get("country", "").lower() if ip_info else None
except Exception:
self.current_region = None
# Optional Abstract functions
# The following functions may be implemented by the Service.
@@ -123,6 +139,9 @@ class Service(metaclass=ABCMeta):
raise TypeError(f"Expected cookies to be a {CookieJar}, not {cookies!r}.")
self.session.cookies.update(cookies)
# Store credential for cache key generation
self.credential = credential
def search(self) -> Generator[SearchResult, None, None]:
"""
Search by query for titles from the Service.
@@ -187,6 +206,52 @@ class Service(metaclass=ABCMeta):
This can be useful to store information on each title that will be required like any sub-asset IDs, or such.
"""
def get_titles_cached(self, title_id: str = None) -> Titles_T:
"""
Cached wrapper around get_titles() to reduce redundant API calls.
This method checks the cache before calling get_titles() and handles
fallback to cached data when API calls fail.
Args:
title_id: Optional title ID for cache key generation.
If not provided, will try to extract from service instance.
Returns:
Titles object (Movies, Series, or Album)
"""
# Try to get title_id from service instance if not provided
if title_id is None:
# Different services store the title ID in different attributes
if hasattr(self, "title"):
title_id = self.title
elif hasattr(self, "title_id"):
title_id = self.title_id
else:
# If we can't determine title_id, just call get_titles directly
self.log.debug("Cannot determine title_id for caching, bypassing cache")
return self.get_titles()
# Get cache control flags from context
no_cache = False
reset_cache = False
if self.ctx and self.ctx.parent:
no_cache = self.ctx.parent.params.get("no_cache", False)
reset_cache = self.ctx.parent.params.get("reset_cache", False)
# Get account hash for cache key
account_hash = get_account_hash(self.credential)
# Use title cache to get titles with fallback support
return self.title_cache.get_cached_titles(
title_id=str(title_id),
fetch_function=self.get_titles,
region=self.current_region,
account_hash=account_hash,
no_cache=no_cache,
reset_cache=reset_cache,
)
@abstractmethod
def get_tracks(self, title: Title_T) -> Tracks:
"""

View File

@@ -0,0 +1,240 @@
from __future__ import annotations
import hashlib
import logging
from datetime import datetime, timedelta
from typing import Optional
from unshackle.core.cacher import Cacher
from unshackle.core.config import config
from unshackle.core.titles import Titles_T
class TitleCacher:
"""
Handles caching of Title objects to reduce redundant API calls.
This wrapper provides:
- Region-aware caching to handle geo-restricted content
- Automatic fallback to cached data when API calls fail
- Cache lifetime extension during failures
- Cache hit/miss statistics for debugging
"""
def __init__(self, service_name: str):
self.service_name = service_name
self.log = logging.getLogger(f"{service_name}.TitleCache")
self.cacher = Cacher(service_name)
self.stats = {"hits": 0, "misses": 0, "fallbacks": 0}
def _generate_cache_key(
self, title_id: str, region: Optional[str] = None, account_hash: Optional[str] = None
) -> str:
"""
Generate a unique cache key for title data.
Args:
title_id: The title identifier
region: The region/proxy identifier
account_hash: Hash of account credentials (if applicable)
Returns:
A unique cache key string
"""
# Hash the title_id to handle complex IDs (URLs, dots, special chars)
# This ensures consistent length and filesystem-safe keys
title_hash = hashlib.sha256(title_id.encode()).hexdigest()[:16]
# Start with base key using hash
key_parts = ["titles", title_hash]
# Add region if available
if region:
key_parts.append(region.lower())
# Add account hash if available
if account_hash:
key_parts.append(account_hash[:8]) # Use first 8 chars of hash
# Join with underscores
cache_key = "_".join(key_parts)
# Log the mapping for debugging
self.log.debug(f"Cache key mapping: {title_id} -> {cache_key}")
return cache_key
def get_cached_titles(
self,
title_id: str,
fetch_function,
region: Optional[str] = None,
account_hash: Optional[str] = None,
no_cache: bool = False,
reset_cache: bool = False,
) -> Optional[Titles_T]:
"""
Get titles from cache or fetch from API with fallback support.
Args:
title_id: The title identifier
fetch_function: Function to call to fetch fresh titles
region: The region/proxy identifier
account_hash: Hash of account credentials
no_cache: Bypass cache completely
reset_cache: Clear cache before fetching
Returns:
Titles object (Movies, Series, or Album)
"""
# If caching is globally disabled or no_cache flag is set
if not config.title_cache_enabled or no_cache:
self.log.debug("Cache bypassed, fetching fresh titles")
return fetch_function()
# Generate cache key
cache_key = self._generate_cache_key(title_id, region, account_hash)
# If reset_cache flag is set, clear the cache entry
if reset_cache:
self.log.info(f"Clearing cache for {cache_key}")
cache_path = (config.directories.cache / self.service_name / cache_key).with_suffix(".json")
if cache_path.exists():
cache_path.unlink()
# Try to get from cache
cache = self.cacher.get(cache_key, version=1)
# Check if we have valid cached data
if cache and not cache.expired:
self.stats["hits"] += 1
self.log.debug(f"Cache hit for {title_id} (hits: {self.stats['hits']}, misses: {self.stats['misses']})")
return cache.data
# Cache miss or expired, try to fetch fresh data
self.stats["misses"] += 1
self.log.debug(f"Cache miss for {title_id}, fetching fresh data")
try:
# Attempt to fetch fresh titles
titles = fetch_function()
if titles:
# Successfully fetched, update cache
self.log.debug(f"Successfully fetched titles for {title_id}, updating cache")
cache = self.cacher.get(cache_key, version=1)
cache.set(titles, expiration=datetime.now() + timedelta(seconds=config.title_cache_time))
return titles
except Exception as e:
# API call failed, check if we have fallback cached data
if cache and cache.data:
# We have expired cached data, use it as fallback
current_time = datetime.now()
max_retention_time = cache.expiration + timedelta(
seconds=config.title_cache_max_retention - config.title_cache_time
)
if current_time < max_retention_time:
self.stats["fallbacks"] += 1
self.log.warning(
f"API call failed for {title_id}, using cached data as fallback "
f"(fallbacks: {self.stats['fallbacks']})"
)
self.log.debug(f"Error was: {e}")
# Extend cache lifetime
extended_expiration = current_time + timedelta(minutes=5)
if extended_expiration < max_retention_time:
cache.expiration = extended_expiration
cache.set(cache.data, expiration=extended_expiration)
return cache.data
else:
self.log.error(f"API call failed and cached data for {title_id} exceeded maximum retention time")
# Re-raise the exception if no fallback available
raise
def clear_all_title_cache(self):
"""Clear all title caches for this service."""
cache_dir = config.directories.cache / self.service_name
if cache_dir.exists():
for cache_file in cache_dir.glob("titles_*.json"):
cache_file.unlink()
self.log.info(f"Cleared cache file: {cache_file.name}")
def get_cache_stats(self) -> dict:
"""Get cache statistics."""
total = sum(self.stats.values())
if total > 0:
hit_rate = (self.stats["hits"] / total) * 100
else:
hit_rate = 0
return {
"hits": self.stats["hits"],
"misses": self.stats["misses"],
"fallbacks": self.stats["fallbacks"],
"hit_rate": f"{hit_rate:.1f}%",
}
def get_region_from_proxy(proxy_url: Optional[str]) -> Optional[str]:
"""
Extract region identifier from proxy URL.
Args:
proxy_url: The proxy URL string
Returns:
Region identifier or None
"""
if not proxy_url:
return None
# Try to extract region from common proxy patterns
# e.g., "us123.nordvpn.com", "gb-proxy.example.com"
import re
# Pattern for NordVPN style
nord_match = re.search(r"([a-z]{2})\d+\.nordvpn", proxy_url.lower())
if nord_match:
return nord_match.group(1)
# Pattern for country code at start
cc_match = re.search(r"([a-z]{2})[-_]", proxy_url.lower())
if cc_match:
return cc_match.group(1)
# Pattern for country code subdomain
subdomain_match = re.search(r"://([a-z]{2})\.", proxy_url.lower())
if subdomain_match:
return subdomain_match.group(1)
return None
def get_account_hash(credential) -> Optional[str]:
"""
Generate a hash for account identification.
Args:
credential: Credential object
Returns:
SHA1 hash of the credential or None
"""
if not credential:
return None
# Use existing sha1 property if available
if hasattr(credential, "sha1"):
return credential.sha1
# Otherwise generate hash from username
if hasattr(credential, "username"):
return hashlib.sha1(credential.username.encode()).hexdigest()
return None

View File

@@ -81,7 +81,7 @@ class Episode(Title):
def __str__(self) -> str:
return "{title}{year} S{season:02}E{number:02} {name}".format(
title=self.title,
year=f" {self.year}" if self.year else "",
year=f" {self.year}" if self.year and config.series_year else "",
season=self.season,
number=self.number,
name=self.name or "",
@@ -95,13 +95,13 @@ class Episode(Title):
# Title [Year] SXXEXX Name (or Title [Year] SXX if folder)
if folder:
name = f"{self.title}"
if self.year:
if self.year and config.series_year:
name += f" {self.year}"
name += f" S{self.season:02}"
else:
name = "{title}{year} S{season:02}E{number:02} {name}".format(
title=self.title.replace("$", "S"), # e.g., Arli$$
year=f" {self.year}" if self.year else "",
year=f" {self.year}" if self.year and config.series_year else "",
season=self.season,
number=self.number,
name=self.name or "",
@@ -197,7 +197,7 @@ class Series(SortedKeyList, ABC):
def __str__(self) -> str:
if not self:
return super().__str__()
return self[0].title + (f" ({self[0].year})" if self[0].year else "")
return self[0].title + (f" ({self[0].year})" if self[0].year and config.series_year else "")
def tree(self, verbose: bool = False) -> Tree:
seasons = Counter(x.season for x in self)

View File

@@ -10,6 +10,7 @@ from pathlib import Path
from typing import Optional, Tuple
import requests
from requests.adapters import HTTPAdapter, Retry
from unshackle.core import binaries
from unshackle.core.config import config
@@ -25,6 +26,22 @@ HEADERS = {"User-Agent": "unshackle-tags/1.0"}
log = logging.getLogger("TAGS")
def _get_session() -> requests.Session:
"""Create a requests session with retry logic for network failures."""
session = requests.Session()
session.headers.update(HEADERS)
retry = Retry(
total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504], allowed_methods=["GET", "POST"]
)
adapter = HTTPAdapter(max_retries=retry)
session.mount("https://", adapter)
session.mount("http://", adapter)
return session
def _api_key() -> Optional[str]:
return config.tmdb_api_key or os.getenv("TMDB_API_KEY")
@@ -59,7 +76,8 @@ def search_simkl(title: str, year: Optional[int], kind: str) -> Tuple[Optional[d
filename += " 2160p.mkv"
try:
resp = requests.post("https://api.simkl.com/search/file", json={"file": filename}, headers=HEADERS, timeout=30)
session = _get_session()
resp = session.post("https://api.simkl.com/search/file", json={"file": filename}, timeout=30)
resp.raise_for_status()
data = resp.json()
log.debug("Simkl API response received")
@@ -139,17 +157,21 @@ def search_tmdb(title: str, year: Optional[int], kind: str) -> Tuple[Optional[in
if year is not None:
params["year" if kind == "movie" else "first_air_date_year"] = year
r = requests.get(
f"https://api.themoviedb.org/3/search/{kind}",
params=params,
headers=HEADERS,
timeout=30,
)
r.raise_for_status()
js = r.json()
results = js.get("results") or []
log.debug("TMDB returned %d results", len(results))
if not results:
try:
session = _get_session()
r = session.get(
f"https://api.themoviedb.org/3/search/{kind}",
params=params,
timeout=30,
)
r.raise_for_status()
js = r.json()
results = js.get("results") or []
log.debug("TMDB returned %d results", len(results))
if not results:
return None, None
except requests.RequestException as exc:
log.warning("Failed to search TMDB for %s: %s", title, exc)
return None, None
best_ratio = 0.0
@@ -196,10 +218,10 @@ def get_title(tmdb_id: int, kind: str) -> Optional[str]:
return None
try:
r = requests.get(
session = _get_session()
r = session.get(
f"https://api.themoviedb.org/3/{kind}/{tmdb_id}",
params={"api_key": api_key},
headers=HEADERS,
timeout=30,
)
r.raise_for_status()
@@ -219,10 +241,10 @@ def get_year(tmdb_id: int, kind: str) -> Optional[int]:
return None
try:
r = requests.get(
session = _get_session()
r = session.get(
f"https://api.themoviedb.org/3/{kind}/{tmdb_id}",
params={"api_key": api_key},
headers=HEADERS,
timeout=30,
)
r.raise_for_status()
@@ -243,16 +265,21 @@ def external_ids(tmdb_id: int, kind: str) -> dict:
return {}
url = f"https://api.themoviedb.org/3/{kind}/{tmdb_id}/external_ids"
log.debug("Fetching external IDs for %s %s", kind, tmdb_id)
r = requests.get(
url,
params={"api_key": api_key},
headers=HEADERS,
timeout=30,
)
r.raise_for_status()
js = r.json()
log.debug("External IDs response: %s", js)
return js
try:
session = _get_session()
r = session.get(
url,
params={"api_key": api_key},
timeout=30,
)
r.raise_for_status()
js = r.json()
log.debug("External IDs response: %s", js)
return js
except requests.RequestException as exc:
log.warning("Failed to fetch external IDs for %s %s: %s", kind, tmdb_id, exc)
return {}
def _apply_tags(path: Path, tags: dict[str, str]) -> None:

View File

@@ -4,8 +4,9 @@ from uuid import UUID
class Vault(metaclass=ABCMeta):
def __init__(self, name: str):
def __init__(self, name: str, no_push: bool = False):
self.name = name
self.no_push = no_push
def __str__(self) -> str:
return f"{self.name} {type(self).__name__}"

View File

@@ -57,7 +57,7 @@ class Vaults:
"""Add a KID:KEY to all Vaults, optionally with an exclusion."""
success = 0
for vault in self.vaults:
if vault != excluding:
if vault != excluding and not vault.no_push:
try:
success += vault.add_key(self.service, kid, key)
except (PermissionError, NotImplementedError):
@@ -68,13 +68,15 @@ class Vaults:
"""
Add multiple KID:KEYs to all Vaults. Duplicate Content Keys are skipped.
PermissionErrors when the user cannot create Tables are absorbed and ignored.
Vaults with no_push=True are skipped.
"""
success = 0
for vault in self.vaults:
try:
success += bool(vault.add_keys(self.service, kid_keys))
except (PermissionError, NotImplementedError):
pass
if not vault.no_push:
try:
success += bool(vault.add_keys(self.service, kid_keys))
except (PermissionError, NotImplementedError):
pass
return success

View File

@@ -15,12 +15,23 @@ set_terminal_bg: false
# false for style - Prime Suspect S07E01 The Final Act - Part One
scene_naming: true
# Whether to include the year in series names for episodes and folders (default: true)
# true for style - Show Name (2023) S01E01 Episode Name
# false for style - Show Name S01E01 Episode Name
series_year: true
# Check for updates from GitHub repository on startup (default: true)
update_checks: true
# How often to check for updates, in hours (default: 24)
update_check_interval: 24
# Title caching configuration
# Cache title metadata to reduce redundant API calls
title_cache_enabled: true # Enable/disable title caching globally (default: true)
title_cache_time: 1800 # Cache duration in seconds (default: 1800 = 30 minutes)
title_cache_max_retention: 86400 # Maximum cache retention for fallback when API fails (default: 86400 = 24 hours)
# Muxing configuration
muxing:
set_title: false
@@ -95,6 +106,8 @@ remote_cdm:
secret: secret_key
# Key Vaults store your obtained Content Encryption Keys (CEKs)
# Use 'no_push: true' to prevent a vault from receiving pushed keys
# while still allowing it to provide keys when requested
key_vaults:
- type: SQLite
name: Local
@@ -104,6 +117,7 @@ key_vaults:
# name: "Remote Vault"
# uri: "https://key-vault.example.com"
# token: "secret_token"
# no_push: true # This vault will only provide keys, not receive them
# - type: MySQL
# name: "MySQL Vault"
# host: "127.0.0.1"
@@ -111,6 +125,7 @@ key_vaults:
# database: vault
# username: user
# password: pass
# no_push: false # Default behavior - vault both provides and receives keys
# Choose what software to use to download data
downloader: aria2c

View File

@@ -10,8 +10,8 @@ from unshackle.core.vault import Vault
class API(Vault):
"""Key Vault using a simple RESTful HTTP API call."""
def __init__(self, name: str, uri: str, token: str):
super().__init__(name)
def __init__(self, name: str, uri: str, token: str, no_push: bool = False):
super().__init__(name, no_push)
self.uri = uri.rstrip("/")
self.session = Session()
self.session.headers.update({"User-Agent": f"unshackle v{__version__}"})

View File

@@ -18,7 +18,15 @@ class InsertResult(Enum):
class HTTP(Vault):
"""Key Vault using HTTP API with support for both query parameters and JSON payloads."""
def __init__(self, name: str, host: str, password: str, username: Optional[str] = None, api_mode: str = "query"):
def __init__(
self,
name: str,
host: str,
password: str,
username: Optional[str] = None,
api_mode: str = "query",
no_push: bool = False,
):
"""
Initialize HTTP Vault.
@@ -28,8 +36,9 @@ class HTTP(Vault):
password: Password for query mode or API token for json mode
username: Username (required for query mode, ignored for json mode)
api_mode: "query" for query parameters or "json" for JSON API
no_push: If True, this vault will not receive pushed keys
"""
super().__init__(name)
super().__init__(name, no_push)
self.url = host
self.password = password
self.username = username

View File

@@ -12,12 +12,12 @@ from unshackle.core.vault import Vault
class MySQL(Vault):
"""Key Vault using a remotely-accessed mysql database connection."""
def __init__(self, name: str, host: str, database: str, username: str, **kwargs):
def __init__(self, name: str, host: str, database: str, username: str, no_push: bool = False, **kwargs):
"""
All extra arguments provided via **kwargs will be sent to pymysql.connect.
This can be used to provide more specific connection information.
"""
super().__init__(name)
super().__init__(name, no_push)
self.slug = f"{host}:{database}:{username}"
self.conn_factory = ConnectionFactory(
dict(host=host, db=database, user=username, cursorclass=DictCursor, **kwargs)

View File

@@ -12,8 +12,8 @@ from unshackle.core.vault import Vault
class SQLite(Vault):
"""Key Vault using a locally-accessed sqlite DB file."""
def __init__(self, name: str, path: Union[str, Path]):
super().__init__(name)
def __init__(self, name: str, path: Union[str, Path], no_push: bool = False):
super().__init__(name, no_push)
self.path = Path(path).expanduser()
# TODO: Use a DictCursor or such to get fetches as dict?
self.conn_factory = ConnectionFactory(self.path)

2
uv.lock generated
View File

@@ -1505,7 +1505,7 @@ wheels = [
[[package]]
name = "unshackle"
version = "1.4.0"
version = "1.4.1"
source = { editable = "." }
dependencies = [
{ name = "appdirs" },