diff --git a/README.md b/README.md index 2c350049..199362ff 100644 --- a/README.md +++ b/README.md @@ -26,13 +26,19 @@ A modular archival tool for movies, TV, and music. Fork of [Devine](https://gith ## Install +The recommended way is a git clone with the install script. The script installs `uv` and unshackle, then runs `unshackle setup` to download the external tools and write a first config. + ```shell -uv tool install git+https://github.com/unshackle-dl/unshackle.git -unshackle --help +git clone https://github.com/unshackle-dl/unshackle.git +cd unshackle +sh install.sh # Windows: .\install.bat +uv run unshackle env check ``` -> [!TIP] -> Prefer `uv run unshackle ...` inside a clone to keep the virtual environment active. +In a clone, run every command as `uv run unshackle ...`. + +> [!NOTE] +> Advanced: `uv tool install git+https://github.com/unshackle-dl/unshackle.git` installs unshackle into the tool folder of `uv`, not a folder that you choose. `unshackle setup` then puts the tools, config, and data in your user data and config directories. See [Installation](https://docs.unshackle.dev/getting-started/installation/) for details. ### Requirements diff --git a/docs/dev/rest-api/quickstart.md b/docs/dev/rest-api/quickstart.md index d802fd9b..3cf14057 100644 --- a/docs/dev/rest-api/quickstart.md +++ b/docs/dev/rest-api/quickstart.md @@ -13,19 +13,22 @@ The full loop runs end to end: start the server, authenticate, submit a download You need a working unshackle install with services and CDM devices already configured, exactly as you would for a normal command-line download. If `unshackle dl` works for you, the API will too, since it runs the identical download pipeline. -=== "Installed as a tool" - - ```shell - uv tool install git+https://github.com/unshackle-dl/unshackle.git - unshackle --help - ``` +If you have no install yet, follow [Installation](../../getting-started/installation.md). Then check your tools: === "From a clone" ```shell - uv run unshackle --help + uv run unshackle env check ``` +=== "Installed as a uv tool" + + ```shell + unshackle env check + ``` + +This page uses the bare `unshackle` command. From a git clone, the recommended install, run each command as `uv run unshackle ...`. + --- ## Step 1: Configure an API secret diff --git a/docs/getting-started/configuration-file.md b/docs/getting-started/configuration-file.md index ec2a3e02..f04e5746 100644 --- a/docs/getting-started/configuration-file.md +++ b/docs/getting-started/configuration-file.md @@ -17,11 +17,16 @@ When unshackle starts, it searches a fixed list of locations and uses the **firs | # | Location | Typical path | |---|----------|--------------| -| 1 | The unshackle package folder | `.../site-packages/unshackle/unshackle.yaml` | -| 2 | The parent of the package folder | `.../site-packages/unshackle.yaml` | +| 1 | The unshackle package folder | `/unshackle/unshackle.yaml` (a `uv` tool install: `.../site-packages/unshackle/unshackle.yaml`) | +| 2 | The parent of the package folder | `/unshackle.yaml` (a `uv` tool install: `.../site-packages/unshackle.yaml`) | | 3 | Your OS user-config directory | see the table below | -The third location, your per-user config directory, is the recommended place for most installations, because it lives outside the package and survives reinstalls and upgrades. Its exact path depends on your operating system: +Where the file belongs depends on how you installed unshackle: + +- **Git clone (recommended).** Keep the file at `unshackle/unshackle.yaml` inside the clone, which is location 1. [`unshackle setup`](installation.md#what-unshackle-setup-does) writes a new config there, and it stays with the clone when you update it. +- **`uv` tool install.** Use location 3, your per-user config directory. A `uv` tool upgrade replaces the package folder, so a file at location 1 or 2 is lost. `unshackle setup` writes a new config there. + +The path of the per-user config directory depends on your operating system: === "Linux" @@ -217,7 +222,10 @@ $ unshackle cfg cdm.default --unset $ unshackle cfg --list ``` -When it writes, `unshackle cfg` targets the config file that unshackle loaded. If none exists yet, it makes `unshackle.yaml` inside the `unshackle` package folder (location 1 above), not your OS user-config directory. To keep the config outside the package, make the file at the user-config path yourself first, then `unshackle cfg` writes to it. +When it writes, `unshackle cfg` targets the config file that unshackle loaded. If none exists yet, it makes `unshackle.yaml` in the `unshackle` package folder (location 1 above): + +- **Git clone.** That is `unshackle/unshackle.yaml` in the clone, which is the correct place. +- **`uv` tool install.** A tool upgrade replaces the package folder and deletes that file. Run `unshackle setup` first, which writes the config to your user config directory, or make the file there yourself. Then `unshackle cfg` writes to it. !!! warning "Editing with `cfg` strips comments" Because `unshackle cfg` rewrites the whole file when it saves, a write removes any comments in `unshackle.yaml`. If you keep important notes as comments, edit the file by hand instead, or keep those notes elsewhere. diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index efbf9e47..3c38a18e 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -2,15 +2,139 @@ unshackle is a modular archival tool for movies, TV, and music. It ships as a single Python package named `unshackle` that installs one console command, also -called `unshackle`. This page covers installing that command, the Python -version it needs, and the external command-line tools it expects to find on your -`PATH`. +called `unshackle`. This page covers the install script, the `unshackle setup` +command, the Python version unshackle needs, and the external command-line tools +it expects to find. !!! note "What you get" Installing the package registers a single entry point, the `unshackle` command. It covers everything: downloading, key-vault management, device provisioning, the REST server, and the helper utilities. +## Install from a git clone (recommended) + +A git clone with the install script is the recommended way to install unshackle. +A clone keeps unshackle, its external tools, its config, its data folders, and +your services together in one folder that you can see. + +The repository has an install script for Linux (`install.sh`) and for Windows +(`install.bat`). The script installs `uv` if it is not found, installs unshackle, +and then runs [`unshackle setup`](#what-unshackle-setup-does). + +=== "Linux" + + ```shell + git clone https://github.com/unshackle-dl/unshackle.git + cd unshackle + sh install.sh + ``` + +=== "Windows" + + ```bat + git clone https://github.com/unshackle-dl/unshackle.git + cd unshackle + .\install.bat + ``` + +In a clone, the script runs `uv sync` and then `uv run unshackle setup`. The last +lines of setup tell you to check your tools with `uv run unshackle env check`. +Run unshackle from the clone with `uv run unshackle ...`. + +To install without a clone, see +[Advanced: install as a uv tool](#advanced-install-as-a-uv-tool). + +## What `unshackle setup` does + +`unshackle setup` prepares a new installation. The install script runs it for +you, and you can run it yourself at any time: + +=== "From a clone" + + ```shell + uv run unshackle setup + ``` + +=== "Installed as a uv tool" + + ```shell + unshackle setup + ``` + +The command does these steps: + +1. It shows which [external tools](#external-tools-on-your-path) are missing, + and where to get the ones that it cannot download. +2. It asks once for each tool or group of tools to download. The required + tools come first. shaka-packager and `mp4decrypt` share one DRM question, so + `mp4decrypt` comes with the required tools, but `env check` still shows it + as optional. The HDR and subtitle groups are optional. The prompts show the + version that setup downloads. On Windows, the subtitle question also shows + the download size of CCExtractor, which is much larger than the other tools. + It puts the downloads in a `binaries` folder + (see [Where setup puts files](#where-setup-puts-files)). +3. If no `unshackle.yaml` exists, it asks for a downloads folder and writes a + minimal config file. +4. It makes the data folders for cookies, WVDs, PRDs, and services. +5. It asks for `.wvd` and `.prd` devices to import. You can give a file or a + folder of devices, and it asks again until you give an empty answer. It + validates each file, copies it into the WVDs or PRDs folder, and prints one + line for each file. If setup wrote a new config in this run, it sets + `cdm.default` to the imported device. If you imported more than one, it + asks which one is the default. +6. It prints a summary of the tools that it found. If it wrote a config, it + gives the path of that config and a link to the + [configuration reference](../reference/configuration/index.md). Then it + tells you to run `env check` next. + +You can run the command again. It skips the tools that it finds and never +overwrites an existing config. + +!!! note "Not every tool has a download" + Setup downloads tools only on Linux and Windows, and only when the tool + has a portable build for your system. For other tools, setup prints where + to get them, for example a package manager. On macOS, setup downloads + nothing and prints these links for each missing tool. + + On Linux, setup cannot download MKVToolNix, which is a required tool. + Install the `mkvtoolnix` package with your package manager (for example + `apt`, `dnf`, or `pacman`). + +!!! warning "CCExtractor on Linux needs FUSE" + On Linux, setup downloads CCExtractor as an AppImage, which needs FUSE to + run. Setup runs the AppImage once after the download. If it does not run, setup removes it and prints the + manual install link. Install FUSE with your package manager and run setup + again, or install the `ccextractor` package instead. + +### Where setup puts files + +| Item | From a git clone | From a `uv` tool install | +| --- | --- | --- | +| External tools | `unshackle/binaries` in the clone | `binaries` in the user data directory | +| New config (`unshackle.yaml`) | `unshackle/unshackle.yaml` in the clone | The user config directory | +| Data folders (cookies, WVDs, PRDs, cache, logs, services, temp) | The default folders in the clone | The user data directory | +| Downloads | The folder that you give at the prompt | The folder that you give at the prompt | + +On Windows, CCExtractor and its libraries go into their own `ccextractor` +subfolder of the tools folder. + +On Linux, the user data directory is usually `~/.local/share/unshackle` and the +user config directory is usually `~/.config/unshackle`. On Windows, both are +`%LOCALAPPDATA%\unshackle`. Run `uv run unshackle env info` to see the paths that +unshackle uses. Its `User_Binaries` row shows the `binaries` folder in the user +data directory. + +Setup writes a new config only when it finds no `unshackle.yaml` in any of the +[config locations](configuration-file.md). It keeps an existing config where it +is. In a clone, the new config sets only `directories.downloads`, so the data +folders stay at their defaults in the clone. + +A `uv` tool upgrade replaces the package folder. For a `uv` tool install, setup +puts the tools in the user data directory and writes a `directories` block that +moves the data folders there, so an upgrade keeps them. If an existing config +keeps data folders inside the package of a `uv` tool install, setup shows a +warning and prints the `directories` lines to add. + ## Requirements at a glance | Requirement | Recommended version | Needed for | @@ -37,38 +161,22 @@ You do not usually need to install a matching Python interpreter by hand, since `uv` can download and manage one for you. If you work inside an existing virtual environment, make sure that it is on a supported version. -## Installing with uv +## Installing by hand -The recommended way to install unshackle is as a `uv` tool. This creates an -isolated environment for the package and puts the `unshackle` command on your -`PATH` without touching your system Python. - -```shell title="Install as a uv tool" -uv tool install git+https://github.com/unshackle-dl/unshackle.git -``` - -Once it finishes, the `unshackle` command is available globally: - -```shell -unshackle --help -``` - -!!! tip "Upgrading and removing" - Because you installed it from a git URL, re-run the same - `uv tool install` command (uv will update in place) to get a newer version, - and use `uv tool uninstall unshackle` to remove it. +If you do not use the install script, install unshackle with `uv` and then run +`unshackle setup`, or install the [external tools](#external-tools-on-your-path) +yourself. ### Running from a clone -If you would rather work from a checkout of the repository (for example to try -an in-development branch or to keep your own local services alongside unshackle), use -`uv run` inside the clone. This keeps the project's virtual environment active -for the duration of the command, so you do not have to activate it yourself. +Use `uv run` inside the clone. It keeps the project's virtual environment active +for the duration of the command, so you do not have to activate it yourself. It +also installs the dependencies on the first run. ```shell title="Run from a git clone" git clone https://github.com/unshackle-dl/unshackle.git cd unshackle -uv run unshackle --help +uv run unshackle setup ``` Every command shown elsewhere in these docs works the same way from a clone. @@ -78,6 +186,38 @@ Prefix it with `uv run`: uv run unshackle env check ``` +### Advanced: install as a uv tool + +This way is for users who know `uv` tools and want the `unshackle` command on +their `PATH` without a clone. + +!!! note "Where a uv tool install keeps files" + `uv tool install` puts unshackle into the tool folder of `uv`, not into a + folder that you choose. `unshackle setup` therefore puts the external + tools, the config, and the data folders in your user data and config + directories (see [Where setup puts files](#where-setup-puts-files)). A git + clone is the easier setup: its tools, config, data, and services stay in + one folder that you can see. On the first run from a `uv` tool install, + `unshackle setup` tells you this and asks if you want to continue. + +Install the tool and run setup: + +```shell title="Install as a uv tool" +uv tool install git+https://github.com/unshackle-dl/unshackle.git +unshackle setup +``` + +The install scripts also do this when you run one alone, without a clone: when +the script does not find the unshackle `pyproject.toml` next to it, it runs +`uv tool install` and then `unshackle setup`. When it finishes, it tells you to +open a new terminal, because the `PATH` change only applies to new terminals. +Run `unshackle env check` there. + +!!! tip "Upgrading and removing" + Because you installed it from a git URL, re-run the same + `uv tool install` command (uv will update in place) to get a newer version, + and use `uv tool uninstall unshackle` to remove it. + !!! note "Developer note" Contributors will also want the development and test dependency groups. The project defines `dev` (linters, type checkers) and `test` (pytest and @@ -89,9 +229,13 @@ uv run unshackle env check unshackle shells out to a number of external command-line programs for media processing and DRM work. It looks for each one first in the `binaries/` folder -inside the installed package, and then falls back to your system `PATH`, so -installing these tools system-wide (or dropping them into that folder) both -work. +inside the installed package, then in the `binaries` folder of your user data +directory, and then on your system `PATH`. `unshackle setup` installs tools into +the first folder in a clone and into the second for a `uv` tool install. Installing a tool system-wide or putting it into one of those folders +both work. + +`unshackle setup` installs the required tools and some optional ones for you. +Use the tables below when you install a tool by hand. ### Required tools @@ -128,7 +272,7 @@ download. For example, `dovi_tool` only matters if you download Dolby Vision vid | dovi_tool | `dovi_tool` | Dolby Vision metadata handling | | HDR10Plus_tool | `hdr10plus_tool` | HDR10+ metadata handling | | SubtitleEdit | `seconv`, `SubtitleEdit` | Subtitle conversion (the `SeConv` CLI from 5.x) | -| CCExtractor | `ccextractor` | Extracting closed captions | +| CCExtractor | `ccextractor`, `ccextractorwin`, `ccextractorwinfull` | Extracting closed captions | | FFplay | `ffplay` | Simple preview player | | MPV | `mpv` | Advanced preview player (used by `util crop`/`util range` previews) | | HolaProxy | `hola-proxy` | Hola proxy provider support | @@ -148,35 +292,12 @@ download. For example, `dovi_tool` only matters if you download Dolby Vision vid ## Verifying your installation -After installation, make sure that the command works and that unshackle can see -the tools it depends on. - -### Make sure the command is available - -```shell -unshackle --help -``` - -This prints the list of available commands (`dl`, `env`, `kv`, -`serve`, `wvd`, `prd`, `search`, `util`, `cfg`, `import`). If your shell reports -that `unshackle` is not found, the `uv tool` bin directory is probably not on -your `PATH`. Operate `uv tool update-shell` and open a new terminal. +After installation, make sure that unshackle can see the tools it depends on. ### Examine the external dependencies -The `env check` subcommand inspects your environment and reports which tools it -found: - -```shell -unshackle env check -``` - -=== "Output" - - It prints a table grouped by category (Core, DRM, HDR, Subtitle, Player, - Network) with a status column: a green check for tools that resolved and a - red cross for those it could not find, plus a summary line such as - `All required tools installed ✓` or `Missing required: `. +The `env check` subcommand is the first check. It inspects your environment and +reports which tools it found: === "From a clone" @@ -184,8 +305,24 @@ unshackle env check uv run unshackle env check ``` -If a required tool shows as missing, install it and put it on your `PATH` (or in -the package's `binaries/` folder), then re-run the check. +=== "Installed as a uv tool" + + ```shell + unshackle env check + ``` + +It prints a table grouped by category (Core, DRM, HDR, Subtitle, Player, +Network) with a status column: a green check for tools that resolved and a red +cross for those it could not find, plus a summary line such as +`All required tools installed ✓` or `Missing required: `. + +If a required tool shows as missing, `env check` tells you to run +`unshackle setup`. You can also install the tool by hand and put it on your +`PATH` (or in one of the `binaries` folders), then re-run the check. + +!!! tip "`unshackle` is not found after a uv tool install" + The bin directory of `uv` tools is probably not on your `PATH`. Run + `uv tool update-shell` and open a new terminal. ### Inspect environment paths @@ -193,7 +330,7 @@ To see where unshackle will read its configuration from and where it stores downloads, caches, cookies, and devices, use: ```shell -unshackle env info +uv run unshackle env info ``` If no configuration file exists yet, this command lists the locations unshackle @@ -202,5 +339,6 @@ searches for one, so you know where to make it. See the ## Next steps -- Set up your [configuration](configuration-file.md) file (`unshackle.yaml`). +- Set up your [configuration](configuration-file.md) file (`unshackle.yaml`), or + let `unshackle setup` write a first one. - Start [downloading](../guide/downloading.md) titles when `env check` finds every required tool. diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index 10900ab5..62819448 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -6,9 +6,10 @@ module and CDM it needs to fetch and decrypt media, operate a basic `unshackle d command, and learn where the finished file lands. !!! note "Before you start" - This guide assumes unshackle is already installed and on your `PATH`. If - `unshackle --help` does not print a help screen, work through - [Installation](installation.md) first. + This guide assumes that you installed unshackle as + [Installation](installation.md) describes. The examples use the bare + `unshackle` command. From a git clone, the recommended install, run each + one as `uv run unshackle ...`, for example `uv run unshackle env check`. !!! warning "unshackle includes no services" unshackle handles manifests, DRM, downloading, and muxing. It does not include a @@ -16,52 +17,47 @@ command, and learn where the finished file lands. ([step 3](#3-write-a-service)). `EXAMPLE` in this guide stands in for the tag you give yours. -## 1. Examine your environment +## 1. Run setup unshackle drives several external tools (FFmpeg, MKVToolNix, shaka-packager, and -others) to decrypt, repack, and mux media. Make sure that they are visible before -you download anything: +others) to decrypt, repack, and mux media, and it needs a config file. The +`unshackle setup` command prepares both. If you used the install script, setup +has already run. You can run it again at any time: + +```shell +unshackle setup +``` + +Setup downloads the missing required tools and, if you agree, groups of optional +tools. If no `unshackle.yaml` exists, it writes a minimal one with your downloads +folder. It also makes the data folders and lets you import `.wvd` and `.prd` +devices, one file or a whole folder at a time. [Installation](installation.md#what-unshackle-setup-does) tells you what +each step does and where the files go. + +Then make sure that unshackle finds the tools: ```shell unshackle env check ``` -This prints a dependency table. The **required** tools (FFmpeg, FFprobe, -MKVToolNix, mkvpropedit, and shaka-packager) must show a green check. Optional -tools such as `dovi_tool` (Dolby Vision) and CCExtractor (closed captions) only -matter for the features that use them. +The **required** tools (FFmpeg, FFprobe, MKVToolNix, mkvpropedit, and +shaka-packager) must show a green check. Optional tools such as `dovi_tool` +(Dolby Vision) and CCExtractor (closed captions) only matter for the features +that use them. -!!! tip - The summary line at the bottom reports `installed/total` and lists anything - required that is still missing, so you know exactly what to install next. +!!! note "Set up by hand" + To do these steps yourself, install the tools from the tables in + [Installation](installation.md#external-tools-on-your-path). Then run + `unshackle env info` to see where unshackle looks for `unshackle.yaml`, and + make the file in one of those locations. The + [configuration guide](configuration-file.md) gives the details. -## 2. Make a minimal config - -unshackle reads a single configuration file named `unshackle.yaml`. To see where it -looks for that file, and where it *would* accept one if you have not made it yet, -use this command: - -```shell -unshackle env info -``` - -If no config exists, this prints the candidate locations. In the order unshackle -examines them, they are: - -1. `unshackle.yaml` inside the unshackle package folder. -2. `unshackle.yaml` in that folder's parent. -3. `unshackle.yaml` in your OS user-config directory - (`~/.config/unshackle/` on Linux, `%LOCALAPPDATA%\unshackle\` on Windows, - `~/Library/Application Support/unshackle/` on macOS). - -The **first** file that exists wins. Make `unshackle.yaml` in one of those -locations. - -### The one config key you must set +## 2. Check the output template unshackle refuses to start a download unless it knows how to name the output file. That means `output_template` is the one setting a first run genuinely requires. -Everything else has a sensible default. A minimal, working config looks like this: +Everything else has a sensible default. The config that setup writes copies `tag` +and `output_template` from the example config. A minimal, working config looks like this: ```yaml title="unshackle.yaml" output_template: @@ -98,7 +94,9 @@ commands. ## 4. Add a CDM for DRM Most streaming services encrypt their titles. To fetch content keys, unshackle -needs a **CDM**, a Widevine device (`.wvd`) or a PlayReady device (`.prd`). Add a +needs a **CDM**, a Widevine device (`.wvd`) or a PlayReady device (`.prd`). If you +imported devices during `unshackle setup`, they are already in place. If setup also +wrote your config, it set `cdm.default` to the device that you chose. Otherwise, add a Widevine device you already have with: ```shell @@ -217,7 +215,8 @@ codec, bitrate, channel-layout, and track-type selection. ## 8. Where the output lands By default, unshackle writes finished files to the `downloads` directory. -`unshackle env info` shows its exact path. The built-in default is a `downloads` +`unshackle env info` shows its exact path. A config from `unshackle setup` uses the +folder that you gave at its prompt. Without that setting, the default is a `downloads` folder one level above the installed `unshackle` package. Override it for one command with `-o`: diff --git a/docs/guide/cli-reference.md b/docs/guide/cli-reference.md index b6e74086..44d7cb12 100644 --- a/docs/guide/cli-reference.md +++ b/docs/guide/cli-reference.md @@ -5,12 +5,16 @@ A complete reference for every `unshackle` command, subcommand, and flag. This p - **[`dl`](#dl)**: the download pipeline (the command you use most). - **[`search`](#search)**: find titles on a service. - **[`import`](#import)**: rebuild a download from an export file. +- **[`setup`](#setup)**: install the external tools and make a first config. - **[`cfg`](#cfg)**, **[`env`](#env)**: manage configuration and the environment. - **[`kv`](#kv)**: Key Vault operations. - **[`wvd`](#wvd)**, **[`prd`](#prd)**: Widevine / PlayReady device management. - **[`serve`](#serve)**: operate the local CDM and REST API server. - **[`util`](#util)**: helper media utilities. +!!! note "Running commands from a clone" + This page uses the bare `unshackle` command. From a git clone, the recommended install, run each command as `uv run unshackle ...`. + !!! tip "Getting help on any command" Every command accepts `-h`, `--help`, or `-?`. For example `unshackle dl --help` or `unshackle wvd new --help`. Each service defines its own arguments for `dl` and `search` (such as a title ID or a query), so examine `unshackle dl SERVICE --help` for those. @@ -268,6 +272,24 @@ The export file can be a `mediaexport` file from `dl --export`, an export from a --- +## `setup` + +Install the external tools and make a first config. + +``` +unshackle setup +``` + +The command takes no options and asks its questions at the prompt. It shows the missing [external tools](../getting-started/installation.md#external-tools-on-your-path) and downloads the ones you accept. It puts them in a `binaries` folder, in the package folder of a git clone or in your user data directory for a `uv` tool install. If no `unshackle.yaml` exists, it writes a first one: inside the clone, or to your user config directory for a `uv` tool install. It then makes the data folders and imports `.wvd` and `.prd` devices, one file or folder per answer, until you give an empty answer. It ends with a summary of the tools, the path of a config that it wrote, and a pointer to `env check`. You can run it again: it skips the tools it finds and never overwrites a config. [Installation](../getting-started/installation.md#what-unshackle-setup-does) gives the full steps and where each file goes. + +!!! example + ```shell + uv run unshackle setup # from a git clone + unshackle setup # from a uv tool install + ``` + +--- + ## `cfg` Read, set, delete, or show configuration values in `unshackle.yaml` without hand-editing YAML. @@ -312,7 +334,7 @@ unshackle env COMMAND [ARGS]... unshackle env check ``` -Prints a dependency table (Category / Tool / Status / Required / Purpose) that shows which external tools unshackle finds on your `PATH`. It also gives a summary of how many required tools are present. +Prints a dependency table (Category / Tool / Status / Required / Purpose) that shows which external tools unshackle finds on your `PATH`. It also gives a summary of how many required tools are present. If a required tool is missing, it tells you to run [`unshackle setup`](#setup). | Category | Tools | |---|---| diff --git a/docs/guide/downloading.md b/docs/guide/downloading.md index 6dd032d4..aca2c613 100644 --- a/docs/guide/downloading.md +++ b/docs/guide/downloading.md @@ -148,8 +148,8 @@ unshackle dl -q 2160 -r HDR10,DV EXAMPLE '...' !!! warning "HYBRID requires dovi_tool" `-r HYBRID` makes a single hybrid track by injecting the Dolby Vision RPU metadata onto an HDR10/HDR10+ base layer with **dovi_tool**. - It requires the `dovi_tool` binary, resolved from unshackle's `binaries/` folder or - your `PATH`. The normal case is a DV track plus an HDR10 or HDR10+ base. With HDR10+ + It requires the `dovi_tool` binary, resolved from unshackle's `binaries/` folder, + the `binaries` folder of your user data directory, or your `PATH`. The normal case is a DV track plus an HDR10 or HDR10+ base. With HDR10+ and no DV, unshackle converts the HDR10+ metadata to DV instead, which also needs `hdr10plus_tool`. With no DV and no HDR10+, the title fails. When HDR10+ is present, unshackle prefers it over HDR10 as the base layer. diff --git a/docs/guide/drm-and-cdm.md b/docs/guide/drm-and-cdm.md index a700e0dc..5c125e4d 100644 --- a/docs/guide/drm-and-cdm.md +++ b/docs/guide/drm-and-cdm.md @@ -407,8 +407,8 @@ encrypted file (for Widevine, PlayReady, and ClearKey-CENC): - **shaka-packager** is the default. - **mp4decrypt** is part of the Bento4 tools. -Both must be discoverable in unshackle's bundled `binaries/` directory or on your -`PATH`. If the selected tool's binary is missing at decrypt time, unshackle raises an +Both must be discoverable in unshackle's bundled `binaries/` directory, the +`binaries` folder of your user data directory, or on your `PATH`. If the selected tool's binary is missing at decrypt time, unshackle raises an "executable not found but is required" error. See [dependencies](../getting-started/installation.md) for installation. diff --git a/docs/guide/troubleshooting.md b/docs/guide/troubleshooting.md index 41a70c3a..e292ce45 100644 --- a/docs/guide/troubleshooting.md +++ b/docs/guide/troubleshooting.md @@ -241,7 +241,7 @@ Each command reports how many files it removed and how much space it freed, then !!! example "A required tool is missing" **Symptom:** decryption, muxing, or analysis fails early, sometimes with a "binary not found" style error. - **Inspect:** use `unshackle env check` and look for a red `✗` on a **required** row. Install the missing tool and re-run. + **Inspect:** use `unshackle env check` and look for a red `✗` on a **required** row. Run `unshackle setup` to install it, or install it by hand, and re-run. !!! example "Decryption produces garbage or fails" **Symptom:** the download completes but the file will not play, or the decrypt step errors. diff --git a/docs/index.md b/docs/index.md index 43633113..363f344b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -47,24 +47,45 @@ at a title and it will: ## Installation at a glance -=== "Install as a tool" +A git clone with the install script is the recommended way to install unshackle. - ```shell - uv tool install git+https://github.com/unshackle-dl/unshackle.git - unshackle --help - ``` - -=== "Run from a clone" +=== "Linux (recommended)" ```shell git clone https://github.com/unshackle-dl/unshackle.git cd unshackle - uv run unshackle --help + sh install.sh + uv run unshackle env check ``` +=== "Windows (recommended)" + + ```bat + git clone https://github.com/unshackle-dl/unshackle.git + cd unshackle + .\install.bat + uv run unshackle env check + ``` + +=== "Advanced: uv tool" + + ```shell + uv tool install git+https://github.com/unshackle-dl/unshackle.git + unshackle setup + ``` + + This installs unshackle into the tool folder of `uv`, not a folder that you + choose. `unshackle setup` then puts the tools, config, and data in your user + data and config directories. + +The install script installs `uv` and unshackle, then runs `unshackle setup`. In a +clone, run every command as `uv run unshackle ...`. The examples on this site use +the bare `unshackle` command. + !!! note "External tools" unshackle shells out to FFmpeg, MKVToolNix, shaka-packager, Bento4, and - `dovi_tool` for decryption, repacking, muxing, and Dolby Vision work. See + `dovi_tool` for decryption, repacking, muxing, and Dolby Vision work. + `unshackle setup` downloads them when a portable build exists for your system. See [Installation](getting-started/installation.md) for the full list and recommended versions. ## A first download diff --git a/install.bat b/install.bat index e470206e..13777cce 100644 --- a/install.bat +++ b/install.bat @@ -45,17 +45,48 @@ echo [OK] uv installed and reachable. :install_deps echo. -uv sync +findstr /b /c:"name = \"unshackle\"" "%~dp0pyproject.toml" >nul 2>&1 +if %errorlevel% neq 0 goto tool_install + +cd /d "%~dp0" +uv sync --compile-bytecode if %errorlevel% neq 0 ( echo [ERR] Dependency install failed. See errors above. pause exit /b 1 ) +set "SETUP=uv run unshackle setup" +set "RERUN=Run it again with: uv run unshackle setup" +set "NOTE=" +goto run_setup + +:tool_install +uv tool install --compile-bytecode git+https://github.com/unshackle-dl/unshackle.git +if %errorlevel% neq 0 ( + echo [ERR] unshackle install failed. See errors above. + pause + exit /b 1 +) +uv tool update-shell +set "TOOL_BIN=" +for /f "delims=" %%B in ('uv tool dir --bin') do set "TOOL_BIN=%%B" +set "SETUP="%TOOL_BIN%\unshackle.exe" setup" +set "RERUN=Run it again in a new terminal with: unshackle setup" +set "NOTE=Open a new terminal so that the unshackle command is on your PATH." + +:run_setup +echo. +%SETUP% +if %errorlevel% neq 0 ( + echo [ERR] unshackle is installed, but setup failed. See errors above. + echo %RERUN% + pause + exit /b 1 +) echo. -echo Installation completed successfully! -echo Try: -echo uv run unshackle --help +echo Installation completed successfully. +if defined NOTE echo %NOTE% echo. pause endlocal diff --git a/install.sh b/install.sh new file mode 100644 index 00000000..e77b4ce4 --- /dev/null +++ b/install.sh @@ -0,0 +1,47 @@ +#!/bin/sh +# Install uv and unshackle, then run `unshackle setup`. +# In a clone it runs `uv sync`. Alone (or piped from curl) it runs `uv tool install`. +set -eu + +REPO="git+https://github.com/unshackle-dl/unshackle.git" + +if ! command -v uv >/dev/null 2>&1; then + echo "[..] uv not found. Installing..." + if command -v curl >/dev/null 2>&1; then + curl -LsSf https://astral.sh/uv/install.sh | sh + else + wget -qO- https://astral.sh/uv/install.sh | sh + fi + for dir in "$HOME/.local/bin" "${XDG_DATA_HOME:+$XDG_DATA_HOME/../bin}" "${XDG_BIN_HOME:-}"; do + if [ -n "$dir" ]; then PATH="$dir:$PATH"; fi + done + export PATH +fi +echo "[OK] $(uv --version)" + +run_setup() { + if [ -f "$0" ]; then + "$@" setup + else + "$@" setup None: + monkeypatch.setattr(config, "update_checks", False) + + result = CliRunner().invoke(main, args) + + assert result.exit_code == 0, result.output + assert result.output.count(f"v {__version__}") == 1 + assert "Usage:" in result.output diff --git a/tests/test_setup.py b/tests/test_setup.py new file mode 100644 index 00000000..627f49a1 --- /dev/null +++ b/tests/test_setup.py @@ -0,0 +1,604 @@ +"""`unshackle setup`: asset choice, archive extraction, config writing and the user binaries folder.""" + +from __future__ import annotations + +import io +import os +import subprocess +import sys +import tarfile +import zipfile +from pathlib import Path +from types import SimpleNamespace +from typing import Callable, Iterable, Iterator + +import pytest +import yaml +from click.testing import CliRunner + +from unshackle.commands import setup as setup_mod +from unshackle.commands.env import get_dependencies +from unshackle.core import binaries +from unshackle.core.config import Config, config + +EXT = ".exe" if sys.platform == "win32" else "" + + +@pytest.mark.parametrize( + ("tool", "system", "machine", "suffix"), + [ + ("shaka-packager", "linux", "x86_64", "/v2.6.1/packager-linux-x64"), + ("shaka-packager", "linux", "aarch64", "/v2.6.1/packager-linux-arm64"), + ("shaka-packager", "win32", "AMD64", "/v2.6.1/packager-win-x64.exe"), + ("FFmpeg", "linux", "x86_64", "-linux64-gpl.tar.xz"), + ("FFmpeg", "win32", "AMD64", "-win64-gpl.zip"), + ("MKVToolNix", "win32", "AMD64", "mkvtoolnix-64-bit-102.0.zip"), + ("dovi_tool", "linux", "arm64", "-aarch64-unknown-linux-musl.tar.gz"), + ("CCExtractor", "linux", "x86_64", "/v0.96.6/ccextractor-minimal-x86_64.AppImage"), + ("CCExtractor", "win32", "AMD64", "/v0.96.6/CCExtractor.0.96.6_win_portable.zip"), + ], +) +def test_asset_url_matches_platform(tool: str, system: str, machine: str, suffix: str) -> None: + url = setup_mod.asset_url(tool, system, machine) + assert url is not None and url.startswith("https://") and url.endswith(suffix) + + +@pytest.mark.parametrize( + ("tool", "system", "machine"), + [ + ("MKVToolNix", "linux", "x86_64"), + ("CCExtractor", "linux", "aarch64"), + ("mp4decrypt", "linux", "aarch64"), + ("shaka-packager", "darwin", "arm64"), + ("FFmpeg", "win32", "ARM64"), + ], +) +def test_asset_url_is_none_without_a_portable_build(tool: str, system: str, machine: str) -> None: + assert setup_mod.asset_url(tool, system, machine) is None + + +def test_every_url_and_hint_is_filled_from_the_version() -> None: + for tool, spec in setup_mod.TOOLS.items(): + hint = setup_mod.versioned(tool, spec["hint"]) + assert hint and "{" not in hint + for key in spec["urls"]: + url = setup_mod.asset_url(tool, *key.split("-", 1)) + assert url is not None and url.startswith("https://") and "{" not in url + assert spec["version"].replace(".", "-") in url or spec["version"] in url + + +def test_label_shows_pinned_versions_only() -> None: + assert setup_mod.label("shaka-packager") == "shaka-packager 2.6.1" + assert setup_mod.label("FFmpeg") == "FFmpeg" + assert setup_mod.label("CCExtractor", "linux") == "CCExtractor 0.96.6" + assert setup_mod.label("CCExtractor", "win32") == "CCExtractor 0.96.6, 88 MB" + + +def make_zip(path: Path, members: dict[str, bytes]) -> Path: + with zipfile.ZipFile(path, "w") as zf: + for name, data in members.items(): + zf.writestr(name, data) + return path + + +def make_tar(path: Path, members: dict[str, bytes]) -> Path: + with tarfile.open(path, "w:gz") as tf: + for name, data in members.items(): + info = tarfile.TarInfo(name) + info.size = len(data) + tf.addfile(info, io.BytesIO(data)) + return path + + +@pytest.mark.parametrize(("archive_name", "make"), [("tool.zip", make_zip), ("tool.tar.gz", make_tar)]) +def test_extract_puts_named_members_flat( + tmp_path: Path, archive_name: str, make: Callable[[Path, dict[str, bytes]], Path] +) -> None: + archive = make( + tmp_path / archive_name, + {"pkg/bin/tool": b"tool", "pkg/bin/other": b"other", "pkg/README": b"readme"}, + ) + dest = tmp_path / "bin" + + placed = setup_mod.extract(archive, dest, ("tool",)) + + assert placed == [dest / "tool"] + assert sorted(p.name for p in dest.iterdir()) == ["tool"] + assert (dest / "tool").read_bytes() == b"tool" + if sys.platform != "win32": + assert os.access(dest / "tool", os.X_OK) + + +@pytest.mark.parametrize(("archive_name", "make"), [("evil.zip", make_zip), ("evil.tar.gz", make_tar)]) +def test_extract_never_writes_outside_dest( + tmp_path: Path, archive_name: str, make: Callable[[Path, dict[str, bytes]], Path] +) -> None: + archive = make(tmp_path / archive_name, {"../../escaped": b"x", "/abs/escaped": b"y"}) + dest = tmp_path / "a" / "b" / "bin" + + setup_mod.extract(archive, dest, ("escaped",)) + + assert not (tmp_path / "escaped").exists() + assert not (tmp_path / "a" / "escaped").exists() + assert [p.name for p in dest.iterdir()] == ["escaped"] + + +def test_extract_moves_a_lone_binary(tmp_path: Path) -> None: + download = tmp_path / "packager-linux-x64" + download.write_bytes(b"elf") + dest = tmp_path / "bin" + + assert setup_mod.extract(download, dest, ("packager",)) == [dest / "packager"] + assert not download.exists() + + +def test_is_clone_follows_pyproject(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + package = tmp_path / "unshackle" + package.mkdir() + monkeypatch.setattr(config.directories, "namespace_dir", package) + assert setup_mod.is_clone() is False + (tmp_path / "pyproject.toml").write_text("") + assert setup_mod.is_clone() is True + + +def test_write_config_tool_mode_moves_data_dirs(tmp_path: Path) -> None: + path = tmp_path / "config" / "unshackle.yaml" + data_root = tmp_path / "data" + + assert setup_mod.write_config(path, tmp_path / "downloads", data_root) is not None + + loaded = Config.from_yaml(path) + assert loaded.output_template + assert loaded.directories.downloads == tmp_path / "downloads" + assert loaded.directories.wvds == data_root / "WVDs" + assert loaded.directories.temp == data_root / "temp" + assert loaded.directories.services == [data_root / "services"] + + +def test_write_config_clone_mode_keeps_package_data_dirs(tmp_path: Path) -> None: + path = tmp_path / "unshackle.yaml" + + setup_mod.write_config(path, tmp_path / "downloads", None) + + data = yaml.safe_load(path.read_text()) + assert data["directories"] == {"downloads": str(tmp_path / "downloads")} + assert Config.from_yaml(path).directories.wvds == Config._Directories.wvds + + +def test_write_config_never_overwrites(tmp_path: Path) -> None: + path = tmp_path / "unshackle.yaml" + path.write_text("tag: MINE\n") + + assert setup_mod.write_config(path, tmp_path / "downloads", tmp_path / "data") is None + assert path.read_text() == "tag: MINE\n" + + +def make_exe(folder: Path, name: str) -> Path: + folder.mkdir(parents=True, exist_ok=True) + path = folder / f"{name}{EXT}" + path.write_bytes(b"") + path.chmod(0o755) + return path + + +def test_find_checks_the_user_binaries_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(binaries, "user_binaries_dir", tmp_path / "user") + tool = make_exe(tmp_path / "user", "setup-test-tool") + + assert binaries.find("setup-test-tool") == tool + + +def test_find_prefers_the_package_binaries_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(binaries, "package_binaries_dir", tmp_path / "unshackle" / "binaries") + monkeypatch.setattr(binaries, "user_binaries_dir", tmp_path / "user") + make_exe(tmp_path / "user", "setup-test-tool") + packaged = make_exe(tmp_path / "unshackle" / "binaries", "setup-test-tool") + + assert binaries.find("setup-test-tool") == packaged + + +@pytest.fixture +def asks(monkeypatch: pytest.MonkeyPatch) -> list[str]: + """Record every Confirm prompt and answer it with no.""" + prompts: list[str] = [] + + def fake_ask(prompt: str, **kwargs: object) -> bool: + prompts.append(prompt) + return False + + monkeypatch.setattr(setup_mod.Confirm, "ask", fake_ask) + return prompts + + +@pytest.mark.parametrize( + ("clone", "has_config", "asked", "keep"), + [(False, False, True, False), (True, False, False, True), (False, True, False, True)], +) +def test_only_a_first_tool_mode_run_recommends_a_clone( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + asks: list[str], + clone: bool, + has_config: bool, + asked: bool, + keep: bool, +) -> None: + monkeypatch.setattr(setup_mod, "get_config_path", lambda: tmp_path / "unshackle.yaml" if has_config else None) + + assert setup_mod.keep_tool_install(clone=clone) is keep + assert asks == (["Continue with the uv tool install?"] if asked else []) + + +def test_declining_the_tool_install_stops_setup(monkeypatch: pytest.MonkeyPatch, asks: list[str]) -> None: + def fail() -> None: + raise AssertionError("setup went past the clone prompt") + + monkeypatch.setattr(setup_mod, "is_clone", lambda: False) + monkeypatch.setattr(setup_mod, "get_config_path", lambda: None) + monkeypatch.setattr(setup_mod, "get_dependencies", fail) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + assert asks == ["Continue with the uv tool install?"] + + +REQUIRED_DEPS = {"FFmpeg", "FFprobe", "MKVToolNix", "mkvpropedit", "shaka-packager"} + + +def fake_deps(missing: Iterable[str]) -> list[dict[str, object]]: + """Return a `get_dependencies()` result for every TOOLS dep, with the deps of `missing` tools not found.""" + return [ + {"name": dep, "binary": None if tool in missing else Path(dep), "required": dep in REQUIRED_DEPS} + for tool, spec in setup_mod.TOOLS.items() + for dep in spec["deps"] + ] + + +def test_tools_deps_exist_in_env_check(monkeypatch: pytest.MonkeyPatch) -> None: + # Set in the module dict: a setattr reads the old value first, and that read runs the PATH search. + for attr in set(binaries.__all__) - {"find", "register", "get_registered_dependencies"}: + monkeypatch.setitem(vars(binaries), attr, None) + + deps = {dep["name"]: dep["required"] for dep in get_dependencies()} + + names = {dep for spec in setup_mod.TOOLS.values() for dep in spec["deps"]} + assert names <= deps.keys() + assert {name for name in names if deps[name]} == REQUIRED_DEPS + + +@pytest.fixture +def sandbox(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> SimpleNamespace: + """Point every path setup touches into tmp_path and record its prompts and installs. + + Every tool is missing and downloadable, and each prompt takes its default answer. + """ + box = SimpleNamespace( + package=tmp_path / "clone" / "unshackle", + user_config=tmp_path / "user_config" / "unshackle.yaml", + user_data=tmp_path / "user_data", + prompts=[], + installs=[], + ) + box.package.mkdir(parents=True) + (box.package / "unshackle-example.yaml").write_bytes( + (config.directories.namespace_dir / "unshackle-example.yaml").read_bytes() + ) + for key, name in setup_mod.DATA_DIRS.items(): + monkeypatch.setattr(config.directories, key, box.package / name) + monkeypatch.setattr(config.directories, "services", [box.package / "services"]) + monkeypatch.setattr(config.directories, "namespace_dir", box.package) + monkeypatch.setattr(setup_mod, "PACKAGE_CONFIG_PATH", box.package / "unshackle.yaml") + monkeypatch.setattr(setup_mod, "USER_CONFIG_PATH", box.user_config) + monkeypatch.setattr(binaries, "package_binaries_dir", box.package / "binaries") + monkeypatch.setattr(binaries, "user_binaries_dir", box.user_data / "binaries") + monkeypatch.setattr(setup_mod, "is_clone", lambda: True) + monkeypatch.setattr(setup_mod, "get_config_path", lambda: None) + monkeypatch.setattr(setup_mod, "get_dependencies", lambda: fake_deps(setup_mod.TOOLS)) + + def confirm(prompt: str, default: bool = False, **kwargs: object) -> bool: + box.prompts.append((prompt, default)) + return default + + monkeypatch.setattr(setup_mod.Confirm, "ask", confirm) + monkeypatch.setattr(setup_mod.Prompt, "ask", lambda prompt, default="", **kwargs: default) + monkeypatch.setattr(setup_mod, "install", lambda tool, url, dest: box.installs.append((tool, dest)) or True) + monkeypatch.setattr(setup_mod, "asset_url", lambda tool: f"https://example.invalid/{tool}") + return box + + +@pytest.mark.parametrize("clone", [True, False]) +def test_setup_writes_into_the_clone_or_the_user_dirs( + sandbox: SimpleNamespace, monkeypatch: pytest.MonkeyPatch, clone: bool +) -> None: + monkeypatch.setattr(setup_mod, "is_clone", lambda: clone) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + if clone: + assert {dest for _, dest in sandbox.installs} == {sandbox.package / "binaries"} + assert (sandbox.package / "unshackle.yaml").is_file() + assert not sandbox.user_config.exists() + assert not sandbox.user_data.exists() + else: + assert {dest for _, dest in sandbox.installs} == {sandbox.user_data / "binaries"} + assert sandbox.user_config.is_file() + assert not (sandbox.package / "unshackle.yaml").exists() + assert (sandbox.user_data / "WVDs").is_dir() + assert [tool for tool, _ in sandbox.installs] == ["FFmpeg", "MKVToolNix", "shaka-packager", "mp4decrypt"] + config_path = sandbox.package / "unshackle.yaml" if clone else sandbox.user_config + assert f"A minimal unshackle.yaml was written to {config_path}." in result.output + assert f"To expand it, see {setup_mod.DOCS_URL}" in result.output + run = "uv run unshackle" if clone else "unshackle" + assert f"Next: run `{run} env check` to confirm the tools." in result.output + + +@pytest.mark.parametrize( + ("missing", "downloadable", "prompt", "default", "installed"), + [ + ( + {"shaka-packager", "mp4decrypt"}, + {"shaka-packager", "mp4decrypt"}, + "Install the DRM tools (shaka-packager 2.6.1, mp4decrypt 1.6.0.641) (required)?", + True, + ["shaka-packager", "mp4decrypt"], + ), + ({"mp4decrypt"}, {"mp4decrypt"}, "Install the DRM tools (mp4decrypt 1.6.0.641)?", False, []), + ( + {"shaka-packager"}, + {"shaka-packager"}, + "Install the DRM tools (shaka-packager 2.6.1) (required)?", + True, + ["shaka-packager"], + ), + ( + {"shaka-packager", "mp4decrypt"}, + {"shaka-packager"}, + "Install the DRM tools (shaka-packager 2.6.1) (required)?", + True, + ["shaka-packager"], + ), + ], +) +def test_drm_tools_share_one_prompt( + sandbox: SimpleNamespace, + monkeypatch: pytest.MonkeyPatch, + missing: set[str], + downloadable: set[str], + prompt: str, + default: bool, + installed: list[str], +) -> None: + monkeypatch.setattr(setup_mod, "get_dependencies", lambda: fake_deps(missing)) + monkeypatch.setattr( + setup_mod, "asset_url", lambda tool: f"https://x.invalid/{tool}" if tool in downloadable else None + ) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + assert sandbox.prompts == [(prompt, default)] + assert [tool for tool, _ in sandbox.installs] == installed + assert ("mp4decrypt: no download" in result.output) == ("mp4decrypt" in missing - downloadable) + + +@pytest.fixture +def fake_devices(monkeypatch: pytest.MonkeyPatch) -> None: + """Accept any device file except one that holds b"bad".""" + + def load(path: Path) -> None: + if Path(path).read_bytes() == b"bad": + raise ValueError("bad magic") + + monkeypatch.setattr("pywidevine.device.Device.load", load) + monkeypatch.setattr("pyplayready.device.Device.load", load) + + +def script_cdm_prompts(monkeypatch: pytest.MonkeyPatch, answers: list[str]) -> list[tuple[str, object]]: + """Answer the CDM prompts in order, then finish; every other prompt takes its default.""" + asked: list[tuple[str, object]] = [] + + def ask(prompt: str, default: str = "", choices: object = None, **kwargs: object) -> str: + asked.append((prompt, choices)) + if prompt.startswith("Downloads"): + return default + return answers.pop(0) if answers else default + + monkeypatch.setattr(setup_mod.Prompt, "ask", ask) + return asked + + +def make_devices(folder: Path, files: dict[str, bytes]) -> Path: + folder.mkdir(parents=True, exist_ok=True) + for name, data in files.items(): + (folder / name).write_bytes(data) + return folder + + +def test_device_files_expands_a_folder_without_recursion(tmp_path: Path) -> None: + folder = make_devices(tmp_path / "in", {"a.wvd": b"", "b.wvd": b"", "c.prd": b"", "notes.txt": b""}) + make_devices(folder / "sub", {"deep.wvd": b""}) + + assert setup_mod.device_files(str(folder)) == [folder / "a.wvd", folder / "b.wvd", folder / "c.prd"] + assert setup_mod.device_files(str(make_devices(tmp_path / "empty", {}))) == [] + + +def test_device_files_keeps_quotes_and_home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("HOME", str(tmp_path)) + monkeypatch.setenv("USERPROFILE", str(tmp_path)) + + assert setup_mod.device_files('"~/x.wvd"') == [tmp_path / "x.wvd"] + + +@pytest.mark.usefixtures("fake_devices") +def test_cdm_prompt_repeats_and_asks_for_the_default( + sandbox: SimpleNamespace, monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + folder = make_devices(tmp_path / "in", {"a.wvd": b"", "b.wvd": b"", "c.prd": b"", "notes.txt": b""}) + single = make_devices(tmp_path / "other", {"d.wvd": b""}) / "d.wvd" + asked = script_cdm_prompts(monkeypatch, [str(folder), f'"{single}"', "", "c"]) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + for name in ("a.wvd", "b.wvd", "c.prd", "d.wvd"): + assert f"Imported: {name}" in result.output + assert [p for p, _ in asked].count(setup_mod.CDM_PROMPT) == 3 + assert ("Default CDM", ["a", "b", "c", "d"]) in asked + assert sorted(p.name for p in (sandbox.package / "WVDs").iterdir()) == ["a.wvd", "b.wvd", "d.wvd"] + assert [p.name for p in (sandbox.package / "PRDs").iterdir()] == ["c.prd"] + assert yaml.safe_load((sandbox.package / "unshackle.yaml").read_text())["cdm"] == {"default": "c"} + + +@pytest.mark.usefixtures("fake_devices") +def test_one_bad_device_does_not_stop_the_rest( + sandbox: SimpleNamespace, monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + folder = make_devices(tmp_path / "in", {"bad.wvd": b"bad", "good.wvd": b"", "good2.prd": b""}) + asked = script_cdm_prompts(monkeypatch, [str(folder), "", "good"]) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + assert "Skipped bad.wvd: not a valid Widevine device (bad magic)" in result.output + assert "Imported: good.wvd" in result.output + assert "Imported: good2.prd" in result.output + assert ("Default CDM", ["good", "good2"]) in asked + assert not (sandbox.package / "WVDs" / "bad.wvd").exists() + + +@pytest.mark.usefixtures("fake_devices") +def test_folder_without_devices_says_so( + sandbox: SimpleNamespace, monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + # Longer than the 80-column console, so a wrapped line would split the path. + folder = make_devices(tmp_path / ("long" * 20) / "in", {"notes.txt": b""}) + script_cdm_prompts(monkeypatch, [str(folder)]) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + assert f"No .wvd or .prd files in {folder}" in result.output + assert "cdm" not in yaml.safe_load((sandbox.package / "unshackle.yaml").read_text()) + + +@pytest.mark.usefixtures("fake_devices") +def test_destination_folder_is_already_in_place(sandbox: SimpleNamespace, monkeypatch: pytest.MonkeyPatch) -> None: + wvds = make_devices(sandbox.package / "WVDs", {"x.wvd": b"device"}) + asked = script_cdm_prompts(monkeypatch, [str(wvds)]) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + assert "Already in place: x.wvd" in result.output + assert (wvds / "x.wvd").read_bytes() == b"device" + assert all(prompt != "Default CDM" for prompt, _ in asked) + assert yaml.safe_load((sandbox.package / "unshackle.yaml").read_text())["cdm"] == {"default": "x"} + + +@pytest.mark.usefixtures("fake_devices") +def test_existing_config_is_never_edited( + sandbox: SimpleNamespace, monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + existing = sandbox.package / "unshackle.yaml" + existing.write_text("tag: MINE\n") + monkeypatch.setattr(setup_mod, "get_config_path", lambda: existing) + folder = make_devices(tmp_path / "in", {"a.wvd": b"", "b.wvd": b""}) + asked = script_cdm_prompts(monkeypatch, [str(folder)]) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + assert "Imported: b.wvd" in result.output + assert existing.read_text() == "tag: MINE\n" + assert "A minimal unshackle.yaml" not in result.output + assert all(prompt != "Default CDM" for prompt, _ in asked) + + +def test_extract_puts_a_tool_and_its_dlls_in_a_subfolder(tmp_path: Path) -> None: + spec = setup_mod.platform_spec("CCExtractor", "win32") + archive = make_zip( + tmp_path / "ccx.zip", + { + name: b"x" + for name in ( + "ccextractorwinfull.exe", + "avcodec-60.dll", + "vcruntime140.dll", + "flutter_windows.dll", + "url_launcher_windows_plugin.dll", + "ccxgui.exe", + "data/app.so", + "tessdata/eng.traineddata", + ) + }, + ) + dest = tmp_path / "bin" / spec["folder"] + + placed = setup_mod.extract(archive, dest, ("ccextractorwinfull.exe",), spec["extra"], spec["exclude"]) + + assert placed == [dest / "ccextractorwinfull.exe"] + assert sorted(p.name for p in dest.iterdir()) == ["avcodec-60.dll", "ccextractorwinfull.exe", "vcruntime140.dll"] + assert [p.name for p in (tmp_path / "bin").iterdir()] == ["ccextractor"] + + +def test_find_resolves_a_tool_in_its_subfolder(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(binaries, "user_binaries_dir", tmp_path / "user") + tool = make_exe(tmp_path / "user" / "ccextractor", "setup-test-ccx") + + assert binaries.find("setup-test-ccx") == tool + + +@pytest.fixture +def fake_appimage(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Add a tool that downloads as an AppImage, and serve its download from tmp_path.""" + monkeypatch.setitem( + setup_mod.TOOLS, + "FakeImage", + {"version": "1.0", "bins": ("fakeimage",), "urls": {}, "hint": "see the fake docs"}, + ) + + def fake_requests(url: str, output_dir: Path, filename: str, **kwargs: object) -> Iterator[dict[str, Path]]: + output_dir.mkdir(parents=True, exist_ok=True) + (output_dir / filename).write_bytes(b"appimage") + yield {"file_downloaded": output_dir / filename} + + monkeypatch.setattr(setup_mod, "requests", fake_requests) + monkeypatch.setattr(setup_mod, "session", lambda: None) + return tmp_path / "bin" + + +@pytest.mark.parametrize("returncode", [0, 1]) +def test_appimage_is_kept_only_when_it_runs( + fake_appimage: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], returncode: int +) -> None: + calls: list[list[str]] = [] + + def fake_run(args: list[str], **kwargs: object) -> subprocess.CompletedProcess[str]: + calls.append(args) + assert kwargs["text"] and kwargs["encoding"] == "utf-8" and kwargs["errors"] == "replace" + return subprocess.CompletedProcess(args, returncode, "", "No suitable fusermount binary found") + + monkeypatch.setattr(setup_mod.subprocess, "run", fake_run) + + ok = setup_mod.install("FakeImage", "https://x.invalid/FakeImage.AppImage", fake_appimage) + + binary = fake_appimage / f"fakeimage{EXT}" + assert calls == [[str(binary), "--version"]] + assert ok is (returncode == 0) + assert binary.exists() is (returncode == 0) + if returncode: + output = capsys.readouterr().out + assert "the AppImage does not run, it needs FUSE" in output + assert "Install it manually: see the fake docs" in output + + +def test_group_prompt_shows_the_download_size(sandbox: SimpleNamespace, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setitem(setup_mod.TOOLS["CCExtractor"], "size", "88 MB") + monkeypatch.setattr(setup_mod, "get_dependencies", lambda: fake_deps({"SubtitleEdit", "CCExtractor"})) + + result = CliRunner().invoke(setup_mod.setup, []) + + assert result.exit_code == 0, result.output + assert sandbox.prompts == [("Install the Subtitle tools (SubtitleEdit 5.2.0, CCExtractor 0.96.6, 88 MB)?", False)] diff --git a/unshackle/commands/env.py b/unshackle/commands/env.py index 152272c3..9ed65126 100644 --- a/unshackle/commands/env.py +++ b/unshackle/commands/env.py @@ -238,6 +238,8 @@ def check() -> None: summary_parts.append(f"[red]Missing required: {', '.join(missing_required)}[/red]") console.print(Padding(" ".join(summary_parts), (0, 3, 1, 3))) + if not all_required_installed: + console.print(Padding("[text]Run [cyan]unshackle setup[/cyan] to install them.[/text]", (0, 3, 1, 3))) @env.command() @@ -370,10 +372,14 @@ def info() -> None: if sys.platform == "win32" and os.getenv(x) } - for name in sorted(dir(config.directories)): - if name.startswith("__") or name == "app_dirs": - continue - attr_value = getattr(config.directories, name) + rows = { + name: getattr(config.directories, name) + for name in dir(config.directories) + if not name.startswith("__") and name != "app_dirs" + } + rows["user_binaries"] = binaries.user_binaries_dir + for name in sorted(rows): + attr_value = rows[name] if isinstance(attr_value, list): paths_str = "\n".join(str(p.resolve()) if isinstance(p, Path) else str(p) for p in attr_value) diff --git a/unshackle/commands/setup.py b/unshackle/commands/setup.py new file mode 100644 index 00000000..c88cefce --- /dev/null +++ b/unshackle/commands/setup.py @@ -0,0 +1,457 @@ +import platform +import shutil +import subprocess +import sys +import tarfile +import tempfile +import zipfile +from fnmatch import fnmatch +from functools import partial +from pathlib import Path, PurePosixPath +from typing import Any, Optional + +import click +import yaml +from rich.prompt import Confirm, Prompt + +from unshackle.commands.env import get_dependencies +from unshackle.core import binaries +from unshackle.core.config import PACKAGE_CONFIG_PATH, USER_CONFIG_PATH, Config, config, get_config_path +from unshackle.core.console import console +from unshackle.core.constants import context_settings +from unshackle.core.downloaders import requests +from unshackle.core.session import session + +GITHUB = "https://github.com" +BTBN = GITHUB + "/BtbN/FFmpeg-Builds/releases/download/{v}/ffmpeg-master-latest" +SHAKA = GITHUB + "/shaka-project/shaka-packager/releases/download/v{v}/packager" +DOVI = GITHUB + "/quietvoid/dovi_tool/releases/download/{v}/dovi_tool-{v}" +HDR10PLUS = GITHUB + "/quietvoid/hdr10plus_tool/releases/download/{v}/hdr10plus_tool-{v}" +BENTO4 = "https://www.bok.net/Bento4/binaries/Bento4-SDK-{dashed}" +SECONV = GITHUB + "/SubtitleEdit/subtitleedit/releases/download/v{v}/SeConv" +CCX = GITHUB + "/CCExtractor/ccextractor/releases/download/v{v}" + +TOOLS: dict[str, dict[str, Any]] = { + "FFmpeg": { + "version": "latest", + "deps": ("FFmpeg", "FFprobe"), + "bins": ("ffmpeg", "ffprobe"), + "urls": { + "linux-x86_64": BTBN + "-linux64-gpl.tar.xz", + "linux-aarch64": BTBN + "-linuxarm64-gpl.tar.xz", + "win32-x86_64": BTBN + "-win64-gpl.zip", + }, + "hint": "https://ffmpeg.org/download.html", + }, + "MKVToolNix": { + "version": "102.0", + "deps": ("MKVToolNix", "mkvpropedit"), + "bins": ("mkvmerge", "mkvpropedit"), + "urls": { + "win32-x86_64": "https://mkvtoolnix.download/windows/releases/{v}/mkvtoolnix-64-bit-{v}.zip", + }, + "hint": "install the mkvtoolnix package (apt, dnf or pacman), or see https://mkvtoolnix.download/downloads.html", + }, + "shaka-packager": { + "group": "DRM", + "version": "2.6.1", + "deps": ("shaka-packager",), + "bins": ("packager",), + "urls": { + "linux-x86_64": SHAKA + "-linux-x64", + "linux-aarch64": SHAKA + "-linux-arm64", + "win32-x86_64": SHAKA + "-win-x64.exe", + }, + "hint": GITHUB + "/shaka-project/shaka-packager/releases/tag/v{v}", + }, + "dovi_tool": { + "group": "HDR", + "version": "2.3.4", + "deps": ("dovi_tool",), + "bins": ("dovi_tool",), + "urls": { + "linux-x86_64": DOVI + "-x86_64-unknown-linux-musl.tar.gz", + "linux-aarch64": DOVI + "-aarch64-unknown-linux-musl.tar.gz", + "win32-x86_64": DOVI + "-x86_64-pc-windows-msvc.zip", + }, + "hint": GITHUB + "/quietvoid/dovi_tool/releases", + }, + "HDR10Plus_tool": { + "group": "HDR", + "version": "1.7.2", + "deps": ("HDR10Plus_tool",), + "bins": ("hdr10plus_tool",), + "urls": { + "linux-x86_64": HDR10PLUS + "-x86_64-unknown-linux-musl.tar.gz", + "linux-aarch64": HDR10PLUS + "-aarch64-unknown-linux-musl.tar.gz", + "win32-x86_64": HDR10PLUS + "-x86_64-pc-windows-msvc.zip", + }, + "hint": GITHUB + "/quietvoid/hdr10plus_tool/releases", + }, + "mp4decrypt": { + "group": "DRM", + "version": "1.6.0.641", + "deps": ("mp4decrypt",), + "bins": ("mp4decrypt",), + "urls": { + "linux-x86_64": BENTO4 + ".x86_64-unknown-linux.zip", + "win32-x86_64": BENTO4 + ".x86_64-microsoft-win32.zip", + }, + "hint": "https://www.bento4.com/downloads/", + }, + "SubtitleEdit": { + "group": "Subtitle", + "version": "5.2.0", + "deps": ("SubtitleEdit",), + "bins": ("seconv",), + "urls": { + "linux-x86_64": SECONV + "-Linux-x64.tar.gz", + "linux-aarch64": SECONV + "-Linux-ARM64.tar.gz", + "win32-x86_64": SECONV + "-Windows-x64.zip", + }, + "hint": GITHUB + "/SubtitleEdit/subtitleedit/releases", + }, + "CCExtractor": { + "group": "Subtitle", + "version": "0.96.6", + "deps": ("CCExtractor",), + "bins": ("ccextractor",), + "urls": { + "linux-x86_64": CCX + "/ccextractor-minimal-x86_64.AppImage", + "win32-x86_64": CCX + "/CCExtractor.{v}_win_portable.zip", + }, + "win32": { + "bins": ("ccextractorwinfull",), + "folder": "ccextractor", + "extra": ("*.dll",), + "exclude": ("flutter_windows.dll", "*_plugin.dll"), + "size": "88 MB", + }, + "hint": "install the ccextractor package, or see https://ccextractor.org/public/general/downloads/", + }, +} + +DATA_DIRS = { + key: getattr(Config._Directories, key).name + for key in ("cookies", "wvds", "prds", "cache", "logs", "exports", "dcsl", "temp") +} +DATA_DIRS["services"] = Config._Directories.services[0].name + +say = partial(console.print, soft_wrap=True) + + +def versioned(tool: str, text: str) -> str: + """Fill the `{v}` and `{dashed}` fields of a tool's URL or hint with its version.""" + version = TOOLS[tool]["version"] + return text.format(v=version, dashed=version.replace(".", "-")) + + +def platform_spec(tool: str, system: Optional[str] = None) -> dict[str, Any]: + """Return a tool's entry with the overrides for this platform applied.""" + entry = TOOLS[tool] + return {**entry, **entry.get(system or sys.platform, {})} + + +def label(tool: str, system: Optional[str] = None) -> str: + """Return the tool name with its pinned version and download size, for prompts.""" + spec = platform_spec(tool, system) + text = tool if spec["version"] == "latest" else f"{tool} {spec['version']}" + return f"{text}, {spec['size']}" if spec.get("size") else text + + +def asset_url(tool: str, system: Optional[str] = None, machine: Optional[str] = None) -> Optional[str]: + """Return the download URL of a tool for a platform, or None when it has no portable build.""" + machine = (machine or platform.machine()).lower() + machine = {"amd64": "x86_64", "arm64": "aarch64"}.get(machine, machine) + url = TOOLS[tool]["urls"].get(f"{system or sys.platform}-{machine}") + return versioned(tool, url) if url else None + + +def extract( + download: Path, dest: Path, files: tuple[str, ...], extra: tuple[str, ...] = (), exclude: tuple[str, ...] = () +) -> list[Path]: + """Put the named files of an archive flat into dest, or move a lone binary there as files[0]. + + It keeps only the base name of each member, so a member path cannot write outside dest. + """ + + def wanted(name: str) -> bool: + return name in files or (any(fnmatch(name, p) for p in extra) and not any(fnmatch(name, p) for p in exclude)) + + dest.mkdir(parents=True, exist_ok=True) + if download.name.endswith(".zip"): + with zipfile.ZipFile(download) as zf: + for info in zf.infolist(): + name = PurePosixPath(info.filename).name + if wanted(name) and not info.is_dir(): + with zf.open(info) as src, (dest / name).open("wb") as out: + shutil.copyfileobj(src, out) + elif download.name.endswith((".tar.gz", ".tar.xz")): + with tarfile.open(download, "r|*") as tf: + for member in tf: + name = PurePosixPath(member.name).name + data = tf.extractfile(member) if member.isfile() and wanted(name) else None + if data: + with data, (dest / name).open("wb") as out: + shutil.copyfileobj(data, out) + else: + shutil.move(download, dest / files[0]) + + placed = [dest / name for name in files if (dest / name).is_file()] + if sys.platform != "win32": + for path in placed: + path.chmod(path.stat().st_mode | 0o755) + return placed + + +def runs(binary: Path) -> bool: + """Return True when `binary --version` exits 0.""" + try: + return ( + subprocess.run( + [str(binary), "--version"], + capture_output=True, + text=True, + encoding="utf-8", + errors="replace", + timeout=30, + ).returncode + == 0 + ) + except (OSError, subprocess.SubprocessError): + return False + + +def install(tool: str, url: str, dest: Path) -> bool: + """Download one tool into dest. Print one line, and the manual source when it fails.""" + spec = platform_spec(tool) + ext = ".exe" if sys.platform == "win32" else "" + files = tuple(f"{name}{ext}" for name in spec["bins"]) + dest = dest / spec["folder"] if spec.get("folder") else dest + say(f"Downloading {tool} ({spec['version']}) ...", end=" ") + try: + with tempfile.TemporaryDirectory() as tmp: + downloaded: Optional[Path] = None + for event in requests(url, Path(tmp), url.rsplit("/", 1)[-1], session=session()): + downloaded = event.get("file_downloaded") or downloaded + if downloaded is None: + raise RuntimeError("the download did not finish") + placed = extract(downloaded, dest, files, spec.get("extra", ()), spec.get("exclude", ())) + if len(placed) != len(files): + raise RuntimeError(f"the archive does not contain {', '.join(files)}") + if url.endswith(".AppImage") and not runs(placed[0]): + placed[0].unlink() + raise RuntimeError("the AppImage does not run, it needs FUSE (fusermount)") + except Exception as e: + say(f"[red]failed[/] ({e})") + say(f" Install it manually: {versioned(tool, TOOLS[tool]['hint'])}") + return False + say("[green]done[/]") + return True + + +def is_clone() -> bool: + """Return True when unshackle runs from a git clone, False for a `uv tool` install.""" + return (config.directories.namespace_dir.parent / "pyproject.toml").exists() + + +def destinations(clone: bool) -> tuple[Path, Path]: + """Return the tools folder and the path for a new config: in the clone, or in the user dirs.""" + if clone: + return binaries.package_binaries_dir, PACKAGE_CONFIG_PATH + return binaries.user_binaries_dir, USER_CONFIG_PATH + + +CLONE_STEPS = ( + " git clone https://github.com/unshackle-dl/unshackle.git\n" + " then run install.sh (Linux) or install.bat (Windows) in the new unshackle folder" +) + + +def keep_tool_install(clone: bool) -> bool: + """On a first run from a uv tool install, recommend a git clone and ask whether to continue.""" + if clone or get_config_path(): + return True + say("A git clone is the easiest setup: its tools, config, data and services stay in one folder you can see.") + say(CLONE_STEPS) + if Confirm.ask("Continue with the uv tool install?", default=True, console=console): + return True + say(f"To set up from a git clone:\n{CLONE_STEPS}") + return False + + +def data_dirs(data_root: Path) -> dict[str, Any]: + """Return the `directories` entries that put the data folders under data_root.""" + dirs: dict[str, Any] = {key: str(data_root / name) for key, name in DATA_DIRS.items()} + dirs["services"] = [dirs["services"]] + return dirs + + +def write_config(path: Path, downloads: Path, data_root: Optional[Path]) -> Optional[dict[str, Any]]: + """Write a minimal config to path and return its data, or None when path exists. + + With a data_root, the data folders move there, out of the package folder. + """ + if path.exists(): + return None + example = yaml.safe_load((config.directories.namespace_dir / "unshackle-example.yaml").read_text("utf-8")) + directories = {"downloads": str(downloads), **(data_dirs(data_root) if data_root else {})} + data = {"tag": example["tag"], "output_template": example["output_template"], "directories": directories} + dump_config(path, data) + return data + + +def dump_config(path: Path, data: dict[str, Any]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(yaml.safe_dump(data, sort_keys=False, allow_unicode=True), encoding="utf-8") + + +DOCS_URL = "https://docs.unshackle.dev/reference/configuration/" +CDM_PROMPT = "Path to a .wvd/.prd file or a folder of them (empty to finish)" + + +def device_files(answer: str) -> list[Path]: + """Expand one answer to the CDM prompt: a folder gives the .wvd and .prd files directly in it.""" + path = Path(answer.strip().strip('"')).expanduser() + if path.is_dir(): + return sorted(file for file in path.iterdir() if file.is_file() and file.suffix.lower() in (".wvd", ".prd")) + return [path] + + +def import_cdm(device: Path, wvds: Path, prds: Path) -> Path: + """Validate a .wvd or .prd file, copy it into its folder and return the copy's path. + + A file that is already its own destination stays where it is. + """ + if device.suffix.lower() == ".wvd": + from pywidevine.device import Device as WidevineDevice + + load, kind, folder = WidevineDevice.load, "Widevine", wvds + elif device.suffix.lower() == ".prd": + from pyplayready.device import Device as PlayReadyDevice + + load, kind, folder = PlayReadyDevice.load, "PlayReady", prds + else: + raise ValueError("not a .wvd or .prd file") + if not device.is_file(): + raise FileNotFoundError("no such file") + try: + load(device) + except Exception as e: + raise ValueError(f"not a valid {kind} device ({str(e).splitlines()[0]})") from e + target = folder / device.name + if target.exists() and target.resolve() == device.resolve(): + return target + if target.exists(): + raise FileExistsError(f"{target} already exists") + folder.mkdir(parents=True, exist_ok=True) + shutil.copy2(device, target) + return target + + +def install_tools(bin_dir: Path) -> tuple[list[str], set[str]]: + """Find the missing tools, offer the ones with a download and install them into bin_dir. + + Returns the missing tools and the ones this run installed. + """ + deps = {dep["name"]: dep for dep in get_dependencies()} + missing = [tool for tool, spec in TOOLS.items() if not all(deps[dep]["binary"] for dep in spec["deps"])] + say(f"Missing tools: {', '.join(missing) or 'none'}") + + urls: dict[str, str] = {} + for tool in missing: + url = asset_url(tool) + if url: + urls[tool] = url + else: + say(f"{tool}: no download for this platform, {versioned(tool, TOOLS[tool]['hint'])}") + + installed: set[str] = set() + for group in dict.fromkeys(spec.get("group", tool) for tool, spec in TOOLS.items()): + tools = [tool for tool in urls if TOOLS[tool].get("group", tool) == group] + if not tools: + continue + required = any(deps[dep]["required"] for tool in tools for dep in TOOLS[tool]["deps"]) + names = ", ".join(label(tool) for tool in tools) + what = names if group in TOOLS else f"the {group} tools ({names})" + if Confirm.ask(f"Install {what}{' (required)' if required else ''}?", default=required, console=console): + installed.update(tool for tool in tools if install(tool, urls[tool], bin_dir)) + return missing, installed + + +def import_devices(wvds: Path, prds: Path) -> list[str]: + """Ask for device files or folders until an empty answer, import each one and return their CDM names.""" + imported: list[str] = [] + while answer := Prompt.ask(CDM_PROMPT, default="", console=console): + files = device_files(answer) + if not files: + say(f"No .wvd or .prd files in {answer}") + for device in files: + try: + copy = import_cdm(device, wvds, prds) + except Exception as e: + say(f"[red]Skipped[/] {device.name}: {e}") + continue + say(f"{'Already in place' if copy.resolve() == device.resolve() else 'Imported'}: {device.name}") + imported.append(copy.stem) + return list(dict.fromkeys(imported)) + + +@click.command(short_help="Install the external tools and make a first config.", context_settings=context_settings) +def setup() -> None: + """ + Install the external tools and make a first config. + + You can run it again: it skips the tools it finds and never overwrites a config. + """ + clone = is_clone() + if not keep_tool_install(clone): + return + data_root = binaries.user_binaries_dir.parent + bin_dir, target = destinations(clone) + say(f"Mode: {'git clone' if clone else 'uv tool install'}") + missing, installed = install_tools(bin_dir) + + written = None + existing = get_config_path() + if existing: + say(f"Config: {existing} (kept)") + package = config.directories.namespace_dir.parent + paths = [getattr(config.directories, key) for key in DATA_DIRS if key != "services"] + paths += config.directories.services + if not clone and any(isinstance(path, Path) and path.is_relative_to(package) for path in paths): + say("[yellow]Warning:[/] some data folders are inside the package, and a tool upgrade deletes them.") + say(f"Add this to {existing}:\n{yaml.safe_dump({'directories': data_dirs(data_root)}, sort_keys=False)}") + else: + default = Path.home() / "Downloads" / "unshackle" + downloads = Path(Prompt.ask("Downloads folder", default=str(default), console=console)).expanduser() + written = write_config(target, downloads, None if clone else data_root) + + if written and not clone: + folders = {key: data_root / DATA_DIRS[key] for key in ("wvds", "prds", "cookies", "services")} + else: + folders = {key: getattr(config.directories, key) for key in ("wvds", "prds", "cookies")} + local_services = [path for path in config.directories.services if isinstance(path, Path)] + if local_services: + folders["services"] = local_services[0] + for key, path in folders.items(): + path.mkdir(parents=True, exist_ok=True) + say(f"{key}: {path}") + + cdms = import_devices(folders["wvds"], folders["prds"]) + if written and cdms: + cdm = cdms[0] + if len(cdms) > 1: + cdm = Prompt.ask("Default CDM", choices=cdms, default=cdm, console=console) + written["cdm"] = {"default": cdm} + dump_config(target, written) + say(f"Default CDM: {cdm}") + + for tool in TOOLS: + say(f"{'[green]✓[/]' if tool not in missing or tool in installed else '[red]✗[/]'} {tool}") + if written: + say(f"A minimal unshackle.yaml was written to {target}.") + say(f"To expand it, see {DOCS_URL}") + run = "uv run unshackle" if clone else "unshackle" + say(f"Next: run `{run} env check` to confirm the tools.") + say(f"Then put a service in a services folder and run `{run} dl `.") diff --git a/unshackle/core/__main__.py b/unshackle/core/__main__.py index 5b64213e..89aaaa02 100644 --- a/unshackle/core/__main__.py +++ b/unshackle/core/__main__.py @@ -20,7 +20,38 @@ from unshackle.core.update_checker import UpdateChecker from unshackle.core.utilities import close_debug_logger, init_debug_logger -@click.command(cls=Commands, invoke_without_command=True, context_settings=context_settings) +def print_banner() -> None: + """Print the ASCII banner with the version line.""" + console.print( + Padding( + Group( + Text( + r"▄• ▄▌ ▐ ▄ .▄▄ · ▄ .▄ ▄▄▄· ▄▄· ▄ •▄ ▄▄▌ ▄▄▄ ." + "\n" + r"█▪██▌•█▌▐█▐█ ▀. ██▪▐█▐█ ▀█ ▐█ ▌▪█▌▄▌▪██• ▀▄.▀·" + "\n" + r"█▌▐█▌▐█▐▐▌▄▀▀▀█▄██▀▐█▄█▀▀█ ██ ▄▄▐▀▀▄·██▪ ▐▀▀▪▄" + "\n" + r"▐█▄█▌██▐█▌▐█▄▪▐███▌▐▀▐█ ▪▐▌▐███▌▐█.█▌▐█▌▐▌▐█▄▄▌" + "\n" + r" ▀▀▀ ▀▀ █▪ ▀▀▀▀ ▀▀▀ · ▀ ▀ ·▀▀▀ ·▀ ▀.▀▀▀ ▀▀▀ ", + style="ascii.art", + ), + f"v [repr.number]{__version__}[/]{f' ({__code_hash__})' if __code_hash__ else ''}" + f" - © 2025-{datetime.now().year} github.com/unshackle-dl/unshackle", + ), + (1, 11, 1, 10), + expand=True, + ), + justify="center", + ) + + +class Main(Commands): + """The top-level group. Its help shows the banner, because its callback does not run for --help.""" + + def format_help(self, ctx: click.Context, formatter: click.HelpFormatter) -> None: + print_banner() + super().format_help(ctx, formatter) + + +@click.command(cls=Main, invoke_without_command=True, context_settings=context_settings) @click.option("-v", "--version", is_flag=True, default=False, help="Print version information.") @click.option("-d", "--debug", is_flag=True, default=False, help="Enable DEBUG level logs and JSON debug logging.") def main(version: bool, debug: bool) -> None: @@ -58,25 +89,7 @@ def main(version: bool, debug: bool) -> None: if "--quiet" in serve_args or "-q" in serve_args: return - console.print( - Padding( - Group( - Text( - r"▄• ▄▌ ▐ ▄ .▄▄ · ▄ .▄ ▄▄▄· ▄▄· ▄ •▄ ▄▄▌ ▄▄▄ ." + "\n" - r"█▪██▌•█▌▐█▐█ ▀. ██▪▐█▐█ ▀█ ▐█ ▌▪█▌▄▌▪██• ▀▄.▀·" + "\n" - r"█▌▐█▌▐█▐▐▌▄▀▀▀█▄██▀▐█▄█▀▀█ ██ ▄▄▐▀▀▄·██▪ ▐▀▀▪▄" + "\n" - r"▐█▄█▌██▐█▌▐█▄▪▐███▌▐▀▐█ ▪▐▌▐███▌▐█.█▌▐█▌▐▌▐█▄▄▌" + "\n" - r" ▀▀▀ ▀▀ █▪ ▀▀▀▀ ▀▀▀ · ▀ ▀ ·▀▀▀ ·▀ ▀.▀▀▀ ▀▀▀ ", - style="ascii.art", - ), - f"v [repr.number]{__version__}[/]{f' ({__code_hash__})' if __code_hash__ else ''}" - f" - © 2025-{datetime.now().year} github.com/unshackle-dl/unshackle", - ), - (1, 11, 1, 10), - expand=True, - ), - justify="center", - ) + print_banner() if version: return diff --git a/unshackle/core/binaries.py b/unshackle/core/binaries.py index 30f41d28..0928a5ff 100644 --- a/unshackle/core/binaries.py +++ b/unshackle/core/binaries.py @@ -3,16 +3,19 @@ import sys from pathlib import Path from typing import Optional +from appdirs import user_data_dir + __shaka_platform = {"win32": "win", "darwin": "osx"}.get(sys.platform, sys.platform) +package_binaries_dir = Path(__file__).resolve().parent.parent / "binaries" +user_binaries_dir = Path(user_data_dir("unshackle", False)) / "binaries" + def find(*names: str, search_dirs: Optional[list[Path]] = None) -> Optional[Path]: """Find the path of the first found binary name.""" - current_dir = Path(__file__).resolve().parent.parent - local_binaries_dir = current_dir / "binaries" - services_dir = current_dir / "services" + services_dir = package_binaries_dir.parent / "services" - dirs_to_check: list[Path] = [local_binaries_dir] + dirs_to_check: list[Path] = [package_binaries_dir, user_binaries_dir] if services_dir.exists(): for s_dir in services_dir.iterdir(): s_bin = s_dir / "binaries" diff --git a/unshackle/core/config.py b/unshackle/core/config.py index 74e6f828..23469191 100644 --- a/unshackle/core/config.py +++ b/unshackle/core/config.py @@ -361,14 +361,15 @@ class Config: return cls(**restore_bool_languages(yaml.safe_load(path.read_text(encoding="utf8")) or {})) +# noinspection PyProtectedMember +PACKAGE_CONFIG_PATH = Config._Directories.namespace_dir / Config._Filenames.root_config +USER_CONFIG_PATH = Path(Config._Directories.app_dirs.user_config_dir) / Config._Filenames.root_config # noinspection PyProtectedMember POSSIBLE_CONFIG_PATHS = ( - # The unshackle Namespace Folder (e.g., %appdata%/Python/Python311/site-packages/unshackle) - Config._Directories.namespace_dir / Config._Filenames.root_config, + PACKAGE_CONFIG_PATH, # The Parent Folder to the unshackle Namespace Folder (e.g., %appdata%/Python/Python311/site-packages) Config._Directories.namespace_dir.parent / Config._Filenames.root_config, - # The AppDirs User Config Folder (e.g., ~/.config/unshackle on Linux, %LOCALAPPDATA%\unshackle on Windows) - Path(Config._Directories.app_dirs.user_config_dir) / Config._Filenames.root_config, + USER_CONFIG_PATH, )