kenzuya/unshackle
1
0
Fork 2
mirror of https://github.com/unshackle-dl/unshackle.git synced 2026-11-13 12:35:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(api): stream download job progress over server-sent events

Browse Source 
Adds GET /api/download/jobs/{job_id}/events with snapshot, status, progress and terminal events, query-param auth for EventSource in both serve modes, atomic progress-file writes, and stream-safe CORS and compression middleware.
This commit is contained in:
imSp4rky
2026-08-12 10:35:43 -06:00
parent 581979c5a8
commit 16ae73e3a0
11 changed files with 312 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/dev/rest-api/authentication.md
+14
View File
@@ -108,6 +108,17 @@ X-Secret-Key: <your-secret>
The value must exactly match one of the configured keys (`api_secret`, or one of the keys under `users`). There is no login step, no session cookie, and no token exchange for API access. The same static key is sent on each request.
!!! info "One route also accepts the key as a query parameter"
The job event stream, `GET /api/download/jobs/{job_id}/events`, also accepts the key in a `secret_key` query parameter. This is for browsers: the `EventSource` API cannot set request headers. The exception applies to that one route only, and it works in both server modes.
```javascript
new EventSource(`http://127.0.0.1:8786/api/download/jobs/${jobId}/events?secret_key=${key}`);
```
The header always wins. If the request has an `X-Secret-Key` header, the server checks that header and ignores the query parameter, so a wrong header is rejected even when the query parameter holds a valid key.
A key in a URL is easier to leak than a key in a header, because proxies and browser history can record it. Use the header wherever your client can set one.
=== "curl"
```bash
@@ -269,6 +280,7 @@ With `--caddy`, unshackle also starts `caddy run` using the `Caddyfile` located
| Item | Value |
|---|---|
| Auth header | `X-Secret-Key` |
| Query-parameter alternative | `?secret_key=` on `GET /api/download/jobs/{job_id}/events` only |
| Default bind address | `127.0.0.1:8786` |
| Exempt endpoint | `GET /api/health` |
| Failure status | `401 Unauthorized` |
@@ -293,6 +305,8 @@ The header and config are identical between modes, but the code that enforces th
Because the API-only middleware guards *all* non-health paths, the Swagger UI at `/api/docs/` is also behind `X-Secret-Key` there. In `--remote-only` mode the Swagger UI is not mounted at all.
Both middlewares read the caller's key through the shared `request_secret_key()` helper in `unshackle/core/api/handlers.py`, which returns the `X-Secret-Key` header and falls back to the `secret_key` query parameter only when the matched route is the job event stream. The integrated server needs one extra branch for this: pywidevine's `authentication` is header-only, so `serve_authentication` compares the query-parameter key itself when the header is absent, and delegates to pywidevine in every other case. That is why a present-but-wrong header is rejected in both modes.
### CORS and the `Authorization` header
A `cors_middleware` runs on every response in both modes (and answers `OPTIONS` preflight requests with an empty response). It emits:
docs/dev/rest-api/endpoints.md
+67
View File
@@ -492,6 +492,73 @@ Fetch a single job with full details (equivalent to a `full=true` list entry).
| `200` | - | Job returned. |
| `404` | `JOB_NOT_FOUND` | No such job. |
### `GET /api/download/jobs/{job_id}/events`
Stream the job's progress as [Server-Sent Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events). Use this instead of polling `GET /api/download/jobs/{job_id}`.
The response has the content type `text/event-stream`. Each event has this format:
```
event: <name>
data: <job JSON>
```
The `data` field of every event is the same job object that `GET /api/download/jobs/{job_id}` returns.
| Event | When it is sent |
| --- | --- |
| `snapshot` | Immediately, as the first event. Gives the job's current state. |
| `status` | The job leaves the queue and starts to download. |
| `progress` | The worker's progress record changed. The server reads that record every 0.5 seconds, so you get at most two of these events each second, and none while the record stays the same. |
| `completed` | The job finished. The server then closes the stream. |
| `failed` | The job failed. The server then closes the stream. |
| `cancelled` | The job was cancelled. The server then closes the stream. |
The server sends a `: keep-alive` comment each 15 seconds while the job is quiet, to keep proxies from closing an idle connection. Ignore these lines.
If the job is already in a terminal state, the server sends the `snapshot` event, then the terminal event, and closes the stream immediately.
!!! tip "Authentication from a browser"
`EventSource` cannot set headers. For this endpoint only, the server also accepts the key in the `secret_key` query parameter. This works in both server modes, the default integrated server and `--api-only`:
```javascript
const events = new EventSource(
`http://127.0.0.1:8786/api/download/jobs/${jobId}/events?secret_key=${key}`
);
events.addEventListener("progress", (e) => console.log(JSON.parse(e.data).progress));
events.addEventListener("completed", () => events.close());
```
The header always wins. Send the key in the query parameter *or* in the header, not both: if an `X-Secret-Key` header is present, the server checks that header and ignores the query parameter. See [Authentication](authentication.md#how-clients-present-credentials).
=== "Request"
```bash
curl -N -H "X-Secret-Key: $KEY" \
"http://127.0.0.1:8786/api/download/jobs/b0f7c8e2-.../events"
```
=== "Response `200`"
```
event: snapshot
data: {"job_id":"b0f7c8e2-...","status":"downloading","progress":42.5,...}
event: progress
data: {"job_id":"b0f7c8e2-...","status":"downloading","progress":48.0,...}
: keep-alive
event: completed
data: {"job_id":"b0f7c8e2-...","status":"completed","progress":100.0,...}
```
| Status | Error code | Meaning |
| --- | --- | --- |
| `200` | - | Event stream started. |
| `404` | `JOB_NOT_FOUND` | No such job. |
### `DELETE /api/download/jobs/{job_id}`
Cancel or remove a job. The behaviour depends on the job's current state:
docs/dev/rest-api/index.md
+4 -2
View File
@@ -22,8 +22,9 @@ calls you can script against:
- **Discovery**: list the services available on the server, run a service's search,
and enumerate the titles and tracks behind a title ID.
- **Downloading**: submit a download as a background *job*, then poll its status and
live progress, cancel it, retry it, or bump it to the front of the queue.
- **Downloading**: submit a download as a background *job*, then follow its status and
live progress (by polling, or by subscribing to its Server-Sent Events stream),
cancel it, retry it, or bump it to the front of the queue.
- **History and housekeeping**: read the log of finished jobs, inspect the server's
effective (redacted) configuration, check external tool versions, and clear caches
or temp files.
@@ -161,6 +162,7 @@ request and response in full.
| `GET` | `/api/download/jobs` | List jobs, with filtering and sorting. |
| `POST` | `/api/download/jobs/clear-finished` | Remove all finished jobs. |
| `GET` | `/api/download/jobs/{job_id}` | Full detail for one job. |
| `GET` | `/api/download/jobs/{job_id}/events` | Stream one job's progress as Server-Sent Events. |
| `DELETE` | `/api/download/jobs/{job_id}` | Cancel a running job, or remove a finished one. |
| `POST` | `/api/download/jobs/{job_id}/retry` | Re-queue a finished job's parameters as a new job. |
| `POST` | `/api/download/jobs/{job_id}/priority` | Move a queued job to the front. |
docs/dev/rest-api/job-lifecycle.md
+5
View File
@@ -177,6 +177,11 @@ The full detail adds `started_time`, `completed_time`, `output_files`, the redac
done
```
!!! tip "Or subscribe instead of polling"
`GET /api/download/jobs/{job_id}/events` pushes the same job object as Server-Sent
Events, and closes the stream when the job ends. See
[Endpoints](endpoints.md#get-apidownloadjobsjob_idevents).
!!! note "What the percentage means"
`progress` climbs from `0` toward `90` while tracks download, then the final stretch
to `100` covers post-download work such as decryption cleanup, repackaging, and
docs/dev/rest-api/quickstart.md
+9 -1
View File
@@ -310,7 +310,15 @@ done
```
!!! tip "Polling cadence"
A few seconds between polls is plenty. The server writes progress updates roughly twice a second internally, but there is no push/streaming channel; you read the current snapshot each time you ask.
A few seconds between polls is plenty. The server updates a job's progress roughly twice a second internally, and each poll reads the current snapshot.
!!! tip "Or let the server push the updates"
`GET /api/download/jobs/{job_id}/events` streams the same job record as Server-Sent Events, and closes the stream when the job ends. It removes the polling loop above. See [Endpoints](endpoints.md#get-apidownloadjobsjob_idevents).
```bash
curl -N "$BASE/api/download/jobs/$JOB/events" \
-H "X-Secret-Key: $SECRET"
```
If a job ends in `failed`, the full record includes `error_message`, `error_code`, and (when the server was started with `--debug-api`) `error_traceback` and `worker_stderr` to help you diagnose it. Sensitive values in these fields are scrubbed before they are returned.