fix(remote): return session cache for device-code and cache-only logins

Answered login prompts and uploaded cache now count as client auth; the client closes on exit and keeps the remote session alive during long downloads.

docs/dev/rest-api/endpoints.md

@@ -839,7 +839,7 @@ Report the install status of the external binaries unshackle depends on (the sam
 !!! note "Developer feature"
     The `/api/session/*` endpoints power unshackle's **remote download** mode. A thin local client authenticates once against a remote server, then fetches titles, tracks, and segments, and proxies DRM licensing back through it. The server mounts these endpoints in `--remote-only` mode, and they are what the built-in `RemoteService` client drives. Most end users never call them directly.
 
-The server holds remote sessions in memory. They expire after `serve.session_ttl` (default 300s), except while awaiting interactive authentication input, which uses a 600s window. A remote session binds to the IP that made it, so the server rejects a request from a different IP with `403 FORBIDDEN`.
+The server holds remote sessions in memory. They expire after `serve.session_ttl` (default 300s) without a request, except while awaiting interactive authentication input, which uses a 600s window. A remote session binds to the IP that made it, so the server rejects a request from a different IP with `403 FORBIDDEN`.
 
 ### Remote session lifecycle overview
 
@@ -874,7 +874,7 @@ The `proxy` value must be a full proxy URI, unless the operator gives your API k
     }
     ```
 
-`server_account` is `true` when the server authenticated with one of its own accounts (`serve.server_accounts`). Otherwise the server uses only what the client sent, and never falls back to its own credentials. Only a client that sent its own cookies or credentials gets cookies, auth headers, and the remote session cache back. For any other remote session (a server account, or an anonymous login through the server's proxy) the tracks and segments responses carry no cookies and no auth headers. The server also drops secret-looking keys from track data, and the cache stays on the server. A server-account service also refuses a client proxy URI, because the server's login would pass through it.
+`server_account` is `true` when the server authenticated with one of its own accounts (`serve.server_accounts`). Otherwise the server uses only what the client sent, and never falls back to its own credentials. Only a client that sent its own cookies, credentials, or cache files gets cookies, auth headers, and the remote session cache back. A client that answered a login prompt (a device code or an OTP) gets them too. This applies only when the login then succeeds and the remote session does not use a server account. For any other remote session (a server account, or an anonymous login through the server's proxy) the tracks and segments responses carry no cookies and no auth headers. The server also drops secret-looking keys from track data, and the cache stays on the server. A server-account service also refuses a client proxy URI, because the server's login would pass through it.
 
 | Status | Error code | Meaning |
 | --- | --- | --- |
@@ -1134,7 +1134,7 @@ Tear down a remote session. The server cancels its input bridge, harvests any up
     }
     ```
 
-The `cache` field is present only if the remote session produced cache files. `404 SESSION_NOT_FOUND` if the remote session does not exist.
+The `cache` field is present only if the remote session produced cache files and the login belongs to the client. `404 SESSION_NOT_FOUND` if the remote session does not exist.
 
 ---
 

docs/dev/rest-api/remote-sessions.md

@@ -195,10 +195,16 @@ hood the client walks a remote session through its lifecycle.
 
 === "5. Download + close"
 
-    The client downloads, decrypts, and muxes locally. On completion it deletes
+    The client downloads, decrypts, and muxes locally. While it works, it sends a
-    the remote session. If the server has updated cache files (for example a refreshed
+    keep-alive request to the remote session at an interval of one third of
-    token), the delete returns them and the client saves them locally, so the **next**
+    `session_ttl`, so a long download does not let the remote session expire. On
-    remote session can skip interactive auth.
+    completion, and also when `dl` exits early (`--list`, Ctrl+C, an error), it
+    deletes the remote session. If the login belongs to the client, the `DELETE`
+    request returns the updated cache files (for example a refreshed token) and the
+    client saves them locally, so the **next** remote session can skip interactive
+    authentication. The login belongs to the client when the client sent
+    cookies, credentials, or cache files, or answered a login prompt (a device code
+    or an OTP) that led to a successful login. A server-account login never does.
 
 !!! tip "Renaming remote titles locally"
     You can rename titles for a remote service you do not have installed locally by
@@ -423,9 +429,10 @@ _sessions/<pbkdf2_hmac(sha256, X-Secret-Key, "unshackle-session-ns", 100000)[:12
 ```
 
 The handler writes forwarded `cache` files into that directory before authentication.
-On `DELETE`, the handler harvests updated cache files (compressing each with zlib
+On `DELETE`, when the login belongs to the client (`client_auth`), the handler
-and base64-encoding, **excluding** `titles_*` files) and returns them under a
+harvests updated cache files (compressing each with zlib and base64-encoding,
-`cache` field, so the client can keep refreshed tokens:
+**excluding** `titles_*` files) and returns them under a `cache` field, so the
+client can keep refreshed tokens:
 
 ```json
 { "status": "ok", "cache": { "tokens": "...base64(zlib(bytes))..." } }

docs/reference/configuration/services.md

@@ -121,7 +121,7 @@ is the [REST API](../../dev/rest-api/index.md) section. These are the config key
 | `services` | list | *(unset)* | Global service allowlist. Omit to allow all. |
 | `remote_only` | bool | `false` | Expose only the remote service session endpoints (health, services, search, session) and disable the rest of the REST API. |
 | `dashboard` | dict | *(unset)* | Developer dashboard: `key` is the API key for the read-only `/api/dashboard/` endpoints (status, remote sessions, remote session logs, jobs, keys, services, health, logs, SSE events). Unset leaves those routes unregistered. See [dashboard endpoints](../../dev/rest-api/dashboard.md). |
-| `session_ttl` | int (s) | `300` | Lifetime of an interactive auth session. |
+| `session_ttl` | int (s) | `300` | Seconds a remote session can stay idle before it expires. A remote client sends keep-alive requests to its remote session during a download, so the remote session does not expire during a long download. |
 | `max_sessions` | int or null | `100` | Maximum concurrent remote sessions; at the cap the server evicts the oldest. Set it to `null` or `0` for no limit, which is also what `/api/dashboard/status` then reports, so a dashboard can tell "no cap" from a cap that happens to sit at the default. |
 | `history_limit` | int | `100` | How many finished jobs to retain in history. |
 | `compression_level` | int | `1` | gzip level for responses. |

tests/remote/unit/test_input_bridge.py

@@ -94,3 +94,16 @@ def test_status_setter() -> None:
     bridge = InputBridge()
     bridge.status = AuthStatus.AUTHENTICATED
     assert bridge.status is AuthStatus.AUTHENTICATED
+
+
+def test_answered_only_after_an_accepted_response() -> None:
+    bridge = InputBridge()
+    assert bridge.submit_response("early") is False
+    assert bridge.answered is False
+    worker = threading.Thread(target=bridge.request_input, args=("code?",))
+    worker.start()
+    while bridge.status != AuthStatus.PENDING_INPUT:
+        time.sleep(0.01)
+    assert bridge.submit_response("CODE") is True
+    worker.join(timeout=2)
+    assert bridge.answered is True

tests/remote/unit/test_session_store.py

@@ -130,3 +130,15 @@ async def test_max_sessions_evicts_oldest(store: SessionStore, monkeypatch: pyte
     assert await store.get("a") is None  # evicted
     assert (await store.get("b")) is b
     assert (await store.get("c")) is c
+
+
+async def test_get_expiry_removes_cache_dir(store: SessionStore, monkeypatch: pytest.MonkeyPatch) -> None:
+    from datetime import datetime, timedelta, timezone
+
+    removed: list = []
+    monkeypatch.setattr(SessionStore, "cleanup_cache_dir", staticmethod(removed.append))
+    entry = await store.create("EXAMPLE", _FakeService())
+    entry.cache_tag = "_sessions/k/s/EXAMPLE"
+    entry.last_accessed = datetime.now(timezone.utc) - timedelta(seconds=store.ttl + 100)
+    assert await store.get(entry.session_id) is None
+    assert removed == ["_sessions/k/s/EXAMPLE"]

tests/test_client_session_auth.py

@@ -168,3 +168,129 @@ def test_safe_inflate_rejects_non_str_via_isinstance():
     for bad in (1, {"a": 1}, ["x"], None):
         with pytest.raises(APIError):
             guard(bad)
+
+
+def _run_session(monkeypatch, tmp_path, data, answer, server_account=False, fail=False):
+    """Run session_create_handler with a service that prompts during login; return the settled remote session."""
+    import asyncio
+    import base64
+    import json
+    import zlib
+
+    from unshackle.core.api import handlers
+    from unshackle.core.api.input_bridge import AuthStatus
+    from unshackle.core.api.session_store import get_session_store
+    from unshackle.core.config import config
+
+    class PromptingService:
+        _input_bridge = None
+        log = None
+        cache = None
+
+        def authenticate(self, cookies, credential):
+            if answer is not None:
+                self._input_bridge.request_input("Enter code")
+            if fail:
+                raise ValueError("bad code")
+
+    monkeypatch.setattr(config.directories, "cache", tmp_path)
+    monkeypatch.setattr(handlers, "validate_service", lambda service, request=None: service)
+    monkeypatch.setattr(handlers, "resolve_handler_proxy", lambda *args: (None, []))
+    monkeypatch.setattr(handlers, "server_account_for", lambda *args: server_account)
+    monkeypatch.setattr(handlers, "next_server_profile", lambda *args: None)
+    monkeypatch.setattr(handlers.Services, "load", lambda service: PromptingService)
+    monkeypatch.setattr(handlers, "create_service_instance", lambda *a, **k: (PromptingService(), None, None))
+    if data.get("cache") == "tokens":
+        data["cache"] = {"tokens_default": base64.b64encode(zlib.compress(b"{}")).decode("ascii")}
+
+    async def run():
+        resp = await handlers.session_create_handler({"service": "EXAMPLE", "title_id": "t", **data})
+        session = get_session_store().peek(json.loads(resp.body)["session_id"])
+        for _ in range(200):
+            if answer is not None and session.input_bridge.status == AuthStatus.PENDING_INPUT:
+                await handlers.session_prompt_post_handler({"response": answer}, session.session_id)
+            if session.auth_status in (AuthStatus.AUTHENTICATED, AuthStatus.FAILED):
+                break
+            await asyncio.sleep(0.01)
+        await get_session_store().delete(session.session_id)
+        return session
+
+    return asyncio.run(run())
+
+
+@pytest.mark.parametrize(
+    "data, answer, server_account, fail, expected",
+    [
+        ({}, None, False, False, False),  # anonymous login through the server: stays the server's
+        ({}, "CODE", False, False, True),  # device code the client approved
+        ({}, "CODE", False, True, False),  # answered, but the login failed
+        ({}, "CODE", True, False, False),  # server account OTP answered by the client
+        ({"cache": "tokens"}, None, False, False, True),  # client uploaded its own earlier login
+    ],
+)
+def test_client_auth_follows_login_provenance(monkeypatch, tmp_path, data, answer, server_account, fail, expected):
+    session = _run_session(monkeypatch, tmp_path, dict(data), answer, server_account, fail)
+    assert session.client_auth is expected
+
+
+def test_prompt_post_rejects_response_the_bridge_refused(monkeypatch):
+    import asyncio
+
+    from unshackle.core.api import handlers
+    from unshackle.core.api.input_bridge import AuthStatus
+
+    bridge = SimpleNamespace(status=AuthStatus.PENDING_INPUT, submit_response=lambda text: False)
+
+    async def fake_get_session(session_id, request):
+        return SimpleNamespace(input_bridge=bridge)
+
+    monkeypatch.setattr(handlers, "get_validated_session", fake_get_session)
+    with pytest.raises(APIError):
+        asyncio.run(handlers.session_prompt_post_handler({"response": ""}, "sess"))
+
+
+def test_login_that_outlives_its_session_leaves_no_cache(monkeypatch, tmp_path):
+    """A login loop that finishes after DELETE must not leave its tokens on the server's disk."""
+    import asyncio
+    import json
+    import threading
+
+    from unshackle.core.api import handlers
+    from unshackle.core.api.session_store import get_session_store
+    from unshackle.core.config import config
+
+    paired = threading.Event()
+    finished = threading.Event()
+
+    class SlowPairingService:
+        _input_bridge = None
+        log = None
+        cache = None
+
+        def authenticate(self, cookies, credential):
+            paired.wait(5)
+            self.cache.get("tokens").set("token")
+            finished.set()
+
+    monkeypatch.setattr(config.directories, "cache", tmp_path)
+    monkeypatch.setattr(handlers, "validate_service", lambda service, request=None: service)
+    monkeypatch.setattr(handlers, "resolve_handler_proxy", lambda *args: (None, []))
+    monkeypatch.setattr(handlers, "server_account_for", lambda *args: False)
+    monkeypatch.setattr(handlers.Services, "load", lambda service: SlowPairingService)
+    monkeypatch.setattr(handlers, "create_service_instance", lambda *a, **k: (SlowPairingService(), None, None))
+
+    async def run():
+        resp = await handlers.session_create_handler({"service": "EXAMPLE", "title_id": "t"})
+        session = get_session_store().peek(json.loads(resp.body)["session_id"])
+        await get_session_store().delete(session.session_id)
+        paired.set()
+        for _ in range(200):
+            if finished.is_set() and session.auth_status.value == "authenticated":
+                break
+            await asyncio.sleep(0.01)
+        await asyncio.sleep(0.05)
+        return session
+
+    session = asyncio.run(run())
+    assert session.auth_status.value == "authenticated"
+    assert not (tmp_path / session.cache_tag).exists()

unshackle/core/api/handlers.py

@@ -3033,7 +3033,9 @@ async def session_create_handler(data: Dict[str, Any], request: Optional[web.Req
             server_account=(profile or "default") if server_account else None,
         )
         session.cache_tag = session_cache_tag
-        session.client_auth = not server_account and (cookies is not None or credential is not None)
+        session.client_auth = not server_account and (
+            cookies is not None or credential is not None or bool(cache_data and session_cache_tag)
+        )
         # Echoed to every dashboard viewer, so cap what an arbitrary client can push into it.
         if isinstance(data.get("client"), dict) and len(json.dumps(data["client"], default=str)) <= 4096:
             session.client = data["client"]
@@ -3044,6 +3046,8 @@ async def session_create_handler(data: Dict[str, Any], request: Optional[web.Req
         async def run_auth() -> None:
             try:
                 await asyncio.to_thread(service_instance.authenticate, cookies, credential)
+                if bridge.answered and not server_account:
+                    session.client_auth = True
                 session.auth_status = AuthStatus.AUTHENTICATED
                 bridge.status = AuthStatus.AUTHENTICATED
             except (Exception, SystemExit) as e:
@@ -3051,6 +3055,9 @@ async def session_create_handler(data: Dict[str, Any], request: Optional[web.Req
                 session.auth_status = AuthStatus.FAILED
                 session.auth_error = redact_secrets(str(e))
                 bridge.status = AuthStatus.FAILED
+            finally:
+                if store.peek(session_id) is not session:
+                    SessionStore.cleanup_cache_dir(session_cache_tag)
 
         asyncio.create_task(run_auth())
 
@@ -3478,7 +3485,8 @@ async def session_prompt_post_handler(
     if bridge is None or bridge.status != AuthStatus.PENDING_INPUT:
         raise APIError(APIErrorCode.INVALID_INPUT, "No prompt pending for this session")
 
-    bridge.submit_response(str(response_text))
+    if not bridge.submit_response(str(response_text)):
+        raise APIError(APIErrorCode.INVALID_INPUT, "No prompt pending for this session")
     return web.json_response({"status": "accepted"})
 
 

unshackle/core/api/input_bridge.py

@@ -41,6 +41,7 @@ class InputBridge:
     _response: Optional[str] = field(default=None, init=False, repr=False)
     _status: AuthStatus = field(default=AuthStatus.AUTHENTICATING, init=False)
     _cancelled: bool = field(default=False, init=False, repr=False)
+    _answered: bool = field(default=False, init=False, repr=False)
     _response_ready: threading.Event = field(default_factory=threading.Event, init=False, repr=False)
     _lock: threading.Lock = field(default_factory=threading.Lock, init=False, repr=False)
 
@@ -100,6 +101,7 @@ class InputBridge:
             if self._status != AuthStatus.PENDING_INPUT:
                 return False
             self._response = response
+            self._answered = True
         self._response_ready.set()
         return True
 
@@ -110,6 +112,12 @@ class InputBridge:
             self._status = AuthStatus.FAILED
         self._response_ready.set()
 
+    @property
+    def answered(self) -> bool:
+        """``True`` once the bridge has accepted a response from the client."""
+        with self._lock:
+            return self._answered
+
     @property
     def status(self) -> AuthStatus:
         with self._lock:

unshackle/core/api/session_store.py

@@ -46,7 +46,9 @@ class SessionEntry:
     owner_key: Optional[str] = None  # X-Secret-Key that owns this session
     cache_tag: Optional[str] = None
     server_account: Optional[str] = None  # profile name when the server lent its own account
-    client_auth: bool = False  # the client sent its own cookies or credentials
+    client_auth: bool = (
+        False  # the client sent its own cookies, credentials, or cache files, or answered a login prompt
+    )
     input_bridge: Optional[InputBridge] = None
     log_buffer: Optional[Any] = None  # SessionLogBuffer mirroring the service's self.log
     auth_status: AuthStatus = AuthStatus.AUTHENTICATED
@@ -169,6 +171,7 @@ class SessionStore:
                 elapsed = (datetime.now(timezone.utc) - entry.last_accessed).total_seconds()
                 if elapsed > self.ttl:
                     log.info(f"Session {sanitize_log(session_id)} expired (elapsed={elapsed:.0f}s, ttl={self.ttl}s)")
+                    self.cleanup_cache_dir(entry.cache_tag)
                     _publish("delete", self._sessions.pop(session_id), "expired")
                     return None
 

unshackle/core/remote_service.py

@@ -7,6 +7,7 @@ Everything else (track selection, download, decrypt, mux) runs locally.
 
 from __future__ import annotations
 
+import atexit
 import base64
 import hashlib
 import logging
@@ -18,7 +19,7 @@ from datetime import date as date_
 from enum import Enum
 from http.cookiejar import CookieJar
 from pathlib import Path
-from threading import Lock
+from threading import Event, Lock, Thread
 from typing import Any, Callable, Dict, Iterator, Optional, Union
 from uuid import UUID
 
@@ -797,6 +798,7 @@ class RemoteService:
         self._segment_filters: Dict[str, tuple[set[str], set[str]]] = {}
         self._log_seq = 0
         self._log_drain_lock = Lock()
+        self._keepalive_stop = Event()
 
         self._session = requests.Session()
         self._session.headers.update(config.headers)
@@ -959,11 +961,33 @@ class RemoteService:
 
         result = self.client.post("/api/session/create", create_data)
         self._session_id = result["session_id"]
+        atexit.register(self.close)
 
         status = result.get("status", "authenticated")
         if status == "authenticating":
             self.poll_auth_completion()
         self.drain_server_logs()
+        self.start_keepalive()
+
+    def start_keepalive(self) -> None:
+        """Send keep-alive requests so that a long download or mux operation does not outlive the server's idle TTL.
+
+        ``GET /api/session/{session_id}`` refreshes the idle timer and reports the TTL, so the thread sends
+        a request at an interval of one third of it. The thread stops when ``close()`` runs. A client that
+        dies stops sending requests, so the server still expires an abandoned remote session.
+        """
+        info = self.client.get_optional(f"/api/session/{self._session_id}")
+        interval = max(float(info.get("expires_in") or 300) / 3, 5.0)
+        url = f"{self.client.server_url}/api/session/{self._session_id}"
+
+        def ping() -> None:
+            while not self._keepalive_stop.wait(interval):
+                try:
+                    self.client.session.get(url, timeout=30)
+                except requests.RequestException:
+                    pass
+
+        Thread(target=ping, name=f"{self.service_tag}-keepalive", daemon=True).start()
 
     def drain_server_logs(self) -> None:
         """Fetch the service's server-side log records for this remote session and re-emit them locally.
@@ -1362,13 +1386,16 @@ class RemoteService:
         pass
 
     def close(self) -> None:
-        if self._session_id:
+        self._keepalive_stop.set()
+        session_id, self._session_id = self._session_id, None
+        if session_id:
             try:
-                result = self.client.delete(f"/api/session/{self._session_id}")
+                result = self.client.delete(f"/api/session/{session_id}")
                 self.save_returned_cache(result.get("cache", {}))
+            except SystemExit:
+                pass
             except Exception as e:
                 self.log.warning(f"Failed to clean up remote session: {e}")
-            self._session_id = None
 
     def save_returned_cache(self, cache_data: Dict[str, str]) -> None:
         """Save cache files returned by the server to the local cache directory.