fix(sanitize): normalise filenames to NFC and keep combining marks

Remove only control characters in unicode mode. A mark after a zero width joiner or a digit was still dropped (#161). In ASCII mode a decomposed title now transliterates like the composed one.

Names change for titles with combining marks, compatibility ideographs or decomposed spellings, so an existing library can get a second folder.
This commit is contained in:
imSp4rky
2026-10-02 13:22:04 -06:00
parent 7b3794333f
commit 34eb94b55d
3 changed files with 59 additions and 46 deletions
+3 -1
View File
@@ -380,7 +380,9 @@ unicode_filenames: true
| Value | Result |
|---|---|
| `false` (default) | `기생충` → transliterated ASCII; accents stripped. |
| `true` | Native characters preserved in filenames and folders. |
| `true` | Native characters preserved in filenames and folders, with the vowel and tone marks of languages such as Thai, Hindi and Bengali. |
With each value, unshackle first normalises the name to Unicode NFC. A title that a service sends in composed form and in decomposed form gives the same name.
Regardless of this setting, unshackle always removes or replaces a set of filesystem-unsafe and structural characters during sanitisation. For example, `/` and `;` become ` & `, and unshackle removes characters like `\ * ! ? , ' " < > | $ # ~`. This is why you must never put path-unsafe characters directly in a template.