FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Windows: from_single_file original_config and fp16_safetensors crash with UnicodeDecodeError under non-UTF-8 locales · Issue #14927 · huggingface/diffusers · GitHub

Repository navigation

Windows: from_single_file original_config and fp16_safetensors crash with UnicodeDecodeError under non-UTF-8 locales #14927

Description

Summary

Two user-facing config reads fall back to the locale encoding and crash with UnicodeDecodeError on Windows machines whose default locale is not UTF-8 (e.g. cp936/GBK):

  • fetch_original_config reading the user-supplied original config file (src/diffusers/loaders/single_file_utils.py) — the entry point used by from_single_file when a local YAML config is passed
  • the fp16_safetensors conversion command reading model_index.json (src/diffusers/commands/fp16_safetensors.py)

Reproduction

On Windows with a non-UTF-8 system locale, put a non-ASCII comment (e.g. CJK text) in the original config YAML and load a checkpoint with from_single_file(..., original_config=...), or run the fp16_safetensors command on a model folder whose model_index.json contains non-ASCII text. Both fail with UnicodeDecodeError: 'gbk' codec can't decode byte ....

Both files are user-authored/Hub-provided UTF-8 documents, so the reads should pin encoding="utf-8" instead of relying on the locale default.

I have a fix in PR #14819.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions


      Back | FazBrowse Home | New Git URL