Summary
Add support for single non-ASCII Unicode code points in CSV dialect character attributes, matching CPython behavior.
Rationale
csv dialect options such as delimiter, quotechar, and escapechar accept exactly one Unicode character in CPython. RustPython currently stores these attributes as u8 in crates/stdlib/src/csv.rs, so valid one-character values outside the byte range (for example, €) are rejected.
Affected area
- crates/stdlib/src/csv.rs
- CSV dialect parsing, validation, reader, and writer configuration
Required changes
Replace the byte-only representation and parsing path for dialect characters with a representation that preserves a full Unicode code point, then propagate that support through the CSV reader and writer implementation. Keep existing validation that rejects empty and multi-character values, while reporting conversion/representation failures accurately.
Acceptance criteria
- delimiter, quotechar, and escapechar accept a single non-ASCII Unicode code point such as €.
- Empty and multi-character strings remain rejected according to CPython-compatible behavior.
- Reader and writer behavior correctly handles the supported Unicode dialect characters.
- Relevant RustPython CSV tests cover successful non-ASCII dialect-character usage and invalid lengths.
Backlinks
Requested by @hyoinandout.
Reactions are currently unavailable
Summary
Add support for single non-ASCII Unicode code points in CSV dialect character attributes, matching CPython behavior.
Rationale
csv dialect options such as delimiter, quotechar, and escapechar accept exactly one Unicode character in CPython. RustPython currently stores these attributes as u8 in crates/stdlib/src/csv.rs, so valid one-character values outside the byte range (for example, €) are rejected.
Affected area
Required changes
Replace the byte-only representation and parsing path for dialect characters with a representation that preserves a full Unicode code point, then propagate that support through the CSV reader and writer implementation. Keep existing validation that rejects empty and multi-character values, while reporting conversion/representation failures accurately.
Acceptance criteria
Backlinks
Requested by @hyoinandout.