| [ Web Proxy ] |
| Viewing: https://www.pythonbyexample.dev/examples/strings | [Back] [Original] |
Text
Compare three words by code-point count and UTF-8 byte count. ASCII characters take one byte each (hello 5 bytes); the in caf is one code point but two UTF-8 bytes; each Thai character takes three. The str type abstracts over all three.
Source
english = "hello"
french = "caf"
thai = ""
for label, word in [("English", english), ("French", french), ("Thai", thai)]:
print(label, word, len(word), len(word.encode("utf-8")))Output
English hello 5 5
French caf 4 5
Thai 6 18Indexing and iteration work with Unicode code points, not encoded bytes. ord() returns the integer code point, which is often displayed in hexadecimal when teaching text encoding.
Source
print(thai[0])
print([hex(ord(char)) for char in thai[:2]])Output
['0xe2a', '0xe27']String methods return new strings because strings are immutable. Encoding turns text into bytes when another system needs a byte representation.
Source
text = " caf "
clean = text.strip()
print(clean)
print(clean.upper())
print(clean.encode("utf-8"))Output
caf
CAF
b'caf\xc3\xa9'Notes
str for text and bytes for binary data.len(text) counts Unicode code points; len(text.encode("utf-8")) counts encoded bytes.English hello 5 5
French caf 4 5
Thai 6 18
['0xe2a', '0xe27']
caf
CAF
b'caf\xc3\xa9'
Execution time appears here after you run the example.
| Web Proxy Viewer | New URL | Original Page |