FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Support for characters beyond BMP and below PUA · AnkiHubSoftware/unidecode@5f1a7a9 · GitHub

Repository navigation

Commit 5f1a7a9

Browse files
committed
Support for characters beyond BMP and below PUA
1 parent b6ca268 commit 5f1a7a9

181 files changed

Lines changed: 4 additions & 4 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎unidecode/__init__.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,8 @@ def unidecode(string):
3131
retval.append(char)
3232
continue
3333

34-
if codepoint > 0xffff:
35-
continue # We don't support characters beyond the BMP.
34+
if codepoint > 0xeffff:
35+
continue # Characters in Private Use Area and above are ignored
3636

3737
section = codepoint >> 8 # Chop off the last two hex digits
3838
position = codepoint % 256 # Last two hex digits
@@ -41,7 +41,7 @@ def unidecode(string):
4141
table = Cache[section]
4242
except KeyError:
4343
try:
44-
mod = __import__('unidecode.x%02x'%(section), [], [], ['data'])
44+
mod = __import__('unidecode.x%03x'%(section), [], [], ['data'])
4545
except ImportError:
4646
Cache[section] = None
4747
continue # No match: ignore this character and carry on.
@@ -51,4 +51,4 @@ def unidecode(string):
5151
if table and len(table) > position:
5252
retval.append( table[position] )
5353

54-
return ''.join(retval)
54+
return ''.join(retval)
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)

Back | FazBrowse Home | New Git URL