FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

encoding: make TextDecoder handle BOM correctly by addaleax · Pull Request #30132 · nodejs/node · GitHub

/ node Public

encoding: make TextDecoder handle BOM correctly - #30132

Closed
addaleax wants to merge 1 commit into
nodejs:masterfrom
addaleax:textdecoder-bom
ConversationCommits (1)ChecksFiles changed
Closed

encoding: make TextDecoder handle BOM correctly#30132
addaleax wants to merge 1 commit into
nodejs:masterfrom
addaleax:textdecoder-bom

Conversation

addaleax commented Oct 26, 2019

Copy link
Copy Markdown
Member

Do not accept the BOM if it comes from a different encoding, and
only discard the BOM after it has actually been read (including
when it is spread over multiple chunks in streaming mode).

Fixes: #25315

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Oct 26, 2019

nodejs-github-bot commented Oct 26, 2019

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/26209/

addaleax added encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. review wanted PRs that need reviews. and removed lib / src Issues and PRs related to general changes in the lib or src directory. labels Nov 1, 2019
encoding: make TextDecoder handle BOM correctly
a30272b 
Do not accept the BOM if it comes from a different encoding, and
only discard the BOM after it has actually been read (including
when it is spread over multiple chunks in streaming mode).

Fixes: nodejs#25315

nodejs-github-bot commented Nov 1, 2019

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/26282/

nodejs-github-bot commented Nov 4, 2019

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/26317/

addaleax added author ready PRs that have at least one approval, no outstanding review comments, and a CI started. and removed review wanted PRs that need reviews. labels Nov 4, 2019

addaleax commented Nov 5, 2019

Copy link
Copy Markdown
Member Author

Landed in 237be2e

addaleax closed this Nov 5, 2019
addaleax added a commit that referenced this pull request Nov 5, 2019
encoding: make TextDecoder handle BOM correctly
237be2e 
Do not accept the BOM if it comes from a different encoding, and
only discard the BOM after it has actually been read (including
when it is spread over multiple chunks in streaming mode).

Fixes: #25315

PR-URL: #30132
Reviewed-By: Gus Caplan <me@gus.host>
addaleax deleted the textdecoder-bom branch November 5, 2019 19:30
srl295 reviewed Nov 5, 2019
View reviewed changes

srl295 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

missed this discussion. Code looks OK, I'm not familiar with this spec. But shouldn't ICU do this in ucnv_toUnicode?

addaleax commented Nov 5, 2019

Copy link
Copy Markdown
Member Author

@srl295 I don’t know, should it drop a BOM at the beginning of the text automatically? The current behaviour is that it doesn’t. Unless maybe there’s a flag in there that we could use and that I don’t know about? (And we do also at least need one way to make it not drop a BOM at the beginning of the text.)

srl295 commented Nov 5, 2019 via email

Copy link
Copy Markdown
Member

addaleax commented Nov 5, 2019

Copy link
Copy Markdown
Member Author

@srl295 I really have no idea whether ICU should or should not support automatic BOM-dropping or where to look up if it already does (other than that the implementation doesn’t look to me like it does), so I don’t feel qualified to open an issue and explain Node’s needs.

All I know is that it’s relatively easy for Node.js to work with what ICU currently provides.

srl295 commented Nov 5, 2019

Copy link
Copy Markdown
Member

@srl295 I really have no idea whether ICU should or should not support automatic BOM-dropping or where to look up if it already does (other than that the implementation doesn’t look to me like it does), so I don’t feel qualified to open an issue and explain Node’s needs.

All I know is that it’s relatively easy for Node.js to work with what ICU currently provides.

OK. I'll file an issue then.

MylesBorins pushed a commit that referenced this pull request Nov 17, 2019
encoding: make TextDecoder handle BOM correctly
c2c74fc 
Do not accept the BOM if it comes from a different encoding, and
only discard the BOM after it has actually been read (including
when it is spread over multiple chunks in streaming mode).

Fixes: #25315

PR-URL: #30132
Reviewed-By: Gus Caplan <me@gus.host>
BridgeAR mentioned this pull request Nov 19, 2019
Merged
targos pushed a commit that referenced this pull request Dec 1, 2019
encoding: make TextDecoder handle BOM correctly
3be9b22 
Do not accept the BOM if it comes from a different encoding, and
only discard the BOM after it has actually been read (including
when it is spread over multiple chunks in streaming mode).

Fixes: #25315

PR-URL: #30132
Reviewed-By: Gus Caplan <me@gus.host>
BethGriggs mentioned this pull request Dec 9, 2019
Closed
MylesBorins pushed a commit that referenced this pull request Dec 17, 2019
encoding: make TextDecoder handle BOM correctly
c80a4d8 
Do not accept the BOM if it comes from a different encoding, and
only discard the BOM after it has actually been read (including
when it is spread over multiple chunks in streaming mode).

Fixes: #25315

PR-URL: #30132
Reviewed-By: Gus Caplan <me@gus.host>
BethGriggs mentioned this pull request Dec 23, 2019
Merged
phated mentioned this pull request Apr 18, 2022
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

srl295 srl295 left review comments
+1 more reviewer
devsnek devsnek approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

addaleax

Labels

author ready PRs that have at least one approval, no outstanding review comments, and a CI started. c++ Issues and PRs that require attention from people who are familiar with C++. encoding Issues and PRs related to the TextEncoder and TextDecoder APIs.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

TextDecoder should not output BOM

4 participants

Back | FazBrowse Home | New Git URL