FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

ARROW-15812: [R] Accept col_names in open_dataset for CSV by wjones127 · Pull Request #14705 · apache/arrow · GitHub

/ arrow Public

ARROW-15812: [R] Accept col_names in open_dataset for CSV - #14705

Merged
thisisnic merged 3 commits into
apache:masterfrom
wjones127:ARROW-15812-csv-col-names
Jan 9, 2023
Merged

ARROW-15812: [R] Accept col_names in open_dataset for CSV#14705
thisisnic merged 3 commits into
apache:masterfrom
wjones127:ARROW-15812-csv-col-names

Conversation

Copy link
Copy Markdown
Member

No description provided.

Copy link
Copy Markdown

wjones127 force-pushed the ARROW-15812-csv-col-names branch from 64cacde to 5897d27 Compare November 28, 2022 22:45
wjones127 marked this pull request as ready for review November 28, 2022 22:51
wjones127 requested a review from thisisnic November 28, 2022 22:51
Comment thread r/R/csv.R Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Given it's not used here, should we just remove the col_types argument entirely?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

OK, I see we use it later when we're listing which ones we do and do not support. Maybe this bit of the code needs refactoring in other ways, as it feels a bit brittle to depend on this here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Though probably out of scope for this PR

Comment thread r/R/dataset-format.R Outdated
Comment thread r/tests/testthat/test-dataset-csv.R Outdated
Comment thread r/tests/testthat/test-dataset-csv.R Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why this change?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Ah looked at this again and I shouldn't have changed it. It was complaining that skip was a readr style arg, but we were internally appending an Arrow-style arg column_names when we passed an explicit schema. It's a little messy, but I basically changed it so internally we choose to add column_names (Arrow-style) or col_names (readr-style) based on what other arguments were passed.

thisisnic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks for making this change! Just a few comments.

wjones127 force-pushed the ARROW-15812-csv-col-names branch from 5178ae5 to a3df30a Compare December 6, 2022 19:35
wjones127 force-pushed the ARROW-15812-csv-col-names branch from a3df30a to 51726f3 Compare December 27, 2022 23:05
thisisnic merged commit 211925c into apache:master Jan 9, 2023

ursabot commented Jan 9, 2023

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 37f5a35 and contender = 211925c. 211925c is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.71% ⬆️0.03%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.4% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 211925c9 ec2-t3-xlarge-us-east-2
[Finished] 211925c9 test-mac-arm
[Finished] 211925c9 ursa-i9-9960x
[Finished] 211925c9 ursa-thinkcentre-m75q
[Finished] 37f5a358 ec2-t3-xlarge-us-east-2
[Finished] 37f5a358 test-mac-arm
[Finished] 37f5a358 ursa-i9-9960x
[Finished] 37f5a358 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

wjones127 deleted the ARROW-15812-csv-col-names branch January 9, 2023 15:38
EpsilonPrime pushed a commit to EpsilonPrime/arrow that referenced this pull request Jan 10, 2023
Lead-authored-by: Will Jones <willjones127@gmail.com>
Co-authored-by: Nic Crane <thisisnic@gmail.com>
Signed-off-by: Nic Crane <thisisnic@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL