FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

fread problem with different number of columns · Issue #1812 · Rdatatable/data.table · GitHub

fread problem with different number of columns #1812

Description

hi there,
I've got a problem importing files where some lines end with double tab:

If the file's first line has extra columns and then other lines have missing columns, then fill=TRUE solves the problem.
However i have one file where the extra columns happen in line 82:

> test[80:84] [1] "1053982\t2014-11-30\t-1\ttimetak\t14" "1053982\t2014-11-30\t-1\tyearcompleted\t4" [3] "1053982\t24-11-2014\t-1\tAMOUNT OTHER WEBSITES\t6\t\t" "1053982\t24-11-2014\t-1\tBBC1_422\t1\t\t" [5] "1053982\t24-11-2014\t-1\tBBC1_501\t13\t\t"

In this case fill=TRUE isn't enough to get around the problem, and i get an error message saying there's text after processing all cols:

> test<-fread(filenames[9], header = FALSE, fill = TRUE, data.table=FALSE, quote='') # problem as double tab only starts in row 82
Error in fread(filenames[9], header = FALSE, fill = TRUE, data.table = FALSE,  : 
  Expecting 5 cols, but line 82 contains text after processing all cols. Try again with fill=TRUE. Another reason could be that fread's logic in distinguishing one or more fields having embedded sep='    ' and/or (unescaped) '\n' characters within unbalanced unescaped quotes has failed. If quote='' doesn't help, please file an issue to figure out if the logic could be improved.

Is there a way i could force fread to either only read the first 5 columns per line and dropping anything after that?
Failing that, can i specify the number of columns in advance to account for the lines with extra tabs, as this way fill=TRUE would work?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions


      Back | FazBrowse Home | New Git URL