| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Documentation: guides, the API reference, the changelog and migration guides.
Upgrading from CsvHelper 33? Version 34 is a rewrite: the migration guide goes through each concept with before and after code.
A CSV reader and writer for .NET, built for speed. It targets .NET 10 and .NET 8, and also runs on .NET Framework 4.7+ and .NET Standard 2.0 and 2.1 (see Target frameworks):
Benchmarks against Sep and Sylvan.Data.Csv are in bench/; results are in Benchmarks below.
For another cross-library view, see the .NET Matrix CSV Processing comparison.
dotnet add package CsvHelperOr in Visual Studio's Package Manager Console: Install-Package CsvHelper.
A summary. The documentation has the full guides, with examples of every mode, attribute and option: Reading, Records, Messy files and Writing.
using CsvHelper;
using var reader = CsvReader.FromFile("data.csv"); // or Create(stream), Create(byte[]), Create(TextReader), FromText(string)
int price = reader.GetFieldIndex("Price"); // header lookup: exact, then case/whitespace-insensitive
while (reader.Read())
{
string name = reader.GetString(0);
double value = reader.Parse<double>(price); // also int, long, decimal, bool, DateTime, Guid, ...
ReadOnlySpan<byte> raw = reader.GetSpan(2); // UTF-8, already unquoted and unescaped, no allocation
int qty = reader.Parse<int>("Quantity"); // by header name; on .NET 8+, any ISpanParsable<T>
TimeSpan t = reader.Parse("Duration", TimeSpan.Parse); // any parser of strings, on every target
}foreach/await foreach give a CsvRow with CsvField values (row[0].ToString(), row["Price"].Parse<decimal>()):
await using var reader = CsvReader.Create(stream);
await foreach (CsvRow row in reader)
Console.WriteLine(row["Name"].ToString());Spans and rows are valid until the next Read/ReadAsync. Async reads (ReadAsync, ReadHeaderAsync) only await when the buffer needs refilling; otherwise they complete synchronously without allocating.
ParallelEnumerate maps records to objects on several threads and returns them in input order:
using var reader = CsvReader.FromFile("data.csv");
List<Order> orders = reader.ParallelEnumerate(r => new Order(r.GetString("Id"), r.Parse<decimal>(1))).ToList();Workers take turns reading and scanning the next batch of records, then map that batch themselves while its bytes are still in their cache. Each worker gets a reader of its own, positioned on one record per call (don't call Read on it), and with PoolStrings the workers share one string pool. The input is read and checked one batch at a time in order, so malformed-record callbacks run one at a time in input order, and an exception is thrown after the results of the records before it.
Mode (on CsvReaderOptions and CsvWriterOptions) chooses how fields are delimited and escaped:
| Mode | Fields | Example |
|---|---|---|
| Rfc4180 (default) | Quoted fields can contain separators and line breaks; "" is a quote. | "a,b","say ""hi""" |
| Escape | No quoting: the escape character (Escape, \ by default) makes the next character text. | a\,b,say "hi" |
| NoEscape | No quoting or escaping: everything but the separator and line breaks is text. | a b,say "hi" |
| FixedWidth | No separators: columns of fixed widths (FieldWidths), padded with spaces. | a b say "hi" |
// MySQL/PostgreSQL style dumps: \, is a comma, \\ a backslash, \ before a line break keeps it in the field.
using var escaped = CsvReader.FromFile("dump.csv", new CsvReaderOptions { Mode = CsvMode.Escape });
// Fixed-width columns: 10, 30 and 8 characters wide. Values are trimmed of their padding.
var options = new CsvReaderOptions { Mode = CsvMode.FixedWidth, FieldWidths = [10, 30, 8], HasHeader = false };
using var reader = CsvReader.FromFile("accounts.txt", options);
while (reader.Read())
{
string id = reader.GetString(0);
decimal balance = reader.Parse<decimal>(2);
}Separator is one ASCII character. For a separator such as ||, or one that is not ASCII, read through a CsvSeparatorStream: it puts one byte (U+001F unless you choose another) in place of each separator, and the reader splits on it.
using var reader = CsvReader.Create(new CsvSeparatorStream(File.OpenRead("items.txt"), "||"));The reader gives the stream its quoting, escape and comment rules, so separators in quoted fields, after an escape character and in comment lines stay text, and overlapping ones are taken from the left (a|||b is a and |b). The stream works on the bytes in the reader's buffer with the reader's 64-byte SIMD blocks (one VPCOMPRESSB per block with AVX-512 VBMI2), so || files keep the UTF-8 fast path (benchmarks). Input that contains the replacement outside quoted fields throws CsvFormatException; UTF-16 and UTF-32 input is not supported.
Mark a partial class, struct or record [CsvRecord] and the CsvHelper source generator, which ships in the package, writes its mapping code at compile time. The columns are looked up once per reader, then each record is read with the same GetString and Parse<T> calls you would write by hand. There is no reflection, so it works with trimming and native AOT, and it runs as fast as hand-written code (CsvHelper_RecordsGenerated in the benchmarks).
[CsvRecord]
public partial class Order
{
public string Id { get; set; } = "";
[CsvColumn("Unit Price")]
public decimal Price { get; set; }
[CsvColumn(Optional = true)]
public int? Quantity { get; set; }
[CsvIgnore]
public decimal Total { get; set; }
}
[CsvRecord]
public partial record Line(string Sku, [CsvColumn(Index = 3)] double Weight, int Count = 1);
using var reader = CsvReader.FromFile("orders.csv");
foreach (Order order in reader.GetRecords<Order>()) { } // also GetRecordsAsync<T>() and ParallelEnumerate<T>()
while (reader.Read()) { Order order = reader.GetRecord<Order>(); } // the current record
foreach (Order order in reader.EnumerateRecords(new Order())) { } // one instance, refilled for every recordMembers. These are read:
[CsvIgnore] skips a member, and [CsvIgnoreBase] on a class the members of its base classes. The parameters of the constructor are read too. That constructor is the one with [CsvConstructor], else the parameterless one, else the one with the most parameters (positional records). Members the constructor sets are not set again.
Columns. A member reads the header column with its name, or with the name in [CsvColumn("name")]. As with GetFieldIndex, an exact match wins, then case and surrounding whitespace are ignored. [CsvColumn("Unit Price", "Price")] also accepts other names, tried in order. [CsvColumn(Index = n)] reads a column by position, which is needed when HasHeader = false, and [CsvColumn("Amount", NameIndex = 1)] the second of several columns with a name. GetRecords looks up the columns before it returns. If any are missing, it throws one KeyNotFoundException that names them all (ValidateHeader<T>() and TryValidateHeader<T>(out missing) check without reading). Members that are Optional, and constructor parameters with a default value, may be missing; they keep their default.
Fixed width. [CsvColumn(Index = n, Width = w)] on every column makes the type's fixed-width layout: FieldWidths = CsvRecordMapper.GetFieldWidths<T>() reads (and writes) it.
Types. These can be read:
Parsing uses the reader's FormatProvider. A member of any other type is a compile error until it gets [CsvIgnore].
Values. [CsvFormat("yyyyMMdd")] reads a date or time in an exact format ([CsvFormat("yyyy-MM-dd", "dd/MM/yyyy")] in either), [CsvCulture("de-DE")] with another culture, [CsvBooleanValues("Ja", "Nein")] with other true and false values. [CsvNullValues("NULL")] reads those values as null (the reader's NullValues apply to every Nullable<T> and string? member), [CsvDefault(0)] gives the value of an empty field, and [CsvConstant("v2")] gives a member a value without reading a column. [CsvValidate(nameof(IsValidAge))] checks the value with a static method of the type.
Errors. A record that cannot be read throws a CsvFormatException that names the record, line, member and field (Cannot read Order.Price from field 1 'Unit Price' of record 3: ...), with the cause as its InnerException. OnRecordError skips such records (it gets the row number, byte offset, exception and raw record) instead of ending the enumeration.
Types you cannot change. [CsvRecordMap(typeof(Supplier))] on a partial class maps a type of another library: the class's members stand in for the type's and carry its attributes. A hand-written CsvRecordMapper<T> subclass reads anything else; [CsvConverter(typeof(MyMapper))] on the type registers it, or CsvRecordMapper.Register<T> at startup.
Options. [CsvOptions(Separator = ';', HasHeader = false, Culture = "de-DE")] keeps a type's dialect with it; CsvReaderOptions.For<T>() and CsvWriterOptions.For<T>() return those options.
Dynamic records. Without a record type, GetRecords<dynamic>() reads each record as a CsvDynamicRecord whose members are its columns (record.Name; values are strings, and record.Parse<int>("Age") parses one). A record holds its bytes, not a dictionary of strings, and record.Name is bound once to its column for every record with that header, so reading them is 16-22x faster than ExpandoObjects (see Benchmarks). WriteRecords writes them back, and ExpandoObjects and dictionaries keyed by column name too.
Writing. writer.WriteRecords(orders) writes the header and then the records, in the columns the mapper reads and with the same names, formats, cultures, boolean and null values, enum names, constants and converters (an ICsvConverter<T> has a Write method too), so the file reads back. The generated record writer calls WriteField for each member, as fast as writing them by hand. WriteHeader<T>(), WriteRecord<T>(record) and WriteRecordsAsync<T> (for IEnumerable<T> and IAsyncEnumerable<T>) write parts of it, and HasHeader = false in the writer options leaves out the header; a hand-written CsvRecordWriter<T> writes types with a hand-written mapper.
The generator reports problems when you compile:
| Id | Problem |
|---|---|
| CSVH001 | The type, or a type that contains it, is not partial. |
| CSVH002 | A member's type cannot be read from a field. |
| CSVH003 | No constructor to use: the type is abstract or static, has no public or internal constructor, or several constructors have the most parameters. |
| CSVH004 | The type is generic, a ref struct, file-local or not accessible from its whole assembly, or it contains itself through nested records. |
| CSVH005 | Warning: [CsvColumn] is on a member that cannot be set. |
| CSVH006 | A required member has [CsvIgnore]. |
| CSVH007 | The project uses C# 8 or earlier. |
| CSVH008 | The fixed-width layout is not valid: a column has no width or two, a width has no index, or the type has nested records. |
| CSVH009 | A [CsvConverter] cannot be used: the wrong interface or base class, no suitable constructor, two on one member, or two assembly converters of one type. |
| CSVH010 | Warning: an attribute does not apply to its member, such as [CsvFormat] on an int, and is ignored. |
| CSVH011 | An attribute argument is not valid, such as a [CsvDefault] or [CsvConstant] that does not convert to the member's type, or a [CsvValidate] without a suitable method. |
| CSVH012 | A [CsvRecordMap] cannot map its type, or a member of the map class stands in for no member of the type. |
The generator needs C# 9 and a Roslyn 4.3 compiler or later (the .NET 6.0.400 SDK, Visual Studio 2022 17.3). A generated module initializer registers each mapper, so GetRecords<T>() finds it with no reflection. A project that references CsvHelper as a project instead of a package adds the generator itself: <ProjectReference Include="...\CsvHelper.Generators.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />.
| Situation | Behavior |
|---|---|
| Separator | Detected from the first lines (, ; tab |), scored by how consistently each appears per line. Excel's sep=; first line is honored and skipped. |
| Encoding | UTF-8 (BOM skipped). UTF-16/UTF-32 byte order marks are detected and transcoded. Encoding option for other code pages. |
| Line endings | \n, \r\n and lone \r, mixed freely. |
| Quotes | RFC 4180: a quote at the start of a field opens a quoted field (separators/line breaks inside are text, "" is a quote). |
| Stray quotes (5'10", ab"c) | Kept as text; they do not open a quoted region. |
| Text after a closing quote ("ab"c) | Appended: abc. |
| Unterminated quote | The field runs to the end of the input. |
| Empty lines | Skipped (SkipEmptyLines). |
| Ragged records | Allowed; missing fields read as empty (MissingField). |
| Header names | Exact match first, then case-insensitive and trimmed. Duplicates allowed (first wins; GetFieldIndex(name, n) finds the others). |
| Numbers | Invariant culture, fast UTF-8 paths for integers and floats; FormatProvider for other cultures. Booleans accept true/false, yes/no, y/n, t/f, 1/0. |
| Option | Values | Use it when |
|---|---|---|
| MalformedData | Recover (default), Throw, Skip | Decide what happens to records with bad quoting. Throw raises CsvFormatException at the bad record (earlier records are still readable). |
| OnMalformedRecord | callback | Log every malformed record (record number, reason, byte offset, raw text) whether it was recovered or skipped. |
| AllowLineBreaksInQuotes | true (default) / false | Set false for files with unbalanced quotes: every line break then ends the record, so one bad quote cannot swallow the rest of the file. |
| FieldCountMismatch | Allow (default), Throw, Skip | Enforce that records have the header's field count. |
| MissingField | ReturnEmpty (default), Throw | Accessing a field past the end of a short record. |
| TrimWhitespace | null (default: trim in FixedWidth mode only), true, false | Trim spaces/tabs around values; "quoted" still unquotes, and escaped whitespace is kept. TrimInsideQuotes trims inside the quotes too, and WhitespaceCharacters says what is trimmed. |
| ShouldSkipRecord, OnRecordError | callbacks | Skip records such as totals; skip records of a record type whose fields do not parse or fail their checks. |
| MaxRecordSize | null (default: no limit) | Stop a record that keeps growing, such as one after an unclosed quote. |
| TrackLineNumbers | false (default) / true | Report the line of each record and of errors (LineNumber). |
| IncludeValuesInExceptions | true (default) / false | Keep field values, which may be personal data, out of exception messages. |
| Mode | Rfc4180 (default), Escape, NoEscape, FixedWidth | NoEscape treats quotes as ordinary text (TSV exports, logs); see Parsing modes. |
| CommentPrefix | null (default) | Skip lines starting with e.g. #. |
| Separator, Quote, HasHeader, Encoding, FormatProvider | Override detection and defaults. | |
| PoolStrings | false (default) / true | Set true when you keep many records in memory: repeated values in a column then share one string instance, which saves memory and GC work (columns of mostly unique values switch this off by themselves). For strings that are used and dropped, the default is faster. |
using var writer = CsvWriter.ToFile("out.csv"); // or Create(stream), Create(TextWriter)
writer.WriteHeader("Id", "Name", "Price", "When");
foreach (var item in items)
{
writer.WriteField(item.Id); // numbers format straight to UTF-8
writer.WriteField(item.Name); // quoted only if it contains , " CR or LF
writer.WriteField(item.Price);
writer.WriteField(item.Timestamp); // DateTime/DateTimeOffset: ISO 8601 ("O") by default
writer.EndRecord(); // or await writer.EndRecordAsync()
}writer.WriteRecords(orders) writes the records of a [CsvRecord] type with its header, and writer.ParallelWriteRecords(orders) formats them on several threads, with the same output (see Writing from several threads). By default the output follows RFC 4180:
Separator can be changed (e.g. ; or tab); quoting then follows the same rules for that separator.
With CommentPrefix = '#', WriteComment writes comment lines, and a record whose first field starts with the prefix has that field quoted so it does not read as a comment. Injection = CsvInjection.Escape (or Strip, Throw) protects text fields that a spreadsheet would run as a formula (=1+2, @SUM(...)), but not numbers.
The other parsing modes write what the reader reads in those modes (records still end with CRLF or NewLine):
In Escape and NoEscape mode a record whose only field is empty is an empty line. Output is UTF-8 (optional BOM). I/O happens only when a record ends with the buffer mostly full, or on Flush/FlushAsync, so async writers never block.
The CsvHelper.Linq package (dotnet add package CsvHelper.Linq) is a LINQ provider for CSV. The reader runs what it can of a query itself: it reads only the members of the records that the query uses, filters records before it reads the rest, skips rows without parsing them and stops early.
using CsvHelper.Linq;
var orders = CsvQueryable.FromFile<Order>("orders.csv");
var big = orders.Where(o => o.Total > 100).Select(o => new { o.Id, o.Customer }).ToList(); // reads 3 columns
// Entity Framework Core is asked only for the customers that orders.csv has.
var report = orders.Join(db.Customers, o => o.CustomerId, c => c.Id, (o, c) => new { o.Id, c.Name }).ToList();
report.WriteCsv("report.csv"); // anonymous types too
orders.Delete(o => o.Status == Status.Cancelled); // the rest of the file is kept byte for byte| Target | Used by | Scanner |
|---|---|---|
| net10.0 | .NET 10 and later | SIMD: AVX-512, AVX2 or 128-bit vectors (SSE, NEON), chosen at startup |
| net8.0 | .NET 8 and 9 | as net10.0, without the AVX-512 VBMI2 compress of the index, whose intrinsics came with .NET 10 |
| net47 | .NET Framework 4.7 and later | 64-bit integers: the masks of each 64-byte block from eight ulongs, then the same pipeline |
| netstandard2.1 | .NET Core 3.0 to .NET 7, Mono, Unity 2021.2 and later | same as net47 |
| netstandard2.0 | everything else: .NET Core 2.x, Unity before 2021.2 | same as net47 |
net47 and netstandard2.0 depend on two Microsoft packages, System.Memory and Microsoft.Bcl.AsyncInterfaces; netstandard2.1, which has spans and async streams built in, only on System.Runtime.CompilerServices.Unsafe. Every build reads and writes exactly the same CSV: the tests run on .NET 10, on .NET 8 and on .NET Framework 4.8 (the net47 build, whose code the netstandard2.0 and netstandard2.1 builds share), and on .NET 10 the 64-bit scanner is also checked against the SIMD ones. The differences between the .NET builds (net10.0 and net8.0) and the others:
The net10.0 and net8.0 builds of CsvHelper and CsvHelper.Linq are marked IsAotCompatible and IsTrimmable, so the trimming and AOT analyzers check them on every build. Reading, writing and [CsvRecord] types work in trimmed and native AOT applications: the generated mappers are registered by a module initializer, not found with reflection. Two things need code generated at run time:
Correctness is checked by fuzzing the reader against an independent character-by-character reference parser for every vector width (512/256/128/scalar only), every input source (memory, chunked streams, strings, text readers, async) and tiny buffer sizes, plus writer → reader round trips.
CsvHelper against Sep 0.17.1 and Sylvan.Data.Csv 1.4.4, measured with BenchmarkDotNet 0.15.8 on .NET 10.0.12, Windows 11, AMD Ryzen 9 9950X3D (one core, AVX-512). Times are means per operation; lower is better. Sep is the baseline. CsvHelper runs with its default options, except where a row says otherwise.
Data sets (deterministic, generated by DataGenerator, 50,000 records each):
Scenarios: Rows finds the records only. Cols touches every field as a span, without creating strings. Records builds an object with 25 string properties per record and keeps them all. Strings creates a string for every field. Floats parses the 25 floats of each record and sums them. Write writes the same records, and every library produces byte-identical RFC 4180 output. Reads come from a MemoryStream unless the data column says otherwise. The Async variants use each library's async API.
| Data | Scenario | Sep | CsvHelper | Sylvan | CsvHelper vs Sep |
|---|---|---|---|---|---|
| PackageAssets | Rows | 2.80 ms | 2.01 ms | 4.79 ms | 1.40x faster |
| PackageAssets | RowsAsync | 3.04 ms | 2.02 ms | 5.23 ms | 1.51x faster |
| PackageAssets | Cols | 3.75 ms | 3.02 ms | 6.73 ms | 1.24x faster |
| PackageAssets | Records | 28.7 ms | 26.9 ms | 30.6 ms | 1.07x faster |
| PackageAssets | RecordsAsync | 28.4 ms | 26.5 ms | 31.2 ms | 1.07x faster |
| PackageAssets | Records, both pooling strings ¹ | 27.1 ms | 19.3 ms | 1.40x faster | |
| PackageAssets, from a string | Cols | 3.41 ms | 3.05 ms | 6.24 ms | 1.12x faster |
| PackageAssets, from a string | Records | 28.0 ms | 27.4 ms | 30.2 ms | 1.02x faster |
| PackageAssets, from a file | Cols | 9.12 ms | 4.05 ms | 8.95 ms | 2.25x faster |
| PackageAssets, from a file | ColsAsync | 22.4 ms | 5.15 ms | 13.4 ms | 4.35x faster |
| Floats | Floats | 19.9 ms | 12.2 ms | 48.3 ms | 1.62x faster |
| Floats | FloatsAsync | 19.7 ms | 13.6 ms | 48.4 ms | 1.45x faster |
| Quoted | Rows | 2.63 ms | 1.46 ms | 6.90 ms | 1.80x faster |
| Quoted | Strings | 6.17 ms | 5.60 ms | 11.4 ms | 1.10x faster |
| Quoted | StringsAsync | 6.43 ms | 4.79 ms | 11.9 ms | 1.34x faster |
| Data | Scenario | Sep | CsvHelper | Sylvan | CsvHelper vs Sep |
|---|---|---|---|---|---|
| PackageAssets | Write | 27.5 ms | 8.64 ms | 12.7 ms | 3.18x faster |
| PackageAssets | WriteAsync | 44.7 ms | 6.77 ms | 15.1 ms | 6.60x faster |
| PackageAssets | WriteRecords ² | 8.66 ms | |||
| PackageAssets | WriteRecordsAsync ² | 9.18 ms | |||
| Floats | Write | 86.3 ms | 23.7 ms | 62.5 ms | 3.64x faster |
| Quoted | Write | 11.0 ms | 5.28 ms | 5.80 ms | 2.08x faster |
¹ Sep with its per-column string pool (SepToString.PoolPerCol()), CsvHelper with PoolStrings = true.
² Objects of a [CsvRecord] type with 25 string fields: CsvHelper's WriteRecords with its generated record writer, which is 2% slower than writing the fields by hand (8.48 ms). Sep and Sylvan do not write objects.
Throughput: CsvHelper finds records at 8.0 GB/s (Sep 5.7 GB/s) and reads every field of PackageAssets at 5.3 GB/s (Sep 4.3 GB/s). It writes 1.9 GB/s of text records (Sep 0.58 GB/s) and 0.62 GB/s of floats (Sep 0.17 GB/s).
Records keeps 1.25 million strings in memory (62 MB allocated per run, the same for every library), so garbage collection takes most of its time, and the libraries end up close. Pooling strings (PoolStrings = true) cuts that to 18.4 MB (Sep's pool: 28.4 MB). It is off by default because for strings that are used and dropped (Strings) a pool lookup costs more than it saves. The writer allocates under 400 bytes per run (Sep 7–10 KB).
How the comparison is kept fair:
The parsing modes other than RFC 4180, on the PackageAssets records (50,000, 25 columns), on the same machine. Sep has no escape mode, and none of the other libraries reads or writes fixed widths, so each row names its own baseline.
| Mode | Scenario | Baseline | CsvHelper | Others | CsvHelper vs baseline |
|---|---|---|---|---|---|
| Escape | Rows | Sylvan 15.1 ms | 2.18 ms | 6.92x faster | |
| Escape | Cols | Sylvan 19.5 ms | 4.95 ms | 3.94x faster | |
| Escape | Records | Sylvan 42.3 ms | 22.1 ms | 1.92x faster | |
| Escape | Write | Sylvan 14.9 ms | 12.5 ms | 1.19x faster | |
| NoEscape | Rows | Sep 2.56 ms | 1.45 ms | Sylvan 2.97 ms | 1.77x faster |
| NoEscape | Cols | Sep 3.53 ms | 2.27 ms | Sylvan 4.93 ms | 1.55x faster |
| NoEscape | Records | Sep 27.7 ms | 19.2 ms | Sylvan 28.8 ms | 1.45x faster |
| NoEscape | Write | Sep 15.9 ms | 8.48 ms | 1.87x faster | |
| FixedWidth | Rows | ReadLine 3.39 ms | 1.91 ms | 1.77x faster | |
| FixedWidth | Cols | ReadLine 13.6 ms | 11.7 ms | 1.17x faster | |
| FixedWidth | Records | ReadLine 70.0 ms | 35.3 ms | 1.99x faster | |
| FixedWidth | Write | PadRight 16.7 ms | 11.8 ms | 1.42x faster |
Every library returns the same results here too, and every writer produces byte-identical output.
The PackageAssets and Quoted data with || in place of every comma (in the Quoted data, the commas inside quoted fields become || too), read through CsvSeparatorStream. Sep and Sylvan do not read such separators. The baseline is CsvHelper on the same data with commas, so the difference is the cost of the stream.
| Data | Scenario | CsvHelper, , | CsvHelper, || |
|---|---|---|---|
| PackageAssets | Rows | 2.06 ms | 3.82 ms |
| PackageAssets | Strings | 7.65 ms | 9.29 ms |
| Quoted | Rows | 1.52 ms | 2.51 ms |
| Quoted | Strings | 4.97 ms | 6.08 ms |
The stream alone replaces separators at 9.6 GB/s on PackageAssets and 6.9 GB/s on the Quoted data with AVX-512 VBMI2 (one VPCOMPRESSB per 64-byte block), and at 5.8 and 4.6 GB/s with the code for AVX2 and SSSE3 on the same machine (a byte shuffle per 8 bytes, which for a separator of two bytes also writes the replacement). It allocates about 300 bytes per stream.
The PackageAssets data read as dynamic records, without a record type (see Dynamic records). Dynamic reads 3 of the 25 fields of every record by name (record.Id, record.Version, record.FileName); DynamicToList keeps every record in a list. They are compared with the usual way to get dynamic records: an ExpandoObject per record, with a string per field added by header name (filled by CsvHelper's reader, so the difference is the record).
| Scenario | CsvHelper | ExpandoObject | CsvHelper vs ExpandoObject |
|---|---|---|---|
| Dynamic | 5.05 ms | 82.1 ms | 16.3x faster |
| Dynamic, from a byte[] | 3.87 ms | ||
| DynamicToList | 5.41 ms | 119 ms | 22.0x faster |
CsvHelper reads dynamic records at 3.2 GB/s (4.1 GB/s from a byte[]). It allocates 29.7 MB per run in Dynamic (14.3 MB from a byte[], whose records reference the input in place; most of the rest are the 150,000 strings of the fields read), against 136 MB for the ExpandoObjects. Kept in a list, its 50,000 records take 23.6 MB and the ExpandoObjects 137 MB. The ExpandoObject times vary more than CsvHelper's (a standard deviation of 9 ms in Dynamic).
ParallelWriteRecords against WriteRecords on the same [CsvRecord] objects (see Writing from several threads), with byte-identical output. Unlike the other benchmarks, these run on all 32 threads of the same machine (16 cores). The data are the PackageAssets records, the Floats records (objects with 25 float fields) and Orders (9.5 MB): 12 fields of dates, a decimal, a Guid, numbers and text. 50,000 records each. The 4, 8 and 16 thread columns set maxDegreeOfParallelism; 32 threads is the default, one per processor.
| Data | WriteRecords | 4 threads | 8 threads | 16 threads | 32 threads |
|---|---|---|---|---|---|
| PackageAssets | 8.23 ms | 2.84 ms (2.9x) | 1.71 ms (4.8x) | 1.84 ms (4.5x) | 1.32 ms (6.2x) |
| Floats | 23.6 ms | 7.16 ms (3.3x) | 3.97 ms (5.9x) | 2.31 ms (10.2x) | 1.62 ms (14.6x) |
| Orders | 6.50 ms | 2.29 ms (2.8x) | 1.29 ms (5.1x) | 0.90 ms (7.2x) | 0.85 ms (7.7x) |
To a file (CsvWriter.ToFile, on a WD Blue SN5000 NVMe drive), with 32 threads:
| Data | WriteRecords | ParallelWriteRecords |
|---|---|---|
| PackageAssets | 12.0 ms | 5.59 ms (2.2x) |
| Floats | 28.2 ms | 5.13 ms (5.5x) |
| Orders | 8.78 ms | 3.54 ms (2.5x) |
To memory, the parallel writer reaches 12.1 GB/s on PackageAssets, 9.1 GB/s on Floats and 11.3 GB/s on Orders. To a file, all three end up at 2.7 to 2.9 GB/s: writing the file is then the limit, so the records that cost the most to format gain the most. A call allocates about 42 KB with 32 workers (their writers and output buffers), against 416 bytes for WriteRecords.
Run them with:
cd bench/CsvHelper.Benchmarks
dotnet run -c Release -- --filter *PackageAssetsRead.* *PackageAssetsReadFromString.* *PackageAssetsReadFile.* *FloatsRead.* *QuotedRead.* *PackageAssetsWrite.* *FloatsWrite.* *QuotedWrite.*
dotnet run -c Release -- --filter *.EscapeRead.* *.NoEscapeRead.* *.FixedWidthRead.* *.EscapeWrite.* *.NoEscapeWrite.* *.FixedWidthWrite.*
dotnet run -c Release -- --filter *SeparatorPackageAssetsRead.* *SeparatorQuotedRead.* --anyCategories Rows Strings
dotnet run -c Release -- --filter *ParallelWrite.* # on every core
dotnet run -c Release -- verify # check that all libraries return the same results
dotnet run -c Release -- quick QuotedRead # quick Stopwatch runner for tuning (not for published numbers)dotnet build -c Release
dotnet test tests/CsvHelper.Tests -c Release # .NET 10, .NET 8 and .NET Framework 4.8 (Windows)
dotnet test tests/CsvHelper.Tests -c Release -f net10.0 # .NET 10 only
pwsh docs/build.ps1 # the documentation site, in docs/_site
pwsh docs/build.ps1 -Preview 34.0.0 # the site as it will be once v34.0.0 is tagged
pwsh docs/build.ps1 -h # all of its optionsCompletely free for commercial use. Dual licensed under the Microsoft Public License (MS-PL) and the Apache License, Version 2.0: choose whichever suits your needs. See LICENSE.txt.
Want to contribute? Great! Here are a few guidelines.
This project exists thanks to all the people who contribute.
You can make one-time or recurring donations through GitHub Sponsors.
A huge thanks to the .NET on AWS Open Source Software Fund for sponsoring CsvHelper!
Thanks to Microsoft for being a sponsor!
| Back | FazBrowse Home | New Git URL |