Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/softcircuits/csvparser
Classes for reading and writing comma-separated-values (CSV) files. Supports special characters in column fields. Options to instead use tab delimiters, etc.
https://github.com/softcircuits/csvparser
csv csv-parser csv-parsing tsv tsv-parser tsv-reader
Last synced: 3 months ago
JSON representation
Classes for reading and writing comma-separated-values (CSV) files. Supports special characters in column fields. Options to instead use tab delimiters, etc.
- Host: GitHub
- URL: https://github.com/softcircuits/csvparser
- Owner: SoftCircuits
- License: mit
- Created: 2019-07-09T01:14:08.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-07-12T00:10:53.000Z (7 months ago)
- Last Synced: 2024-10-14T07:54:00.320Z (4 months ago)
- Topics: csv, csv-parser, csv-parsing, tsv, tsv-parser, tsv-reader
- Language: C#
- Size: 759 KB
- Stars: 14
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: ReadMe.md
- License: License.txt
Awesome Lists containing this project
README
# CSV Parser
[](https://www.nuget.org/packages/SoftCircuits.CsvParser/)
```
Install-Package SoftCircuits.CsvParser
```
## Overview
CsvParser is a .NET library that makes it easy to work with comma-separated-value (CSV) files. (And can be customized to work with other delimiters). CsvParser includes basic classes to read and write CSV data, and also higher-level classes that automatically map class properties to CSV columns. The library correctly handles column values that contain embedded commas, quotes or other special characters. It even supports column values that span multiple lines. CsvParser is very fast and is designed to handle large data files efficiently. The object mapping classes run up to four times faster than the popular CsvHelper library.
## CsvWriter and CsvReader Classes
These classes provide the simplest way to read and write CSV files. The example below writes several records of data to a CSV file.
```cs
// Write some data to disk
using CsvWriter writer = new(path);
// Write header
writer.Write("Name", "Email", "Phone", "Birthday");
// Write data
writer.Write("Bill Smith", "[email protected]", "555-1234", "10/29/1982 12:00:00 AM");
writer.Write("Susan Carpenter", "[email protected]", "555-2345", "2/17/1995 12:00:00 AM");
writer.Write("Jim Windsor", "[email protected]", "555-3456", "4/23/1989 12:00:00 AM");
writer.Write("Jill Morrison", "[email protected]", "555-4567", "5/2/1979 12:00:00 AM");
writer.Write("Gary Wright", "[email protected]", "555-5678", "2/18/1984 12:00:00 AM");
```
Note that the `CsvWriter.Write()` method accepts any number of string parameters. It is also overloaded to handle `string[]` and `IEnumerable` arguments. (Note: The legacy `WriteRow()` and `WriteRowAsync()` methods can be used to accomplish the same task but `Write()` and `WriteAsync()` are preferred.)
The next example reads all records from a CSV file.
```cs
// Read the data from disk
using CsvReader reader = new(path);
while (reader.Read())
Console.WriteLine(string.Join(", ", reader.Columns));
```
When the `CsvReader.Read()` method returns true, the `Columns` property holds the values that were read. This method returns false when the end of the file has been reached. (Note: The legacy `ReadRow()` and `ReadRowAsync()` methods can be used to accomplish the same task but `Read()` and `ReadAsync()` are preferred and have slightly better performance.)
## CsvWriter<T> and CsvReader<T> Classes
These are higher level classes that will automatically map data between class properties and CSV columns. The following example defines a class, declares a collection with several instances of that class, then uses `CsvWriter` to write the data to a CSV file, and `CsvReader` to read it back again.
```cs
// This class will represent the data in the CSV file
class Person
{
public string Name { get; set; }
public string Email { get; set; }
public string Phone { get; set; }
public DateTime Birthday { get; set; }
}
// Define some sample data
List People = new()
{
new Person { Name = "Bill Smith", Email = "[email protected]", Phone = "555-1234", Birthday = new DateTime(1982, 10, 29) },
new Person { Name = "Susan Carpenter", Email = "[email protected]", Phone = "555-2345", Birthday = new DateTime(1995, 2, 17) },
new Person { Name = "Jim Windsor", Email = "[email protected]", Phone = "555-3456", Birthday = new DateTime(1989, 4, 23) },
new Person { Name = "Jill Morrison", Email = "[email protected]", Phone = "555-4567", Birthday = new DateTime(1979, 5, 2) },
new Person { Name = "Gary Wright", Email = "[email protected]", Phone = "555-5678", Birthday = new DateTime(1984, 2, 18) },
};
// Write the data to disk
// Note: Since all records are already in memory, you could replace the
// foreach loop with: writer.Write(People)
using (CsvWriter writer = new(path))
{
writer.WriteHeaders();
foreach (Person person in People)
writer.Write(person);
}
// Read the data from disk
List people = new();
using (CsvReader reader = new(path))
{
// Read header and use to determine column order
reader.ReadHeaders(true);
// Read data
Person? person;
while ((person = reader.Read()) != null)
people.Add(person);
}
```
It is important to note in the above example where the code that writes the data calls `CsvWriter.WriteHeaders()`. This writes a row with the name of each column. (The library gets the column names from the properties of the `Person` class.) The code that reads the data calls `CsvReader.ReadHeaders()` to read that header data. Because the argument to `CsvReader.ReadHeaders()` is `true`, this tells the code to use the header data to determine how to map the columns. For example, it can determine the column order and also detect if one or more properties are not mapped to any column.
If you can be sure the CSV file being read was created using the code above, the argument to `CsvReader.ReadHeaders()` could be false because you could be confident that the columns would be in the order expected, etc. But if someone else is supplying the CSV file, setting the `CsvReader.ReadHeaders()` argument to `true` would allow it to work if the supplier put the columns in a different order.
Correctly mapping the class properties to the CSV columns is critical for these classes to work correctly. Here, the code maps the class properties to columns based on the column headers. The following sections will discuss other ways to map class properties to columns.
## ColumnMap Attribute
The `ColumnMapAttribute` can be applied to any class property or field to specify how that property or field is mapped to a CSV column. This attribute accepts any of the following arguments:
- **Name:**
Specifies a column name, allowing the column name to be different from the class property name.
- **Index:**
Specifies a property's 0-based column position. To ensure expected results, it is generally best to set the column Index for all properties when setting this property.
- **Exclude:**
Specifies whether or not the class property should be excluded, and not written to or read from any column.
- **ConverterType:**
Data converters convert individual class properties to strings and back again from strings to class properties. The CsvParser library includes converters for all basic data types (including `Guid` and `DateTime`), basic nullable data types, basic data type arrays and basic nullable data type arrays. But you can override the data converter used for any class property. For example, you might want to write your own data converter to support custom property types, or when you are working with data not formatted as expected by the built-in data converters. A good example of this are `DateTime` properties because there are so many ways to format date and time values.
To override a data converter, create a class that implements the `IDataConverter` interface. The easiest way to do this in a type-safe manner is to derive your class from `DataConverter`, where `T` is the type of the property you are converting. The `DataConverter` class has two abstract methods, `ConvertToString()` and `TryConvertFromString()`, which must be overridden in your derived class.
Finally, set the `ConverterType` argument of the property's `ColumnMap` attribute to your custom convert class type. Note that if you set this property to a type that does not implement `IDataConverter`, an `ArgumentOutOfRangeException` exception is thrown at runtime.
The example below uses the `ColumnMap` attribute to customize the `Person` class. It sets the `Index` properties such that the CSV columns appear in the reverse order from how the properties are declared in the class, it excludes the `Phone` property, and it causes the `Birthday` header to use the name *DOB*. It also specifies a custom converter for the `Birthday` property that stores the date (no time) in a very compact format.
```cs
// Add column mapping attributes to our data class
class Person
{
[ColumnMap(Index = 2)]
public string Name { get; set; }
[ColumnMap(Index = 1)]
public string Email { get; set; }
[ColumnMap(Exclude = true)]
public string Phone { get; set; }
[ColumnMap(Index = 0, Name = "DOB", ConverterType = typeof(DateTimeConverter))]
public DateTime Birthday { get; set; }
}
// Create a custom data converter for DateTime values
// Stores a date-only value (no time) in a very compact format
class DateTimeConverter : DataConverter
{
public override string ConvertToString(DateTime value)
{
int i = ((value.Day - 1) & 0x1f) |
(((value.Month - 1) & 0x0f) << 5) |
(value.Year) << 9;
return i.ToString("x");
}
public override bool TryConvertFromString(string s, out DateTime value)
{
try
{
int i = Convert.ToInt32(s, 16);
value = new DateTime(i >> 9, ((i >> 5) & 0x0f) + 1, (i & 0x1f) + 1);
return true;
}
catch (Exception)
{
value = DateTime.Now;
return false;
}
}
}
using (CsvWriter writer = new(path))
{
writer.WriteHeaders();
foreach (Person person in People)
writer.Write(person);
}
// Read the data from disk
List people = new();
using (CsvReader reader = new(path))
{
// Read header and use to determine column order
reader.ReadHeaders(false);
// Read data
Person? person;
while ((person = reader.Read()) != null)
people.Add(person);
}
```
Note that the code above will work correctly without the calls to the `CsvWriter.WriteHeaders()` and `CsvReader.ReadHeaders()` methods. We still included them for the benefit of anyone looking at the contents of the CSV file. But they are optional here because we used the `ColumnMap` attribute to provide enough information about column order, etc. You can also see that we passed `false` to the `CsvReader.ReadHeaders()` method. Again, this is because we have all the column information we need. However, if the file is being supplied by someone else, as described earlier, you might want to change this argument to `true`, so that it can handle unexpected cases such as where columns are in a different order or are omitted. (If `true` is passed to `CsvReader.ReadHeaders()` here, it would override any existing `Index` and `Exclude` mapping properties.)
Also note that, in the collection read back from disk, the `Phone` property will always be `null` because we excluded that property and it was not written to the CSV file.
## MapColumns() Method
In some cases, you may want to set `ColumnMap` attributes for a class you cannot directly modify. For example, the class might be part of a library you are using and you don't have the source code. In these cases, you can use the `CsvWriter.MapColumns()` and `CsvReader.MapColumns()` methods.
The example below creates a custom class that derives from `ColumnMaps`, where `T` is the type of class being written or read. The constructor of this class must call `MapColumn()` for each class property that it maps. This method supports a fluent interface to set the various mapping properties for each class property.
The code that writes the CSV file calls the `CsvWriter.MapColumns()` method to register the custom mappings before any data is written. The code that reads the CSV file calls the `CsvReader.MapColumns()` method in the same way. Both must use the same mapping in order for the data to be interpreted correctly. The easiest way to do this is to pass the same class to both methods.
```cs
// Create our custom mapping class
class PersonMaps : ColumnMaps
{
public PersonMaps()
{
// Note that only those properties set, and only those columns referenced
// will be modified. All columns and settings not referenced here retain
// their previous values.
MapColumn(m => m.Name).Index(2);
MapColumn(m => m.Email).Index(1);
MapColumn(m => m.Phone).Exclude(true);
MapColumn(m => m.Birthday).Index(0).Name("DOB").Converter(new DateTimeConverter());
}
}
// Write data to disk
using (CsvWriter writer = new(path))
{
// Register our custom mapping
writer.MapColumns();
writer.WriteHeaders();
foreach (Person person in People)
writer.Write(person);
}
// Read data from disk
List people = new();
using (CsvReader reader = new(path))
{
// Register our custom mapping
reader.MapColumns();
// Read header
reader.ReadHeaders(false);
// Read data
Person? person;
while ((person = reader.Read()) != null)
people.Add(person);
}
```
This example does exactly the same thing as the previous example but without modifying the `Person` class.
## CsvSettings Class
You can customize the way the library behaves by passing your own instance of the `CsvSettings` class to any of the reader or writer constructors, as demonstrated in the following example. This code uses the `CsvSettings` class to read a tab-separated-values (TSV) file. It sets the `ColumnDelimiter` property to a tab. It also sets it to use single quotes instead of double quotes (something not useful very often, but is fully supported).
```cs
// Set custom settings
CsvSettings settings = new()
{
ColumnDelimiter = '\t',
QuoteCharacter = '\''
};
// Apply custom settings to CsvReader
using CsvReader reader = new(path, settings);
while (reader.Read())
Console.WriteLine(string.Join(", ", reader.Columns));
```
## Additional Information
This code was originally derived from the article [Reading and Writing CSV Files in C#](http://www.blackbeltcoder.com/Articles/files/reading-and-writing-csv-files-in-c).