{"id":13493773,"url":"https://github.com/dfurtado/dataclass-csv","last_synced_at":"2025-04-04T15:08:01.999Z","repository":{"id":45434157,"uuid":"159055716","full_name":"dfurtado/dataclass-csv","owner":"dfurtado","description":"Map CSV to Data Classes","archived":false,"fork":false,"pushed_at":"2024-01-30T15:10:52.000Z","size":97,"stargazers_count":190,"open_issues_count":11,"forks_count":18,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-05-01T15:23:24.260Z","etag":null,"topics":["csv","csv-parser","dataclass","dataclasses","python","python-dataclasses","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dfurtado.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.md","dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["dfurtado"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2018-11-25T17:15:01.000Z","updated_at":"2024-06-18T18:20:03.162Z","dependencies_parsed_at":"2024-06-18T18:19:52.705Z","dependency_job_id":"efbbd75b-8eee-45d7-8d9f-3bfffcb58264","html_url":"https://github.com/dfurtado/dataclass-csv","commit_stats":{"total_commits":75,"total_committers":9,"mean_commits":8.333333333333334,"dds":"0.30666666666666664","last_synced_commit":"2dc71be81cb253eb10aba5ba70c6cebe42ab0301"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfurtado%2Fdataclass-csv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfurtado%2Fdataclass-csv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfurtado%2Fdataclass-csv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfurtado%2Fdataclass-csv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dfurtado","download_url":"https://codeload.github.com/dfurtado/dataclass-csv/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247198459,"owners_count":20900080,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","csv-parser","dataclass","dataclasses","python","python-dataclasses","python3"],"created_at":"2024-07-31T19:01:18.666Z","updated_at":"2025-04-04T15:08:01.971Z","avatar_url":"https://github.com/dfurtado.png","language":"Python","readme":"[![Build Status](https://travis-ci.org/dfurtado/dataclass-csv.svg?branch=master)](https://travis-ci.org/dfurtado/dataclass-csv)\n[![pypi](https://img.shields.io/pypi/v/dataclass-csv.svg)](https://pypi.python.org/pypi/dataclass-csv)\n[![Downloads](https://pepy.tech/badge/dataclass-csv)](https://pepy.tech/project/dataclass-csv)\n\n\n\n# Dataclass CSV\n\nDataclass CSV makes working with CSV files easier and much better than working with Dicts. It uses Python's Dataclasses to store data of every row on the CSV file and also uses type annotations which enables proper type checking and validation.\n\n\n## Main features\n\n- Use `dataclasses` instead of dictionaries to represent the rows in the CSV file.\n- Take advantage of the `dataclass` properties type annotation. `DataclassReader` use the type annotation to perform validation of the data of the CSV file.\n- Automatic type conversion. `DataclassReader` supports `str`, `int`, `float`, `complex`, `datetime` and `bool`, as well as any type whose constructor accepts a string as its single argument.\n- Helps you troubleshoot issues with the data in the CSV file. `DataclassReader` will show exactly in which line of the CSV file contain errors.\n- Extract only the data you need. It will only parse the properties defined in the `dataclass`\n- Familiar syntax. The `DataclassReader` is used almost the same way as the `DictReader` in the standard library.\n- It uses `dataclass` features that let you define metadata properties so the data can be parsed exactly the way you want.\n- Make the code cleaner. No more extra loops to convert data to the correct type, perform validation, set default values, the `DataclassReader` will do all this for you.\n- In additon of the `DataclassReader` the library also provides a `DataclassWriter` which enables creating a CSV file\nusing a list of instances of a dataclass.\n\n\n## Installation\n\n```shell\npipenv install dataclass-csv\n```\n\n## Getting started\n\n## Using the DataclassReader\n\nFirst, add the necessary imports:\n\n```python\nfrom dataclasses import dataclass\n\nfrom dataclass_csv import DataclassReader\n```\n\nAssuming that we have a CSV file with the contents below:\n```text\nfirstname,email,age\nElsa,elsa@test.com, 11\nAstor,astor@test.com, 7\nEdit,edit@test.com, 3\nElla,ella@test.com, 2\n```\n\nLet's create a dataclass that will represent a row in the CSV file above:\n```python\n@dataclass\nclass User:\n    firstname: str\n    email: str\n    age: int\n```\n\nThe dataclass `User` has 3 properties, `firstname` and `email` is of type `str` and `age` is of type `int`.\n\nTo load and read the contents of the CSV file we do the same thing as if we would be using the `DictReader` from the `csv` module in the Python's standard library. After opening the file we create an instance of the `DataclassReader` passing two arguments. The first is the `file` and the second is the dataclass that we wish to use to represent the data of every row of the CSV file. Like so:\n\n```python\nwith open(filename) as users_csv:\n    reader = DataclassReader(users_csv, User)\n    for row in reader:\n        print(row)\n```\n\nThe `DataclassReader` internally uses the `DictReader` from the `csv` module to read the CSV file which means that you can pass the same arguments that you would pass to the `DictReader`. The complete argument list is shown below:\n\n```python\ndataclass_csv.DataclassReader(\n    f,\n    cls,\n    fieldnames=None,\n    restkey=None,\n    restval=None,\n    dialect='excel',\n    *args,\n    **kwds\n)\n```\n\nAll keyword arguments support by `DictReader` are supported by the `DataclassReader`, with the addition of:\n\n`validate_header` - The `DataclassReader` will raise a `ValueError` if the CSV file cointain columns with the same name. This\nvalidation is performed to avoid data being overwritten. To skip this validation set `validate_header=False` when creating a\ninstance of the `DataclassReader`, see an example below:\n\n```python\nreader = DataclassReader(f, User, validate_header=False)\n```\n\nIf you run this code you should see an output like this:\n\n```python\nUser(firstname='Elsa', email='elsa@test.com', age=11)\nUser(firstname='Astor', email='astor@test.com', age=7)\nUser(firstname='Edit', email='edit@test.com', age=3)\nUser(firstname='Ella', email='ella@test.com', age=2)\n```\n\n### Error handling\n\nOne of the advantages of using the `DataclassReader` is that it makes it easy to detect when the type of data in the CSV file is not what your application's model is expecting. And, the `DataclassReader` shows errors that will help to identify the rows with problem in your CSV file.\n\nFor example, say we change the contents of the CSV file shown in the **Getting started** section and, modify the `age` of the user Astor, let's change it to a string value:\n\n```text\nAstor, astor@test.com, test\n```\n\nRemember that in the dataclass `User` the `age` property is annotated with `int`. If we run the code again an exception will be raised with the message below:\n\n```text\ndataclass_csv.exceptions.CsvValueError: The field `age` is defined as \u003cclass 'int'\u003e but\nreceived a value of type \u003cclass 'str'\u003e. [CSV Line number: 3]\n```\n\nNote that apart from telling what the error was, the `DataclassReader` will also show which line of the CSV file contain the data with errors.\n\n### Default values\n\nThe `DataclassReader` also handles properties with default values. Let's modify the dataclass `User` and add a default value for the field `email`:\n\n```python\nfrom dataclasses import dataclass\n\n\n@dataclass\nclass User:\n    firstname: str\n    email: str = 'Not specified'\n    age: int\n```\n\nAnd we modify the CSV file and remove the email for the user Astor:\n\n```python\nAstor,, 7\n```\n\nIf we run the code we should see the output below:\n\n```text\nUser(firstname='Elsa', email='elsa@test.com', age=11)\nUser(firstname='Astor', email='Not specified', age=7)\nUser(firstname='Edit', email='edit@test.com', age=3)\nUser(firstname='Ella', email='ella@test.com', age=2)\n```\n\nNote that now the object for the user Astor have the default value `Not specified` assigned to the email property.\n\nDefault values can also be set using `dataclasses.field` like so:\n\n```python\nfrom dataclasses import dataclass, field\n\n\n@dataclass\nclass User:\n    firstname: str\n    email: str = field(default='Not specified')\n    age: int\n```\n\n### Mapping dataclass fields to columns\n\nThe mapping between a dataclass property and a column in the CSV file will be done automatically if the names match, however, there are situations that the name of the header for a column is different. We can easily tell the `DataclassReader` how the mapping should be done using the method `map`. Assuming that we have a CSV file with the contents below:\n\n```text\nFirst Name,email,age\nElsa,elsa@test.com, 11\n```\n\nNote that now, the column is called **First Name** and not **firstname**\n\nAnd we can use the method `map`, like so:\n\n```python\nreader = DataclassReader(users_csv, User)\nreader.map('First name').to('firstname')\n```\n\nNow the DataclassReader will know how to extract the data from the column **First Name** and add it to the to dataclass property **firstname**\n\n### Supported type annotation\n\nAt the moment the `DataclassReader` support `int`, `str`, `float`, `complex`, `datetime`, and `bool`. When defining a `datetime` property, it is necessary to use the `dateformat` decorator, for example:\n\n```python\nfrom dataclasses import dataclass\nfrom datetime import datetime\n\nfrom dataclass_csv import DataclassReader, dateformat\n\n\n@dataclass\n@dateformat('%Y/%m/%d')\nclass User:\n    name: str\n    email: str\n    birthday: datetime\n\n\nif __name__ == '__main__':\n\n    with open('users.csv') as f:\n        reader = DataclassReader(f, User)\n        for row in reader:\n            print(row)\n```\n\nAssuming that the CSV file have the following contents:\n\n```text\nname,email,birthday\nEdit,edit@test.com,2018/11/23\n```\n\nThe output would look like this:\n\n```text\nUser(name='Edit', email='edit@test.com', birthday=datetime.datetime(2018, 11, 23, 0, 0))\n```\n\n### Fields metadata\n\nIt is important to note that the `dateformat` decorator will define the date format that will be used to parse date to all properties\nin the class. Now there are situations where the data in a CSV file contains two or more columns with date values in different formats. It is possible\nto set a format specific for every property using the `dataclasses.field`. Let's say that we now have a CSV file with the following contents:\n\n```text\nname,email,birthday, create_date\nEdit,edit@test.com,2018/11/23,2018/11/23 10:43\n```\n\nAs you can see the `create_date` contains time information as well.\n\nThe `dataclass` User can be defined like this:\n\n```python\nfrom dataclasses import dataclass, field\nfrom datetime import datetime\n\nfrom dataclass_csv import DataclassReader, dateformat\n\n\n@dataclass\n@dateformat('%Y/%m/%d')\nclass User:\n    name: str\n    email: str\n    birthday: datetime\n    create_date: datetime = field(metadata={'dateformat': '%Y/%m/%d %H:%M'})\n```\n\nNote that the format for the `birthday` field was not speficied using the `field` metadata. In this case the format specified in the `dateformat`\ndecorator will be used.\n\n### Handling values with empty spaces\n\nWhen defining a property of type `str` in the `dataclass`, the `DataclassReader` will treat values with only white spaces as invalid. To change this\nbehavior, there is a decorator called `@accept_whitespaces`. When decorating the class with the `@accept_whitespaces` all the properties in the class\nwill accept values with only white spaces.\n\nFor example:\n\n```python\nfrom dataclass_csv import DataclassReader, accept_whitespaces\n\n@accept_whitespaces\n@dataclass\nclass User:\n    name: str\n    email: str\n    birthday: datetime\n    created_at: datetime\n```\n\nIf you need a specific field to accept white spaces, you can set the property `accept_whitespaces` in the field's metadata, like so:\n\n```python\n@dataclass\nclass User:\n    name: str\n    email: str = field(metadata={'accept_whitespaces': True})\n    birthday: datetime\n    created_at: datetime\n```\n\n### User-defined types\n\nYou can use any type for a field as long as its constructor accepts a string:\n\n```python\nclass SSN:\n    def __init__(self, val):\n        if re.match(r\"\\d{9}\", val):\n            self.val = f\"{val[0:3]}-{val[3:5]}-{val[5:9]}\"\n        elif re.match(r\"\\d{3}-\\d{2}-\\d{4}\", val):\n            self.val = val\n        else:\n            raise ValueError(f\"Invalid SSN: {val!r}\")\n\n\n@dataclasses.dataclass\nclass User:\n    name: str\n    ssn: SSN\n```\n\n\n## Using the DataclassWriter\n\nReading a CSV file using the `DataclassReader` is great and gives us the type-safety of Python's dataclasses and type annotation, however, there are situations where we would like to use dataclasses for creating CSV files, that's where the `DataclassWriter` comes in handy.\n\nUsing the `DataclassWriter` is quite simple. Given that we have a dataclass `User`:\n\n```python\nfrom dataclasses import dataclass\n\n\n@dataclass\nclass User:\n    firstname: str\n    lastname: str\n    age: int\n```\n\nAnd in your program we have a list of users:\n\n```python\n\nusers = [\n    User(firstname=\"John\", lastname=\"Smith\", age=40),\n    User(firstname=\"Daniel\", lastname=\"Nilsson\", age=10),\n    User(firstname=\"Ella\", \"Fralla\", age=4)\n]\n```\n\nIn order to create a CSV using the `DataclassWriter` import it from `dataclass_csv`:\n\n```python\nfrom dataclass_csv import DataclassWriter\n```\n\nInitialize it with the required arguments and call the method `write`:\n\n```python\nwith open(\"users.csv\", \"w\") as f:\n    w = DataclassWriter(f, users, User)\n    w.write()\n```\n\nThat's it! Let's break down the snippet above.\n\nFirst, we open a file called `user.csv` for writing. After that, an instance of the `DataclassWriter` is created. To create a `DataclassWriter` we need to pass the `file`, the list of `User` instances, and lastly, the type, which in this case is `User`.\n\nThe type is required since the writer uses it when trying to figure out the CSV header. By default, it will use the names of the\nproperties defined in the dataclass, in the case of the dataclass `User` the title of each column\nwill be `firstname`, `lastname` and `age`.\n\nSee below the CSV created out of a list of `User`:\n\n```text\nfirstname,lastname,age\nJohn,Smith,40\nDaniel,Nilsson,10\nElla,Fralla,4\n```\n\nThe `DataclassWriter` also takes a `**fmtparams` which accepts the same parameters as the `csv.writer`, for more\ninformation see: https://docs.python.org/3/library/csv.html#csv-fmt-params\n\nNow, there are situations where we don't want to write the CSV header. In this case, the method `write` of\nthe `DataclassWriter` accepts an extra argument, called `skip_header`. The default value is `False` and when set to\n`True` it will skip the header.\n\n#### Modifying the CSV header\n\nAs previously mentioned the `DataclassWriter` uses the names of the properties defined in the dataclass as the CSV header titles, however,\ndepending on your use case it makes sense to change it. The `DataclassWriter` has a `map` method just for this purpose.\n\n Using the `User` dataclass with the properties `firstname`, `lastname` and `age`. The snippet below shows how to change `firstname` to `First name` and `lastname` to `Last name`:\n\n ```python\n with open(\"users.csv\", \"w\") as f:\n    w = DataclassWriter(f, users, User)\n\n    # Add mappings for firstname and lastname\n    w.map(\"firstname\").to(\"First name\")\n    w.map(\"lastname\").to(\"Last name\")\n\n    w.write()\n ```\n\n The CSV output of the snippet above will be:\n\n```text\nFirst name,Last name,age\nJohn,Smith,40\nDaniel,Nilsson,10\nElla,Fralla,4\n```\n\n## Copyright and License\n\nCopyright (c) 2018 Daniel Furtado. Code released under BSD 3-clause license\n\n## Credits\n\nThis package was created with [Cookiecutter](https://github.com/audreyr/cookiecutter) and the [audreyr/cookiecutter-pypackage](https://github.com/audreyr/cookiecutter-pypackage) project template.\n","funding_links":["https://github.com/sponsors/dfurtado"],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfurtado%2Fdataclass-csv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdfurtado%2Fdataclass-csv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfurtado%2Fdataclass-csv/lists"}