{"id":28804491,"url":"https://github.com/advancedpricing/delimit","last_synced_at":"2026-03-10T10:32:20.477Z","repository":{"id":291986025,"uuid":"979438350","full_name":"advancedpricing/delimit","owner":"advancedpricing","description":"CSV/TSV/PSV/XLSX reader/writer utilizing Ecto like module definitions","archived":false,"fork":false,"pushed_at":"2026-03-03T22:01:58.000Z","size":233,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-03-04T01:41:19.352Z","etag":null,"topics":["csv","elixir","elixir-lang"],"latest_commit_sha":null,"homepage":"https://hex.pm/packages/delimit","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/advancedpricing.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-07T14:07:27.000Z","updated_at":"2026-03-03T22:02:01.000Z","dependencies_parsed_at":"2025-07-04T12:07:22.046Z","dependency_job_id":"406adc4c-3ebe-4d0a-a706-89bfacaed60a","html_url":"https://github.com/advancedpricing/delimit","commit_stats":null,"previous_names":["advancedpricing/delimit"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/advancedpricing/delimit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advancedpricing%2Fdelimit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advancedpricing%2Fdelimit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advancedpricing%2Fdelimit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advancedpricing%2Fdelimit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/advancedpricing","download_url":"https://codeload.github.com/advancedpricing/delimit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advancedpricing%2Fdelimit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30330582,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T05:25:20.737Z","status":"ssl_error","status_checked_at":"2026-03-10T05:25:17.430Z","response_time":106,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","elixir","elixir-lang"],"created_at":"2025-06-18T09:03:56.188Z","updated_at":"2026-03-10T10:32:20.455Z","avatar_url":"https://github.com/advancedpricing.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Delimit\n\n[![Hex.pm Version](https://img.shields.io/hexpm/v/delimit.svg)](https://hex.pm/packages/delimit)\n[![Hex.pm License](https://img.shields.io/hexpm/l/delimit.svg)](https://github.com/jcowgar/delimit/blob/main/LICENSE)\n\nDelimit is a powerful yet elegant library for reading and writing delimited data files (CSV, TSV, PSV, SSV) in Elixir. Inspired by Ecto, it allows you to define schemas for your delimited data, providing strong typing with structs, validation, and transformation capabilities. By defining the structure of your data, Delimit enables type-safe parsing and generation with minimal boilerplate code.\n\n## Features\n\n- **Schema-based approach**: Define the structure of your delimited files using Ecto-like schemas\n- **Strong typing with structs**: Convert between string values and proper Elixir types in type-safe structs\n- **Full TypeSpecs**: Automatically generated type specifications for your schemas\n- **Streaming support**: Process large files efficiently with Elixir streams\n- **Customizable parsing**: Configure delimiters, headers, type conversion, and more\n- **Embedded schemas**: Nest schemas for complex data structures\n- **Custom transformations**: Add your own read/write functions for special data formats\n- **Memory efficient**: Stream large files without loading everything into memory\n\n## Installation\n\nAdd `delimit` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:delimit, \"~\u003e 0.1.0\"}\n  ]\nend\n```\n\nThen fetch your dependencies:\n\n```bash\nmix deps.get\n```\n\n## Quick Start\n\n### Define a schema\n\nDefine a schema that represents the structure of your delimited file:\n\n```elixir\ndefmodule MyApp.Person do\n  use Delimit\n\n  layout do\n    field :first_name, :string\n    field :last_name, :string\n    field :age, :integer\n    field :salary, :float\n    field :birthday, :date, format: \"{YYYY}-{0M}-{0D}\"\n    field :active, :boolean\n    field :notes, :string, nil_on_empty: true\n  end\nend\n```\n\nThis automatically creates a struct with type specifications:\n\n```elixir\n@type t :: %__MODULE__{\n  first_name: String.t(),\n  last_name: String.t(),\n  age: integer(),\n  salary: float(),\n  birthday: Date.t(),\n  active: boolean(),\n  notes: String.t()\n}\n```\n\n### Reading data\n\nRead data from a file:\n\n```elixir\n# Read all records at once - returns a list of structs\npeople = MyApp.Person.read(\"people.csv\")\nfirst_person = List.first(people) # returns a %MyApp.Person{} struct\n\n# Stream records for better memory efficiency\npeople_stream =\n  \"large_file.csv\"\n  |\u003e MyApp.Person.stream()\n  |\u003e Stream.filter(fn person -\u003e person.age \u003e 30 end)\n  |\u003e Stream.map(fn person -\u003e %{person | salary: person.salary * 1.1} end)\n  |\u003e Enum.to_list()\n\n# Read from a string\ncsv_data = \"first_name,last_name,age\\nJohn,Doe,42\"\npeople = MyApp.Person.read_string(csv_data)\n```\n\n### Writing data\n\nWrite data to a file:\n\n```elixir\npeople = [\n  %MyApp.Person{first_name: \"John\", last_name: \"Doe\", age: 42,\n    salary: 50000.0, birthday: ~D[1980-01-15], active: true, notes: \"Senior developer\"},\n  %MyApp.Person{first_name: \"Jane\", last_name: \"Smith\", age: 35,\n    salary: 60000.0, birthday: ~D[1987-05-22], active: true, notes: nil}\n]\n\n# Write all records at once\n:ok = MyApp.Person.write(\"people.csv\", people)\n\n# Write to a string\ncsv_string = MyApp.Person.write_string(people)\n\n# Stream data to a file (memory efficient)\nstream = Stream.map(1..1000, fn i -\u003e\n  %MyApp.Person{\n    first_name: \"User#{i}\",\n    last_name: \"Test\",\n    age: 20 + rem(i, 50),\n    salary: 30_000.0 + (i * 100),\n    birthday: Date.add(~D[2000-01-01], i),\n    active: rem(i, 2) == 0,\n    notes: \"Generated user #{i}\"\n  }\nend)\n\n:ok = MyApp.Person.stream_to_file(\"users.csv\", stream)\n```\n\n## Field Types\n\nDelimit supports the following field types:\n\n| Type        | Description            | Example                        |\n| ----------- | ---------------------- | ------------------------------ |\n| `:string`   | Basic string values    | `field :name, :string`         |\n| `:integer`  | Integer numbers        | `field :age, :integer`         |\n| `:float`    | Floating point numbers | `field :salary, :float`        |\n| `:boolean`  | Boolean values         | `field :active, :boolean`      |\n| `:date`     | Date values            | `field :birthday, :date`       |\n| `:datetime` | DateTime values        | `field :created_at, :datetime` |\n\n## Field Options\n\nEach field can have additional options:\n\n\u003e **Note:** Date and DateTime fields use [Timex](https://hexdocs.pm/timex/Timex.Format.DateTime.Formatters.Default.html) format patterns for parsing and formatting.\n\n```elixir\n# Default value when field is missing\nfield :age, :integer, default: 0\n\n# Custom header name in CSV file\nfield :email, :string, label: \"contact_email\"\n\n# Format for date/datetime fields (using Timex format patterns)\nfield :birthday, :date, format: \"{0M}/{0D}/{YYYY}\"\n\n# Convert empty strings to nil\nfield :notes, :string, nil_on_empty: true\n\n# Custom values for boolean fields\nfield :status, :boolean, true_values: [\"Y\", \"Yes\"], false_values: [\"N\", \"No\"]\n\n# Custom conversion functions with explicit struct type\nfield :tags, :string,\n  read_fn: \u0026String.split(\u00261, \"|\"),\n  write_fn: \u0026Enum.join(\u00261, \"|\"),\n  struct_type: {:list, :string}\n```\n\n## Advanced Usage\n\nThis section covers more advanced features and techniques for getting the most out of Delimit.\n\n### Type Specifications\n\nDelimit automatically generates typespecs for your schemas, including support for complex field types:\n\n```elixir\ndefmodule MyApp.User do\n  use Delimit\n\n  layout do\n    field :name, :string\n    # File contains comma-separated tags, but in memory it's a list\n    field :tags, :string,\n      read_fn: \u0026String.split(\u00261, \",\"),\n      write_fn: \u0026Enum.join(\u00261, \",\"),\n      struct_type: {:list, :string}\n\n    # Map type with string keys and integer values\n    field :scores, :string,\n      read_fn: \u0026parse_scores/1,\n      write_fn: \u0026serialize_scores/1,\n      struct_type: {:map, :string, :integer}\n  end\n\n  defp parse_scores(str), do: # Parse string to map\n  defp serialize_scores(map), do: # Convert map to string\nend\n```\n\n### Embedded Schemas\n\nYou can nest schemas using the `embeds_one` macro:\n\n```elixir\ndefmodule MyApp.Address do\n  use Delimit\n\n  layout do\n    field :street, :string\n    field :city, :string\n    field :state, :string\n    field :postal_code, :string\n  end\nend\n\ndefmodule MyApp.Customer do\n  use Delimit\n\n  layout do\n    field :name, :string\n    field :email, :string\n    embeds_one :address, MyApp.Address\n    embeds_one :billing_address, MyApp.Address, prefix: \"billing_\"\n  end\nend\n\n# This will handle headers like:\n# name,email,street,city,state,postal_code,billing_street,billing_city,billing_state,billing_postal_code\n#\n# And create structs like:\n# %MyApp.Customer{\n#   name: \"John Doe\",\n#   email: \"john@example.com\",\n#   address: %MyApp.Address{street: \"123 Main St\", ...},\n#   billing_address: %MyApp.Address{street: \"456 Billing St\", ...}\n# }\n```\n\n### Using Standard Formats\n\nDelimit provides built-in support for common file formats:\n\n```elixir\n# Read tab-separated values with the format option\npeople = MyApp.Person.read(\"people.tsv\", format: :tsv)\n\n# Read comma-separated values (also the default)\npeople = MyApp.Person.read(\"people.csv\", format: :csv)\n\n# Write pipe-separated values\n:ok = MyApp.Person.write(\"people.psv\", people, format: :psv)\n```\n\nSupported formats include:\n\n- `:csv` - Comma-separated values with double-quote escaping\n- `:tsv` - Tab-separated values with double-quote escaping\n- `:psv` - Pipe-separated values with double-quote escaping\n- `:ssv` - Semi-colon-separated values with double-quote escaping\n\n### Parser Configuration Options\n\nDelimit provides several customization options for parsing and generating delimited files:\n\n#### Delimiter Options\n\n```elixir\n# Read tab-separated values with explicit delimiter\npeople = MyApp.Person.read(\"people.tsv\", delimiter: \"\\t\")\n\n# Write pipe-separated values with explicit delimiter\n:ok = MyApp.Person.write(\"people.psv\", people, delimiter: \"|\")\n\n# Use a specific escape character (default is double-quote)\npeople = MyApp.Person.read(\"people.csv\", escape: \"\\\"\")\n\n# Set line ending for generated files (default is \\n)\n:ok = MyApp.Person.write(\"people.csv\", people, line_ending: \"\\r\\n\")\n```\n\n#### Headers and Content Processing\n\n```elixir\n# Control whether headers are included (default is true)\npeople = MyApp.Person.read(\"people.csv\", headers: false)\n:ok = MyApp.Person.write(\"people.csv\", people, headers: false)\n\n# Skip a specific number of lines at the beginning of a file\npeople = MyApp.Person.read(\"people.csv\", skip_lines: 3)\n\n# Skip lines dynamically based on content (like comments)\npeople = MyApp.Person.read(\"people.csv\",\n  skip_while: fn line -\u003e String.starts_with?(line, \"#\") end)\n\n# Control whether to trim whitespace from fields\npeople = MyApp.Person.read(\"people.csv\", trim_fields: true)\n```\n\n#### Combining Multiple Options\n\nOptions can be combined for complete customization:\n\n```elixir\n# Multiple options can be combined\npeople = MyApp.Person.read(\"people.csv\",\n  delimiter: \";\",\n  escape: \"\\\"\",\n  skip_lines: 2,\n  trim_fields: true,\n  headers: true\n)\n```\n\n## License\n\nThis project is licensed under the LGPL-3 License - see the LICENSE file for details.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadvancedpricing%2Fdelimit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadvancedpricing%2Fdelimit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadvancedpricing%2Fdelimit/lists"}