{"id":36418448,"url":"https://github.com/robjg/dido","last_synced_at":"2026-01-11T17:01:27.416Z","repository":{"id":1323331,"uuid":"1268536","full_name":"robjg/dido","owner":"robjg","description":"Data In/Data Out in many formats","archived":false,"fork":false,"pushed_at":"2025-12-02T19:00:55.000Z","size":13591,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-05T18:56:50.507Z","etag":null,"topics":["csv-parser","data","etl","java","json-parser"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robjg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2011-01-18T20:47:57.000Z","updated_at":"2025-12-03T05:56:00.000Z","dependencies_parsed_at":"2024-04-23T06:55:56.488Z","dependency_job_id":"4a040a94-7b5c-4295-805d-cfcdbb061047","html_url":"https://github.com/robjg/dido","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/robjg/dido","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robjg%2Fdido","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robjg%2Fdido/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robjg%2Fdido/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robjg%2Fdido/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robjg","download_url":"https://codeload.github.com/robjg/dido/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robjg%2Fdido/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28314259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T14:58:17.114Z","status":"ssl_error","status_checked_at":"2026-01-11T14:55:53.580Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv-parser","data","etl","java","json-parser"],"created_at":"2026-01-11T17:01:17.814Z","updated_at":"2026-01-11T17:01:27.398Z","avatar_url":"https://github.com/robjg.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"Dido\n====\n\n- [Overview](#overview)\n- [Some Java Examples](#some-java-examples)\n- [No Code Dido](#no-code-dido)\n- [More Info](#more-info)\n- [Building](#building)\n- [Background](#background)\n\n### Overview\n\nDido stands for Data-In/Data-Out. It is a framework for making data from different sources\nlook the same so that it can be copied, processed and compared.\n\nDido is available in Maven. To get started simply include [dido-all](https://mvnrepository.com/artifact/uk.co.rgordon/dido-all)\nwhich will provide all the stable modules in one dependency.\n\n### Some Java Examples\n\nGiven this CSV:\n```\nApple,5,19.50\nOrange,2,35.24\nPear,3,26.84\n```\n\nWe can read it in:\n```java\n        List\u003cDidoData\u003e didoData;\n\n        try (DataIn in = DataInCsv.fromPath(Path.of(\"Fruit.csv\"))) {\n\n            didoData = in.stream().collect(Collectors.toList());\n        }\n\n        assertThat(didoData, contains(\n                DidoData.of(\"Apple\", \"5\", \"19.50\"),\n                DidoData.of(\"Orange\", \"2\", \"35.24\"),\n                DidoData.of(\"Pear\", \"3\", \"26.84\")));\n```\n\nAnd we can write it out as Json\n```java\n        try (DataOut out = DataOutJson.with()\n                .outFormat(JsonDidoFormat.LINES)\n                .toOutputStream(System.out)) {\n\n            didoData.forEach(out);\n        }\n```\n\nGiving us:\n```\n{\"f_1\":\"Apple\",\"f_2\":\"5\",\"f_3\":\"19.50\"}\n{\"f_1\":\"Orange\",\"f_2\":\"2\",\"f_3\":\"35.24\"}\n{\"f_1\":\"Pear\",\"f_2\":\"3\",\"f_3\":\"26.84\"}\n```\n\nWe can give our data a schema:\n```java\n        DataSchema schema = DataSchema.builder()\n                .addNamed(\"Fruit\", String.class)\n                .addNamed(\"Qty\", int.class)\n                .addNamed(\"Price\", double.class)\n                .build();\n```\n\nAnd now when we copy from CSV to JSON\n```java\n        try (DataIn in = DataInCsv.with()\n                .schema(schema)\n                .fromPath(Path.of(\"Fruit.csv\"));\n             DataOut out = DataOutJson.with()\n                     .outFormat(JsonDidoFormat.LINES)\n                     .toOutputStream(System.out)) {\n\n            in.forEach(out);\n        }\n```\n\nWe get:\n```\n{\"Fruit\":\"Apple\",\"Qty\":5,\"Price\":19.5}\n{\"Fruit\":\"Orange\",\"Qty\":2,\"Price\":35.24}\n{\"Fruit\":\"Pear\",\"Qty\":3,\"Price\":26.84}\n```\n\n\n### No Code Dido\n\nDido comes with Jobs and Types for creating Data Processing Pipelines in [Oddjob](https://github.com/robjg/oddjob/)\nwithout code using Oddjob's UI - *Oddjob Explorer*\n\nHere is Oddjob Explorer running the first example above.\n\n![Csv to Json in Oddjob](docs/images/OddjobCsvJson.jpg)\n\nSee [Dido in Oddjob](docs/DIDO-ODDJOB.md) for getting started with Dido in Oddjob.\n\nSee [The Reference](docs/reference/README.md) for details of all the Oddjob configurations in Dido. \n\n### More Info\n\n[dido-data](docs/DIDO-DATA.md) provides the definition of Data on which the rest of Dido is based.\n[dido-operators](docs/DIDO-OPERATORS.md) provide functions for processing data.\n\nFor Reading Data in and Out in different formats: \n - [dido-csv](docs/DIDO-CSV.md) - For reading and writing CSV data.  \n - [dido-json](docs/DIDO-JSON.md) - For reading and writing JSON. \n - [dido-sql](docs/DIDO-SQL.md) - For reading and writing to Databases.\n - [dido-poi](docs/DIDO-POI.md) - For reading and writing to Excel sheets.\n - [dido-text](docs/DIDO-TEXT.md) - For writing to Ascii Formatted Text Tables.\n\n[dido-objects](docs/DIDO-OBJECTS.md) for converting to and from Java Objects.\n\n\n### Building\n\nSee [Building](BUILDING.md)\n\n### Background\n\nFor a more information on why Dido was created please see\n[Background](BACKGROUND.md)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobjg%2Fdido","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobjg%2Fdido","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobjg%2Fdido/lists"}