{"id":23632503,"url":"https://github.com/netzwerg/paleo","last_synced_at":"2025-08-18T08:05:50.534Z","repository":{"id":145536814,"uuid":"42098141","full_name":"netzwerg/paleo","owner":"netzwerg","description":"Immutable Java 8 data frames with typed columns (including primitives)","archived":false,"fork":false,"pushed_at":"2021-12-14T15:19:07.000Z","size":1119,"stargazers_count":71,"open_issues_count":6,"forks_count":16,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-30T06:01:58.652Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/netzwerg.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-09-08T07:54:06.000Z","updated_at":"2024-10-28T07:01:22.000Z","dependencies_parsed_at":"2023-06-01T04:45:47.161Z","dependency_job_id":null,"html_url":"https://github.com/netzwerg/paleo","commit_stats":null,"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/netzwerg/paleo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netzwerg%2Fpaleo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netzwerg%2Fpaleo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netzwerg%2Fpaleo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netzwerg%2Fpaleo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/netzwerg","download_url":"https://codeload.github.com/netzwerg/paleo/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netzwerg%2Fpaleo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270962391,"owners_count":24675965,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-28T03:29:20.514Z","updated_at":"2025-08-18T08:05:50.523Z","avatar_url":"https://github.com/netzwerg.png","language":"Java","funding_links":[],"categories":["数据科学"],"sub_categories":["计算机视觉"],"readme":"# NO LONGER MAINTAINED – USE AT YOUR OWN RISK!\n\n# Paleo image:https://travis-ci.org/netzwerg/paleo.svg?branch=master[\"Build Status\", link=\"https://travis-ci.org/netzwerg/paleo\"]\n:latest-release-version: 0.14.0\n\nImmutable Java 8 data frames with typed columns.\n\nA data frame is composed of `0..n` named columns, which all contain the same number of row values. Each column has a fixed\ndata type, which allows for type-safe value access. The following column types are supported out-of-the-box:\n\n* **Int**: Primitive `int` values\n* **Long**: Primitive `long` values\n* **Double**: Primitive `double` values\n* **Boolean**: Primitive `boolean` values\n* **String**: `java.lang.String` values\n* **Timestamp**: `java.time.Instant` values\n* **Category**: Categorical `String` values (aka factors)\n\nColumns can be created via simple factory methods, through a fluent builder API, or from text files.\n\n# Hello Paleo\n\nThe `paleo-core` module provides all classes to identify, create, and structure typed columns: \n\n[source,java]\n----\n// Type-safe column identifiers\nfinal StringColumnId NAME = StringColumnId.of(\"Name\");\nfinal CategoryColumnId COLOR = CategoryColumnId.of(\"Color\");\nfinal DoubleColumnId SERVING_SIZE = DoubleColumnId.of(\"Serving Size (g)\");\n\n// Convenient column creation\nStringColumn nameColumn = StringColumn.ofAll(NAME, \"Banana\", \"Blueberry\", \"Lemon\", \"Apple\");\nCategoryColumn colorColumn = CategoryColumn.ofAll(COLOR, \"Yellow\", \"Blue\", \"Yellow\", \"Green\");\nDoubleColumn servingSizeColumn = DoubleColumn.ofAll(SERVING_SIZE, 118, 148, 83, 182);\n\n// Grouping columns into a data frame\nDataFrame dataFrame = DataFrame.ofAll(nameColumn, colorColumn, servingSizeColumn);\n\n// Typed random access to individual values (based on rowIndex / columnId)\nString lemon = dataFrame.getValueAt(2, NAME);\ndouble appleServingSize = dataFrame.getValueAt(3, SERVING_SIZE);\n\n// Typed stream-based access to all values\nDoubleStream servingSizes = servingSizeColumn.valueStream();\ndouble maxServingSize = servingSizes.summaryStatistics().getMax();\n\n// Smart column implementations\nSet\u003cString\u003e colors = colorColumn.getCategories();\n----\n\n# Parsing From Text / File\n\nThe `paleo-io` module parses data frames from tab-delimited or comma-separated text representations. The structure of the\ndata frame (i.e. the names and types of its columns) can be defined in one of two ways:\n\n## Header Rows\n\nIn its simplest format, the tab-delimited text representation directly contains column names and types in a header.\nThe first row specifies the column names, the second row specifies the column types (actual data starting on third row):\n\n----\n1 Name    Color\n2 String  Category\n3 Banana  Yellow\n...\nn Apple   Green\n----\n\nThe contents can then be parsed via `Parser.tsv(Reader in)` or `Parser.csv(Reader in)`, e.g. like:\n\n[source,java]\n----\nfinal String EXAMPLE =\n            \"Name\\tColor\\tServing Size (g)\\n\" +\n            \"String\\tCategory\\tDouble\\n\" +\n            \"Banana\\tYellow\\t118\\n\" +\n            \"Blueberry\\tBlue\\t148\\n\" +\n            \"Lemon\\tYellow\\t83\\n\" +\n            \"Apple\\tGreen\\t182\";\n\nDataFrame dataFrame = Parser.tsv(new StringReader(EXAMPLE));\n----\n\n## External JSON Schema\n\nGenerally it is advisable to separate the structural information from the actual data. Paleo therefore supports the\ndefinition of an external JSON schema. The format is inspired by the\nhttp://dataprotocols.org/json-table-schema[JSON Table Schema]:\n\n[source,json]\n----\n{\n  \"title\": \"Example Schema\",\n  \"dataFileName\": \"data.txt\",\n  \"charsetName\": \"ISO-8859-1\", // \u003c1\u003e\n  \"fields\": [\n    {\n      \"name\": \"Name\",\n      \"type\": \"String\"\n    },\n    {\n      \"name\": \"Color\",\n      \"type\": \"Category\"\n    },\n    {\n      \"name\": \"Serving Size\",\n      \"type\": \"Double\",\n      \"metaData\": { \"unit\": \"g\" }\n    },\n    {\n      \"name\": \"Exemplary Date\",\n      \"type\": \"Timestamp\",\n      \"format\": \"yyyyMMddHHmmss\"\n    }\n  ]\n}\n----\n\u003c1\u003e Optionally specify an encoding\n\nDedicated parsing methods allow to first parse the schema from JSON, and subsequently use it to create a `DataFrame`.\nA given base directory is used to load the actual data (i.e. to resolve the location of the configured `dataFileName`):\n\n[source,java]\n----\nSchema schema = Schema.parseJson(new StringReader(EXAMPLE_SCHEMA));\nDataFrame dataFrame = Parser.tsv(schema, baseDir);\n----\n\n## Working With Parsed Data Frames\n\nOnce a `DataFrame` instance has been parsed, its data can be accessed through a type-safe API:\n\n[source,java]\n----\nfinal String EXAMPLE =\n            \"Name\\tColor\\tServing Size (g)\\n\" +\n            \"String\\tCategory\\tDouble\\n\" +\n            \"Banana\\tYellow\\t118\\n\" +\n            \"Blueberry\\tBlue\\t148\\n\" +\n            \"Lemon\\tYellow\\t83\\n\" +\n            \"Apple\\tGreen\\t182\";\n\nDataFrame dataFrame = Parser.tsv(new StringReader(EXAMPLE));\n\n// Lookup typed identifiers by column index\nfinal StringColumnId NAME = dataFrame.getColumnId(0, ColumnType.STRING);\nfinal CategoryColumnId COLOR = dataFrame.getColumnId(1, ColumnType.CATEGORY);\nfinal DoubleColumnId SERVING_SIZE = dataFrame.getColumnId(2, ColumnType.DOUBLE);\n\n// Use identifier to access columns \u0026 values\nStringColumn nameColumn = dataFrame.getColumn(NAME);\nIndexedSeq\u003cString\u003e nameValues = nameColumn.getValues();\n\n// ... or access individual values via row index / column id \nString yellow = dataFrame.getValueAt(2, COLOR);\n----\n\n# Usage\n\nAll modules are available via https://bintray.com/netzwerg/maven/paleo/view[Bintray/JCenter].\n\n## Repository Configuration\n\nGradle:\n\n[source,groovy]\n----\nrepositories {\n    jcenter()\n}\n----\n\nMaven `settings.xml`:\n\n[source,xml]\n----\n\u003crepository\u003e\n    \u003csnapshots\u003e\n      \u003cenabled\u003efalse\u003c/enabled\u003e\n    \u003c/snapshots\u003e\n    \u003cid\u003ecentral\u003c/id\u003e\n    \u003cname\u003ebintray\u003c/name\u003e\n    \u003curl\u003ehttp://jcenter.bintray.com\u003c/url\u003e\n\u003c/repository\u003e\n----\n\n## Using the `paleo-core` module\n\nGradle:\n\n[source,groovy]\n[subs=\"attributes\"]\n----\ncompile 'ch.netzwerg:paleo-core:{latest-release-version}'\n----\n\nMaven:\n\n[source,xml]\n[subs=\"specialcharacters,attributes\"]\n----\n\u003cdependency\u003e\n    \u003cgroupId\u003ech.netzwerg\u003c/groupId\u003e\n    \u003cartifactId\u003epaleo-core\u003c/artifactId\u003e\n    \u003cversion\u003e{latest-release-version}\u003c/version\u003e\n    \u003ctype\u003ejar\u003c/type\u003e\n\u003c/dependency\u003e\n----\n\n## Using the `paleo-io` module\n\nOptional (requires `paleo-core`)\n\nGradle:\n\n[source,groovy]\n[subs=\"attributes\"]\n----\ncompile 'ch.netzwerg:paleo-io:{latest-release-version}'\n----\n\nMaven:\n\n[source,xml]\n[subs=\"specialcharacters,attributes\"]\n----\n\u003cdependency\u003e\n    \u003cgroupId\u003ech.netzwerg\u003c/groupId\u003e\n    \u003cartifactId\u003epaleo-io\u003c/artifactId\u003e\n    \u003cversion\u003e{latest-release-version}\u003c/version\u003e\n    \u003ctype\u003ejar\u003c/type\u003e\n\u003c/dependency\u003e\n----\n\n# Vavr\n\nPaleo makes extensive use of the http://www.vavr.io/[Vavr library]. Vavr provides\nawesome collection classes which offer functionality way beyond the standard JDK. Working with the Vavr classes\nis highly recommended, but it is always possible to back out and convert to JDK standards (e.g. with `toJavaList()`).\n\n# Factory-Methods vs. Builders\n\nPaleo tries to make the best compromise between parsing speed, index-based value lookup, and memory usage. That's why\nit offers two ways to create columns: Static factory methods allow for convenient construction if all values are already\navailable. Individual column builders should be used if columns are constructed via successive value addition. Please be\naware that the builders are not thread-safe.\n\n# Why The Name?\n\nThe backing data structures are all about **raw** values and **primitive** types \u0026mdash; this somehow reminded me of\nthe paleo diet.\n\n# Contributions\n\nPull requests are very welcome.\nPlease note that by submitting a pull request, you agree to license your contribution under the \"Apache License Version 2.0\".\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetzwerg%2Fpaleo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetzwerg%2Fpaleo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetzwerg%2Fpaleo/lists"}