{"id":18644147,"url":"https://github.com/indix/dfs-datastores","last_synced_at":"2025-04-11T12:30:57.238Z","repository":{"id":8111326,"uuid":"9527106","full_name":"indix/dfs-datastores","owner":"indix","description":"Dead-simple vertical partitioning, compression, appends, and consolidation of data on a distributed filesystem.","archived":false,"fork":false,"pushed_at":"2021-03-23T05:42:51.000Z","size":388,"stargazers_count":0,"open_issues_count":1,"forks_count":6,"subscribers_count":64,"default_branch":"master","last_synced_at":"2025-03-25T13:39:31.621Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/indix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-04-18T17:05:16.000Z","updated_at":"2020-05-18T04:34:47.000Z","dependencies_parsed_at":"2022-09-14T13:22:16.442Z","dependency_job_id":null,"html_url":"https://github.com/indix/dfs-datastores","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indix%2Fdfs-datastores","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indix%2Fdfs-datastores/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indix%2Fdfs-datastores/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indix%2Fdfs-datastores/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/indix","download_url":"https://codeload.github.com/indix/dfs-datastores/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248401935,"owners_count":21097328,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T06:10:11.407Z","updated_at":"2025-04-11T12:30:56.940Z","avatar_url":"https://github.com/indix.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://snap-ci.com/indix/dfs-datastores/branch/master/build_image)](https://snap-ci.com/indix/dfs-datastores/branch/master)\n\n# dfs-datastores\n\nA dramatically simpler and more powerful way to store records on a distributed filesystem.\n\nTo run the tests:\n\n```scala\nsbt test\n```\n\nTo publish the changes\n* Update the minor / majar version in version.sbt\n* `sbt publish` to publish the artifact to artifactory\n\n### Using Incremental PailConsolidate\n\nPlease make sure to add this argument to the PailConsolidate when you want to run the incremental version. The reason for doing so is because by default the S3N implementation does full recursive directory and file fetch without an ability to control the listing limit.\n\n```\n-Dfs.s3n.impl=com.indix.hadoop.fs.s3native.LimitedListingS3NFileSystem\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findix%2Fdfs-datastores","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Findix%2Fdfs-datastores","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findix%2Fdfs-datastores/lists"}