{"id":28401955,"url":"https://github.com/fal-ai/lavender-data","last_synced_at":"2025-06-25T17:31:50.021Z","repository":{"id":291978605,"uuid":"953878768","full_name":"fal-ai/lavender-data","owner":"fal-ai","description":"Load \u0026 manage evolving datasets efficiently","archived":false,"fork":false,"pushed_at":"2025-06-16T02:08:53.000Z","size":11529,"stargazers_count":14,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-16T03:38:28.432Z","etag":null,"topics":["data","dataloader","ml","torch"],"latest_commit_sha":null,"homepage":"https://docs.lavenderdata.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fal-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-24T08:19:38.000Z","updated_at":"2025-06-16T02:08:56.000Z","dependencies_parsed_at":"2025-05-07T14:44:52.452Z","dependency_job_id":"57af3188-98df-4c8b-beca-c05fbefcc5c0","html_url":"https://github.com/fal-ai/lavender-data","commit_stats":null,"previous_names":["fal-ai/lavender-data"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fal-ai/lavender-data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fal-ai%2Flavender-data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fal-ai%2Flavender-data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fal-ai%2Flavender-data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fal-ai%2Flavender-data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fal-ai","download_url":"https://codeload.github.com/fal-ai/lavender-data/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fal-ai%2Flavender-data/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261397571,"owners_count":23152496,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","dataloader","ml","torch"],"created_at":"2025-06-01T14:08:18.782Z","updated_at":"2025-06-25T17:31:50.009Z","avatar_url":"https://github.com/fal-ai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://github.com/fal-ai/lavender-data/raw/main/assets/logo.webp\" alt=\"Lavender Data Logo\" width=\"50%\" /\u003e\n\u003c/p\u003e\n\n\u003ch2\u003e\n    \u003cp align=\"center\"\u003e\n        Load \u0026 evolve datasets efficiently\n    \u003c/p\u003e\n\u003c/h2\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://pypi.org/project/lavender-data/\"\u003e\n        \u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/lavender-data.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://discord.gg/fal-ai\"\u003e\n        \u003cimg alt=\"Discord\" src=\"https://img.shields.io/badge/Discord-chat-2eb67d.svg?logo=discord\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/fal-ai/lavender-data/blob/main/LICENSE\"\u003e\n        \u003cimg alt=\"License\" src=\"https://img.shields.io/badge/License-Apache%202.0-green.svg\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cbr /\u003e\n\n\u003cp align=\"center\"\u003e\n    Please visit our docs for more information.\n    \u003cbr /\u003e\n    \u003ca href=\"https://docs.lavenderdata.com/\"\u003e\n        docs.lavenderdata.com\n    \u003c/a\u003e\n\u003c/p\u003e\n\n## Quick Start\n\n### Installation\n\n```bash\npip install lavender-data\n```\n\n#### Start the server\n\n```bash\nlavender-data server start --init\n```\n\n```\nlavender-data is running on 0.0.0.0:8000\nUI is running on http://localhost:3000\nAPI key created: la-...\n```\n\nSave the API key to use it in the next steps.\n\n```bash\nexport LAVENDER_API_URL=http://0.0.0.0:8000\nexport LAVENDER_API_KEY=la-...\n```\n\n### Create an example dataset\n\n```bash\nlavender-data client \\\n  datasets create \\\n  --name my_dataset \\\n  --uid-column-name id \\\n  --shardset-location https://docs.lavenderdata.com/example-dataset/images/\n```\n\n### Iterate over the dataset\n\n```python\nimport lavender_data.client as lavender\n\nlavender.init()\n\niteration = lavender.LavenderDataLoader(\n    dataset_name=\"my_dataset\",\n    shuffle=True,\n    shuffle_block_size=10,\n)\n\nfor i in iteration:\n    print(i[\"id\"])\n```\n\n\u003cp align=\"center\"\u003e\n    Please visit our docs for more information.\n    \u003cbr /\u003e\n    \u003ca href=\"https://docs.lavenderdata.com/\"\u003e\n        docs.lavenderdata.com\n    \u003c/a\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffal-ai%2Flavender-data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffal-ai%2Flavender-data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffal-ai%2Flavender-data/lists"}