{"id":19947341,"url":"https://github.com/firstbatchxyz/dria-sdk","last_synced_at":"2025-05-03T17:33:47.184Z","repository":{"id":257806260,"uuid":"865402302","full_name":"firstbatchxyz/dria-sdk","owner":"firstbatchxyz","description":"Dria SDK is for building and executing synthetic data generation pipelines on Dria Knowledge Network. ","archived":false,"fork":false,"pushed_at":"2025-04-02T18:50:26.000Z","size":2750,"stargazers_count":22,"open_issues_count":14,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-02T19:17:09.170Z","etag":null,"topics":["decentralized-ai","multi-agent","synthetic-data","synthetic-dataset-generation"],"latest_commit_sha":null,"homepage":"https://docs.dria.co","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/firstbatchxyz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-30T13:26:54.000Z","updated_at":"2025-04-02T18:50:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"4fa74d80-3ba5-4413-83b8-4018682eee26","html_url":"https://github.com/firstbatchxyz/dria-sdk","commit_stats":null,"previous_names":["firstbatchxyz/dria-sdk"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fdria-sdk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fdria-sdk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fdria-sdk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firstbatchxyz%2Fdria-sdk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/firstbatchxyz","download_url":"https://codeload.github.com/firstbatchxyz/dria-sdk/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252227168,"owners_count":21714955,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decentralized-ai","multi-agent","synthetic-data","synthetic-dataset-generation"],"created_at":"2024-11-13T00:35:37.441Z","updated_at":"2025-05-03T17:33:47.176Z","avatar_url":"https://github.com/firstbatchxyz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/firstbatchxyz/.github/refs/heads/master/branding/dria-logo-square.svg\" alt=\"logo\" width=\"168\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eDria SDK\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/dria/\"\u003e\u003cimg src=\"https://badge.fury.io/py/dria.svg\" alt=\"PyPI version\"\u003e\u003c/a\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://docs.dria.co\"\u003e\u003cimg src=\"https://img.shields.io/badge/docs-online-brightgreen.svg\" alt=\"Documentation Status\"\u003e\u003c/a\u003e\n      \u003ca href=\"https://discord.gg/dria\" target=\"_blank\"\u003e\n        \u003cimg alt=\"Discord\" src=\"https://dcbadge.vercel.app/api/server/dria?style=flat\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n**Dria SDK** is a scalable and versatile toolkit for creating and managing synthetic datasets for AI. With Dria, you can orchestrate multi-step pipelines that pull data from both web and siloed sources, blend them with powerful AI model outputs, and produce high-quality synthetic datasets—**no GPU required**.  \n\n---\n\n## Why Dria?\n\n- **Dataset Generation**: Easily build synthetic data pipelines using Dria’s flexible APIs.\n- **Multi-Agent Network**: Orchestrate complex tasks and data retrieval using specialized agents for web search and siloed APIs.\n- **No GPUs Needed**: Offload your compute to the network, accelerating your workflows without personal GPU hardware.\n- **Customizable**: Define custom Pydantic schemas to shape the output of your datasets precisely.\n- **Model-Rich**: Use different Large Language Models (LLMs) such as OpenAI, Gemini, Ollama or others to synthesize data.\n- **Grounding \u0026 Diversity**: Add real-world context to your synthetic datasets with integrated web and siloed data retrieval.\n\n---\n\n## Installation\n\nDria SDK is available on PyPI. You can install it with:\n```bash\npip install dria\n```\n\nIt’s recommended to use a virtual environment (e.g., `virtualenv` or `conda`) to avoid version conflicts with other packages.\n\n---\n\n## Quick Start\n\nHere’s a minimal example to get you started with Dria:\n\n```python\nimport asyncio\nfrom dria import Prompt, DatasetGenerator, DriaDataset, Model\nfrom pydantic import BaseModel, Field\n\n# 1. Define schema\nclass Tweet(BaseModel):\n    topic: str = Field(..., title=\"Topic\")\n    tweet: str = Field(..., title=\"Tweet\")\n\n# 2. Create a dataset\ndataset = DriaDataset(name=\"tweet_test\", description=\"A dataset of tweets!\", schema=Tweet)\n\n# 3. Prepare instructions\ninstructions = [\n    {\"topic\": \"BadBadNotGood\"},\n    {\"topic\": \"Decentralized Synthetic Data\"}\n]\n\n# 4. Create a Prompt\nprompter = Prompt(prompt=\"Write a tweet about {{topic}}\", schema=Tweet)\n\n# 5. Generate data\ngenerator = DatasetGenerator(dataset=dataset)\n\nasyncio.run(\n    generator.generate(\n        instructions=instructions,\n        singletons=prompter,\n        models=Model.GPT4O\n    )\n)\n\n# Convert to Pandas\ndf = dataset.to_pandas()\nprint(df)\n```\n\n**Output**:\n```\n                         topic                                              tweet\n0                BadBadNotGood  🎶 Thrilled to have discovered #BadBadNotGood! ...\n1  Decentralized Synthetic Data  Exploring the future of #AI with decentralized...\n```\n\n---\n\n## Usage\n\n### 1. Define Your Dataset Schema\n\nUse [Pydantic](https://pydantic-docs.helpmanual.io/) models to define the structure of your synthetic data. For example:\n\n```python\nfrom pydantic import BaseModel, Field\n\nclass Tweet(BaseModel):\n    topic: str = Field(..., title=\"Topic\")\n    tweet: str = Field(..., title=\"Tweet\")\n```\n\n### 2. Create a Dataset\n\nInstantiate a `DriaDataset` by specifying its name, description, and the Pydantic schema:\n\n```python\nfrom dria import DriaDataset\n\ndataset = DriaDataset(name=\"tweet_test\", description=\"A dataset of tweets!\", schema=Tweet)\n```\n\n### 3. Write a Prompt\n\nUse `Prompt` objects to define how to generate data from an instruction. You can reference fields using double-curly braces:\n\n```python\nfrom dria import Prompt\n\nprompter = Prompt(\n    prompt=\"Write a tweet about {{topic}}\",\n    schema=Tweet\n)\n```\n\n### 4. Generate Synthetic Data\n\nCreate a `DatasetGenerator` and call `generate`:\n\n```python\nfrom dria import DatasetGenerator, Model\n\ngenerator = DatasetGenerator(dataset=dataset)\n\ninstructions = [{\"topic\": \"Cats\"}, {\"topic\": \"Dogs\"}]\n\nawait generator.generate(\n    instructions=instructions,\n    singletons=prompter,\n    models=Model.GPT4O  # Example model\n)\n```\n\n- `instructions`: A list of dictionaries, each used to fill the placeholders in your `Prompt`.\n- `singletons`: A single prompt (or list of prompts) that is applied to all instructions.\n- `models`: The model or list of models you want to use.\n\n### 5. Convert to Pandas\n\nFinally, convert your generated dataset to a Pandas DataFrame:\n\n```python\nimport pandas as pd\n\ndf = dataset.to_pandas()\nprint(df)\n```\n\n\u003e Dria supports a wide range of data type exports. You can see the full list [here](https://docs.dria.co/how-to/dria_datasets_exports). You will need to have some tokens in your balance, which will be approved automatically if required by the register command.\n\n---\n\n## Advanced Usage\n\n### Available Models\n\nDria supports a wide range of models from OpenAI, Gemini, Ollama, and more. You can see the full list [here](https://docs.dria.co/how-to/models).\n\n### Writing Workflows and Custom Pipelines\n\nDria allows you to write custom workflows and pipelines using the `Workflow` class. You can see an example of this [here](https://docs.dria.co/how-to/workflows).\n\n\n### Structured Outputs\n\nDria allows you to define custom schemas for your outputs using Pydantic. This allows you to generate highly structured data that can be used for a wide range of applications.\n\nYou can see an example of this [here](https://docs.dria.co/how-to/structured_outputs/).\n\n### Parallelization \u0026 Offloading\n\nBecause Dria tasks can be dispatched to a distributed network of agents, you can leverage **massive parallelization** without owning any GPUs. This is especially helpful for large-scale data generation tasks:\n- Avoid timeouts or rate limits by distributing tasks.\n- Scale to thousands or millions of records quickly.\n\n---\n\n## Contributing\n\nContributions are more than welcome! To get started:\n\n1. **Fork** the repository on GitHub.  \n2. **Clone** your fork locally and create a new branch for your feature or fix.\n3. **Install** dependencies in a virtual environment:  \n   ```bash\n   poetry install\n   ```\n4. **Make Changes** and **Test** them thoroughly.\n5. **Submit a Pull Request** describing your changes.\n\nWe value all contributions—from bug reports and suggestions to feature implementations.\n\n---\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE). Feel free to use it in your personal or commercial projects.\n\n---","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirstbatchxyz%2Fdria-sdk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffirstbatchxyz%2Fdria-sdk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirstbatchxyz%2Fdria-sdk/lists"}