{"id":38776534,"url":"https://github.com/eyra/feldspar","last_synced_at":"2026-05-11T22:01:46.222Z","repository":{"id":209155295,"uuid":"723364320","full_name":"eyra/feldspar","owner":"eyra","description":"Easily build custom apps for the Next platform","archived":false,"fork":false,"pushed_at":"2026-05-04T20:23:45.000Z","size":42546,"stargazers_count":15,"open_issues_count":10,"forks_count":56,"subscribers_count":0,"default_branch":"develop","last_synced_at":"2026-05-04T22:23:42.156Z","etag":null,"topics":["custom-app","data-donation","next-platform","port"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eyra.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-25T12:41:07.000Z","updated_at":"2026-05-04T20:21:15.000Z","dependencies_parsed_at":"2026-03-03T03:09:54.653Z","dependency_job_id":null,"html_url":"https://github.com/eyra/feldspar","commit_stats":null,"previous_names":["eyra/feldspar"],"tags_count":112,"template":false,"template_full_name":null,"purl":"pkg:github/eyra/feldspar","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eyra%2Ffeldspar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eyra%2Ffeldspar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eyra%2Ffeldspar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eyra%2Ffeldspar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eyra","download_url":"https://codeload.github.com/eyra/feldspar/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eyra%2Ffeldspar/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32914533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-11T17:09:15.040Z","status":"ssl_error","status_checked_at":"2026-05-11T17:08:45.420Z","response_time":120,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["custom-app","data-donation","next-platform","port"],"created_at":"2026-01-17T12:14:22.812Z","updated_at":"2026-05-11T22:01:46.217Z","avatar_url":"https://github.com/eyra.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Feldspar\n\nFeldspar is an integration mechanism for building data donation applications that can be hosted on the [Next](https://next.eyra.co/) platform. It enables researchers to create custom data extraction and donation flows using Python and React.\n\n## Digital Trace Data Donation (Port)\n\nMore information about the Port program can be found [here](https://eyra.notion.site/Port-Program-4bbf0bbc466547af95f05c609405c4b2?pvs=4).\n\nFeldspar enables researchers to:\n\n- Extract only the data of interest through local processing (on the participant's device) using Python (Pyodide)\n- Prompt participants for questions about the data\n- Enable participants to inspect the extracted data before donation\n- Enable participants to delete table rows before donation\n- Consent or decline to donate the extracted data\n\n## Getting Started\n\n### Prerequisites\n\n- Fork or clone this repo\n- Install [Node.js](https://nodejs.org/en)\n- Install [pnpm](https://pnpm.io/installation) (Fast, disk space efficient package manager)\n- Install [Python](https://www.python.org/) (Version 3.11 or higher)\n- Install [Poetry](https://python-poetry.org/)\n- Install [Earthly CLI](https://earthly.dev/get-earthly)\n\n### Installation\n\n1. Install dependencies:\n\n   ```sh\n   pnpm install\n   ```\n\n2. Run the project locally with hot reloading (builds Python package and starts the development server):\n\n   ```sh\n   pnpm run start\n   ```\n\n3. Access the application at [http://localhost:3000](http://localhost:3000)\n\n## Customizing the Python Code\n\nThe core of Feldspar's functionality is in the Python script at `packages/python/port/script.py`. This script defines the flow of the data donation process.\n\n### Basic Structure\n\n1. Fork the repository to create your own version\n2. Navigate to `packages/python/port/script.py`\n3. Modify the `process(sessionId)` function to customize your data donation flow\n\nA basic donation flow typically includes:\n\n1. Prompt the participant to select a file\n2. Extract relevant data from the file\n3. Present the extracted data in a consent form\n4. Process the participant's consent decision\n\n### Example: Modifying the File Selection Screen\n\n```python\ndef prompt_file(extensions):\n    description = props.Translatable({\n        \"en\": \"Please select your data export file.\",\n        \"de\": \"Bitte wählen Sie Ihre Datenexportdatei aus.\",\n        \"it\": \"Seleziona il tuo file di esportazione dati.\",\n        \"es\": \"Por favor, seleccione su archivo de exportación de datos.\",\n        \"nl\": \"Selecteer uw data-exportbestand.\"\n    })\n    return props.PropsUIPromptFileInput(description, extensions)\n```\n\n### Working with Assets\n\nAdd any static assets your script needs to `packages/python/port/assets/`. Access them in your script:\n\n```python\nfrom port.api.assets import *\n\ndef process(sessionId):\n    # Path to an asset\n    path = asset_path(\"my_file.txt\")\n\n    # Open an asset directly\n    file = open_asset(\"my_file.txt\")\n\n    # Read asset contents\n    content = read_asset(\"my_file.txt\")\n```\n\n### Data Frame Size limits\n\nRow limits for data frames in the Props UI:\n- Consent Table: default maximum of 10,000 rows (configurable)\n- UI hard cap: 50,000 rows (cannot be exceeded)\n\nFor larger datasets, pre-aggregate or sample before display, and review your informed consent and privacy guidelines.\n\n### Local extraction debugging (CLI)\n\nYou can run the extraction locally against a real zip file — no browser or Pyodide needed:\n\n```bash\ncd packages/python\npoetry run python -m port.script path/to/file.zip\n```\n\nThis drives `extract_data()` directly and prints each extracted table to the terminal. Useful for quickly verifying that your extraction logic works before testing it in the browser.\n\n### Adding Dependencies\n\nIf you need additional Python packages, add them to `packages/python/pyproject.toml` in the `tool.poetry.dependencies` section.\n\n## Adding New Visual Components (Advanced)\n\nFeldspar allows you to add custom UI components that can be used in your Python script. This is a more advanced feature that requires understanding both Python and React.\n\n### Step 1: Define Component Types\n\nCreate a new folder in `packages/data-collector/src/components/my_component/` and add a `types.ts` file:\n\n```typescript\nexport interface PropsUIPromptMyComponent {\n  __type__: \"PropsUIPromptMyComponent\";\n  title: string;\n  // Add any other properties your component needs\n}\n```\n\n### Step 2: Create React Component\n\nAdd a `component.tsx` file to implement your component:\n\n```typescript\nimport React from \"react\";\nimport { PropsUIPromptMyComponent } from \"./types\";\nimport { ReactFactoryContext } from \"@eyra/feldspar\";\n\ntype Props = PropsUIPromptMyComponent \u0026 ReactFactoryContext;\n\nexport const MyComponent: React.FC\u003cProps\u003e = ({ title, resolve }) =\u003e {\n  return (\n    \u003cdiv\u003e\n      \u003ch1\u003e{title}\u003c/h1\u003e\n      \u003cbutton\n        onClick={() =\u003e resolve?.({ __type__: \"PayloadTrue\", value: true })}\n      \u003e\n        Continue\n      \u003c/button\u003e\n    \u003c/div\u003e\n  );\n};\n```\n\n### Step 3: Create Component Factory\n\nAdd a new file at `packages/data-collector/src/factories/my_component.tsx`:\n\n```typescript\nimport { PromptFactory, ReactFactoryContext } from \"@eyra/feldspar\";\nimport React from \"react\";\nimport { MyComponent } from \"../components/my_component/component\";\nimport { PropsUIPromptMyComponent } from \"../components/my_component/types\";\n\nexport class MyComponentFactory implements PromptFactory {\n  create(body: unknown, context: ReactFactoryContext) {\n    if (this.isMyComponent(body)) {\n      return \u003cMyComponent {...body} {...context} /\u003e;\n    }\n    return null;\n  }\n\n  private isMyComponent(body: unknown): body is PropsUIPromptMyComponent {\n    return (\n      (body as PropsUIPromptMyComponent).__type__ === \"PropsUIPromptMyComponent\"\n    );\n  }\n}\n```\n\n### Step 4: Register Your Component\n\nUpdate `packages/data-collector/src/App.tsx` to include your new factory:\n\n```typescript\nimport { DataSubmissionPageFactory, ScriptHostComponent } from \"@eyra/feldspar\";\nimport { HelloWorldFactory } from \"./factories/hello_world\";\nimport { MyComponentFactory } from \"./factories/my_component\";\n\nfunction App() {\n  return (\n    \u003cdiv className=\"App\"\u003e\n      \u003cScriptHostComponent\n        workerUrl=\"./py_worker.js\"\n        standalone={process.env.NODE_ENV !== \"production\"}\n        factories={[\n          new DataSubmissionPageFactory({\n            promptFactories: [\n              new HelloWorldFactory(),\n              new MyComponentFactory(), // Add your new factory here\n            ],\n          }),\n        ]}\n      /\u003e\n    \u003c/div\u003e\n  );\n}\n\nexport default App;\n```\n\n### Step 5: Use Your Component in Python\n\nAdd a class to your `script.py` to create your component:\n\n```python\nfrom dataclasses import dataclass\n\n@dataclass\nclass PropsUIPromptMyComponent:\n    title: str\n\n    def toDict(self):\n        dict = {}\n        dict[\"__type__\"] = \"PropsUIPromptMyComponent\"\n        dict[\"title\"] = self.title\n        return dict\n\ndef process(sessionId):\n    result = yield render_data_submission_page(\n        PropsUIPromptMyComponent(\"My Custom Component\")\n    )\n    # Handle the result...\n```\n\n## Creating a Release\n\nWhen your data donation application is ready for deployment:\n\n1. Create a release package:\n\n   ```sh\n   ./release.sh\n   ```\n\n2. Find the generated ZIP file in the `releases/` directory, named with the current date and sequential number (e.g., `feldspar_2023-07-15_1.zip`)\n\n3. This ZIP file can be deployed to:\n   - The Next platform\n   - A self-hosted environment\n   - Any server that can host static files and store the donated data\n\nTo use the release in the Next platform, add a \"Donate task\" and select the generated ZIP file as the \"Flow application\".\n\n## Important Disclaimer\n\nPlease review the [disclaimer](./DISCLAIMER.md) in this repository for important information about technical limitations, logging behavior, and data handling considerations.\n\n## Funding\n\nFeldspar is part of the Port program for data donation and has been funded by the UU, PDI-SSH ([D3i project](https://datadonation.eu/)), and [Eyra](https://www.eyra.co/).\n\n## Contributing\n\nWe welcome contributions to make Feldspar better. Please read our [contributing guidelines](https://github.com/eyra/feldspar/blob/master/CONTRIBUTING.md) for details on how to submit issues, feature requests, and pull requests.\n\n\u003c!-- Original detailed API examples and technical specifications can be included here --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feyra%2Ffeldspar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feyra%2Ffeldspar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feyra%2Ffeldspar/lists"}