{"id":34075267,"url":"https://github.com/whiskyechobravo/karboni","last_synced_at":"2026-04-24T13:31:30.233Z","repository":{"id":327105728,"uuid":"1005888222","full_name":"whiskyechobravo/karboni","owner":"whiskyechobravo","description":"Mirror a Zotero library into a SQL database","archived":false,"fork":false,"pushed_at":"2025-12-16T05:11:08.000Z","size":64,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-16T10:32:55.651Z","etag":null,"topics":["academia","cli-app","database","python","scholar","sql","zotero","zotero-api"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/whiskyechobravo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-21T02:54:46.000Z","updated_at":"2025-12-16T05:11:12.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/whiskyechobravo/karboni","commit_stats":null,"previous_names":["whiskyechobravo/karboni"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/whiskyechobravo/karboni","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/whiskyechobravo%2Fkarboni","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/whiskyechobravo%2Fkarboni/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/whiskyechobravo%2Fkarboni/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/whiskyechobravo%2Fkarboni/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/whiskyechobravo","download_url":"https://codeload.github.com/whiskyechobravo/karboni/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/whiskyechobravo%2Fkarboni/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32225732,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T13:21:15.438Z","status":"ssl_error","status_checked_at":"2026-04-24T13:21:15.005Z","response_time":64,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["academia","cli-app","database","python","scholar","sql","zotero","zotero-api"],"created_at":"2025-12-14T09:12:33.195Z","updated_at":"2026-04-24T13:31:30.226Z","avatar_url":"https://github.com/whiskyechobravo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Karboni\n\nMirror a Zotero library into a SQL database.\n\n\n## Features\n\n- Fast one-way synchronization from Zotero to a SQL database.\n- Fetch library items, collections, tags, saved searches metadata.\n- Download file attachments.\n- Fetch formatted references in multiple bibliographic styles, for multiple\n  locales.\n- Fetch multiple export formats.\n- Fetch the full text content of items.\n- Fetch the labels of item types, field names and creator types, for multiple\n  locales.\n- Python API for managing synchronization.\n- Command line interface for managing synchronization.\n- Support for a wide range of database systems (through SQLAlchemy).\n\n\n## Installation\n\nIt is recommended that you install the package in a virtual environment.\n\nThe installation steps might look like the following. Replace `DIR` with the\ndesired path for your new virtual environment.\n\n### Unix/macOS\n\nCreate the virtual environment:\n\n```sh\npython3 -m venv DIR\n```\n\nActivate the virtual environment:\n\n```sh\nsource DIR/bin/activate\n```\n\nInstall Karboni:\n\n```sh\npython3 -m pip install karboni\n```\n\n### Windows\n\nCreate the virtual environment:\n\n```\npy -m venv DIR\n```\n\nActivate the virtual environment:\n\n```\nDIR\\Scripts\\activate\n```\n\nInstall Karboni:\n\n```\npy -m pip install karboni\n```\n\n\n## Command line interface\n\nIn order to use the command line interface, you must first configure your Zotero\ncredentials. With a text editor, create a `.env` file in your working directory\nwith the following content:\n\n```\nZOTERO_LIBRARY_PREFIX=your_library_prefix\nZOTERO_LIBRARY_ID=your_library_id\nZOTERO_API_KEY=your_api_key\n```\n\nReplace `your_library_prefix` with `users` for a personal library, or `groups`\nfor a group library.\n\nReplace `your_library_id` with the identifier of your library. For a personal\nlibrary the value is your user ID, as found on\nhttps://www.zotero.org/settings/keys (you must be logged-in). For a group\nlibrary this value is the group ID of the library, as found in the URL of the\nlibrary (e.g., the group ID of the library at\n`https://www.zotero.org/groups/1234567/example` is `1234567`).\n\nReplace `your_api_key` with your Zotero API key. You may create one for your\nlibrary on https://www.zotero.org/settings/keys/new (you must be logged-in).\nKarboni does not need to write to your library. Thus, we recommend that your API\nkey be read-only, and that it does not grant any more access to your Zotero data\nthan strictly necessary.\n\nBy default, Karboni commands will manage data in a `data/karboni` directory\nunder your current directory, and use SQLite as the relational database. You may\nchange those defaults by setting the following variables in your `.env` file:\n\n- `KARBONI_DATA_PATH`. Defaults to\n  `./data/karboni/ZOTERO_LIBRARY_PREFIX-ZOTERO_LIBRARY_ID/`. If the directory\n  does not already exists, Karboni will create it.\n- `KARBONI_DATABASE_URL`. Defaults to\n  `sqlite:///data/karboni/ZOTERO_LIBRARY_PREFIX-ZOTERO_LIBRARY_ID/library.sqlite`.\n  When using SQLite, the directory specified in the database URL must either\n  exist prior to running the Karboni command, or match the directory specified\n  by `KARBONI_DATA_PATH`. For other relational databases, see the [SQLAlchemy\n  documentation on database\n  URLs](https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls).\n  While SQLite support is readily available through the Python standard library,\n  other database backends usually require that you install additional Python\n  packages.\n\nOnce the required variables have been set, you may use Karboni commands. If you\nhave installed Karboni in a virtual environment, make sure it is active before\nattempting to use the commands (see the activation command in the Installation\nsection). Some example commands below.\n\nInitialize the mirror database (create the tables):\n\n```sh\nkarboni init\n```\n\nSynchronize from Zotero:\n\n```sh\nkarboni sync\n```\n\nList the available commands and general options:\n\n```sh\nkarboni --help\n```\n\nList the options of a specific command:\n\n```sh\nkarboni COMMAND --help\n```\n\nA more complex example, synchronizing from Zotero with some data options enabled\n— format references in APA and Vancouver styles, fetch BibTeX and RIS formats,\ndownload file attachments, fetch any available full text:\n\n```sh\nkarboni sync --style apa --style vancouver --export-format bibtex --export-format ris --files --fulltext\n```\n\nOnce an initial synchronization has completed, subsequent invocations of the\n`karboni sync` command will perform incremental synchronization by default,\ni.e., fetching just the modified data from Zotero. However, that only works if\nyou use the same data options as on the initial synchronization. To change the\ndata options, add the `--full` option to force a full synchronization. For\nexample:\n\n```sh\nkarboni sync --style apa-5th-edition --full\n```\n\nFor the formatting styles available for the `--style` option, refer to the\n[Zotero Style Repository](https://www.zotero.org/styles/).\n\nFor the export formats available for the `--export-format` option, refer to the\n[Zotero API documentation on export\nformats](https://www.zotero.org/support/dev/web_api/v3/basics#item_export_formats).\n\nFor the locales available for the `--locale` option, refer to the [Citation\nStyle Language locales](https://github.com/citation-style-language/locales).\nNote that some styles use a fixed locale and will ignore the `--locale` option.\n\n\n## Python interface\n\nThe `karboni` Python module provides the main entry points, with functions such\nas `initialize()` and `synchronize()`.\n\nIf you wish to use the SQLAlchemy ORM to query the database, you might want to\nimport models from `karboni.database.schema`.\n\n\n## Design choices\n\nHere are some of the design choices that have guided the development of Karboni:\n\n- Perform Zotero API requests and file IO asynchronously to minimize idle time.\n- Use SQLite as the baseline database system (reducing the need for additional\n  dependencies), but interface it through SQLAlchemy in order to support other\n  databases as well.\n- Since Karboni itself only needs a few simple database operations, encapsulate\n  SQLAlchemy under a thin abstraction layer to decouple the synchronization\n  process from the database toolkit.\n- Stay close to the Zotero schema. Store data in the JSON format provided by the\n  Zotero API whenever possible, for consistency and better adaptability to\n  future Zotero schema changes. Add SQL columns where they can be useful to the\n  synchronization process or to allow basic queries.\n- Don't fuss too much with database-level referential integrity constraints.\n  Leave that to Zotero. In particular, the keys of parent items and parent\n  collections are not validated (this simplifies the synchronization process).\n- Don't worry about database schema migrations. The database is just a mirror,\n  thus its tables can be wiped when necessary and re-synchronized from Zotero.\n- Synchronization of file attachments is not atomic. If library synchronization\n  finishes but file downloads fail, we accept that and don't rollback the\n  database changes.\n\n\n## Known limitations\n\n- Database operations are synchronous because SQLAlchemy cannot (at least not\n  easily) share a session between concurrent tasks.\n- During transactions, SQLite locks database access from other threads or\n  processes. When synchronizing from Zotero, Karboni applies all changes in a\n  single transaction (to allow rollback in case of failure), which means the\n  database can remain locked for some time. To ensure availability during\n  synchronization, use a database system that has more advanced locking\n  mechanisms (such as PostgreSQL or MariaDB/MySQL).\n- Python 3.11+ is required (it facilitates exception handling with asynchronous\n  tasks).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwhiskyechobravo%2Fkarboni","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwhiskyechobravo%2Fkarboni","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwhiskyechobravo%2Fkarboni/lists"}