{"id":22302602,"url":"https://github.com/dataoneorg/bookkeeper","last_synced_at":"2026-02-04T22:33:51.235Z","repository":{"id":43829359,"uuid":"235205484","full_name":"DataONEorg/bookkeeper","owner":"DataONEorg","description":"Bookkeeper keeps track of DataONE product subscriptions and quotas for researchers using the extended services.","archived":false,"fork":false,"pushed_at":"2024-11-04T06:06:03.000Z","size":1393,"stargazers_count":1,"open_issues_count":25,"forks_count":2,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-07-29T03:56:48.299Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataONEorg.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-01-20T21:49:21.000Z","updated_at":"2023-04-13T07:46:17.000Z","dependencies_parsed_at":"2024-08-08T03:16:32.296Z","dependency_job_id":"fc811de0-daf3-4703-b912-9dd3c88bf57f","html_url":"https://github.com/DataONEorg/bookkeeper","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/DataONEorg/bookkeeper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataONEorg%2Fbookkeeper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataONEorg%2Fbookkeeper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataONEorg%2Fbookkeeper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataONEorg%2Fbookkeeper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataONEorg","download_url":"https://codeload.github.com/DataONEorg/bookkeeper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataONEorg%2Fbookkeeper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29098235,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T21:05:08.033Z","status":"ssl_error","status_checked_at":"2026-02-04T21:04:53.031Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-03T18:39:59.976Z","updated_at":"2026-02-04T22:33:51.220Z","avatar_url":"https://github.com/DataONEorg.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"DataONE Bookkeeper Service\n==========================\n\nBookkeeper is a RESTful service that is part of the DataONE_ Coordinating Node\ninfrastructure. It is used to manage products, customers, orders, and quotas related to the\nDataONE Plus and Hosted Repositories services.  See the ``docs`` for more details.\n\n.. _DataONE: https://dataone.org/\n\nInstallation via Helm\n---------------------\n\nThe bookkeeper app is packaged as a Helm chart and can be deployed on a Kubernetes cluster with\na release name of `bookkeeper` using helm. First, one manual step is to edit `./helm/config/bookkeeper.yml` \nand set the proper database password. Then install via help with:\n\n.. code:: bash\n\n   $ helm upgrade --install -n bookkeeper bookkeeper ./helm\n\nNote that the helm install is not currently putting the database in place, and so some of the steps \nbelow need to be performed manually to get the database set up. These will all be migrated to helm with time.\n\nInstallation\n------------\n\n#. Download Bookkeeper and expand the archive\n\n.. code:: bash\n\n    $ curl -LO \"https://github.com/DataONEorg/bookkeeper/archive/master.zip\" \u0026\u0026 \\\n      unzip master.zip\n      \n#. Install PostgreSQL 9+. See the homebrew installation instructions.\n\n.. code:: bash\n\n    $ brew install postgresql\n    # Configure PostgreSQL appropriately to connect (pg_hba.conf, etc.)\n\n#. Create a database and user\n\n.. code:: bash\n\n    $ sudo su - postgres -c \"createdb -O bookkeeper bookkeeper\"\n    $ sudo su - postgres -c \"createuser bookkeeper -P\"\n\n#. Install FlywayDB_\n\n.. _FlywayDB: https://flywaydb.org\n\nFlyway is used to manage the SQL database schema.\n\n    **Note**: Adjust the values in ``db/flyway.conf`` to your database and password\n\n.. code:: bash\n\n    $ flyway migrate -configFiles=bookkeeper-master/src/main/resources/db \n\nThe database schema should be created, and you're ready to start the application.\n\n#. Compile and start bookkeeper\n\n.. code:: bash\n\n    $ cd  bookkeeper-master \u0026\u0026 mvn package\n    $ java -jar target/bookkeeper-1.0-SNAPSHOT.jar server bookkeeper.yml\n    \nGetting Started\n---------------\n\nThe following example uses ``bash`` and ``curl`` to demonstrate client connections.\n\nInteracting with the service requires authentication for most REST method\ncalls using a JSON Web Token.  Log in at https://search.daataone.org to get\na token (in ``My Profile \u003e Settings \u003e Authentication Token``).  All messages are transferred as JSON.\n\n#. Set your token\n\n.. code::\n    \n    $ token=\"\u003cpaste-your-token-here\u003e\"\n\n#. Get a list of ``Products`` that can be ordered. The ``/products`` endpoint is one that doesn't require authentication so pricing pages can be built.\n\n.. code::\n    \n    $ curl -H \"Accept: application/json\" \"http://localhost:8080/bookkeeper/v1/products\"\n\nThis returns a ``ProductList``:\n\n.. code-block:: json\n    \n    {\n        \"products\": [\n            {\n                \"id\": 1,\n                \"object\": \"product\",\n                \"active\": true,\n                \"amount\": 32000,\n                \"caption\": \"Faculty or research lab\",\n                \"currency\": \"USD\",\n                \"created\": 1579898043,\n                \"description\": \"Create a customized portal for your work and projects.   Help others understand and access your data.\",\n                \"interval\": \"year\",\n                \"name\": \"Individual\",\n                \"statementDescriptor\": \"DataONE Subscription Plan - Individual\",\n                \"type\": \"service\",\n                \"unitLabel\": \"membership\",\n                \"url\": \"https://products.dataone.org/plus\",\n                \"metadata\": {\n                    \"features\": [\n                        {\n                            \"name\": \"branded_portal\",\n                            \"label\": \"Branded Portals\",\n                            \"description\": \"Showcase your research, data, results, and usage metrics by building a custom web portal.\",\n                            \"quota\": {\n                                \"object\": \"quota\",\n                                \"name\": \"portal\",\n                                \"softLimit\": \"1\",\n                                \"hardLimit\": \"1\",\n                                \"unit\": \"portal\"\n                            }\n                        },\n                        {\n                            \"name\": \"custom_search_filters\",\n                            \"label\": \"Custom Search Filters\",\n                            \"description\": \"Create custom search filters in your portal to allow   scientists to search your holdings using filters appropriate to your field of science.\"\n                        },\n                        {\n                            \"name\": \"fair_data_assessment\",\n                            \"label\": \"FAIR Data Assessments\",\n                            \"description\": \"Access quality metric reports using the FAIR data suite of checks.\"\n                        },\n                        {\n                            \"name\": \"custom_quality_service\",\n                            \"label\": \"Custom Quality Metrics\",\n                            \"description\": \"Create a suite of custom quality metadata checks specific to your datasets.\"\n                        },\n                        {\n                            \"name\": \"aggregated_metrics\",\n                            \"label\": \"Aggregated Metrics\",\n                            \"description\": \"Access and share reports on aggregated usage metrics such as dataset views, data downloads, and dataset citations.\"\n                        },\n                        {\n                            \"name\": \"dataone_voting_member\",\n                            \"label\": \"DataONE Voting Member\",\n                            \"description\": \"Vote on the direction and priorities at DataONE Community meetings.\"\n                        }\n                    ]\n                }\n            },\n            ...\n        ]\n    }\n    \n#. Create a ``Customer`` with the given name, surname, and ORCID of the logged in user (saved as ``customer.json``):\n\n.. code-block:: json\n  \n    {\n        \"object\": \"customer\",\n        \"givenName\": \"Christopher\",\n        \"surName\": \"Jones\",\n        \"email\": \"cjones@nceas.ucsb.edu\",\n        \"subject\": \"http://orcid.org/0000-0002-8121-2343\"\n    }\n\nThen ``POST`` it to the ``/customers`` endpoint:\n\n.. code-block:: bash\n\n    curl -X POST \\\n        -H \"Authorization: Bearer ${token}\" \\\n        -H \"Content-Type: application/json\" \\\n        -H \"Accept: application/json\" \\\n        -d \"@customer.json\" \\\n        \"http://localhost:8080/bookkeeper/v1/customers\"\n\nThe customer object is returned with an ``id`` attribute which is used to  create an ``Order``.\n\n.. code-block:: json\n  \n    {\n        \"id\": 1,\n        \"object\": \"customer\",\n        \"givenName\": \"Christopher\",\n        \"surName\": \"Jones\",\n        \"email\": \"cjones@nceas.ucsb.edu\",\n        \"subject\": \"http://orcid.org/0000-0002-8121-2343\"\n    }\n\n#. Create an ``Order`` (``order.json``), and update it as many times as needed. ``POST`` the order to the ``/orders`` endpoint, and ``PUT`` it to ``/orders/:id`` for updates.\n\n.. code:: json\n\n    {\n        \"object\": \"order\",\n        \"customer\": 1,\n        \"status\": \"created\",\n        \"amount\": 32000,\n        \"items\": [{\n            \"object\": \"order_item\",\n            \"type\": \"sku\",\n            \"parent\": 1, \n            \"quantity\": 1\n            }]\n    }\n    \nThen ``POST`` it to the ``/orders`` endpoint:\n\n.. code-block:: bash\n\n    curl -X POST \\\n        -H \"Authorization: Bearer ${token}\" \\\n        -H \"Content-Type: application/json\" \\\n        -H \"Accept: application/json\" \\\n        -d \"@order.json\" \\\n        \"http://localhost:8080/bookkeeper/v1/orders\"\n\nThis returns:\n\n.. code:: json\n\n    {\n        \"id\": 1,\n        \"object\": \"order\",\n        \"amount\": 32000,\n        \"amountReturned\": 0,\n        \"charge\": {},\n        \"created\": 1579986378,\n        \"customer\": 1,\n        \"items\": [\n            {\n                \"object\": \"order_item\",\n                \"amount\": 32000,\n                \"currency\": \"USD\",\n                \"description\": \"DataONE Subscription Plan - Individual\",\n                \"parent\": 1,\n                \"quantity\": 1,\n                \"type\": \"sku\"\n            }\n        ],\n        \"metadata\": {},\n        \"status\": \"created\",\n        \"statusTransitions\": {},\n        \"updated\": 0,\n        \"totalAmount\": 32000\n    }\n    \n#. Confirm the ``Order``.  Confirming currently sets a **trial period** rather than paying for the order.  Payments will be added into this workflow later.\n\n.. code-block:: bash\n\n    curl -X POST \\\n        -H \"Authorization: Bearer ${token}\" \\\n        -H \"Content-Type: application/json\" \\\n        -H \"Accept: application/json\" \\\n        \"http://localhost:8080/bookkeeper/v1/orders/1/pay\"\n\n#. You have confirmed the order, and it is in the ``paid`` state.  This returns:\n\n.. code:: json\n\n    {\n        \"id\": 1,\n        \"object\": \"order\",\n        \"amount\": 32000,\n        \"amountReturned\": 0,\n        \"charge\": {},\n        \"created\": 1579986378,\n        \"customer\": 1,\n        \"items\": [{\n            \"object\": \"order_item\",\n            \"amount\": 32000,\n            \"currency\": \"USD\",\n            \"description\": \"DataONE Subscription Plan - Individual\",\n            \"parent\": 1,\n            \"quantity\": 1,\n            \"type\": \"sku\"\n            }],\n            \"metadata\": {},\n            \"status\": \"paid\",\n            \"statusTransitions\": {},\n            \"updated\": 1579992719,\n            \"totalAmount\": 32000\n        }\n    }\n    \n#. View your quotas.  Once the order is paid, your quotas are set.\n\n.. code:: bash\n\n    curl \\\n        -H \"Authorization: Bearer ${token}\" \\\n        -H \"Accept: application/json\" \\\n        \"http://localhost:8080/bookkeeper/v1/quotas\"\n\nThis returns a ``QuotaList``:\n\n.. code:: json\n\n    {\n        \"quotas\": [{\n            \"id\": 4,\n            \"object\": \"quota\",\n            \"name\": \"portal\",\n            \"softLimit\": 1.0,\n            \"hardLimit\": 1.0,\n            \"usage\": 0.0,\n            \"unit\": \"portal\",\n            \"orderId\": 1,\n            \"subject\": \"http://orcid.org/0000-0002-8121-2341\"\n        }]\n    }\n\nCreating an object\n------------------\n\nOnce quotas are established through an order, the quotas are\nenforced by participating repositories.  The following diagram shows the sequence of calls made\nwhen a portal document is uploaded to a repository.\n\n..\n    @startuml ./docs/images/create-portal.png\n    !include ./docs/plantuml-styles.txt\n\n    autonumber \"\u003cfont color=999999\u003e\"\n    title \"Uploading a portal document\"\n    actor Researcher\n    participant Client\n    participant Repository \u003c\u003cService\u003e\u003e\n    participant Bookkeeper \u003c\u003cService\u003e\u003e\n    participant \"CN\" \u003c\u003cService\u003e\u003e\n\n    Researcher o-\u003e Client: Chooses \"Edit\" for a listed portal\n    activate Client\n        Client -\u003e Bookkeeper: getUsage(session, sid, quotaName)\n    deactivate Client\n\n    activate Bookkeeper\n        Bookkeeper --\u003e Client: usage\n    deactivate Bookkeeper\n\n    note right\n        Checks usages of the given pid or sid\n        for the given quota name \"portal\"\n    end note\n\n    activate Client\n        Client --\u003e Researcher: editor view\n    deactivate Client\n\n    activate Researcher\n        Researcher -\u003e Client : Chooses \"Save\" after editing a portal\n    deactivate Researcher\n\n    note right\n        Note that the Client can optionally\n        call Bookkeeper's hasRemaining() to\n        proactively check quotas before\n        allowing a new portal editor session.\n    end note\n\n    activate Client\n        Client -\u003e Repository : create(session, pid, object, sysmeta)\n    deactivate Client\n\n    activate Repository\n        note left\n            For all calls, the JWT\n            authentication token in the\n            Authorization HTTP header\n            represents the session\n        end note\n        note right\n            The Client sends the quota subject\n            in the X-DataONE-QuotaSubject HTTP header\n        end note\n\n        Repository -\u003e Bookkeeper : \"hasRemaining(session, quotaSubject, \\nrequestorSubject, quotaName, requestedUsage)\"\n    deactivate Repository\n\n    activate Bookkeeper\n        note right\n            Bookkeeper caches subjectInfo\n            about the requestor to minimize\n            calls to the CN. Also, for portal\n            documents, call hasRemaining()\n            twice, once to check portal quotas,\n            once to check storage quotas.\n        end note\n        Bookkeeper -\u003e CN : getSubjectInfo(session, requestorSubject)\n    deactivate Bookkeeper\n\n    activate CN\n        CN --\u003e Bookkeeper: subjectInfo\n    deactivate CN\n\n    activate Bookkeeper\n        note left\n            If the insert doesn't exceed \n            the quota limit, the quota is\n            returned, otherwise an exception.\n        end note\n        Bookkeeper --\u003e Repository : quota\n    deactivate Bookkeeper\n\n    activate Repository\n        Repository -\u003e Repository : create(session, pid, object, sysmeta)\n        Repository --\u003e Client : pid\n        activate Client\n            Client --\u003e Researcher : Indicates portal is saved\n        deactivate Client\n        Repository --\u003e Bookkeeper : updateUsage(session, quotaId, usage)\n    deactivate Repository\n    \n    activate Bookkeeper\n        note right\n            The repository asynchronously \n            updates the usage for the portal count\n            by sending the usage object with the\n            instanceId (portal id) and quotaId. \n            Note the quotaId comes from the quota \n            returned from hasRemaining().  If the\n            portal has both a pid and sid, call\n            updateUsage() twice. The call with the sid \n            usage should have a quantity of zero.\n        end note\n        Bookkeeper --\u003e Repository : quota\n    deactivate Bookkeeper\n    \n    @enduml\n\n\n.. image:: docs/images/create-portal.png\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdataoneorg%2Fbookkeeper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdataoneorg%2Fbookkeeper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdataoneorg%2Fbookkeeper/lists"}