{"id":24867076,"url":"https://github.com/simonsobs/librarian","last_synced_at":"2026-02-10T17:11:51.111Z","repository":{"id":88678773,"uuid":"241146810","full_name":"simonsobs/librarian","owner":"simonsobs","description":"The Simons Observatory Librarian, previously forked from the HERA Librarian and completely re-written for the needs of (A)SO","archived":false,"fork":false,"pushed_at":"2026-02-02T20:54:22.000Z","size":3574,"stargazers_count":2,"open_issues_count":30,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-03T10:46:26.112Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"HERA-Team/librarian","license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonsobs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-02-17T15:52:09.000Z","updated_at":"2026-02-02T20:54:26.000Z","dependencies_parsed_at":"2024-12-20T15:42:01.522Z","dependency_job_id":"ce1075bd-8d8d-4ca6-b11b-899cddf563dc","html_url":"https://github.com/simonsobs/librarian","commit_stats":null,"previous_names":[],"tags_count":35,"template":false,"template_full_name":null,"purl":"pkg:github/simonsobs/librarian","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsobs%2Flibrarian","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsobs%2Flibrarian/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsobs%2Flibrarian/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsobs%2Flibrarian/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonsobs","download_url":"https://codeload.github.com/simonsobs/librarian/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsobs%2Flibrarian/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29309194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-10T16:09:25.305Z","status":"ssl_error","status_checked_at":"2026-02-10T16:08:52.170Z","response_time":65,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-01T01:56:35.169Z","updated_at":"2026-02-10T17:11:51.093Z","avatar_url":"https://github.com/simonsobs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The Librarian\n\n[![Coverage Status](https://coveralls.io/repos/github/simonsobs/librarian/badge.svg?branch=main)](https://coveralls.io/github/simonsobs/librarian?branch=main)\n[![Docs Status](https://readthedocs.org/projects/solibrarian/badge/?version=latest\u0026style=flat)](https://solibrarian.readthedocs.io)\n\nThe Librarian is a system for data transfer orchestration designed for the\nSimons Observatory. The Librarian is based upon the HERA Librarian framework\n(and even retains its namesake for the client library), but was entirely\nre-written for this new workload. The Librarian's architechture is documented\nin the Scipy 2024 proceeding describing the software.\n\nThe goal of the Librarian is to track and transfer primary data products\ngenerated at remote sites through a push-based system. The first push is\nfrom a client using the `librarian upload` command, ingesting files into\nthe system. From there, redundant copies may be made locally or remotely,\nwith support for SneakerNet (i.e. by movement of physical media) transfers.\n\nThere are a number of sub-packages in this repository:\n\n- `hera_librarian`, containing the client code for the system. This is how\n  most users communicate with the librarian.\n- `librarain_server`, containing the main HTTP REST API server and database\n  models.\n- `librarian_background`, containing the code for background tasks that\n  run on a fixed cadence, and have direct database access.\n- `librarian_server_scripts`, containing the initialization and example\n  run scripts that are used for the persistent service.\n- `tests` containing a large suite of tests for the application.\n- `alembic` containing database migration scripts.\n\nThese individual components are all documented separately. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonsobs%2Flibrarian","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonsobs%2Flibrarian","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonsobs%2Flibrarian/lists"}