{"id":23366857,"url":"https://github.com/davidcain/bibliophile","last_synced_at":"2025-08-02T00:07:48.923Z","repository":{"id":37901351,"uuid":"88684110","full_name":"DavidCain/bibliophile","owner":"DavidCain","description":"Find books at my local library that I want to read","archived":false,"fork":false,"pushed_at":"2023-01-23T20:57:16.000Z","size":4560,"stargazers_count":4,"open_issues_count":18,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-10T14:16:25.868Z","etag":null,"topics":["aws-lambda","goodreads","goodreads-api","python"],"latest_commit_sha":null,"homepage":"https://biblio.dcain.me","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DavidCain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-19T00:54:53.000Z","updated_at":"2024-07-23T14:31:34.000Z","dependencies_parsed_at":"2023-01-24T23:01:15.922Z","dependency_job_id":null,"html_url":"https://github.com/DavidCain/bibliophile","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/DavidCain/bibliophile","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCain%2Fbibliophile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCain%2Fbibliophile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCain%2Fbibliophile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCain%2Fbibliophile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DavidCain","download_url":"https://codeload.github.com/DavidCain/bibliophile/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidCain%2Fbibliophile/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268315948,"owners_count":24231059,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-lambda","goodreads","goodreads-api","python"],"created_at":"2024-12-21T14:18:54.725Z","updated_at":"2025-08-02T00:07:48.896Z","avatar_url":"https://github.com/DavidCain.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Find me something to read!\n\n[![A list of titles available at my local library][reading-list-img]][biblio]\n\nI wrote this utility to extract the most value from two services I love dearly:\n\n- Goodreads\n- My local public library\n\n## How I use it\n\nWhenever I come across a title I'd like to read some day, I store it on my\nGoodreads shelf. When I'd like to visit my local library branch, I visit\n[biblio.dcain.me][biblio] to see which titles are available to be checked out.\n\n## Why?\n\nMy local library branch does not have the most extensive collection. Instead of\nmeandering the stacks until I find a book I like, or fruitlessly querying the\ncatalog to see if that interesting new book is on the shelf, I'd much rather\nhave a script do the hard work for me.\n\n# Can I use this?\n\nThe [web interface][biblio] currently supports Alameda County, San Francisco,\nand Seattle, but if you live near one of the ~190 public libraries using the\nBiblioCommons system, then running this software locally should work for you.\nIt relies on undocumented APIs, so your mileage may vary.\n\nYou can also use just the [Python backend][bibliophile-backend] locally.\n\n# How does this work?\n\n- The [`bibliophile` Python package][bibliophile-backend] does the legwork of\n  querying Goodreads \u0026 BiblioCommons (respectively, these are the services\n  needed to find which books I'm interested in, and which books are available\n  at the library).\n- This repository defines some Lambda functions that are deployed to public\n  endpoints, accessible at api.dcain.me\n  - Function are configured with an API Gateway to enable a REST API.\n  - `serverless` provides automated deployment \u0026 configuration on AWS.\n- A [web app][bibliophile-frontend] provides the user interface on [biblio.dcain.me][biblio].\n\n## Deploying AWS Lambda functions\n\n1. [Download Docker][docker].\n   We choose to Dockerize the pip environment because both `grequests` and\n   `lxml` (dependencies of `bibliophile`) are C-based, and need to compile\n   binaries for use on the Lambda VM. By using Docker, we can make use of an\n   image that mirrors exactly what AWS will run in Lambda-land.\n2. Create an IAM user for `serverless` with permissions to create\n   CloudFormation stacks, S3 buckets, Lambda functions, and more.\n   Standard practice with `serverless` is to just grant the user an\n   administrator policy, though this is not ideal security.\n3. Create access keys for the `serverless` user\n4. (one-time) create the `customDomain` that the API will be served on:\n   ```\n   npm run serverless create_domain\n   ```\n   (Note that new domains may take up to 40 minutes to initialize)\n5. Deploy the latest version of the endpoint:\n   ```\n   npm run deploy\n   ```\n\nOnce the above is done, a simple end-to-end test:\n\n```\ncurl -X POST 'https://api.dcain.me/bibliophile/read_shelf' \\\n    --header \"Content-Type: application/json\" \\\n    --data '{\"userId\": \"41926065\", \"shelf\": \"to-read\"}'\n```\n\n### Customization\n\nConfiguration for `serverless deploy` is contained in `serverless.yml`.\nIf you want to deploy this service to your own domain, you'll need to\ntweak settings in there (namely, changing domain names).\n\n# TODO\n\nThis is a pet project I work on whenever I'm so inclined.\n\nAccordingly, there are a lot of TODOs at any given moment...\n\n## Support shelves with over 200 books\n\nRight now, this tool only reads the first 200 books on the `to-read` shelf\n(the Goodreads API prevents reading more). To support larger shelves, we should\ncall the API endpoint a few times (while not breaking the \"1 query per second\" rule\nGoodreads has on API integrations.\n\n## Batch catalog queries\n\nAfter we've fetched the Goodreads shelf, we could make catalog queries, say, 20 books at a time.\nThe backend already parallelizes execution of large searches, but this would allow us to have\nmore iterative results which appear on-screen as data is available.\n\n## Search for similar ISBNs\n\nCurrently, the search algorithm prefers an exact match on ISBN. This results in fewer\nresults than you'd expect, since popular titles are generally released with several ISBNs\n(for example, paperback and hardcover editions get different ISBNs).\n\n### Why not a search by author \u0026 title?\n\nFor famous authors/works, the number of incorrect results are just too numerous.\nEven when limiting to books (to exclude all the matching DVDs and audio recordings),\nother results appear alongside the \"real\" result.\n\n### Prefer first ISBN, but allow others\n\nAn excellent way around this problem is to utilize Goodreads \"other editions\" feature:\n\n1. For each book, see if there's an exact match in the catalog. If so? Great, we're done.\n2. If there's no match, check for other editions' ISBNs. If no other editions, exit.\n3. Search by title \u0026 author, and accept any results which have a known ISBN.\n\nAn endpoint exists to do this (`work.editions`), though it requires special permission.\nI'm waiting to hear back from Goodreads staff.\n\n[bibliophile-backend]: https://github.com/DavidCain/bibliophile-backend\n[bibliophile-frontend]: https://github.com/DavidCain/bibliophile-frontend\n[docker]: https://www.docker.com/products/docker-desktop\n[reading-list-img]: screenshots/reading_list.png\n[biblio]: https://biblio.dcain.me\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidcain%2Fbibliophile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidcain%2Fbibliophile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidcain%2Fbibliophile/lists"}