{"id":15636565,"url":"https://github.com/capjamesg/cv-book-svg","last_synced_at":"2026-01-08T04:58:33.386Z","repository":{"id":222450201,"uuid":"756537262","full_name":"capjamesg/cv-book-svg","owner":"capjamesg","description":"Turn an image of a bookshelf into an interactive SVG.","archived":false,"fork":false,"pushed_at":"2024-07-31T21:21:16.000Z","size":223,"stargazers_count":126,"open_issues_count":1,"forks_count":16,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-28T19:16:37.276Z","etag":null,"topics":["books","computer-vision","image-analysis","library","personal-library"],"latest_commit_sha":null,"homepage":"https://capjamesg.github.io/cv-book-svg/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/capjamesg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-12T21:09:12.000Z","updated_at":"2025-02-26T20:33:43.000Z","dependencies_parsed_at":"2024-08-01T01:42:41.446Z","dependency_job_id":null,"html_url":"https://github.com/capjamesg/cv-book-svg","commit_stats":{"total_commits":21,"total_committers":4,"mean_commits":5.25,"dds":"0.23809523809523814","last_synced_commit":"0e2ad1307eee127f72b45c9881b0fe8e717ba4dc"},"previous_names":["capjamesg/cv-book-svg"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/capjamesg%2Fcv-book-svg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/capjamesg%2Fcv-book-svg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/capjamesg%2Fcv-book-svg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/capjamesg%2Fcv-book-svg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/capjamesg","download_url":"https://codeload.github.com/capjamesg/cv-book-svg/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246238216,"owners_count":20745493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["books","computer-vision","image-analysis","library","personal-library"],"created_at":"2024-10-03T11:05:08.494Z","updated_at":"2026-01-08T04:58:33.336Z","avatar_url":"https://github.com/capjamesg.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Make your bookshelf clickable\n\nUse computer vision to generate an SVG that you can overlay onto a photo of your bookshelf that lets you click on each book to find out more information.\n\n## Demo\n\n[Try the demo](https://capjamesg.github.io/cv-book-svg/annotations.html)\n\nhttps://github.com/capjamesg/cv-book-svg/assets/37276661/ec57bf18-4182-4dce-870f-6bef81809e80\n\n## How it Works\n\nThis tool uses computer vision to identify and segment each book spine in an image of a bookshelf. Then, each book spine is sent to GPT-4 with Vision to read the book title and, if possible, the author.\n\nThis information is then sent to the Google Books API. The book ISBN, author name, and other meta information is retrieved from this API.\n\nAn SVG is then created using the segmented book spines. Each book is assigned a polygon which, when clicked, takes you to the Google Books page associated with a book.\n\nThis script uses the following vision tools:\n\n- Grounding DINO (zero-shot object detection model)\n- Segment Anything (image segmentation model)\n- GPT-4 with Vision API\n- OpenCV Python\n\nIt takes around 20 seconds to generate the polygons that map to the location of each book on an M1 Macbook Air. It then takes a few seconds to process each book with the OpenAI GPT-4 with Vision API.\n\nFor a bookshelf with 11 books, the script takes around one minute to run.\n\nThe script returns a HTML file with an SVG file that is overlaid onto the source image.\n\n## How to Use\n\nFirst, clone this project and install the required dependencies:\n\n```\ngit clone https://github.com/capjamesg/cv-book-svg\ncd cv-book-svg\npip3 install -r requirements.txt\n```\n\nThen, run the main script:\n\n```\npython3 grounded.py --image=example.jpg --output=annotation.html\n```\n\nThis script takes an image as input (PNG, JPEG) and outputs a HTML document.\n\n## Limitations\n\nThis system may:\n\n- Not identify all books on a bookshelf (thin books are more likely to not be identified).\n- Generate a link to the wrong Google Books URL (which will happen if a book is not available on Google Books, or if a book has a generic title like \"Poems of Emily Dickinson\", which could on its own refer to several publications).\n- Mis-identify some books.\n\n## Notes\n\n- `video.py` contains a work-in-progress system for identifying all unique books in a video.\n\n## License\n\nThis project is licensed under an [MIT license](LICENSE).\n\n## Contributing\n\nFound a bug? Have an idea that you'd like to see in the project? Open an Issue in this GitHub repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcapjamesg%2Fcv-book-svg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcapjamesg%2Fcv-book-svg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcapjamesg%2Fcv-book-svg/lists"}