{"id":16668492,"url":"https://github.com/ekzhang/dispict","last_synced_at":"2025-04-10T01:13:08.600Z","repository":{"id":63675831,"uuid":"560095392","full_name":"ekzhang/dispict","owner":"ekzhang","description":"Design a growing artistic exhibit of your own making, with semantic search powered by OpenAI CLIP","archived":false,"fork":false,"pushed_at":"2025-01-08T04:48:30.000Z","size":1419,"stargazers_count":77,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-10T01:13:01.322Z","etag":null,"topics":["aesthetics","art","clip","computer-vision","creative","graphic-design","machine-learning","modal","museum","python"],"latest_commit_sha":null,"homepage":"https://dispict.com","language":"Svelte","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ekzhang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-31T18:16:43.000Z","updated_at":"2025-01-22T17:46:46.000Z","dependencies_parsed_at":"2023-01-31T09:15:51.313Z","dependency_job_id":"aa275cde-6e86-4e1c-aa39-f1b2359cabb3","html_url":"https://github.com/ekzhang/dispict","commit_stats":{"total_commits":60,"total_committers":1,"mean_commits":60.0,"dds":0.0,"last_synced_commit":"cce5156e20128834b57aaaf6885f51a8fd90658e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fdispict","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fdispict/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fdispict/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fdispict/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ekzhang","download_url":"https://codeload.github.com/ekzhang/dispict/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248137891,"owners_count":21053775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aesthetics","art","clip","computer-vision","creative","graphic-design","machine-learning","modal","museum","python"],"created_at":"2024-10-12T11:25:33.055Z","updated_at":"2025-04-10T01:13:08.575Z","avatar_url":"https://github.com/ekzhang.png","language":"Svelte","funding_links":[],"categories":["Svelte"],"sub_categories":[],"readme":"# Dispict: a creative aesthetics tool\n\nDesign a growing artistic exhibit of your own making, with semantic search\npowered by OpenAI CLIP. Bring your own labels and context.\n\n[![dispict cover image](./public/assets/social-image.jpg)](https://dispict.com)\n\n**[dispict.com](https://dispict.com)** greets you with a blank canvas. You begin\ntyping. Your writing becomes a _label_, and related artworks appear spatially\naround the text you wrote. As you pan and zoom around the gallery, you can try\nother labels to see how the artwork shifts in aesthetic quality.\n\nFocus on a single work to see its context: artist, setting, history, and\nnarrative descriptions. This crucially allows you to learn about the story of\nthe art being presented.\n\n## Motivation\n\nThere's currently a lot of excitement about computers helping creatives find\ninspiration by generating original art pieces from text prompts\n([1](https://openai.com/dall-e-2/), [2](https://www.midjourney.com/),\n[3](https://stability.ai/blog/stable-diffusion-public-release)). But these lose\nthe unique, genuine part of walking through an art museum where every work has\nbeen lovingly created by humans, and the viewer is surrounded by _insight_ and\n_intention_. What if computers could connect us with masterpieces made by\nartists of the past?\n\nThe Harvard Art Museums' online collection is huge, containing over 200,000\ndigitized works. This is far more than can be easily taken in by a single\nperson. So instead, we apply computation to what it's good at: finding patterns\nand connections.\n\n**Creativity and curiosity require associative thinking.** Just like the\ntechnological innovations of centuries past have changed the aesthetic character\nof fine art from literal portraiture to more flexible modes of self-expression,\n_Dispict_ hopes to be technology that explores the honest, intimate relationship\nof the creative process with artistic discovery.\n\n## Technical Details\n\n_Dispict_ uses real-time machine learning. It's built on contrastive\nlanguage-image pretraining (CLIP) and nearest-neighbor search, served from\nPython (on a [Modal](https://modal.com/) endpoint) with a handcrafted\n[Svelte](https://svelte.dev/) frontend.\n\n### Development\n\nIf you want to hack on dispict yourself, you can run the frontend development\nserver locally using [Node v16](https://nodejs.org/) or higher:\n\n```shell\nnpm install\nnpm run dev\n```\n\nThis will automatically connect to the serverless backend recommendation system\nhosted on Modal. To additionally change this part of the code, you need to\ncreate a Modal account, then install [Python 3.10+](https://www.python.org/) and\nfollow these steps:\n\n1. Run the Jupyter notebooks `notebooks/load_data.ipynb` and\n   `notebooks/data_cleaning.ipynb` to download data from the Harvard Art\n   Museums. This will produce two files named `data/artmuseums[-clean].json`.\n2. Run `SKIP_WEB=1 modal run main.py` to spawn a parallel Modal job that\n   downloads and embeds all images in the dataset using\n   [CLIP](https://openai.com/blog/clip/), saving the results to\n   `data/embeddings.hdf5`.\n3. Run `modal deploy main.py` to create the web endpoint, which then gives you a\n   public URL such as `https://ekzhang--dispict-suggestions.modal.run`.\n\nYou can start sending requests to the URL to get recommendations. For example,\n`GET /?text=apple` will find artwork related to apples, such as the image shown\nbelow.\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://harvardartmuseums.org/collections/object/230725\"\u003e\n\u003cimg src=\"https://nrs.harvard.edu/urn-3:HUAM:756527\" alt=\"'West Indian Girl' by Childe Hassam\" width=\"600\"\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\nTo point the web application at your new backend URL, you can set an environment\nvariable to override the default backend.\n\n```shell\nVITE_APP_API_URL=https://[your-app-endpoint].modal.run npm run dev\n```\n\n## Acknowledgements\n\nCreated by Eric Zhang ([@ekzhang1](https://twitter.com/ekzhang1)) for\n[Neuroaesthetics](https://mbb.harvard.edu/) at Harvard. All code is licensed\nunder [MIT](LICENSE), and data is generously provided by the\n[Harvard Art Museums](https://www.harvardartmuseums.org/) public access\ncollection.\n\nI learned a lot from Jono Brandel's [_Curaturae_](https://curaturae.com/) when\ndesigning this.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekzhang%2Fdispict","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fekzhang%2Fdispict","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekzhang%2Fdispict/lists"}