https://github.com/cjunwon/spotify-image-generator
Generates personalized cover art images based on user's top artists
https://github.com/cjunwon/spotify-image-generator
dall-e image-generator object-detection openai spotify spotipy tensorflow tensorflow-lite tensorflow2
Last synced: about 1 year ago
JSON representation
Generates personalized cover art images based on user's top artists
- Host: GitHub
- URL: https://github.com/cjunwon/spotify-image-generator
- Owner: cjunwon
- Created: 2022-12-29T06:30:34.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-01-10T00:21:32.000Z (over 3 years ago)
- Last Synced: 2025-02-08T15:32:50.594Z (about 1 year ago)
- Topics: dall-e, image-generator, object-detection, openai, spotify, spotipy, tensorflow, tensorflow-lite, tensorflow2
- Language: Jupyter Notebook
- Homepage:
- Size: 10.9 MB
- Stars: 2
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Spotify Image Generator
Generates personalized cover art based on user data.
## TABLE OF CONTENTS
* [Background](#background)
* [Objective](#objective)
* [Tools and Packages](#tools)
* [Model Implementation](#model-implementation)
* [Conclusion](#conclusion)
* [Challenges and Future Work](#challenges-and-futurework)
## BACKGROUND
Spotify defaults to a simple collage of four album covers as cover art for a playlist created by user. In order to change the playlist cover art, the user has to select and upload their personal images through their desktop Spotify application. In order to help this selection process, this project generates a completely new cover art based on the user's music preferences.
## OBJECTIVE
* Collect user data
* Pull user data using Spotify API
* Use Spotify API functions to specify queries
* Parse through JSON format output to gather necessary data
* Test different image generator models
* Find objects within cover art using TensorFlow object detection
* Implement Artistic Style Transfer with TensorFlow Lite
* Implement OpenAI's DALL·E2 model
## TOOLS
**Language Used:** Python
Task
Technique
Tools/Packages Used
Data Collection
User top artists, artists' top song, top song's album, album's cover art data extraction through Spotify API
Spotify API (Spotipy)
Data Pre-processing
Sorted JSON output into lists
numpy
Model Implementation
Extracted objects within cover art using TensorFlow object detection. Implmented Artistic Style Transfer with TensorFlow Lite & OpenAI's DALL-E2 model to generate user-specific cover art.
TensorFlow, DALL·E 2
## MODEL-IMPLEMENTATION
Spotify API Connection
(Authenticate user's Spotify account)
↓
Spotipy Queries
(Extract and store as lists: User's top artists → artist's top song → album that the top song belongs to → cover art for each album)
↓
Test Artistic Style Transfer with TensorFlow Lite
(Pull sample cover arts using Spotipy and demonstrate Artistic Style Transfer implementation and functionality)
↓
TensorFlow object detection
(Find prevelant objects within selected top cover art using object detection. Store output into lists.)
↓
OpenAI's DALL·E 2 model
(Use object output list from above step to generate a sentence/prompt for the DALL·E 2 model to process.)
↓
Apply Artistic Style Transfer with TensorFlow Lite
(Run the DALL·E 2 output image through Artistic Style Transfer model to generate final image)
## CONCLUSION
Although the end result of the models did not succeed in creating what we may consider 'artistic' or 'stylistic' art, this project set up a scalable framework for future improvement and also demonstrated the power and relative simplicity of recent development in image generation using deep learning.
## CHALLENGES-AND-FUTUREWORK
* Extract genres for each song/album that was fed into the model, and use adjectives associated with the genre or song/album title as part of the prompt for the DALL·E 2 model. This may provide more context for the model to work with and may produce a greater variety as output.