https://github.com/mneedham/uk-charts-neo4j

https://github.com/mneedham/uk-charts-neo4j

Last synced: 12 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/mneedham/uk-charts-neo4j
• Owner: mneedham
• Created: 2020-01-03T08:01:37.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2022-12-08T03:23:22.000Z (about 2 years ago)
• Last Synced: 2024-04-14T09:10:07.074Z (9 months ago)
• Language: Python
• Size: 404 KB
• Stars: 2
• Watchers: 3
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.adoc

Awesome Lists containing this project

README

        
= Exploring the UK Charts with Neo4j

The code in this repository goes with the <> blog post.

== Downloading and scraping charts

If you want to download the different charts, you can do this by running the following script:

[source, bash]

----

python scripts/download.py

----

The HTML page for each chart will be written into files in the `raw/charts` directory.

We now need to call the Spotify API to pull in the full list of collaborating artists for each track.

I've written a blog post explaining how to https://markhneedham.com/blog/2020/01/02/spotify-api-making-my-first-call/[create a Spotify application and get the corresponding credentials^].

Once we've skimmed that blog post, let's create a file called `.env` and populate it with our Spotify application credentials:

_.env_

[source, bash]

----

export CLIENT_ID=""

export CLIENT_SECRET=""

----

We can then run the following command to add these values as environment variables:

[source,bash]

----

source .env

----

Now let's execute the following script to pull down the collaborating artists:

[source, bash]

----

python scripts/download_songs_spotify.py

----

This script writes its results into the `raw/songs` directory.

And finally, let's run the following script, which will create a JSON document per chart entry in `import/items.json`:

[source,bash]

----

python scripts/scrape.py

----