{"id":15056805,"url":"https://github.com/najuzilu/dm-apachecassandra","last_synced_at":"2026-01-02T21:02:59.000Z","repository":{"id":168729799,"uuid":"333444276","full_name":"najuzilu/DM-ApacheCassandra","owner":"najuzilu","description":"Data Modeling with Apache Cassandra","archived":false,"fork":false,"pushed_at":"2021-07-31T15:57:28.000Z","size":858,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-14T07:45:52.844Z","etag":null,"topics":["apache-cassandra","data-engineering","data-modeling","python","sql","star-schema"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/najuzilu.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-27T14:19:23.000Z","updated_at":"2023-06-29T12:33:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"93724e62-bdff-4790-97ba-ca493c2ddfc5","html_url":"https://github.com/najuzilu/DM-ApacheCassandra","commit_stats":{"total_commits":15,"total_committers":2,"mean_commits":7.5,"dds":0.06666666666666665,"last_synced_commit":"dc6808b14eda42448b2c747f41da35f59e75907e"},"previous_names":["najuzilu/dm-apachecassandra"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/najuzilu%2FDM-ApacheCassandra","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/najuzilu%2FDM-ApacheCassandra/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/najuzilu%2FDM-ApacheCassandra/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/najuzilu%2FDM-ApacheCassandra/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/najuzilu","download_url":"https://codeload.github.com/najuzilu/DM-ApacheCassandra/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243544665,"owners_count":20308168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-cassandra","data-engineering","data-modeling","python","sql","star-schema"],"created_at":"2024-09-24T21:56:36.179Z","updated_at":"2026-01-02T21:02:58.994Z","avatar_url":"https://github.com/najuzilu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Language](https://img.shields.io/badge/language-python--3.8-blue) [![Contributors][contributors-shield]][contributors-url] [![Stargazers][stars-shield]][stars-url] [![Forks][forks-shield]][forks-url] [![Issues][issues-shield]][issues-url] [![MIT License][license-shield]][license-url] [![LinkedIn][linkedin-shield]][linkedin-url]\n\n\u003cbr /\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/najuzilu/DM-ApacheCassandra\"\u003e\n        \u003cimg src=\"./images/logo.png\" alt=\"Logo\" width=\"300\" height=\"200\"\u003e\n    \u003c/a\u003e\n    \u003ch3 align=\"center\"\u003eData Modeling with Cassandra\u003c/h3\u003e\n\u003c/p\u003e\n\n## About The Project\n\nSparkify wants to analyze the data they've been collecting on songs and user activity on their new music streaming app. The analysis team is particularly interested in understanding what songs users are listening to. Currently, there is no easy way to query the data to generate the results, since the data reside in a directory of CSV files on user activity on the app.\n\nThey'd like a data engineer to create an Apache Cassandra database which can create queries on song play data to answer the questions, and wish to bring you on the project. Your role is to create a database for this analysis. You'll be able to test your database by running queries given to you by the analytics team from Sparkify to create the results.\n\n## Description\n\nYou will create an Apache Cassandra database which will host data collected on songs and user activity on Sparkify's new music streaming app.\n\n### Tools\n\n* python\n* Apache Cassandra\n\n## Datasets\n\nYou will work with one dataset: `event_data/`. The directory of CSV files is partitioned by date. Here are examples of filepaths to two files in the dataset:\n\n```\nevent_data/2018-11-08-events.csv\nevent_data/2018-11-09-events.csv\n```\n\nEach CSV file contains event data from users for a specific day. The CSV is comprised of the following fields:\n\n| Field          |   Description              |\n| :------------- | :------------------------- |\n| artist        | artist name               |\n| auth          | tracks whether the user logged in or logged out |\n| firstName     |   user first name |\n| gender |  user gender |\n| itemInSession | number of items for a specific session |\n| lastName | user last name |\n| length | length of session/event |\n| level | tracks whether the user paid for the session or if the session was free |\n| location | user location |\n| method | HTTP methods |\n| page | tracks the page name such as 'NextSong', 'Home', 'Logout', 'Settings', 'Downgrade', 'Login', 'Help', 'Error', 'Upgrade' |\n| registration | registration timestamp |\n| sessionId | session id |\n| song | song name |\n| status | tracks the status of the request such as 200, 307, 404 |\n| ts | timestamp in millisecond |\n| userId | user id |\n\n**[Note:](#)**\nPrior to populating the tables, you will merge all the CSV data under `event_datafile_new.csv`. This merged data includes\n\n![image_event_datafile_new](./images/image.jpeg)\n\n\u003c!-- ## ERD Model--\u003e\n\n\u003c!-- ## Project Structure --\u003e\n\n## Getting Started\n\nClone this repository\n\n```bash\ngit clone https://github.com/najuzilu/DM-ApacheCassandra.git\n```\n\n### Prerequisites\n\n* conda\n* python 3.8\n* cassandra-driver\n* pandas\n\nCreate a virtual environment through Anaconda using\n\n```bash\nconda env create --file environment.yml\n```\n\n## Project Steps\n\n1. Run `create_tables.py` to create the tables\n    ```bash\n    python create_tables.py\n    ```\n2. Run `etl.py` to execute the ETL pipeline and load the data in the database\n    ```bash\n    python etl_tables.py\n    ```\n3. Run `stylized_facts.py` to make sure that all the tables have been populated successfully.\n    ```bash\n    python stylized_facts.py\n    ```\n\n## Authors\n\nYuna Luzi - @najuzilu\n\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n\n\u003c!-- Links ---\u003e\n\n[contributors-shield]: https://img.shields.io/github/contributors/najuzilu/DM-ApacheCassandra.svg?style=flat-square\n[contributors-url]: https://github.com/najuzilu/DM-ApacheCassandra/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/najuzilu/DM-ApacheCassandra.svg?style=flat-square\n[forks-url]: https://github.com/najuzilu/DM-ApacheCassandra/network/members\n[stars-shield]: https://img.shields.io/github/stars/najuzilu/DM-ApacheCassandra.svg?style=flat-square\n[stars-url]: https://github.com/najuzilu/DM-ApacheCassandra/stargazers\n[issues-shield]: https://img.shields.io/github/issues/najuzilu/DM-ApacheCassandra.svg?style=flat-square\n[issues-url]: https://github.com/najuzilu/DM-ApacheCassandra/issues\n[license-shield]: https://img.shields.io/badge/License-MIT-yellow.svg\n[license-url]: https://github.com/najuzilu/DM-ApacheCassandra/blob/master/LICENSE\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=flat-square\u0026logo=linkedin\u0026colorB=555\n[linkedin-url]: https://www.linkedin.com/in/yuna-luzi/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnajuzilu%2Fdm-apachecassandra","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnajuzilu%2Fdm-apachecassandra","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnajuzilu%2Fdm-apachecassandra/lists"}