{"id":18553567,"url":"https://github.com/quantumudit/analyzing-books","last_synced_at":"2025-04-09T23:30:51.069Z","repository":{"id":49387050,"uuid":"426692640","full_name":"quantumudit/Analyzing-Books","owner":"quantumudit","description":"This project focuses on scraping data related to books by their genre from the \"Books To Scrape\" website; performing necessary transformations on the scraped data and then analyzing \u0026 visualizing it using Jupyter Notebook and Power BI.","archived":false,"fork":false,"pushed_at":"2023-02-14T10:01:10.000Z","size":26994,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-24T13:21:23.200Z","etag":null,"topics":["beautifulsoup","data-analytics","power-bi","python","webscraping"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quantumudit.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-10T16:18:57.000Z","updated_at":"2023-05-20T15:04:29.000Z","dependencies_parsed_at":"2022-09-04T18:11:14.888Z","dependency_job_id":null,"html_url":"https://github.com/quantumudit/Analyzing-Books","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumudit%2FAnalyzing-Books","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumudit%2FAnalyzing-Books/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumudit%2FAnalyzing-Books/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quantumudit%2FAnalyzing-Books/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quantumudit","download_url":"https://codeload.github.com/quantumudit/Analyzing-Books/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248129527,"owners_count":21052588,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beautifulsoup","data-analytics","power-bi","python","webscraping"],"created_at":"2024-11-06T21:17:34.648Z","updated_at":"2025-04-09T23:30:46.058Z","avatar_url":"https://github.com/quantumudit.png","language":"Jupyter Notebook","funding_links":["https://www.buymeacoffee.com/quantumudit"],"categories":[],"sub_categories":[],"readme":"# ![Project Logo][project_logo]\n\n---\n\n\u003ch4 align=\"center\"\u003eScraping \u0026 Analyzing books from \u003ca href=\"https://books.toscrape.com/\" target=\"_blank\"\u003eBooks To Scrape\u003c/a\u003e website with \u003ca href=\"https://en.wikipedia.org/wiki/Python_(programming_language)\" target=\"_blank\"\u003ePython\u003c/a\u003e and \u003ca href=\"https://en.wikipedia.org/wiki/Microsoft_Power_BI\" target=\"_blank\"\u003ePower BI\u003c/a\u003e\u003c/h4\u003e\n\n\u003cp align='center'\u003e\n\u003cimg src=\"https://i.ibb.co/KxfMMsP/built-with-love.png\" alt=\"built-with-love\" border=\"0\"\u003e\n\u003cimg src=\"https://i.ibb.co/MBDK1Pk/powered-by-coffee.png\" alt=\"powered-by-coffee\" border=\"0\"\u003e\n\u003cimg src=\"https://i.ibb.co/CtGqhQH/cc-nc-sa.png\" alt=\"cc-nc-sa\" border=\"0\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#overview\"\u003eOverview\u003c/a\u003e •\n  \u003ca href=\"#prerequisites\"\u003ePrerequisites\u003c/a\u003e •\n  \u003ca href=\"#architecture\"\u003eArchitecture\u003c/a\u003e •\n  \u003ca href=\"#demo\"\u003eDemo\u003c/a\u003e •\n  \u003ca href=\"#support\"\u003eSupport\u003c/a\u003e •\n  \u003ca href=\"#license\"\u003eLicense\u003c/a\u003e\n\u003c/p\u003e\n\n## Overview\n\nThe objective of this project is to gather information about books and their attributes from the website [Books To Scrape][website_link].\n\nThe collected data will then undergo a thorough exploratory data analysis, aimed at gaining valuable insights. \n\nThese insights will be effectively visualized using the Power BI tool to provide a clear and comprehensive understanding of the data. The project's primary focus is to extract information from the website and turn it into useful insights and visual representations.\n\n[![Website Snippet][website_snippet]][website_link]\n\nThe repository directory structure is as follows:\n\nAnalyzing-Books\u003cbr\u003e\n├─ 01_SCRAPER\u003cbr\u003e\n├─ 02_ETL\u003cbr\u003e\n├─ 03_DATA\u003cbr\u003e\n├─ 04_ANALYSIS\u003cbr\u003e\n├─ 05_DASHBOARD\u003cbr\u003e\n├─ 06_RESOURCES\u003cbr\u003e\n\nThe type of content present in the directories is as follows:\n\n### 01_SCRAPER\n\nThis directory comprises a [Python script](01_SCRAPER/main.py) designed to extract information from the website and a [flat file](01_SCRAPER/scraped_data.csv) that stores the data obtained through the scraping process. \n\nThe script in this directory automates the data scraping process, making it easier to collect information from the website in question. The flat file serves as a storage space for the scraped data, allowing for easy access and manipulation of the information gathered. \n\nThis setup facilitates the efficient and organized management of the data obtained from the website.\n\n### 02_ETL\n\nThis directory houses a [Jupyter Notebook](02_ETL/books_data_transformation.ipynb) that undertakes an ETL (Extract, Transform, Load) process on the data obtained through scraping. \n\nThe purpose of this process is to convert the raw data into a form that is suitable for analysis. The Jupyter Notebook performs transformations on the scraped dataset to clean, organize, and structure the data into a format that is ready for analysis. \n\nThis notebook is characterized by its thorough documentation, with each step of the analysis process clearly explained and described. \n\nThe data cleaning and transformation steps, in particular, are carefully documented, ensuring that the thought process and decisions made during these processes are easily understood. \n\nThis attention to detail in the documentation makes the notebook a valuable resource for anyone looking to understand how the data was cleaned and transformed to generate meaningful insights.\n\nFinally, the transformed data is exported into the [03_DATA](03_DATA/) directory, making it easily accessible for further examination and analysis.\n\nThis Jupyter Notebook serves as a crucial step in the data preparation process, enabling the effective and efficient transformation of raw data into a form that can provide valuable insights.\n\n### 03_DATA\n\nThis directory contains the data that can be directly used for data analysis and visualization.\n\nThe contents of this directory include only the pristine and organized [data](/03_DATA/books_data.csv), ready to be utilized for data analysis and visualization. \n\nThis data has been thoroughly processed and scrubbed of any errors or inconsistencies, ensuring that it can be relied upon to provide accurate and meaningful insights. \n\n### 04_ANALYSIS\n\nThis directory contains the python notebooks that analyzes the clean dataset to generate insights.\n\nThis directory contains a [Jupyter Notebook](/04_ANALYSIS/books_data_analysis.ipynb) that analyzes the clean dataset to uncover valuable insights. \n\nThis notebook performs complex data analysis and has been crafted to make it easy to work with the clean data. \n\nThe notebook is thoroughly annotated and the results of each analysis are clearly documented within the text cells. This detailed documentation makes it easy to follow the thought process and understand the insights generated through the analysis. \n\nThe well-documented nature of the notebook makes it a valuable resource for anyone looking to gain a deeper understanding of the data and the insights it contains.\n\n### 05_DASHBOARD\n\nThis directory houses a straightforward [markdown file](5_DASHBOARD/Books%20Analysis%20Dashboard.md) that includes an embedded link to a Power BI report. \n\nThis report serves as a visual representation of the data and provides a dynamic, interactive experience for exploring and analyzing the information. \n\nThe simplicity of the markdown file, combined with the robust capabilities of Power BI, make it easy to access and interact with the data in a way that is both intuitive and insightful. \n\nWhether you are looking to gain a high-level overview of the data, or to drill down into specific details, the Power BI report provides an effective and engaging way to work with the data.\n\n### 06_RESOURCES\n\nThis directory serves as a repository for various visual elements used in this project, including images, icons, layouts, styling files, etc. \n\nThese elements play a crucial role in the overall presentation and visualization of the data and help to bring the insights generated by the analysis to life. \n\nBy having these elements easily accessible in a central location, the project is streamlined and efficient, allowing for faster and more effective data analysis and visualization.\n\n## Prerequisites\n\nTo fully grasp the concepts and processes involved in this project, it is recommended to have a solid understanding of the following skills:\n\n- Fundamental knowledge of Python and Jupyter Notebook\n- Familiarity with the Python libraries listed in the [requirements.txt][requirements] file\n- Basic proficiency in HTML and CSS\n- Basic familiarity with browser developer tools\n- An understanding of the basics of Power BI\n\nHaving these skills as a foundation will help to ensure a smooth and effective experience while working on this project.\n\n\u003e The selection of applications and their installation process may differ depending on personal preferences and computer configurations.\n\n## Architecture\n\nThe architecture of this project is straightforward and can be easily understood through the accompanying diagram, as seen below:\n\n![Process Architecture][process_workflow]\n\nThe project architecture consists of the following steps:\n\n- **Data scraping**: Data is collected from a website using a Python script and stored in a flat file.\n- **Data cleaning**: The raw data is processed and cleaned through the use of an ETL-specific Jupyter Notebook.\n- **Data visualization**: The cleaned and analysis-ready dataset is used for exploratory data analysis (EDA) through Jupyter Notebook and creating a comprehensive and insightful report through Power BI.\n\nThese steps are designed to be straightforward and efficient, allowing for quick and effective analysis of the data and generation of meaningful insights.\n\n## Demo\n\nThe following illustration demonstrates the process of collecting data from the website through scraping:\n\n![Scraping Graphic][scraping_graphic]\n\nAccess the interactive Power BI dashboard by clicking on this link here:\n\n[![Power BI Dashboard][dashboard_image]][dashboard_link]\n\n## Support\n\nIf you have any questions, concerns, or suggestions, feel free to reach out to me through any of the following channels:\n\n[![Linkedin Badge][linkedinbadge]][linkedin] [![Twitter Badge][twitterbadge]][twitter]\n\nIf you find my work valuable, you can show your appreciation by [buying me a coffee][buy_me_a_coffee]\n\n\u003ca href=\"https://www.buymeacoffee.com/quantumudit\" target=\"_blank\"\u003e\n\u003cimg src=\"https://i.ibb.co/9cyrq6m/buy-me-a-coffee.png\" alt=\"buy-me-a-coffee\" border=\"0\" width=\"170\" height=\"50\"\u003e\n\u003c/a\u003e\n\n## License\n\n\u003ca href = 'https://creativecommons.org/licenses/by-nc-sa/4.0/' target=\"_blank\"\u003e\n    \u003cimg src=\"https://i.ibb.co/mvmWGkm/by-nc-sa.png\" alt=\"by-nc-sa\" border=\"0\" width=\"88\" height=\"31\"\u003e\n\u003c/a\u003e\n\nThis license allows reusers to distribute, remix, adapt, and build upon the material in any medium or format for noncommercial purposes only, and only so long as attribution is given to the creator. If you remix, adapt, or build upon the material, you must license the modified material under identical terms.\n\n\u003c!-- Image Links --\u003e\n\n[project_logo]: 06_RESOURCES/project_cover_image.png\n[process_workflow]: 06_RESOURCES/process_architecture.png\n[scraping_graphic]: 06_RESOURCES/scraping_graphic.gif\n[website_snippet]: 06_RESOURCES/website_snip.png\n[dashboard_image]: 06_RESOURCES/dashboard_image.png\n\n\u003c!-- External Links --\u003e\n\n[website_link]: https://books.toscrape.com/\n[requirements]: ./requirements.txt\n[dashboard_link]: https://app.powerbi.com/view?r=eyJrIjoiMjdlZjJjYmUtNjEyMC00ODVjLTk4Y2YtMWEzYmI4MDZlNjljIiwidCI6IjcwODlkNGIxLTQyMmUtNDYzZi1hNGM3LTViY2FiOTk0MGRiZCJ9\n\n\u003c!-- Profile Links --\u003e\n\n[linkedin]: https://www.linkedin.com/in/uditkumarchatterjee/\n[twitter]: https://twitter.com/quantumudit\n[buy_me_a_coffee]: https://www.buymeacoffee.com/quantumudit\n\n\u003c!-- Shields Profile Links --\u003e\n\n[linkedinbadge]: https://img.shields.io/badge/-uditkumarchatterjee-0e76a8?style=flat\u0026labelColor=0e76a8\u0026logo=linkedin\u0026logoColor=white\n[twitterbadge]: https://img.shields.io/badge/-@quantumudit-1ca0f1?style=flat\u0026labelColor=1ca0f1\u0026logo=twitter\u0026logoColor=white\u0026link=https://twitter.com/quantumudit\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantumudit%2Fanalyzing-books","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquantumudit%2Fanalyzing-books","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantumudit%2Fanalyzing-books/lists"}