{"id":15356694,"url":"https://github.com/filips123/popularrepositoriesanalysis","last_synced_at":"2025-04-05T00:23:32.326Z","repository":{"id":248305872,"uuid":"826943186","full_name":"filips123/PopularRepositoriesAnalysis","owner":"filips123","description":"Analyzing popular GitHub repositories for UVP project","archived":false,"fork":false,"pushed_at":"2024-07-16T18:44:40.000Z","size":8325,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T11:19:25.298Z","etag":null,"topics":["data-analysis","data-science","data-visualization","popular-repositories","top-repositories"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/filips123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-10T17:33:25.000Z","updated_at":"2024-07-16T18:44:43.000Z","dependencies_parsed_at":"2024-07-13T21:31:55.737Z","dependency_job_id":"65c4712b-ea25-46e5-a8b8-f2d78a09625e","html_url":"https://github.com/filips123/PopularRepositoriesAnalysis","commit_stats":null,"previous_names":["filips123/popularrepositoriesanalysis"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filips123%2FPopularRepositoriesAnalysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filips123%2FPopularRepositoriesAnalysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filips123%2FPopularRepositoriesAnalysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/filips123%2FPopularRepositoriesAnalysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/filips123","download_url":"https://codeload.github.com/filips123/PopularRepositoriesAnalysis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247267650,"owners_count":20911020,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","data-science","data-visualization","popular-repositories","top-repositories"],"created_at":"2024-10-01T12:29:38.194Z","updated_at":"2025-04-05T00:23:32.304Z","avatar_url":"https://github.com/filips123.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Popular Repositories Analysis\n\nAnalyzing popular GitHub repositories for *Uvod v programiranje* (UVP) project at FMF UL.\n\n## About\n\nThe aim of this project is to analyze the most popular GitHub repositories.\nThis has been done by downloading a list of top 5000 repositories and users using the GitHub search API and downloading additional properties about each repository and user.\nThese properties include the owner, name, description, topics, languages, stars, etc. for repositories, and name, homepage, company, location, etc. for users.\nAll repositories and users have then been analyzed in Jupyter Notebooks using the Pandas library.\n\nIf you want to quickly view the analysis, [check the instructions below](#viewing-analysis).\n\n## Installation\n\nThis project requires Python 3.10 or later and [Poetry](https://python-poetry.org/) dependency manager 1.8.0 or later.\n\nYou can clone this repository and install the required dependencies using Poetry:\n\n```shell\ngit clone https://github.com/filips123/PopularRepositoriesAnalysis.git\ncd PopularRepositoriesAnalysis\npoetry install\n```\n\n## Scraping Data\n\nBefore running the API scrapers, you need to create [a read-only token](https://github.com/settings/tokens/new?description=Popular%20Repositories%20Analysis) and store it inside the `scraper/auth.txt` file.\n\nTo collect all the required data, you can run the API scrapers with the following commands:\n\n```shell\n# Repositories Scraper\npython scraper/main.py repositories stars # Top repositories by stars\npython scraper/main.py repositories forks # Top repositories by forks\n\n# Users Scraper\npython scraper/main.py users followers # Top users by forks\npython scraper/main.py users repositories # Top users by repositories\n```\n\nBy default, 1000 results will be fetched for each query.\nEach of the commands will take approximately 10–30 minutes to complete.\nThe collected data will be stored inside the `data` directory.\n\nThere is also an additional HTML scraper, which can collect similar data about a specific repository as the API scraper, but by downloading and parsing HTML.\nHowever, this scraper is less reliable and cannot collect all data, so it is recommended to use the API scrapers instead.\n\nThe HTML scraper can be run with the following command:\n\n```shell\npython scrapers/manual.py YOUR-REPOSITORY-URL\n```\n\n## Existing Data\n\nThe repository already provides the following lists:\n\n* Top 5000 repositories by stars, collected on 2024-07-14.\n* Top 5000 repositories by forks, collected on 2024-07-14.\n* Top 5000 users by followers, collected on 2024-07-14.\n* ~~Top 5000 users by repositories, collected on 2024-07-14.~~ *Incorrect due to a GitHub API bug.*\n\n## Viewing Analysis\n\nThe analysis has been done using Jupyter Notebooks in the [`analysis`](analysis) directory.\nIt is recommended to clone the repository and view them locally, as GitHub cannot properly render all notebook features.\nAlternatively, you can view them online using nbviewer:\n\n* [Analysis of top GitHub repositories](https://nbviewer.org/github/filips123/PopularRepositoriesAnalysis/blob/main/analysis/repositories.ipynb)\n* [Analysis of top GitHub users](https://nbviewer.org/github/filips123/PopularRepositoriesAnalysis/blob/main/analysis/users.ipynb)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilips123%2Fpopularrepositoriesanalysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffilips123%2Fpopularrepositoriesanalysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilips123%2Fpopularrepositoriesanalysis/lists"}