{"id":15782609,"url":"https://github.com/0xsirsaif/appgallery_spy","last_synced_at":"2025-03-31T16:45:13.851Z","repository":{"id":181254141,"uuid":"666382923","full_name":"0xsirsaif/appgallery_spy","owner":"0xsirsaif","description":null,"archived":false,"fork":false,"pushed_at":"2023-07-14T22:30:52.000Z","size":7197,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-05T19:20:15.824Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0xsirsaif.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-14T11:28:11.000Z","updated_at":"2023-07-14T11:36:27.000Z","dependencies_parsed_at":"2024-10-26T00:42:46.155Z","dependency_job_id":"b1c62bf4-30fc-4f98-aaa6-96df133a662b","html_url":"https://github.com/0xsirsaif/appgallery_spy","commit_stats":null,"previous_names":["0xsirsaif/appgallery_spy"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xsirsaif%2Fappgallery_spy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xsirsaif%2Fappgallery_spy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xsirsaif%2Fappgallery_spy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xsirsaif%2Fappgallery_spy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0xsirsaif","download_url":"https://codeload.github.com/0xsirsaif/appgallery_spy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246500093,"owners_count":20787637,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-04T19:20:24.636Z","updated_at":"2025-03-31T16:45:13.829Z","avatar_url":"https://github.com/0xsirsaif.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AppGallerySpy\n\n`appgalleryspy` is a powerful tool designed to download app reviews from Huawei's AppGallery. It utilizes Selenium to crawl the Huawei website and retrieve user reviews. This tool is built as a command-line interface (CLI) using Typer, making it easy to use and integrate into your workflows.\n\n## Deliverables\n\n- [x] Developed a CLI application using `Typer`, facilitating easy downloading of app reviews from Huawei's AppGallery\n- [x] Utilized semantic commit messages to maintain a clear and descriptive commit history.\n- [x] Successfully understood the structure and behavior of Huawei's AppGallery site and implemented effective crawling strategies to retrieve app reviews.\n- [x] Dockerized the project, enabling easy testing of the application in a containerized environment using docker-compose.\n- [x] Implemented a FastAPI application to serve as an api to retrieve app recent reviews.\n- [x] Implemented a `--limit` option to limit the number of reviews to be downloaded.\n- [x] Utilized `pytest` and `pytest-cov` for running tests and generating coverage reports to assess the test coverage of the codebase.\n- [x] Incorporated `pre-commit` hooks to ensure consistent formatting, type checking, and linting before each commit.\n- [x] Automated formatting and versioning of the package using a `Makefile`.\n- [x] Published the package on PyPI at https://pypi.org/project/appgallery_spy/\n- [x] Documentation including installation instructions, usage guidelines, known issues, and testing instructions.\n\n## Installation\n\n### Inside a Docker Container\n\n- the web service is available at http://localhost:8004\n- the api documentation is available at http://localhost:8004/docs\n\n1. Run `docker-compose up --build` to build the image and run the container\n2. Run `docker-compose exec web bash` to enter the container\n3. The `appgallery_spy` CLI is now available inside the container\n4. Or run `python -m appgallery_spy` inside the container too\n\n### Published CLI\n\n1. Install the package using `pip install appgallery_spy`\n2. Run `appgallery_spy --help` to see the available commands and options\n\n```bash\nUsage: python -m appgallery_spy.cli [OPTIONS] APP_ID                                                                                                                                            \n                                                                                                                                                                                                 \n╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮\n│ *    app_id      TEXT  [default: None] [required]                                                                                                                                             │\n╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮\n│ --limit                     INTEGER                          [default: 3]                                                                                                                     │                                                                      │\n│ --help                                                       Show this message and exit.                                                                                                      │\n╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n```\n\n### Manually\n\n1. Clone the repository, `git clone git@github.com:0xsirsaif/appgallery_spy.git`, and `cd` into it\n2. Create a virtual environment, `python -m venv venv`, and activate it, `source venv/bin/activate`\n3. Install the dependencies, `pip install -r requirements.txt`\n4. Run `python -m appgallery_spy --help` to see the available commands and options\n\n## Usage\n\n- Run `python -m appgallery_spy [APP_ID]` to download reviews for the app with the given `APP_ID`\n- Run `python -m appgallery_spy [APP_ID]` with the `--limit` option to limit the number of scrolls to be performed\n\n## Testing\n\n- To run all tests, run the following command:\n```bash\n$ python -m pytest --cov tests/ -vvs\n```\n\n## Known Issues\n\n- Inserting data after crawling: The MongoDB client raises a ServerSelectionTimeoutError, preventing successful data insertion. Work is in progress to fix this issue and will be resolved soon. Apologies for any inconvenience caused.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xsirsaif%2Fappgallery_spy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0xsirsaif%2Fappgallery_spy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xsirsaif%2Fappgallery_spy/lists"}