{"id":18639575,"url":"https://github.com/giograno/reviews_crawler","last_synced_at":"2026-04-28T17:01:47.650Z","repository":{"id":75262700,"uuid":"69534993","full_name":"giograno/reviews_crawler","owner":"giograno","description":"Java-based tool for mining the Google Play Store and gather user reviews","archived":false,"fork":false,"pushed_at":"2018-09-03T07:47:59.000Z","size":18026,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2026-04-24T00:34:50.625Z","etag":null,"topics":["android","mining","playstore"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/giograno.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-09-29T05:48:12.000Z","updated_at":"2020-06-05T09:04:20.000Z","dependencies_parsed_at":"2023-03-13T20:16:33.736Z","dependency_job_id":null,"html_url":"https://github.com/giograno/reviews_crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/giograno/reviews_crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giograno%2Freviews_crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giograno%2Freviews_crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giograno%2Freviews_crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giograno%2Freviews_crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/giograno","download_url":"https://codeload.github.com/giograno/reviews_crawler/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/giograno%2Freviews_crawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32390067,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T14:34:11.604Z","status":"ssl_error","status_checked_at":"2026-04-28T14:32:37.009Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","mining","playstore"],"created_at":"2024-11-07T05:49:23.748Z","updated_at":"2026-04-28T17:01:47.635Z","avatar_url":"https://github.com/giograno.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Reviews Crawling Tool\n\nThis tool has been developed with the purpose to mine reviews from the Google Play Store.\nIt's entirely written in Java, and relies on [Selenium](http://www.seleniumhq.org/) and on [PhantomJS](http://phantomjs.org).\n\nThe tool can save the reviews in a csv format as well storing them using a MongoDB instance.\n\n## How to use\nFirst of all, you need to download ([here](http://phantomjs.org/download.html)) the right PhantomJS executable according to your underlying operating system. It have to be placed in the same directory of the jar and the properties file.\n### Configuration\nThe tool must be set through its configuration file. You need to specify the input and the output file name. \n\nMost parameters are self explained in the `config.properties` file. The most important ones are detailed in the following paragraph.\n\n#### Parameters\n\u003e export_to=...\n\nFor this option you can select the option `file` or `mongodb`. The first one will save the reviews on a csv file, the second one will use a mongodb instance.\n\n\u003e input_file=xxx.csv\n\nThis is the file with the list of app of which the reviews need to be extracted. The file need to report the package name of the app as the first field in the csv. \n### How to run\nIn order to start the reviews mining, you have to run the following command:\n\n`java -jar extractor.jar extractor=reviews`\n\nThe tool will start to extract the reviews for the apps specified in the `input_file` parameter.\n\n#### Extract from mongodb database\n\nWhen you are collecting you reviews storing them on a mongodb database, you can export the reviews for the apps you are interested to with the following command:\n\n`java -jar extractor.jar extractor=export`\n\nYou have specify the name of the csv file in output through the `output_file` field in the `config.properties`. Similarly, the `input_file` field should contain a list of the app for which you want to extract the reviews.\n\n\n### SURF Tool\nWith this tool it is possible to mine user reviews in a format that is compatible with the [SURF tool][surf].\nTo be able to do this, you need to setup che configuration file as follow:\n\n```\nextractor=reviews\nexport_to=file\nformat=surf\n```\n\nThe ouput will be a `review.xml` file in the main directory where the command is launched.\nThe app in input are the ones listed in the `input_file` file.\nWhether you are interested in mining a single application, you can run the tool with such command:\n\n\u003e java -jar \u003cname.jar\u003e extractor=reviews app=\u003cid_app\u003e\n\n[surf]:http://www.ifi.uzh.ch/en/seal/people/panichella/tools/SURFTool.html","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiograno%2Freviews_crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgiograno%2Freviews_crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgiograno%2Freviews_crawler/lists"}