{"id":27596509,"url":"https://github.com/jshyunbin/comment_crawler","last_synced_at":"2025-10-13T05:32:03.871Z","repository":{"id":168973497,"uuid":"584583412","full_name":"jshyunbin/comment_crawler","owner":"jshyunbin","description":"Web crawler for online shopping mall comments using python selectolax and requests.","archived":false,"fork":false,"pushed_at":"2023-01-16T03:12:25.000Z","size":19,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-22T12:58:50.671Z","etag":null,"topics":["python","webcrawler"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jshyunbin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-01-03T01:32:15.000Z","updated_at":"2023-01-09T06:39:15.000Z","dependencies_parsed_at":"2023-05-28T09:15:05.289Z","dependency_job_id":null,"html_url":"https://github.com/jshyunbin/comment_crawler","commit_stats":null,"previous_names":["jshyunbin/comment_crawler"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jshyunbin/comment_crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jshyunbin%2Fcomment_crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jshyunbin%2Fcomment_crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jshyunbin%2Fcomment_crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jshyunbin%2Fcomment_crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jshyunbin","download_url":"https://codeload.github.com/jshyunbin/comment_crawler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jshyunbin%2Fcomment_crawler/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262933317,"owners_count":23386784,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","webcrawler"],"created_at":"2025-04-22T12:51:00.305Z","updated_at":"2025-10-13T05:31:58.834Z","avatar_url":"https://github.com/jshyunbin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🐛 Comment Crawler\n\nThis web crawler targets korean online shopping mall comments. \n\n```zsh\npython src/main.py -url \"http://item.gmarket.co.kr/Item?goodscode=2405613985\u0026ver=638083484789237196\"\n```\n\n## Supporting Websites\n\n- [G마켓](http://www.gmarket.co.kr)\n- [11번가](https://www.11st.co.kr) (wip)\n- [Naver shopping](https://shopping.naver.com/home)\n- [coupang](https://www.coupang.com) (wip)\n\n## How to use\n\nComment crawler uses flags to specify \n\n| Flags         | type    | description                                                      | default  |\n|---------------|---------|------------------------------------------------------------------|----------|\n| url           | string  | URL link of the online shopping website.                         | None     |\n| max_collect   | int     | Maximum number of comments to collect. -1 if no limit.           | -1       |\n| collect_empty | boolean | Collects empty comments if set to True                           | True     |\n| browser       | enum    | Browser used for selenium. Can choose between Safari and Firefox | 'Safari' |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjshyunbin%2Fcomment_crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjshyunbin%2Fcomment_crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjshyunbin%2Fcomment_crawler/lists"}