{"id":21151904,"url":"https://github.com/uranusx86/dcard-crawler-analyzer","last_synced_at":"2025-07-14T19:41:53.285Z","repository":{"id":74942615,"uuid":"177236562","full_name":"uranusx86/Dcard-Crawler-Analyzer","owner":"uranusx86","description":"get Dcard \u0026 Meteor forum content and analyze !","archived":false,"fork":false,"pushed_at":"2019-07-18T14:21:38.000Z","size":41,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-14T14:47:37.018Z","etag":null,"topics":["crawl","crawler","dcard","nlp","python","word-cloud","word-count","word-frequency"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uranusx86.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-23T02:52:11.000Z","updated_at":"2019-07-18T14:21:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"d928f647-a001-43d5-92d4-c90e643f7cac","html_url":"https://github.com/uranusx86/Dcard-Crawler-Analyzer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/uranusx86/Dcard-Crawler-Analyzer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uranusx86%2FDcard-Crawler-Analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uranusx86%2FDcard-Crawler-Analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uranusx86%2FDcard-Crawler-Analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uranusx86%2FDcard-Crawler-Analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uranusx86","download_url":"https://codeload.github.com/uranusx86/Dcard-Crawler-Analyzer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uranusx86%2FDcard-Crawler-Analyzer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265336610,"owners_count":23749193,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawl","crawler","dcard","nlp","python","word-cloud","word-count","word-frequency"],"created_at":"2024-11-20T10:19:50.688Z","updated_at":"2025-07-14T19:41:53.263Z","avatar_url":"https://github.com/uranusx86.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dcard-Crawler-Analyzer\nget Dcard \u0026 Meteor forum content and analyze ! \u003cbr\u003e\nAnd also provide a docker image version\n\n# Dependency\npython3\n\n## pypi package\n* flask\n* flask_script\n* flask_migrate\n* Flask-SQLAlchemy\n* gunicorn\n* requests\n* cfscrape\n* beautifulsoup4\n* jieba\n\n## apt package\n* nodejs 8+\n\n# Install\n```bash=\n# install dependency\napt-get install nodejs\npip install requirements.txt   # it highly recommend install packages in the virual env\n\n# environment variable\nexport APP_SETTINGS=\"config.DevelopmentConfig\"\nexport DATABASE_URL=\"sqlite:///WHERE_YOU_WANT_PUT_DB\"\n\n# database migration\ncd app\npython3 manage.py db init\npython3 manage.py db migrate\npython3 manage.py db upgrade\n```\n\n# Run\n```bash=\ncd app\npython3 dcard.py  # for Dcard\npython3 meteor.py   # for Meteor\n```\n\n# Docker image\nYou can also build crawler from dockerfile\n```bash=\ndocker build . --tag uranusx86/forum_crawler --no-cache\ndocker run -dt --name forumcrawler -p 80:8000 uranusx86/forum_crawler\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Furanusx86%2Fdcard-crawler-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Furanusx86%2Fdcard-crawler-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Furanusx86%2Fdcard-crawler-analyzer/lists"}