{"id":31782063,"url":"https://github.com/ituki-t/django_selenium_lab","last_synced_at":"2026-05-04T09:32:41.584Z","repository":{"id":317483485,"uuid":"1062431176","full_name":"Ituki-t/Django_Selenium_Lab","owner":"Ituki-t","description":"Djangoの裏でSeleniumを用いて情報を取得し，取得した情報をDBに保存。保存された情報をDjangoアプリで一覧表示をする","archived":false,"fork":false,"pushed_at":"2025-10-01T05:45:27.000Z","size":36,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-01T07:25:43.913Z","etag":null,"topics":["celery","django","python","redis","selenium"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ituki-t.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-23T08:46:27.000Z","updated_at":"2025-10-01T05:53:57.000Z","dependencies_parsed_at":"2025-10-01T07:25:45.546Z","dependency_job_id":"26abc0e1-740e-4094-91a1-ed8f78d84b23","html_url":"https://github.com/Ituki-t/Django_Selenium_Lab","commit_stats":null,"previous_names":["ituki-t/django_selenium_lab"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Ituki-t/Django_Selenium_Lab","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ituki-t%2FDjango_Selenium_Lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ituki-t%2FDjango_Selenium_Lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ituki-t%2FDjango_Selenium_Lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ituki-t%2FDjango_Selenium_Lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ituki-t","download_url":"https://codeload.github.com/Ituki-t/Django_Selenium_Lab/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ituki-t%2FDjango_Selenium_Lab/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279003388,"owners_count":26083579,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["celery","django","python","redis","selenium"],"created_at":"2025-10-10T09:14:19.068Z","updated_at":"2025-10-10T09:14:21.720Z","avatar_url":"https://github.com/Ituki-t.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## About this project\n- Djangoアプリの裏で`Celery`を用いて`Selenium`で情報を取得\n- `Selenium`で取得した情報を`Sqlite3g`に保存\n- Djangoアプリで取得した情報を一覧表示\n- ~~githubには`tasks.py`をgitの管理から外して`tasks_sample.py`を管理していく~~\n\n## pip install\n```bash\npip install django\npip install \"celery[redis]\" redis # Celery + Redisサポート, Python用のRedisクライアント\npip install django-celery-results # Celeryの実行結果をDBへ保存\npip install selenium\npip install webdriver-manager # ChromeDriverを自動ダウンロードツール\n```\n\n## Redis\n- Docker で起動する\n```bash\ndocker run -d --name redis -p 6379:6379 redis:7\n```\n- Redis 動作確認\n```bash\npython manage.py shell\nimport redis\nr = redis.Redis(host=\"localhost\", port=6379, db=0)\nprint(\"PING:\", r.ping())\nexit()\n```\n\n\n## Celery\n### ワーカー起動\n```bash\ncelery -A django_selenium_lab worker -l info\n```\n- Windowsはこっちを推奨しい\n```bash\ncelery -A django_selenium_lab worker -l info --pool=solo\n```\n- Celery Workerでシラバスの収集を行うことができた。Celery関係の設定をいじったら，celery worker を再起動させる必要がある\n- Celeryワーカーは起動時のコードと環境変数をプロセスに読み込んだまま走り続けるらしい...\n\n### workerとの接続確認\n```python\n# python manage.py shell\nfrom scraper.tasks import add\nr = add.delay(40, 2)\nr.id, r.status           # （ここで 'PENDING' でもOK）\nr.get(timeout=10)        # ← 42 が返れば接続も処理もOK\n```\n\n### Celery Woker と django-db の接続確認\n```python\nfrom scraper.tasks import store_result_test\nt = 'test用textです。'\nstore_result_test(t)\nc = CeleryTestModel.objects.all()\nprint(c) # 「test用textです。」が表示されたらOk\n```\n- ※ 管理画面でも確認可能\n\n### django-celery-result\n- Celery Woker の処理を Django のDBに保存するには必要らしい\n- `tasks.py`で`models.py`からモデルをインストールしてそのテーブルにtaskの処理を保存しているわけだからあまり必要性を感じない\n- 必要なのか？\n#### マイグレーション\n```python\npython manage.py migrate django_celery_results\n```\n\n\n### MEMO\nPython Celeryで非同期タスクを実行するには、tasks.pyで定義したタスクをviews.pyなどから呼び出します\u003cbr\u003e\n以下例([Celery 5.5.3 documentation » Django » First steps with Django](https://docs.celeryq.dev/en/v5.5.3/django/first-steps-with-django.html))より\n```python\n# views.py\ndef create_user(request):\n    # Note: simplified example, use a form to validate input\n    user = User.objects.create(username=request.POST['username'])\n    send_email.delay(user.pk) # この行でtasks.pyを呼び出しているってこと？\n    return HttpResponse('User created')\n\n# task.py\n@shared_task\ndef send_email(user_pk):\n    user = User.objects.get(pk=user_pk)\n    # send email ...\n```\n\n\n## Installing requirements\n```bash\n$ pip install -r requirements.txt\n```\n## pip freeze\n```bash\npip freeze \u003e requirements.txt\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fituki-t%2Fdjango_selenium_lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fituki-t%2Fdjango_selenium_lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fituki-t%2Fdjango_selenium_lab/lists"}