{"id":27038333,"url":"https://github.com/neulhan/piro-webtoon","last_synced_at":"2025-07-29T03:06:54.842Z","repository":{"id":102379685,"uuid":"236403923","full_name":"Neulhan/piro-webtoon","owner":"Neulhan","description":null,"archived":false,"fork":false,"pushed_at":"2020-01-28T08:05:15.000Z","size":6084,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-05T02:33:36.301Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Neulhan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-27T01:52:23.000Z","updated_at":"2020-01-28T08:05:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"cdabde48-902b-4985-829c-155b6652d4c2","html_url":"https://github.com/Neulhan/piro-webtoon","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Neulhan/piro-webtoon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neulhan%2Fpiro-webtoon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neulhan%2Fpiro-webtoon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neulhan%2Fpiro-webtoon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neulhan%2Fpiro-webtoon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Neulhan","download_url":"https://codeload.github.com/Neulhan/piro-webtoon/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neulhan%2Fpiro-webtoon/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267621584,"owners_count":24116900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-05T02:30:43.546Z","updated_at":"2025-07-29T03:06:54.831Z","avatar_url":"https://github.com/Neulhan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🗺piro_crawling\n\n```python\nprint('피로그래밍 12기 크롤링 강의 페이지입니다.')\n```\n\n## 사용환경\n- jupyter notebook    (.ipynb)\n- google colaboratory (.ipynb)\n\n\n## request\n파이썬 코드를 통해서 웹 페이지에 HTTP 요청을 보냄\n\n### urllib\n```python\nimport urllib\n\nurllib_case = urllib.request.urlopen(url)\nhtml_text = urllib_case.read().decode(\"utf-8\")\n```\n[파이썬 binary 파일에 대해](https://wikidocs.net/15101)\n\n### requests\n```python\nimport requests\n\nhtml_text = requests.get(url).text\n\n# html_text 에는 str 형식의 html 문서가 담긴다\n```\n\n[urllib vs requests 정리된 블로그](https://brownbears.tistory.com/299)\n\n## bs4.Beautifulsoup\n\n[beautifulsoup란 무엇인지에 대해 잘 정리된 블로그](https://velog.io/@neulhan/%EC%B4%88%EB%B3%B4%EB%8F%84-%ED%95%A0-%EC%88%98-%EC%9E%88%EB%8A%94-python%EC%9C%BC%EB%A1%9C-%EB%84%A4%EC%9D%B4%EB%B2%84%EC%97%90%EC%84%9C-%EC%8B%A4%EC%8B%9C%EA%B0%84-%EA%B2%80%EC%83%89%EC%96%B4-%EC%A0%95%EB%B3%B4-%EA%B0%80%EC%A0%B8%EC%98%A4%EA%B8%B0-2-BeautifulSoup-1uk4asqet0)\n```python \nfrom bs4 import BeautifulSoup as bs\n\n# beautiful soup 객체 생성\nsoup = bs(html_text, 'html.parser')\n\n# html 안에서 선택자를 통해 특정 태그들 가져오기\nselected_elements = soup.select('selector')\n\n# 가져온 태그들 활용하기\n# 1. .text로 내용 추출\n# 2. .attrs\n# 3. .get\n```\n## pandas 활용하기\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneulhan%2Fpiro-webtoon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneulhan%2Fpiro-webtoon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneulhan%2Fpiro-webtoon/lists"}