{"id":19785100,"url":"https://github.com/donghquinn/daangn_scraping","last_synced_at":"2025-04-30T22:32:44.303Z","repository":{"id":61231770,"uuid":"549404104","full_name":"donghquinn/daangn_scraping","owner":"donghquinn","description":"당근마켓 인기 중고 매물 - 서울특별시 전체 매물 데이터 스크레이핑","archived":true,"fork":false,"pushed_at":"2023-02-15T03:10:00.000Z","size":82722,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-28T13:23:51.913Z","etag":null,"topics":["api","data-scrape","nodejs"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/donghquinn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-11T06:15:49.000Z","updated_at":"2023-03-31T04:08:56.000Z","dependencies_parsed_at":"2022-10-13T06:47:56.634Z","dependency_job_id":null,"html_url":"https://github.com/donghquinn/daangn_scraping","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donghquinn%2Fdaangn_scraping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donghquinn%2Fdaangn_scraping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donghquinn%2Fdaangn_scraping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donghquinn%2Fdaangn_scraping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/donghquinn","download_url":"https://codeload.github.com/donghquinn/daangn_scraping/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251791959,"owners_count":21644494,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","data-scrape","nodejs"],"created_at":"2024-11-12T06:13:30.768Z","updated_at":"2025-04-30T22:32:39.294Z","avatar_url":"https://github.com/donghquinn.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 당근마켓 인기 물품 데이터 스크레이핑\n\n## 사용 라이브러리\n\n: cheerio, axios, winston, winston-rotate-daily-file, nodemon, ts-node,\n\n## 스크레이핑 하는 데이터 종류\n\n게시글 URL, 카테고리, 지역명\n\n## 동작 구조\n\n60분(한 시간)에 한 번씩 데이터를 가져와 파싱 후 DB에 저장.\n\n- 당근마켓 서울 지역 인기 매물 페이지의 모든 목록의 카테고리와 url 링크를 스크레이핑 한다.\n- url 링크를 이용해 해당 게시물 내부에서 지역 정보를 긁어온다.\n- 이렇게 가져온 데이터를 데이터베이스에 인서트한다.\n\nV1은 전체 데이터에 대한 API\n\nV2는 각각 점수들에 대한 통계를 낸 API. 가장 많은 지역과 카테고리에 대해 각각의 API를 열어둔다.\n\n- 카테고리: /v2/score/category\n- 지역: /v2/score/region\n\n## 환경변수 설명\n\n- DB_HOST= DB 서버 주소\n- DB_PORT= DB 서버 포트\n- DB_NAME= 데이터베이스 이름\n- DB_PASS= DB 유저 비밀번호\n- DB_USER: DB 유저 이름\n- TABLE: 스크레이핑 한 데이터 저장 테이블 이름\n\n- NODE_ENV: 구동 환경 ( development | production)\n- APP_PORT: 구동 포트\n\n- TIME_SET: 데이터를 스크레이핑 할 주기 선택\n\n## 빌드\n\nyarn run build\n\n## 구동 명령\n\nyarn run start\n\n## 도커를 이용한 구동\n\ndocker-compose up -d\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonghquinn%2Fdaangn_scraping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdonghquinn%2Fdaangn_scraping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonghquinn%2Fdaangn_scraping/lists"}