{"id":24387184,"url":"https://github.com/lier0011/job_listings_python_crawler","last_synced_at":"2026-05-03T18:35:53.123Z","repository":{"id":269151703,"uuid":"906579612","full_name":"lier0011/job_listings_python_crawler","owner":"lier0011","description":"[coursera] web scraping project using python","archived":false,"fork":false,"pushed_at":"2024-12-21T17:48:50.000Z","size":18,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-12T21:36:04.692Z","etag":null,"topics":["csv","git","json","python3","regex"],"latest_commit_sha":null,"homepage":"https://www.coursera.org/projects/scrape-job-postings-data-analyst","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lier0011.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-21T09:49:51.000Z","updated_at":"2024-12-21T17:48:51.000Z","dependencies_parsed_at":"2024-12-21T10:28:41.799Z","dependency_job_id":"60efdd25-8721-4c57-9011-feb4091defbe","html_url":"https://github.com/lier0011/job_listings_python_crawler","commit_stats":null,"previous_names":["lier0011/job_listings_python_crawler"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lier0011/job_listings_python_crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lier0011%2Fjob_listings_python_crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lier0011%2Fjob_listings_python_crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lier0011%2Fjob_listings_python_crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lier0011%2Fjob_listings_python_crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lier0011","download_url":"https://codeload.github.com/lier0011/job_listings_python_crawler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lier0011%2Fjob_listings_python_crawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32580161,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T06:36:36.687Z","status":"ssl_error","status_checked_at":"2026-05-03T06:36:09.306Z","response_time":103,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","git","json","python3","regex"],"created_at":"2025-01-19T12:41:43.862Z","updated_at":"2026-05-03T18:35:53.107Z","avatar_url":"https://github.com/lier0011.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# job_listings_python_crawler\n\n## INTRO\nThis program takes keywords and location as input parameter and do the job search on the \njob platform reed.co.uk. The result is provided as a csv file in the data subdirectory.\n\nThese are some screenshots:\n* starting the program\n![image](https://github.com/user-attachments/assets/34b283c0-cfe4-4403-9083-f730675beae1)\n\n* csv output\n![image](https://github.com/user-attachments/assets/a52f2230-480d-4c08-b439-7f3979b46bde)\n\n\n## SYNTAX\nTo run the program:\n\u003cpre\u003epython3 main.py [keywords] [location]\u003c/pre\u003e\n\nFor example if we want to search for Typist in Manchester\n\u003cpre\u003epython3 main.py Typist Manchester\u003c/pre\u003e  \n\nIf the keywords contains whitespace than you have to escape it using quotation mark\n\u003cpre\u003epython3 main.py \"IT Support\" Blackpool\u003c/pre\u003e\n\n## DESCRIPTION \n\nA coursera project to showcase how to scrape job listings from a popular job search platform.\nAt first, I thought I want to scrape the data from de.indeed.com. I did some research, analysis\nand simple tests using curl on cli, just to check if I can access the website easily.\nUnfortunately, I always got HTTP 403 forbidden. Hence I decided to access other alternative, reed.co.uk. \n\nRelevant information are going to be extracted, such as: job titles, companies, locations, job descriptions, etc.\n\nLink to coursera Project: https://www.coursera.org/projects/scrape-job-postings-data-analyst\n\nHere are the steps to do which I copied from coursera for this project:  \n\n* Setup our development environment\n* Understand the basics of web scraping\n* Analyze the website structure of our job search platform\n* Write the Python code to extract job data from our job search platform\n* Save the data to a CSV file\n* Test our web scraper and refine our code as needed\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flier0011%2Fjob_listings_python_crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flier0011%2Fjob_listings_python_crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flier0011%2Fjob_listings_python_crawler/lists"}