{"id":28360462,"url":"https://github.com/mazzasaverio/data-job-parser","last_synced_at":"2026-04-25T23:34:22.518Z","repository":{"id":295461470,"uuid":"990081301","full_name":"mazzasaverio/data-job-parser","owner":"mazzasaverio","description":"Job posting parser with structured outputs","archived":false,"fork":false,"pushed_at":"2025-05-26T07:37:10.000Z","size":181,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-06-23T00:39:06.926Z","etag":null,"topics":["data-engineering","job-parser","openai","pydantic-v2","structured-outputs","uv"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mazzasaverio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-25T13:16:20.000Z","updated_at":"2025-05-26T07:37:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"7297c3ad-9951-49f3-b5be-5ae43bd281f1","html_url":"https://github.com/mazzasaverio/data-job-parser","commit_stats":null,"previous_names":["mazzasaverio/data-job-parser"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/mazzasaverio/data-job-parser","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fdata-job-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fdata-job-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fdata-job-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fdata-job-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mazzasaverio","download_url":"https://codeload.github.com/mazzasaverio/data-job-parser/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzasaverio%2Fdata-job-parser/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32280975,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-25T18:29:39.964Z","status":"ssl_error","status_checked_at":"2026-04-25T18:29:32.149Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineering","job-parser","openai","pydantic-v2","structured-outputs","uv"],"created_at":"2025-05-28T11:08:21.341Z","updated_at":"2026-04-25T23:34:22.512Z","avatar_url":"https://github.com/mazzasaverio.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data Job Parser\n\n[![PyPI Downloads](https://img.shields.io/pypi/dm/data-job-parser)](https://pypi.org/project/data-job-parser/)\n[![Python Version](https://img.shields.io/badge/python-3.8%2B-blue)](https://www.python.org/downloads/)\n[![PyPI Version](https://img.shields.io/pypi/v/data-job-parser)](https://pypi.org/project/data-job-parser/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Tests](https://github.com/mazzasaverio/data-job-parser/workflows/CI/badge.svg)](https://github.com/mazzasaverio/data-job-parser/actions)\n\nExtract structured data from job postings using OpenAI's structured output capabilities.\n\n## Features\n\n- 🎯 **Smart Extraction**: Extract structured information from any job posting URL\n- 🧠 **AI-Powered**: Uses OpenAI's GPT models with structured output for accurate parsing\n- 📊 **Comprehensive Data**: Covers all job posting aspects including salary, skills, requirements\n- 🌐 **Advanced Scraping**: Playwright-based scraping handles JavaScript-heavy sites\n- 💾 **File Storage**: Save as markdown and JSON with SHA-1 hash filenames for deduplication\n- 🔄 **Reliability**: Automatic retries with exponential backoff for robust operation\n- 📝 **Observability**: Detailed logging with Logfire integration\n- 🐍 **Modern Python**: Full type hints and Python 3.8+ support\n\n## Installation\n\n```bash\npip install data-job-parser\n```\n\nAfter installation, install Playwright browsers:\n\n```bash\nplaywright install chromium\n```\n\n## Quick Start\n\n### Basic Usage\n\n```python\nfrom data_job_parser import JobPostingParser\n\n# Initialize with OpenAI API key\nparser = JobPostingParser(api_key=\"your-openai-api-key\")\n\n# Parse a job posting\njob_data = parser.parse(\"https://example.com/job-posting\")\n\n# Access structured data\nprint(f\"Title: {job_data.title}\")\nprint(f\"Company: {job_data.company}\")\nprint(f\"Location: {job_data.location.city}, {job_data.location.country}\")\nprint(f\"Salary: {job_data.salary.min_amount}-{job_data.salary.max_amount} {job_data.salary.currency}\")\nprint(f\"Skills: {', '.join(job_data.required_skills)}\")\n```\n\n### Save Files\n\n```python\n# Parse and save both markdown and JSON\njob_data, markdown_path, json_path = await parser.parse_async(\n    \"https://jobs.pradagroup.com/job/Milan-Data-Engineer/1199629101/\",\n    save_markdown=True,\n    save_json=True\n)\n\nprint(f\"Markdown: {markdown_path}\")\nprint(f\"JSON: {json_path}\")\n```\n\n### Batch Processing\n\n```python\nfrom data_job_parser import JobPostingParser\n\nparser = JobPostingParser(api_key=\"your-api-key\")\nurls = [\"https://job1.com\", \"https://job2.com\", \"https://job3.com\"]\n\nfor url in urls:\n    try:\n        job_data, md_path, json_path = parser.parse(\n            url, \n            save_markdown=True, \n            save_json=True\n        )\n        print(f\"✅ {job_data.title} at {job_data.company}\")\n    except Exception as e:\n        print(f\"❌ Failed to parse {url}: {e}\")\n```\n\n## Configuration\n\n### Environment Variables\n\nCreate a `.env` file in your project root:\n\n```bash\n# Required\nOPENAI_API_KEY=your-openai-api-key\n\n# Optional - Logging\nLOGFIRE_TOKEN=your-logfire-token\n\n# Optional - Model Configuration\nOPENAI_MODEL=gpt-4-turbo-preview\n\n# Optional - Playwright Settings\nPLAYWRIGHT_HEADLESS=true\nPLAYWRIGHT_TIMEOUT=60000\n```\n\n### API Key Setup\n\n**Option 1: Parameter**\n```python\nparser = JobPostingParser(api_key=\"your-api-key\")\n```\n\n**Option 2: Environment Variable**\n```bash\nexport OPENAI_API_KEY=\"your-api-key\"\n```\n```python\nparser = JobPostingParser()  # Auto-loads from env\n```\n\n### Model Selection\n\n```python\n# Use different OpenAI model\nparser = JobPostingParser(\n    api_key=\"your-api-key\", \n    model=\"gpt-4o\"  # or gpt-3.5-turbo, etc.\n)\n```\n\n### File Storage\n\nFiles are saved with SHA-1 hash filenames to prevent duplicates:\n\n```\ndata/\n├── markdown/\n│   └── a1b2c3d4e5f6789.md\n└── json/\n    └── a1b2c3d4e5f6789.json\n```\n\n## Data Model\n\nThe parser extracts comprehensive job information:\n\n**Core Information**\n- Title, company, location, description\n- Work type (full-time, part-time, contract)\n- Work mode (remote, hybrid, on-site)\n- Experience level required\n\n**Compensation \u0026 Benefits**\n- Salary range with currency\n- Benefits and perks\n- Stock options, bonuses\n\n**Skills \u0026 Requirements**\n- Required technical skills\n- Preferred/nice-to-have skills\n- Education requirements\n- Years of experience needed\n\n**Additional Details**\n- Team size and department\n- Application process\n- Company culture information\n\n## Error Handling\n\nThe parser includes robust error handling:\n\n```python\nfrom data_job_parser import JobPostingParser\nfrom data_job_parser.exceptions import ParsingError, ScrapingError\n\nparser = JobPostingParser(api_key=\"your-api-key\")\n\ntry:\n    job_data = parser.parse(\"https://example.com/job\")\nexcept ScrapingError as e:\n    print(f\"Failed to scrape URL: {e}\")\nexcept ParsingError as e:\n    print(f\"Failed to parse content: {e}\")\nexcept Exception as e:\n    print(f\"Unexpected error: {e}\")\n```\n\n## Development\n\n### Setup Development Environment\n\n```bash\n# Clone repository\ngit clone https://github.com/mazzasaverio/data-job-parser.git\ncd data-job-parser\n\n# Install with uv (recommended)\nuv sync --dev\n\n# Or with pip\npip install -e \".[dev]\"\n```\n\n### Run Tests\n\n```bash\n# Run all tests\nuv run pytest\n\n# With coverage report\nuv run pytest --cov=src/data_job_parser --cov-report=html\n\n# Run specific test file\nuv run pytest tests/test_parser.py -v\n```\n\n### Code Quality\n\n```bash\n# Format code\nuv run ruff format .\n\n# Lint code\nuv run ruff check .\n\n# Type checking\nuv run mypy src/\n```\n\n### Release Process\n\n1. **Update version** in both files:\n   - `src/data_job_parser/__init__.py`\n   - `pyproject.toml`\n\n2. **Run quality checks**:\n   ```bash\n   uv run pytest\n   uv run ruff check .\n   uv run mypy src/\n   ```\n\n3. **Commit and tag**:\n   ```bash\n   git add .\n   git commit -m \"chore: bump version to X.Y.Z\"\n   git push origin main\n   \n   git tag vX.Y.Z\n   git push origin vX.Y.Z\n   ```\n\n4. **Automated deployment**: GitHub Actions will automatically:\n   - Run tests\n   - Build package  \n   - Publish to PyPI\n   - Create GitHub release\n\n## Contributing\n\nWe welcome contributions! Please follow these steps:\n\n1. **Fork** the repository\n2. **Create** feature branch: `git checkout -b feature/amazing-feature`\n3. **Make** your changes with tests\n4. **Run** quality checks: `uv run pytest \u0026\u0026 uv run ruff check .`\n5. **Commit** changes: `git commit -m 'feat: add amazing feature'`\n6. **Push** branch: `git push origin feature/amazing-feature`\n7. **Open** a Pull Request\n\n### Development Guidelines\n\n- Write tests for new features\n- Follow existing code style\n- Update documentation as needed\n- Use conventional commit messages\n\n## Requirements\n\n- **Python**: 3.8+\n- **OpenAI API Key**: Required for parsing\n- **Internet Connection**: For web scraping and API calls\n\n## Changelog\n\nSee [CHANGELOG.md](CHANGELOG.md) for version history and changes.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- **OpenAI** for structured output capabilities\n- **Playwright** for robust web scraping\n- **Pydantic** for data validation\n- **Logfire** for observability\n\n---\n\n**Made with ❤️ by [Saverio Mazza](https://github.com/mazzasaverio)**","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazzasaverio%2Fdata-job-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmazzasaverio%2Fdata-job-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazzasaverio%2Fdata-job-parser/lists"}