{"id":23431039,"url":"https://github.com/jroakes/seodp","last_synced_at":"2025-04-12T23:20:24.029Z","repository":{"id":258829668,"uuid":"866049371","full_name":"jroakes/SEODP","owner":"jroakes","description":"The SEO Data Platform automates SEO analysis, aggregating data from Google Analytics 4, Search Console, Page Speed Insights, and rendered content. Powered by Google Gemini AI, it emails actionable insights weekly or monthly, streamlining SEO workflows.","archived":false,"fork":false,"pushed_at":"2024-10-20T19:21:21.000Z","size":2990,"stargazers_count":22,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-05T00:46:52.641Z","etag":null,"topics":["automation","llm","seo"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jroakes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-01T15:02:23.000Z","updated_at":"2025-03-28T00:18:47.000Z","dependencies_parsed_at":"2024-10-22T00:23:56.600Z","dependency_job_id":null,"html_url":"https://github.com/jroakes/SEODP","commit_stats":null,"previous_names":["jroakes/seodp"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jroakes%2FSEODP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jroakes%2FSEODP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jroakes%2FSEODP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jroakes%2FSEODP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jroakes","download_url":"https://codeload.github.com/jroakes/SEODP/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248643879,"owners_count":21138521,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","llm","seo"],"created_at":"2024-12-23T09:49:14.067Z","updated_at":"2025-04-12T23:20:24.005Z","avatar_url":"https://github.com/jroakes.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SEO Data Platform\n\n![SEO Data Platform Cover](cover.png)\n\n## Description\n\nThe SEO Data Platform is a comprehensive tool designed to automate the collection, analysis, and reporting of SEO-related data. It integrates with various data sources such as Google Analytics 4, Google Search Console, and PageSpeed Insights to provide actionable insights for improving website performance and search engine rankings.\n\n## Features\n\n- Automated data collection from multiple sources\n- Integration with Google Analytics 4, Google Search Console, and PageSpeed Insights\n- URL content extraction and analysis\n- LLM-powered insights generation using Google's Gemini API\n- Configurable reporting topics and significance thresholds\n- Scheduled data processing and report generation\n- Email reporting functionality\n\n## Installation\n\n1. Clone the repository:\n```\ngit clone https://github.com/jroakes/SEODP.git \ncd SEODP\n```\n\n2. Install the required dependencies:\n`pip install -r requirements.txt`\nYou can also use poetry.\n\n3. Set up the configuration file:\n- Edit `seodpconfig.yaml` with your specific settings\n\n4. Set up environment variables:\n- Create a `.env` file in the project root\n- Add the required API keys and credentials (see Configuration section)\n\n\n## Usage\n\nTo start the SEO Data Platform:\n\n`python src/seodp/main.py --start`\n\n\nFor other command-line options:\n\n`python src/seodp/main.py --help`\n\n\n## Configuration\n\n### Environment Variables\n\nSet the following environment variables in your `.env` file:\n\n- `SERVICE_ACCOUNT_FILE_PATH`: Path to your Google service account JSON file\n- `SUBJECT_EMAIL`: Email address for Google API authentication\n- `SCRAPINGBEE_API_KEY`: Your ScrapingBee API key\n- `GEMINI_API_KEY`: Your Google Gemini API key\n- `MAILTRAP_LOGIN`: Your Mailtrap login\n- `MAILTRAP_PASSWORD`: Your Mailtrap password\n- `MAILTRAP_SENDER_EMAIL`: Sender email for Mailtrap\n- `RECIPIENT_EMAIL`: Recipient email for reports\n\nSee `.env.example` example file.\n\n### Configuration File\n\nEdit `seodpconfig.yaml` to customize:\n\n- URLs to analyze\n- Reporting schedule\n- Data sources and API settings\n- Reporting topics and thresholds\n\n## Project Structure\n\n- `src/seodp/`: Main source code directory\n  - `main.py`: Entry point of the application\n  - `settings.py`: Configuration loading and management\n  - `lib/`: Core functionality modules\n    - `extractors/`: Data extraction modules for different sources\n    - `manager/`: Data processing and management modules\n    - `api/`: API client modules (e.g., Gemini, email)\n\n## Contributing\n\nContributions to the SEO Data Platform are welcome! Please follow these steps:\n\n1. Fork the repository\n2. Create a new branch for your feature\n3. Commit your changes\n4. Push to your fork\n5. Submit a pull request\n\nPlease ensure your code adheres to the project's coding standards and include tests for new functionality.\n\n### Ideas for Contributing\n\n- [ ] Don't re-hit PSI when backfilling data on initial run\n- [ ] Async!!!!!!!!!!!!!!!!!!!!!!!!!!! processing\n- [x] ~~Use something called Pydantic (Hey Joe!)~~\n- [ ] Add YoY as well PoP for comparison\n- [ ] Fine tune some models\n- [ ] Add a SEMRush, Ahrefs, or Nozzle extractor???!!!\n- [ ] Build a one-click deployer on Digital Ocean or something\n\n## License\nMIT","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjroakes%2Fseodp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjroakes%2Fseodp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjroakes%2Fseodp/lists"}