{"id":15102648,"url":"https://github.com/code3743/social-scraper","last_synced_at":"2026-01-19T14:33:34.728Z","repository":{"id":256047811,"uuid":"854206873","full_name":"code3743/social-scraper","owner":"code3743","description":"Social Scraper is a Node.js console application that leverages Playwright to scrape data from various social media platforms. ","archived":false,"fork":false,"pushed_at":"2024-09-10T20:14:36.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T11:42:09.429Z","etag":null,"topics":["nodejs","playwright","scraping","social-media","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/code3743.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-08T16:54:22.000Z","updated_at":"2024-09-10T20:14:39.000Z","dependencies_parsed_at":"2024-09-10T22:44:04.097Z","dependency_job_id":null,"html_url":"https://github.com/code3743/social-scraper","commit_stats":{"total_commits":17,"total_committers":1,"mean_commits":17.0,"dds":0.0,"last_synced_commit":"21022bb038ffc3421b3ffa52b8884d9f1e7c287b"},"previous_names":["code3743/social-scraper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/code3743/social-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/code3743%2Fsocial-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/code3743%2Fsocial-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/code3743%2Fsocial-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/code3743%2Fsocial-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/code3743","download_url":"https://codeload.github.com/code3743/social-scraper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/code3743%2Fsocial-scraper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28571801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T12:50:50.164Z","status":"ssl_error","status_checked_at":"2026-01-19T12:50:42.704Z","response_time":67,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nodejs","playwright","scraping","social-media","web-scraping"],"created_at":"2024-09-25T19:03:54.887Z","updated_at":"2026-01-19T14:33:34.713Z","avatar_url":"https://github.com/code3743.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Social Scraper\n\n## Description\n\n**Social Scraper** is a console tool developed in **Node.js** that uses **Playwright** to perform web scraping on various social media platforms.\n\nCurrently, **Social Scraper** supports the following providers:\n1. **X (Twitter)**\n2. **Instagram**\n\n## Features\n\n- **Multi-Platform Support**: Compatible with different social media platforms through specific providers.\n- **Structured Storage**: Saves the results in JSON files, organized by provider name and date.\n- **Session Management**: Handles active sessions for efficient scraping.\n\n## Installation\n\n### Prerequisites\n\n- **Node.js** (version 18 or higher)\n- **npm** \n\n### Installation Steps\n\n1. **Clone the Repository**\n\n    ```bash\n    git clone https://github.com/code3743/social-scraper.git\n    ```\n\n2. **Navigate to the Project Directory**\n\n    ```bash\n    cd social-scraper\n    ```\n\n3. **Install Dependencies**\n\n    ```bash\n    npm install\n    ```\n\n## Usage\n\n**Social Scraper** is a console-based tool. To run it, use the following command:\n\n```bash\nnode app.js\n```\n\n### Results\n\nThe results are stored in the `/results` folder as JSON files in the format `providerName-date.json`. Each file contains an array of posts with the following structure:\n\n- **id**: The post's identifier.\n- **content**: The textual content of the post.\n- **media**: An array of URLs for associated media.\n- **metadata**: An object containing additional relevant information.\n\n## Contribution\n\nIf you would like to contribute to **Social Scraper**, please follow these steps:\n\n1. **Fork the Repository**\n2. **Create a Branch for Your Feature or Bug Fix**\n\n    ```bash\n    git checkout -b feature/new-feature\n    ```\n\n3. **Make Your Changes and Commit Them**\n\n    ```bash\n    git commit -m \"Description of changes\"\n    ```\n\n4. **Push to Your Branch**\n\n    ```bash\n    git push origin feature/new-feature\n    ```\n\n5. **Open a Pull Request**\n\n## License\n\nThis project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcode3743%2Fsocial-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcode3743%2Fsocial-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcode3743%2Fsocial-scraper/lists"}