{"id":27947835,"url":"https://github.com/miiguellssantos/x-api-research","last_synced_at":"2026-07-21T15:33:17.033Z","repository":{"id":291847724,"uuid":"978974612","full_name":"miiguellssantos/X-API-Research","owner":"miiguellssantos","description":"A Node.js tool to bulk-collect tweets via Twitter API v2, handle rate limits and safe interruption, and export results incrementally to CSV.","archived":false,"fork":false,"pushed_at":"2025-05-06T20:03:16.000Z","size":631,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-07T14:48:37.860Z","etag":null,"topics":["javascript","x-api"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miiguellssantos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-06T19:49:31.000Z","updated_at":"2025-05-06T20:03:20.000Z","dependencies_parsed_at":"2025-05-06T21:32:11.301Z","dependency_job_id":null,"html_url":"https://github.com/miiguellssantos/X-API-Research","commit_stats":null,"previous_names":["miiguellssantos/x-api-research"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/miiguellssantos/X-API-Research","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miiguellssantos%2FX-API-Research","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miiguellssantos%2FX-API-Research/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miiguellssantos%2FX-API-Research/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miiguellssantos%2FX-API-Research/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miiguellssantos","download_url":"https://codeload.github.com/miiguellssantos/X-API-Research/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miiguellssantos%2FX-API-Research/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279018110,"owners_count":26086281,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["javascript","x-api"],"created_at":"2025-05-07T14:37:36.995Z","updated_at":"2025-10-14T06:31:32.028Z","avatar_url":"https://github.com/miiguellssantos.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Twitter Data Collector\n\nA Node.js script that uses the Twitter API v2 to search and collect tweets in bulk, save progress incrementally to CSV, and handle rate limits and safe interruption (Ctrl+C).\n\n## Features\n\n- **Authentication** with Twitter v2 Bearer Token  \n- **Incremental CSV export** with headers  \n- **Safe interruption** (SIGINT) and progress saving  \n- **Rate-limit handling** with automatic wait until reset  \n- **Configurable query**, date range, and output folder  \n\n## Prerequisites\n\n- Node.js v14+  \n- A Twitter Developer account with Elevated or Academic Research access  \n- A valid Bearer Token  \n\n## Installation\n\n1. Clone this repository  \n   ```bash\n   git clone https://github.com/miiguellssantos/X-API-Research.git\n   cd X-API-Research\n   ```\n\n2. Install dependencies  \n   ```bash\n   npm install twitter-api-v2 luxon node-notifier\n   ```\n\n3. Add your Bearer Token  \n   - Open `twitterAPI.js`  \n   - Replace the placeholder in the `BEARER_TOKEN` constant with your actual token  \n\n## Configuration\n\n- **Queries**  \n  Edit the `queries` array in `twitterAPI.js` to define one or more search queries.  \n- **Date range**  \n  Adjust `startDate` and `endDate` (ISO 8601 format) in `main()` to control the period.  \n- **Output path**  \n  Change the `path.resolve(...)` in `main()` to point to your desired CSV directory.\n\n## Usage\n\n```bash\nnode twitterAPI.js\n```\n\n- The script logs authentication status, each batch progress, and any rate-limit waits.  \n- Press **Ctrl+C** to safely stop the run; progress will be saved up to the last batch.\n\n## CSV Output Format\n\n| Geo    | Created At            | Likes | Retweets | Tweet ID          | Username | Text           | User Location | Followers Count |\n|--------|-----------------------|-------|----------|-------------------|----------|----------------|---------------|-----------------|\n| JSON   | `YYYY-MM-DDTHH:MM:SSZ`| int   | int      | string            | string   | string         | string        | int             |\n\n## Extending\n\n- Add more tweet or user fields by modifying `params` in `getTweetDataWithStopSupport()`.  \n- Re-enable desktop notifications (using `node-notifier`) by uncommenting the `notifier.notify(...)` block.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiiguellssantos%2Fx-api-research","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiiguellssantos%2Fx-api-research","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiiguellssantos%2Fx-api-research/lists"}