{"id":17593845,"url":"https://github.com/rfs-adreno/twitter-x-scraper","last_synced_at":"2025-04-30T04:23:33.393Z","repository":{"id":211256496,"uuid":"709516800","full_name":"RFS-ADRENO/twitter-x-scraper","owner":"RFS-ADRENO","description":"scrape from twitter/x and post to facebook page, control via telegram bot","archived":false,"fork":false,"pushed_at":"2024-04-29T09:42:28.000Z","size":82,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T12:33:51.162Z","etag":null,"topics":["facebook","facebook-graph-api","javascript","nodejs","puppeteer","scraper","telegram","telegram-bot","twitter"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RFS-ADRENO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-24T20:53:57.000Z","updated_at":"2024-08-16T10:06:37.000Z","dependencies_parsed_at":"2024-10-23T02:02:25.764Z","dependency_job_id":null,"html_url":"https://github.com/RFS-ADRENO/twitter-x-scraper","commit_stats":null,"previous_names":["rfs-adreno/twitter-x-scraper"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RFS-ADRENO%2Ftwitter-x-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RFS-ADRENO%2Ftwitter-x-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RFS-ADRENO%2Ftwitter-x-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RFS-ADRENO%2Ftwitter-x-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RFS-ADRENO","download_url":"https://codeload.github.com/RFS-ADRENO/twitter-x-scraper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251640025,"owners_count":21619871,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["facebook","facebook-graph-api","javascript","nodejs","puppeteer","scraper","telegram","telegram-bot","twitter"],"created_at":"2024-10-22T06:43:22.397Z","updated_at":"2025-04-30T04:23:33.358Z","avatar_url":"https://github.com/RFS-ADRENO.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# twitter-x-scraper\nscrape from twitter/x and post to facebook page, control via telegram bot\n\n\n## Installation\n\nLine by line:\n\n\n```bash\ngit clone https://github.com/RFS-ADRENO/twitter-x-scraper.git\ncd twitter-x-scraper\n\nnpm install\n```\n\n\n## Configuration\n\nRename `.env.example` to `.env` and fill in the values.\n\nLinux:\n```bash\nmv .env.example .env\n```\n\nWindows:\n```bash\nrename .env.example .env\n```\n\n### Telegram Bot\n\nCreate new telegram bot via [BotFather](https://t.me/BotFather) and get the token. \u003cbr/\u003e\nFor more information, see [here](https://core.telegram.org/bots/tutorial).\n\nCreate new telegram channel and add the bot. \u003cbr/\u003e\nNow fill in the values in `.env` file\n\n```env\nTELEGRAMBOT_TOKEN=\"your_token_here\"\nWHITELIST_CHAT_ID=\"your_channel_id_here\"\n```\n\nStart the bot\n\n```bash\nnpm start\n```\n\nGo to your channel and send `/uid`, the bot will send you your id. \u003cbr/\u003e\nGrab the id and fill in the `.env` file\n\n```env\nADMIN_IDS=\"id1,id2,id3\"\n```\n\n\u003e As you can see, you can add multiple ids, just separate them with comma.\nThe bot will only accept commands if the channel id is equal to `WHITELIST_CHAT_ID` and the sender id is equal to one of the `ADMIN_IDS`.\n\n### Facebook App\n\nFirst you need to create a facebook app from [here](https://developers.facebook.com/apps/). \u003cbr/\u003e\n\nFollow the steps below while creating the app:\n\nChoose `Other`\n\n![step-1](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/7c7e0c71-fe50-46ac-a7ea-fb4c7a5dbe5d)\n\nChose `Business`\n\n![step-2](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/09084a87-7270-4c7f-be68-ce86ef263093)\n\nThen fill in the app name and contact email and click `Create App`\n\n![step-3](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/dd4608c1-292f-4cb9-b53f-fab098f432fb)\n\n\nNow, you need a Privacy Policy URL. \u003cbr/\u003e\nYou can create one from [here](https://app.freeprivacypolicy.com/wizard/privacy-policy) for free. \u003cbr/\u003e\n\nThen navigate to `App settings` \u003e `Basic` and fill in the Privacy Policy URL and click `Save Changes`\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/fbdea232-c3f8-4076-b932-e54a5de19298)\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/12c40c02-b51a-47a1-8a6f-a52aebc226a5)\n\n\u003cbr /\u003e\n\nNow toggle the `App Mode` from `Development` to `Live`\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/6744ef5d-fd62-4333-97ab-b5cc95650cab)\n\n\n### Facebook Page\n\nThis bot support posting to multiple facebook pages. \u003cbr/\u003e\nFirst, create a page from [here](https://www.facebook.com/pages/create) if you haven't already. \u003cbr/\u003e\n\nThen go to [Graph API Explorer](https://developers.facebook.com/tools/explorer/) and select your app from the dropdown menu. \u003cbr/\u003e\nThen click `Get Token` \u003e `Get Page Access Token`\n\nFollow the instructions and select pages that you want to post to. \u003cbr/\u003e\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/64dcf716-7aa9-4ade-ba5e-db763722d31a)\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/93108cef-c5b7-4bab-b455-14ff5cd4289c)\n\nAfter you connected pages to your app, click `Get Token` or `User Token`, this time you will see your page names. \u003cbr/\u003e\nSelect the page that you want to post, add `pages_manage_posts` permission. \u003cbr/\u003e\nThen click `Generate Access Token` then copy the token.\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/4842542d-36c3-4150-801a-947e6d2d8a8d)\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/84d1e462-3731-45bd-bc9e-dff43ea815f8)\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/cd47218b-9b60-45d2-8a68-f704b9e0f110)\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/1ace9d85-2e1a-4fb8-a07c-5b356cb6263a)\n\nThen copy the token and paste it in `.env` file\n\n\u003e Because this bot support posting to multiple pages, you need to follow the naming rule.\n\n```env\nFBTOKEN_PAGE_ONE = \"your_token_here\"\nFBTOKEN_PAGE_TWO = \"your_token_here\"\n```\n\n\n### Start the program\n\nNow all done, you can start the program\n\n```bash\nnpm start\n```\n\nGo to your telegram channel and send `/pages` to see the list of pages that you connected to your app. \u003cbr/\u003e\n\n## Commands\n\n| Command | Description |\n| --- | --- |\n| `/uid` | Get your telegram id |\n| `/pages` | Get the list of pages that you connected to your app |\n| `/startTop` | Start scraping from top section |\n| `/startLatest` | Start scraping from latest section |\n| `/stop` | Stop scraping |\n\n### Example\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/1828f148-30df-4be6-927c-cf85072876dc)\n\n![image](https://github.com/RFS-ADRENO/twitter-x-scraper/assets/77768272/cf32b338-2de3-402c-85dd-6e9bc33a5865)\n\n`startTop` and `startLatest` commands will add a job to the queue. \u003cbr/\u003e\nThe job will be executed when the previous job is finished. \u003cbr/\u003e\n\n`/startTop pageNumber | hashtag1 hashtag2 ...` \u003cbr/\u003e\n`/startLatest pageNumber | hashtag1 hashtag2 ...`\n\nfor example: `/startTop 1 | openai gpt`\nThis will scrape from top section of tweets that contains both `openai` and `gpt` hashtags.\n\n\n## LICENSE\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frfs-adreno%2Ftwitter-x-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frfs-adreno%2Ftwitter-x-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frfs-adreno%2Ftwitter-x-scraper/lists"}