Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rfs-adreno/twitter-x-scraper
scrape from twitter/x and post to facebook page, control via telegram bot
https://github.com/rfs-adreno/twitter-x-scraper
facebook facebook-graph-api javascript nodejs puppeteer scraper telegram telegram-bot twitter
Last synced: 2 months ago
JSON representation
scrape from twitter/x and post to facebook page, control via telegram bot
- Host: GitHub
- URL: https://github.com/rfs-adreno/twitter-x-scraper
- Owner: RFS-ADRENO
- License: mit
- Created: 2023-10-24T20:53:57.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-04-29T09:42:28.000Z (8 months ago)
- Last Synced: 2024-04-29T10:55:00.838Z (8 months ago)
- Topics: facebook, facebook-graph-api, javascript, nodejs, puppeteer, scraper, telegram, telegram-bot, twitter
- Language: JavaScript
- Homepage:
- Size: 80.1 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# twitter-x-scraper
scrape from twitter/x and post to facebook page, control via telegram bot
## Installation
Line by line:
```bash
git clone https://github.com/RFS-ADRENO/twitter-x-scraper.git
cd twitter-x-scraper
npm install
```
## Configuration
Rename `.env.example` to `.env` and fill in the values.
Linux:
```bash
mv .env.example .env
```
Windows:
```bash
rename .env.example .env
```
### Telegram Bot
Create new telegram bot via [BotFather](https://t.me/BotFather) and get the token.
For more information, see [here](https://core.telegram.org/bots/tutorial).
Create new telegram channel and add the bot.
Now fill in the values in `.env` file
```env
TELEGRAMBOT_TOKEN="your_token_here"
WHITELIST_CHAT_ID="your_channel_id_here"
```
Start the bot
```bash
npm start
```
Go to your channel and send `/uid`, the bot will send you your id.
Grab the id and fill in the `.env` file
```env
ADMIN_IDS="id1,id2,id3"
```
> As you can see, you can add multiple ids, just separate them with comma.
The bot will only accept commands if the channel id is equal to `WHITELIST_CHAT_ID` and the sender id is equal to one of the `ADMIN_IDS`.
### Facebook App
First you need to create a facebook app from [here](https://developers.facebook.com/apps/).
Follow the steps below while creating the app:
Choose `Other`
Chose `Business`
Then fill in the app name and contact email and click `Create App`
Now, you need a Privacy Policy URL.
You can create one from [here](https://app.freeprivacypolicy.com/wizard/privacy-policy) for free.
Then navigate to `App settings` > `Basic` and fill in the Privacy Policy URL and click `Save Changes`
Now toggle the `App Mode` from `Development` to `Live`
### Facebook Page
This bot support posting to multiple facebook pages.
First, create a page from [here](https://www.facebook.com/pages/create) if you haven't already.
Then go to [Graph API Explorer](https://developers.facebook.com/tools/explorer/) and select your app from the dropdown menu.
Then click `Get Token` > `Get Page Access Token`
Follow the instructions and select pages that you want to post to.
After you connected pages to your app, click `Get Token` or `User Token`, this time you will see your page names.
Select the page that you want to post, add `pages_manage_posts` permission.
Then click `Generate Access Token` then copy the token.
Then copy the token and paste it in `.env` file
> Because this bot support posting to multiple pages, you need to follow the naming rule.
```env
FBTOKEN_PAGE_ONE = "your_token_here"
FBTOKEN_PAGE_TWO = "your_token_here"
```
### Start the program
Now all done, you can start the program
```bash
npm start
```
Go to your telegram channel and send `/pages` to see the list of pages that you connected to your app.
## Commands
| Command | Description |
| --- | --- |
| `/uid` | Get your telegram id |
| `/pages` | Get the list of pages that you connected to your app |
| `/startTop` | Start scraping from top section |
| `/startLatest` | Start scraping from latest section |
| `/stop` | Stop scraping |
### Example
`startTop` and `startLatest` commands will add a job to the queue.
The job will be executed when the previous job is finished.
`/startTop pageNumber | hashtag1 hashtag2 ...`
`/startLatest pageNumber | hashtag1 hashtag2 ...`
for example: `/startTop 1 | openai gpt`
This will scrape from top section of tweets that contains both `openai` and `gpt` hashtags.
## LICENSE
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details