Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/nickysemenza/message-analyzer

💬 📊 Facebook Messenger history scraper + analyzer
https://github.com/nickysemenza/message-analyzer

chats facebook scraper

Last synced: 14 days ago
JSON representation

💬 📊 Facebook Messenger history scraper + analyzer

Awesome Lists containing this project

README

        

#Facebook Messenger Analyzer
> Download, Parse, and Analyze

This project consists of a few parts / 'modules'
* Scraper (node.js) that downloads facebook messages into MySQL
* Analyzer (node) that does data processing, storing intermediary values in redis
* Web (React.js) that displays data nicely

The Scraper code can work on its own if desired.

##Howto run the scraper
* Node & Yarn
* MySQL, with utf8mb4 support
* Redis

1. clone, cd into project repo, `yarn install`
2. `cp .env.example .env`, add your own credentials for Facebook and MySQL
4. `node auth.js` to auth with facebook and save the state in `appstate.json` (only need to do this once until the cookies expire)
6. Scroll down to the end of scraper.js and run your code from within login method!

#TODO
* [ ] make script automatically download all your threads in a loop
* [x] add support for other metadata (changing emoji/color/title) -requires tweaking facebook-chat-api
* [ ] stop downloading a thread when there are back to back sql duplicate errors, this can be a hacky way of getting a delta update of a thread since the last time we downloaded it
* [x] chat titles for 1:1 convos are blank, so maybe have a script hint them based on fbid->name lookup and update that title value for 1:1 threads