https://github.com/himanshub16/lekhpal
Monitor and catalog Twitter feed matching your desired keywords
https://github.com/himanshub16/lekhpal
analytics data data-catalog data-filtering mongodb twitter twitter-streaming-api
Last synced: 2 months ago
JSON representation
Monitor and catalog Twitter feed matching your desired keywords
- Host: GitHub
- URL: https://github.com/himanshub16/lekhpal
- Owner: himanshub16
- License: mit
- Created: 2018-02-23T21:30:24.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2018-02-25T04:24:25.000Z (about 7 years ago)
- Last Synced: 2025-01-02T08:14:07.110Z (4 months ago)
- Topics: analytics, data, data-catalog, data-filtering, mongodb, twitter, twitter-streaming-api
- Language: JavaScript
- Size: 25.4 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Lekhpal
Lekhpal monitors Twitter for certain filters and catalogs the result in a MongoDB database.
It provides a ReST API to add/remove trackers, which are custom set of keywords to monitor, and query the result back.
## Features
* **ReSTful** and **stateless** handling of keywords to monitor Twitter for.
* API on initialization, starts monitoring for keywords where you last left it.
* Delete keyword to stop monitoring via ReST.
* Query tweets stored in database.
* Paginated API to limit large amount of data.
* Download as **JSON/CSV** supported.
## Technical aspects
* Uses Twitter's streaming API
* MongoDB database as JSON store and for efficient query.
* NodeJS to have an efficient asynchronous implementation.
## Instructions
1. Create a database for this in MongoDB.
```mongo
create database lekhpal;
use lekhpal;
db.createCollection('tweets')
db.createCollection('trackers')
```
2. Modify environment variables in `.env.sh`.
3. Source env file and run `index.js`.
```sh
source .env.sh
node index.js
```
## Routes
* ```
POST /find
Content-Type: application/json
```
**Parameters**
| Parameter | Type | Description |
| -- | -- | -- |
| query | Object | The query to make in MongoDB's query format |
| projection | Object | The fields to project in MongoDB's projection format |
| format | string | One of 'json' or 'csv' |
| page_no | number | Default 1, The page number to get result from |
| page_size | number | Optional; The page size required |
* ```
POST /add_tracker
Content-Type: application/json
```
**Parameters**
| Parameter | Type | Description |
|-| - |-|
| words_to_track | Array | The list of words to track. This creates a new tracker and starts listening to it |
* ```
DELETE /remove_tracker
Content-Type: application/json
```
**Parameters**
| Parameter | Type | Description |
|-| - |-|
| tracker_id | string | The id of the tracker to delete |
* ```
GET /list_trackers
Content-Type: application/json
```