Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/joweich/chat-miner
Parsers and visualizations for chats
https://github.com/joweich/chat-miner
chat facebook-messenger messenger parser python signal telegram whatsapp whatsapp-statistics wordcloud
Last synced: about 11 hours ago
JSON representation
Parsers and visualizations for chats
- Host: GitHub
- URL: https://github.com/joweich/chat-miner
- Owner: joweich
- License: mit
- Created: 2021-12-11T22:05:49.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-26T11:22:22.000Z (about 2 months ago)
- Last Synced: 2024-10-29T15:22:37.451Z (about 1 month ago)
- Topics: chat, facebook-messenger, messenger, parser, python, signal, telegram, whatsapp, whatsapp-statistics, wordcloud
- Language: Python
- Homepage:
- Size: 3 MB
- Stars: 566
- Watchers: 10
- Forks: 57
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- project-awesome - joweich/chat-miner - Parsers and visualizations for chats (Python)
README
-----------------
# chat-miner: turn your chats into artwork
[](https://pypi.org/project/chat-miner/)
[](https://opensource.org/licenses/MIT)
[](https://pepy.tech/project/chat-miner)
[](https://codecov.io/gh/joweich/chat-miner)
[](https://github.com/psf/black)
-----------------
**chat-miner** provides lean parsers for every major platform transforming chats into dataframes. Artistic visualizations allow you to explore your data and create artwork from your chats.
## 1. Installation
Latest release including dependencies can be installed via PyPI:
```sh
pip install chat-miner
```
If you're interested in contributing, running the latest source code, or just like to build everything yourself:
```sh
git clone https://github.com/joweich/chat-miner.git
cd chat-miner
pip install .
```
## 2. Exporting chat logs
Have a look at the official tutorials for [WhatsApp](https://faq.whatsapp.com/1180414079177245/), [Signal](https://github.com/carderne/signal-export), [Telegram](https://telegram.org/blog/export-and-more), [Facebook Messenger](https://www.facebook.com/help/messenger-app/713635396288741), or [Instagram Chats](https://help.instagram.com/181231772500920) to learn how to export chat logs for your platform.
## 3. Parsing
Following code showcases the ``WhatsAppParser`` module.
The usage of ``SignalParser``, ``TelegramJsonParser``, ``FacebookMessengerParser``, and ``InstagramJsonParser`` follows the same pattern.
```python
from chatminer.chatparsers import WhatsAppParser
parser = WhatsAppParser(FILEPATH)
parser.parse_file()
df = parser.parsed_messages.get_df(as_pandas=True) # as_pandas=False returns polars dataframe
```
**Note:**
Depending on your source system, Python requires to convert the filepath to a raw string.
```python
import os
FILEPATH = r"C:\Users\Username\chat.txt" # Windows
FILEPATH = "/home/username/chat.txt" # Unix
assert os.path.isfile(FILEPATH)
```
## 4. Visualizing
```python
import chatminer.visualizations as vis
import matplotlib.pyplot as plt
```
### 4.1 Heatmap: Message count per day
```python
fig, ax = plt.subplots(2, 1, figsize=(9, 3))
ax[0] = vis.calendar_heatmap(df, year=2020, cmap='Oranges', ax=ax[0])
ax[1] = vis.calendar_heatmap(df, year=2021, linewidth=0, monthly_border=True, ax=ax[1])
```
### 4.2 Sunburst: Message count per daytime
```python
fig, ax = plt.subplots(1, 2, figsize=(7, 3), subplot_kw={'projection': 'polar'})
ax[0] = vis.sunburst(df, highlight_max=True, isolines=[2500, 5000], isolines_relative=False, ax=ax[0])
ax[1] = vis.sunburst(df, highlight_max=False, isolines=[0.5, 1], color='C1', ax=ax[1])
```
### 4.3 Wordcloud: Word frequencies
```python
fig, ax = plt.subplots(figsize=(8, 3))
stopwords = ['these', 'are', 'stopwords']
kwargs={"background_color": "white", "width": 800, "height": 300, "max_words": 500}
ax = vis.wordcloud(df, ax=ax, stopwords=stopwords, **kwargs)
```
### 4.4 Radarchart: Message count per weekday
```python
if not vis.is_radar_registered():
vis.radar_factory(7, frame="polygon")
fig, ax = plt.subplots(1, 2, figsize=(7, 3), subplot_kw={'projection': 'radar'})
ax[0] = vis.radar(df, ax=ax[0])
ax[1] = vis.radar(df, ax=ax[1], color='C1', alpha=0)
```
## 5. Natural Language Processing
### 5.1 Add Sentiment
```python
from chatminer.nlp import add_sentiment
df_sentiment = add_sentiment(df)
```
### 5.2 Example Plot: Sentiment per Author in Groupchat
```python
df_grouped = df_sentiment.groupby(['author', 'sentiment']).size().unstack(fill_value=0)
ax = df_grouped.plot(kind='bar', stacked=True, figsize=(8, 3))
```
## 6. Command Line Interface
The CLI supports parsing chat logs into csv files.
As of now, you **can't** create visualizations from the CLI directly.
Example usage:
```bash
$ chatminer -p whatsapp -i exportfile.txt -o output.csv
```
Usage guide:
```
usage: chatminer [-h] [-p {whatsapp,instagram,facebook,signal,telegram}] [-i INPUT] [-o OUTPUT]
options:
-h, --help
Show this help message and exit
-p {whatsapp,instagram,facebook,signal,telegram}, --parser {whatsapp,instagram,facebook,signal,telegram}
The platform from which the chats are imported
-i INPUT, --input INPUT
Input file to be processed
-o OUTPUT, --output OUTPUT
Output file for the results
```