Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jakubvalenta/automatic-diary
Create one timeline from various digital sources
https://github.com/jakubvalenta/automatic-diary
calendar csv diary journal log productivity
Last synced: 3 months ago
JSON representation
Create one timeline from various digital sources
- Host: GitHub
- URL: https://github.com/jakubvalenta/automatic-diary
- Owner: jakubvalenta
- License: gpl-3.0
- Created: 2019-03-12T21:53:38.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-12-09T05:20:56.000Z (almost 2 years ago)
- Last Synced: 2024-05-16T02:39:52.901Z (6 months ago)
- Topics: calendar, csv, diary, journal, log, productivity
- Language: Python
- Size: 524 KB
- Stars: 12
- Watchers: 3
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: COPYING
Awesome Lists containing this project
- project-awesome - jakubvalenta/automatic-diary - Create one timeline from various digital sources (Python)
README
# Automatic Diary
Automatic Diary is a script that creates one timeline from various digital
sources describing your life.
_On the screenshot above, you can see: calendar events (blue), completed todo
list items (yellow), software development work (gray), sport activity (green),
and films watched (red)._
## Why
Automatic Diary can be useful to those who:
(a) have bad memory,\
(b) want to remember what they did,\
(c) but don't have the time to write a real diary.
## What is collected
- [X] Calendar events (via CalDAV service and iCalendar file reading)
- [X] Emails sent and received (via Maildir reading)
- [X] Sport activity (via spreadsheet parsing)
- [X] Facebook and Twitter posts (via Facebook/Twitter data archive parsing)
- [X] Software development work (via Git repository log)
- [X] Completed todo list items (via todo.txt parsing)
- [X] Films watched (via ČSFD website parsing and Trakt.tv)
- [X] Custom diary notes (via plain-text and Org-mode files parsing)
## What is not collected
Automatic Diary could be extended to collect also the following data but right
now it is considered low priority:
- [ ] Custom notes and ideas
- [ ] Articles and blog posts written
- [ ] Phone calls and text messages (SMS)
- [ ] Social media private messages and messaging apps
- [ ] Browser history
- [ ] Music listened (via Last.fm)
- [ ] OpenStreetMap contributions
- [ ] Money transfers
- [ ] News headlines
- [ ] Weather
## Installation
### Mac
``` shell
$ brew install python
$ pip install poetry
$ make setup
```
### Arch Linux
``` shell
# pacman -S python-poetry libsecret
$ make setup
```
### Other systems
Install these dependencies manually:
- Python >= 3.7
- poetry
- libsecret (Linux keyring) -- required by the `caldav` provider
Then run:
``` shell
$ make setup
```
## Configuration
Before you run Automatic Diary, you need to configure all the providers (sources
from which the data for your timeline will be read). The following providers are
supported:
- [caldav](#caldav)
- [csfd](#csfd)
- [csv](#csv)
- [facebook](#facebook)
- [git](#git)
- [icalendar](#icalendar)
- [maildir](#maildir)
- [orgmode](#orgmode)
- [orgmodelist](#orgmodelist)
- [todotxt](#todotxt)
- [trakt](#trakt)
- [twitter](#twitter)
- [txt](#txt)
All providers are configured using a single `config.json` file. Use
[config-sample.json](./config-sample.json) as a template for your own
configuration.
### caldav
- Input: CalDAV server
- Output: Names and locations of calendar events
- Configuration:
``` json
{
"url": "",
"username": "",
"password_key": "",
"password_val": "",
"cache_dir": ""
}
```
### csfd
- Input: User profile on [ČSFD](https://www.csfd.cz/) (film database website, something
like IMDB)
- Output: Titles of films rated
- Configuration:
``` json
{
"profile_url": "",
"cache_dir": ""
}
```
### csv
- Input: CSV spreadsheet (.csv) file
- Output: Rows formatted using a template
- Configuration
``` json
{
"path": "",
"date_source": "{{}}",
"date_format": "",
"text_source": ""
}
```
### facebook
- Input: Downloaded Facebook archive
- Output: Texts of statuses
- Configuration
``` json
{
"path": "",
"username": ""
}
```
### git
- Input: Directory with checked-out Git repositories and an author name
- Output: Commit messages by the author from all repositories
- Configuration
``` json
"config": {
"base_path": "",
"author": ""
}
```
### icalendar
- Input: Calendar events stored offline in iCalendar (.ics) files
- Output: Names and locations of calendar events
- Configuration:
``` json
{
"paths": [
"",
...
]
}
```
Not that events from all the listed .ics files will be merged -- duplicate
events removed.
### maildir
- Input: Emails stored offline in the Maildir format
- Output: Subjects of emails
- Configuration:
``` json
{
"received_pathname": "",
"sent_pathname": ""
}
```
### orgmode
- Input: Emacs Org-mode (.org) file in format:
* <2019-01-17 Thu>
Lorem ipsum
foo.
bar
* <2019-01-18 Fri>
spam spam
...
- Output: Example:
``` csv
2019-01-17,Lorem ipsum foo.
2019-01-17,bar
2019-01-18,spam spam
```
- Configuration:
``` json
{
"path": ""
}
```
### orgmodelist
- Input: Emacs Org-mode (.org) file in format:
- Lorem ipsum foo. <2019-01-17 Thu>
- bar <2019-01-18 Fri 11:30>
...
- Output: Example:
``` csv
2019-01-17,Lorem ipsum foo.
2019-01-18T11:00:00+02:00,bar
```
- Configuration:
``` json
{
"path": ""
}
```
### todotxt
- Input: Todo.txt completed tasks file (done.txt)
- Output: Texts of completed tasks
- Configuration
``` json
{
"path": ""
}
```
### trakt
- Input: [Manually create an application within Trakt.tv](https://trakt.tv/oauth/applications/new). Manual login with OAuth when running this provider. A recommended `Redirect URI` is `urn:ietf:wg:oauth:2.0:oob`
- Output: Texts of watched films and shows
- Configuration
``` json
{
"key_id": "",
"key_secret": "",
"app_id": ""
}
```
### twitter
- Input: Downloaded Twitter archive
- Output: Texts of tweets
- Configuration
``` json
{
"path": ""
}
```
### txt
- Input: Plain text (.txt) file in format:
```
2015-12-02 St
Some
Text
Lorem
Ipsum
Hierarchy
Spam
2015-12-03 Čt
Foobar
...
```
- Output: Example:
``` csv
2015-12-02,Some
2015-12-02,Text
2015-12-02,Lorem: Ipsum: Hierarchy
2015-12-02,Lorem: Ipsum: Spam
2015-12-03,Foobar
```
- Configuration
``` json
{
"path": ""
}
```
## Usage
### Generating CSV
The basic output of Automatic Diary is a CSV file. Generate it by running:
``` shell
$ ./automatic-diary
```
Example:
``` shell
$ ./automatic-diary ~/.config/automatic-diary/config.json ~/Desktop/automatic_diary.csv
```
The CSV output is in format:
``` csv
,,,
```
Example CSV output:
``` csv
2019-01-23T15:31:54-08:00,git,human-activities,model: Log exceptions while scanning
2019-01-24T09:00:00+01:00,caldav,https://dav.mailbox.org/caldav/da39a3ee5e6b,DHL Packstation
2019-01-25,todotxt,done.txt,Opravit Ondrovi kolo
```
See the help for all command line options:
``` shell
$ ./automatic-diary --help
```
### Visualization
The output CSV file can also be rendered as an HTML document which looks kind of
like a calendar. See the screenshot above. Generate this HTML document by
running:
``` shell
$ ./automatic-diary-visualize
```
Example:
``` shell
$ ./automatic-diary-visualize ~/Desktop/automatic_diary.csv ~/Desktop/automatic_diary.html
```
See the help for all command line options:
``` shell
$ ./automatic-diary-visualize --help
```
## Development
### Testing and linting
``` shell
$ make test
$ make lint
```
### Help
``` shell
$ make help
```
## Contributing
__Feel free to remix this project__ under the terms of the GNU General Public
License version 3 or later. See [COPYING](./COPYING) and [NOTICE](./NOTICE).