https://github.com/alvinkuruvilla/kmlog
A sleek and lightweight Python TUI application to collect typing dynamics data from real people
https://github.com/alvinkuruvilla/kmlog
datacollection keylogging python3 sql yaml
Last synced: about 1 year ago
JSON representation
A sleek and lightweight Python TUI application to collect typing dynamics data from real people
- Host: GitHub
- URL: https://github.com/alvinkuruvilla/kmlog
- Owner: AlvinKuruvilla
- License: mit
- Created: 2021-10-31T23:54:31.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2025-04-04T02:34:51.000Z (about 1 year ago)
- Last Synced: 2025-04-04T03:25:45.679Z (about 1 year ago)
- Topics: datacollection, keylogging, python3, sql, yaml
- Language: HTML
- Homepage:
- Size: 77.8 MB
- Stars: 2
- Watchers: 3
- Forks: 3
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
README
# KMLog
KMLog is an interactive cli application for interfacing with tools which collect typing dynamics
## ⚠️ Disclaimer ⚠️
I am very much aware that KMLogger can be used for nefarious purposes. This project is used solely for controlled data collection purposes. Do not use this for illegal purposes, period. Please read the LICENSE for licensing information
### Table of Contents
**[Rationale](#rationale)**
**[Code Breakdown](#code-breakdown)**
**[MySQL Installation](#mysql-installation)**
**[MySQL Setup](#mysql-setup)**
**[User Information Storage in YAML](#yaml)**
**[Usage](#usage)**
**[Using the Headless Keylogger](#using-the-headless-keylogger)**
**[Testing the HTML Pages](#testing-the-html-pages-locally)**
**[License](#license)**
**[Contributing](#contributing)**
**[Media](#media)**
**[Credits](#credits)**
**[Participate](#participate)**
## Rationale
The goal of this project is to create a sleek and lightweight interface to collect specific typing dynamics data from real people
## Code Breakdown
### The _"base"_ folder
This folder holds python files for the project's core functionality:
- csv_writer.py: A wrapper class that allows for writing data to a csv file
- log.py: This is a wrapper class of the loguru module for more colorful and expressive logging
- util.py: This file holds some utility functions to make the tui prettier and more functional.
- backends: This folder hold files that wrap specific subsystems, one of which, the project uses (or could use) for operations (querying, storage, etc...)
- sql_driver.py: This file holds a wrapper class to make some sql operations nicer
- yaml_driver.py: This file holds a wrapper class to make yaml operations nicer
- user_ops: This folder holds files that wrap specific user-related functions and operations
- generic_ops.py: A set of generic functions to get user information and properly format it
- sql_ops.py: A specialized set of MySQL specific functions and operations to properly add and retrieve user information from a MySQL database
- yml_ops.py: A specialized set of YAML specific functions and operations to properly add and retrieve user information from a YAML file
### The _"tools"_ folder
This folder holds the keylogger and mouse tracker to collect typing dynamics data.
### The _"logs"_ folder
This folder holds the log and csv files generated by the keylogger and the mouse tracker
### The _".env"_ file
The use of the .env file was a strategic decision to allow for future users of this code base to be able to adapt it to their needs with as little code modifications as possible. The use of the .env file also improves the secrecy and security associated with connecting to the mysql database as none of the connection information is written into the code, but rather read in from created environment variables.
The .env file contains the following variables:
- KM_USER: The username to connect to the database
- PASSWORD: The password to connect to the database
- DB_HOST: The host used to connect to the database
- DB_NAME: The name of the database which will be operated on
- TABLE: The name of the database table which will be operated on
See sample.env for an example .env configuration
```diff
NOTE: If you decide to change the names of the environment variables make sure that they do not conflict with an Operating System specific predefined variables. For example USER in MacOS and USERNAME in Windows -
```
## MySQL Installation
### [Homebrew](https://formulae.brew.sh/formula/mysql#default)
For macs install homebrew and run:
```sh
brew install mysql
```
### [MySQL Installer](https://dev.mysql.com/doc/refman/8.0/en/windows-installation.html)
The MySQL documentation recommends using the MySQL Installer for Windows
## MySQL Setup
To ensure that mysql is installed correctly run:
```sh
mysql --version
```
### **Windows**
The MySQL installer should take care of everything
To start the mysql interactive cli run:
```sh
mysql -u [YOUR_USERNAME] -p
```
### **MacOS**
Start MySQL server by running:
```sh
brew services start mysql
```
Secure installation of MySQL Server:
```sh
mysql_secure_installation
```
Like for Windows, to start the MySQL cli run:
```sh
mysql -u [YOUR_USERNAME] -p
```
## User Information Storage in YAML
The YAML backed system is a simpler local storage alternative to the MySQL database
The YAML system uses the users inputted user-id to search through the "users" folder to see if a YAML file already exists with a matching file name. If it does we know the user already exists in the "database" and we can just pull their information from the file. If the file does not exist, we can create one and add it to the "database"
The YAML file stores data in the following format:
```yaml
user_id: Integer
first_name: String
last_name: String
age: Integer
gender: String
handedness: String
education: String
platform: String
```
See 123.yaml for more details
## Usage
### Build with pip
1. Clone this repository.
```bash
git clone https://github.com/AlvinKuruvilla/kmlog.git
```
2. Install dependencies
```bash
pip3 -r requirements.txt
```
3. Run the main file
```bash
python3 kmlog.py
```
### Automated Build
1. Clone this repository.
```bash
git clone https://github.com/AlvinKuruvilla/kmlog.git
```
2. Install dependencies
```bash
pip3 -r requirements.txt
```
3. Create the kmlog.spec file and build the executable
```bash
./build.sh
```
> This puts the executable in the dist/ folder
4. Run kmlog
```bash
./kmlog
```
To fix any problems with input recording, give your terminal or the executable Input Monitoring access
On Apple devices, navigate to System Preferences > Go to Privacy and Security > Go to the Input Monitoring tab > Select your terminal application or kmlog executable (if they are not already there) and then toggle one of them on.
### Running from Github release
1. Download the binary from the release page.
2. Change the permissions of the binary file so it is executable
- This could be done with a command like:
```bash
chmod 777 kmlog
```
3. If your shell kills the executable when trying to run it, this is because Apple does not allow programs to run which are from unidentifiable developers. To fix this, navigate to System Preferences > Go to Privacy and Security > Go to the General tab > Click Run Anyway
## Using the headless keylogger
The headless keylogger is a local webserver which utilizes one of the social media clones in fake_pages/ as a frontend by proxy through tasks.html.
The webserver utilizes the FP_USER_ID environment variable which __MUST__ be set in a `.env` file for the server to work properly (The server will throw an error if the variable is not set)
## Testing the HTML pages locally
The HTML pages can be tested locally by following these steps:
1. Start a python webserver in the root directory by:
```bash
python3 -m http.server 8000
```
2. Navigate to 127.0.0.1:8000/ (or whatever port you specified) and you should see a directory of the project root
3. If you want to test one of the social media clone pages in isolation you can just navigate to the respective site's `index.html`
OR
3. If you wish to test the data collection flow you can start the process at [consent.html](pages/hosting/consent.html) and follow the flow from there.
> ℹ **_NOTE:_** Make sure to have the headless kmlog server running as it is required for the tasks page to work properly:
```bash
python3 src/kmlog_headless.py
```
> ⚠️ **_NOTE:_** Do not use live server to test the tasks page as there is an issue with how it handles the fetch requests, which causes the page to refresh on any keyboard input or focus change
## License
[MIT](https://choosealicense.com/licenses/mit/)
## Contributing
See [Contributing](CONTRIBUTING.md)
## Media
The logo was designed from [Hatchful by Shopify](https://hatchful.shopify.com)
## Credits
[The original Facebook clone](https://github.com/KashanAdnan/Facebook-Clone)
[The original Instagram clone](https://github.com/leocosta1/instagram-clone)
## Participate
Click [here](docs/hosting/consent.html) to participate in data collection