{"id":20411652,"url":"https://github.com/alvinkuruvilla/kmlog","last_synced_at":"2025-04-12T16:26:08.466Z","repository":{"id":39128091,"uuid":"423287178","full_name":"AlvinKuruvilla/kmlog","owner":"AlvinKuruvilla","description":"A sleek and lightweight Python TUI application to collect typing dynamics data from real people ","archived":false,"fork":false,"pushed_at":"2025-04-04T02:34:51.000Z","size":81588,"stargazers_count":2,"open_issues_count":6,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-04T03:25:45.679Z","etag":null,"topics":["datacollection","keylogging","python3","sql","yaml"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlvinKuruvilla.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-31T23:54:31.000Z","updated_at":"2025-04-04T02:34:55.000Z","dependencies_parsed_at":"2023-02-08T19:15:51.232Z","dependency_job_id":"3b5180a4-1792-4681-8065-8ed556b59bee","html_url":"https://github.com/AlvinKuruvilla/kmlog","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlvinKuruvilla%2Fkmlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlvinKuruvilla%2Fkmlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlvinKuruvilla%2Fkmlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlvinKuruvilla%2Fkmlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlvinKuruvilla","download_url":"https://codeload.github.com/AlvinKuruvilla/kmlog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248595498,"owners_count":21130536,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datacollection","keylogging","python3","sql","yaml"],"created_at":"2024-11-15T05:53:34.454Z","updated_at":"2025-04-12T16:26:08.460Z","avatar_url":"https://github.com/AlvinKuruvilla.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"![KM_Icon](media/logo_transparent.png)\n\n# KMLog\n\nKMLog is an interactive cli application for interfacing with tools which collect typing dynamics\n\n## ⚠️ Disclaimer ⚠️\n\nI am very much aware that KMLogger can be used for nefarious purposes. This project is used solely for controlled data collection purposes. Do not use this for illegal purposes, period. Please read the LICENSE for licensing information\n\n### Table of Contents\n\n**[Rationale](#rationale)**\u003cbr\u003e\n**[Code Breakdown](#code-breakdown)**\u003cbr\u003e\n**[MySQL Installation](#mysql-installation)**\u003cbr\u003e\n**[MySQL Setup](#mysql-setup)**\u003cbr\u003e\n**[User Information Storage in YAML](#yaml)**\u003cbr\u003e\n**[Usage](#usage)**\u003cbr\u003e\n**[Using the Headless Keylogger](#using-the-headless-keylogger)**\u003cbr\u003e\n**[Testing the HTML Pages](#testing-the-html-pages-locally)**\u003cbr\u003e\n**[License](#license)**\u003cbr\u003e\n**[Contributing](#contributing)**\u003cbr\u003e\n**[Media](#media)**\u003cbr\u003e\n**[Credits](#credits)**\u003cbr\u003e\n**[Participate](#participate)**\u003cbr\u003e\n\n## Rationale\n\nThe goal of this project is to create a sleek and lightweight interface to collect specific typing dynamics data from real people\n\n## Code Breakdown\n\n### The _\"base\"_ folder\n\nThis folder holds python files for the project's core functionality:\n\n- csv_writer.py: A wrapper class that allows for writing data to a csv file\n- log.py: This is a wrapper class of the loguru module for more colorful and expressive logging\n- util.py: This file holds some utility functions to make the tui prettier and more functional.\n- backends: This folder hold files that wrap specific subsystems, one of which, the project uses (or could use) for operations (querying, storage, etc...)\n  - sql_driver.py: This file holds a wrapper class to make some sql operations nicer\n  - yaml_driver.py: This file holds a wrapper class to make yaml operations nicer\n- user_ops: This folder holds files that wrap specific user-related functions and operations\n  - generic_ops.py: A set of generic functions to get user information and properly format it\n  - sql_ops.py: A specialized set of MySQL specific functions and operations to properly add and retrieve user information from a MySQL database\n  - yml_ops.py: A specialized set of YAML specific functions and operations to properly add and retrieve user information from a YAML file\n\n### The _\"tools\"_ folder\n\nThis folder holds the keylogger and mouse tracker to collect typing dynamics data.\n\n### The _\"logs\"_ folder\n\nThis folder holds the log and csv files generated by the keylogger and the mouse tracker\n\n### The _\".env\"_ file\n\nThe use of the .env file was a strategic decision to allow for future users of this code base to be able to adapt it to their needs with as little code modifications as possible. The use of the .env file also improves the secrecy and security associated with connecting to the mysql database as none of the connection information is written into the code, but rather read in from created environment variables.\nThe .env file contains the following variables:\n\n- KM_USER: The username to connect to the database\n- PASSWORD: The password to connect to the database\n- DB_HOST: The host used to connect to the database\n- DB_NAME: The name of the database which will be operated on\n- TABLE: The name of the database table which will be operated on\n\nSee sample.env for an example .env configuration\n\n```diff\nNOTE: If you decide to change the names of the environment variables make sure that they do not conflict with an Operating System specific predefined variables. For example USER in MacOS and USERNAME in Windows -\n```\n\n## MySQL Installation\n\n### [Homebrew](https://formulae.brew.sh/formula/mysql#default)\n\nFor macs install homebrew and run:\n\n```sh\nbrew install mysql\n```\n\n### [MySQL Installer](https://dev.mysql.com/doc/refman/8.0/en/windows-installation.html)\n\nThe MySQL documentation recommends using the MySQL Installer for Windows\n\n## MySQL Setup\n\nTo ensure that mysql is installed correctly run:\n\n```sh\nmysql --version\n```\n\n### **\u003cu\u003eWindows\u003c/u\u003e**\n\nThe MySQL installer should take care of everything\nTo start the mysql interactive cli run:\n\n```sh\nmysql -u [YOUR_USERNAME] -p\n```\n\n### **\u003cu\u003eMacOS\u003c/u\u003e**\n\nStart MySQL server by running:\n\n```sh\nbrew services start mysql\n```\n\nSecure installation of MySQL Server:\n\n```sh\nmysql_secure_installation\n```\n\nLike for Windows, to start the MySQL cli run:\n\n```sh\nmysql -u [YOUR_USERNAME] -p\n```\n\n## User Information Storage in YAML\n\nThe YAML backed system is a simpler local storage alternative to the MySQL database\nThe YAML system uses the users inputted user-id to search through the \"users\" folder to see if a YAML file already exists with a matching file name. If it does we know the user already exists in the \"database\" and we can just pull their information from the file. If the file does not exist, we can create one and add it to the \"database\"\nThe YAML file stores data in the following format:\n\n```yaml\nuser_id: Integer\nfirst_name: String\nlast_name: String\nage: Integer\ngender: String\nhandedness: String\neducation: String\nplatform: String\n```\n\nSee 123.yaml for more details\n\n## Usage\n\n### Build with pip\n\n1. Clone this repository.\n\n```bash\ngit clone https://github.com/AlvinKuruvilla/kmlog.git\n```\n\n2. Install dependencies\n\n```bash\npip3 -r requirements.txt\n```\n\n3. Run the main file\n\n```bash\npython3 kmlog.py\n```\n\n### Automated Build\n\n1. Clone this repository.\n\n```bash\ngit clone https://github.com/AlvinKuruvilla/kmlog.git\n```\n\n2. Install dependencies\n\n```bash\npip3 -r requirements.txt\n```\n\n3. Create the kmlog.spec file and build the executable\n\n```bash\n./build.sh\n```\n\u003e This puts the executable in the dist/ folder\n\n4. Run kmlog\n\n```bash\n./kmlog\n```\n\nTo fix any problems with input recording, give your terminal or the executable Input Monitoring access\n\nOn Apple devices, navigate to System Preferences \u003e Go to Privacy and Security \u003e Go to the Input Monitoring tab \u003e Select your terminal application or kmlog executable (if they are not already there) and then toggle one of them on.\n\n### Running from Github release\n\n1. Download the binary from the release page.\n\n2. Change the permissions of the binary file so it is executable\n\n   - This could be done with a command like:\n\n   ```bash\n     chmod 777 kmlog\n   ```\n\n3. If your shell kills the executable when trying to run it, this is because Apple does not allow programs to run which are from unidentifiable developers. To fix this, navigate to System Preferences \u003e Go to Privacy and Security \u003e Go to the General tab \u003e Click Run Anyway\n\n## Using the headless keylogger\n\nThe headless keylogger is a local webserver which utilizes one of the social media clones in fake_pages/ as a frontend by proxy through tasks.html.\n\nThe webserver utilizes the FP_USER_ID environment variable which __MUST__ be set in a `.env` file for the server to work properly (The server will throw an error if the variable is not set)\n\n## Testing the HTML pages locally\n\nThe HTML pages can be tested locally by following these steps:\n\n1. Start a python webserver in the root directory by:\n  ```bash\n     python3 -m http.server 8000\n  ```\n\n2. Navigate to 127.0.0.1:8000/ (or whatever port you specified) and you should see a directory of the project root\n\n3. If you want to test one of the social media clone pages in isolation you can just navigate to the respective site's `index.html`\n\nOR\n\n3. If you wish to test the data collection flow you can start the process at [consent.html](pages/hosting/consent.html) and follow the flow from there.\n\u003e ℹ **_NOTE:_**  Make sure to have the headless kmlog server running as it is required for the tasks page to work properly:\n  ```bash\n     python3 src/kmlog_headless.py\n  ```\n\u003e  ⚠️ **_NOTE:_**  Do not use live server to test the tasks page as there is an issue with how it handles the fetch requests, which causes the page to refresh on any keyboard input or focus change\n\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n\n## Contributing\n\nSee [Contributing](CONTRIBUTING.md)\n\n## Media\n\nThe logo was designed from [Hatchful by Shopify](https://hatchful.shopify.com)\n\n\n## Credits\n[The original Facebook clone](https://github.com/KashanAdnan/Facebook-Clone)\n\n[The original Instagram clone](https://github.com/leocosta1/instagram-clone)\n\n## Participate\nClick [here](docs/hosting/consent.html) to participate in data collection","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falvinkuruvilla%2Fkmlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falvinkuruvilla%2Fkmlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falvinkuruvilla%2Fkmlog/lists"}