{"id":15050582,"url":"https://github.com/kerekesdavid/pycolytics","last_synced_at":"2025-07-13T15:42:35.187Z","repository":{"id":255267093,"uuid":"847891151","full_name":"KerekesDavid/pycolytics","owner":"KerekesDavid","description":"A tiny event logging webservice for software analytics.","archived":false,"fork":false,"pushed_at":"2024-11-05T09:52:32.000Z","size":47,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-10T02:14:14.624Z","etag":null,"topics":["app-analytics","event-logging","fastapi","game-analytics","pyhton","software-analytics","sqlite","webservice"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KerekesDavid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"kerekesdavid","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2024-08-26T18:36:02.000Z","updated_at":"2025-02-01T19:02:32.000Z","dependencies_parsed_at":"2024-09-12T22:35:53.106Z","dependency_job_id":"48062537-433c-4dd1-ab97-ebae188593be","html_url":"https://github.com/KerekesDavid/pycolytics","commit_stats":null,"previous_names":["kerekesdavid/pycolytics"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KerekesDavid%2Fpycolytics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KerekesDavid%2Fpycolytics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KerekesDavid%2Fpycolytics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KerekesDavid%2Fpycolytics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KerekesDavid","download_url":"https://codeload.github.com/KerekesDavid/pycolytics/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248142903,"owners_count":21054671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["app-analytics","event-logging","fastapi","game-analytics","pyhton","software-analytics","sqlite","webservice"],"created_at":"2024-09-24T21:27:36.203Z","updated_at":"2025-04-10T02:14:21.675Z","avatar_url":"https://github.com/KerekesDavid.png","language":"Python","funding_links":["https://ko-fi.com/kerekesdavid","https://ko-fi.com/E1E712JJXK'"],"categories":[],"sub_categories":[],"readme":"# Pycolytics\n\nA tiny webservice for logging software analytics events.\nIt takes HTTP requests, and puts them into an SQLite database.\n\nThe goal of this library is to be easy to set up and easy to use.\nMinimal dependencies, no complicated database setups.\n\n- Easy setup on any linux machine, or in WSL:\n\n  ```sh\n  git clone git@github.com:KerekesDavid/pycolytics.git\n  cd pycolytics\n  pip install -r requirements.txt\n  ```\n\n- Launches out of the box:\n\n  ```sh\n  fastapi dev\n  ```\n\n- After launch, API docs are available at: \u003chttp://127.0.0.1:8000/docs\u003e\n\nPycolytics is written in python, based on\n[SQLite](https://github.com/sqlite/sqlite) and\n[FastAPI](https://github.com/fastapi/fastapi), and was inspired by\n[Attolytics](https://github.com/ttencate/attolytics/).\n\n## Client Plugin for Godot 4.2+\n\n- I wrote a plugin for Godot: just install it and call a single function to log\n  an event! You can find it in the\n  [asset library](https://godotengine.org/asset-library/asset/3292), or on\n  [github](https://github.com/KerekesDavid/pycolytics-godot).\n\n- If you have written clients for anything else, I would be more than happy to\n  feature them here!\n\n## Configuration\n\nEdit the .env file, or specify these parameters as environment variables:\n\n```sh\n# Name of the database file to write into.\nSQLITE_FILE_PATH=\"databases/database.db\"\n\n#  A list of secret keys. The server won't accept events\n#    that do not contain one of these keys in the request body.\nAPI_KEYS=[\"I-am-an-unsecure-dev-key-REPLACE_ME\"]\n\n# Requests from the same IP above this rate will be rejected.\n# See https://limits.readthedocs.io/en/stable/quickstart.html#rate-limit-string-notation\nRATE_LIMIT=\"60/minute\"\n```\n\n## API\n\nThe server will listen to POST requests at `http://ip:port/v1.0/event`,\nand will expect a request body in the following format:\n\n```json\n{\n  \"event_type\": \"string\",\n  \"application\": \"string\",\n  \"version\": \"string\",\n  \"platform\": \"string\",\n  \"user_id\": \"string\",\n  \"session_id\": \"string\",\n  \"value\": {\n    \"event_description\": \"Life, the universe and everything.\",\n    \"event_data\": 42\n  },\n  \"api_key\": \"I-am-an-unsecure-dev-key-REPLACE_ME\"\n}\n```\n\nThere is also a more performant batch interface at\n`http://ip:port/v1.0/events`, expecting a list of events:\n\n```json\n[\n  {\"event_type\": ...},\n  {\"event_type\": ...},\n  ...\n]\n```\n\nAn example curl call for logging an event:\n\n```sh\ncurl -X 'POST' \\\n  'http://127.0.0.1:8000/v1.0/event' \\\n  -H 'accept: */*' \\\n  -H 'Content-Type: application/json' \\\n  -d '{\n  \"event_type\": \"string\",\n  \"application\": \"string\",\n  \"version\": \"string\",\n  \"platform\": \"string\",\n  \"user_id\": \"string\",\n  \"session_id\": \"string\",\n  \"value\": {\n    \"event_description\": \"Life, the universe and everything.\"\n    \"event_data\": 42\n    },\n  \"api_key\": \"I-am-an-unsecure-dev-key-REPLACE_ME\"\n}'\n```\n\nThe `value` field can contain an arbitrary JSON with event details.\n\nThe POST request will return `204: No Content` on successful inserts.\n\n## Database\n\nThe database will contain an `event` table with all logged events.\nThe columns are:\n\n```sql\nevent_type VARCHAR NOT NULL\nplatform VARCHAR NOT NULL\nversion VARCHAR NOT NULL\nuser_id VARCHAR NOT NULL\nsession_id VARCHAR NOT NULL\nvalue JSON NOT NULL\nid INTEGER NOT NULL, PRIMARY KEY\ntime DATETIME NOT NULL\n```\n\nIt can be opened using any sqlite database browser,\nor in python using the built in sqlite package.\n\nMy personal choice for performing data analytics is a\n[jupyter notebook](https://jupyter.org/) using\n[pandas](https://pandas.pydata.org/). They have a wonderful cheat sheet\n[here](https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf).\n\n## Launching a Production Server\n\nSetting up a permanent server as a service is also quite simple.\n\nThe method I share here has some minimal extra complications, but it ensures\nsome level of separation from other parts of the system using systemd's\n`DynamicUser` parameter. It might come handy in case there is a vulnerability\nin FastAPI.\n\n(Contributions to this section are very welcome, I'm barely a fledgelig server admin.)\n\n- Install pycolytics, and set up a virtualenv.\n  A usual place for this would be `/srv/pycolytics` for example.\n\n- Create a systemd service file: `/etc/systemd/system/pycolytics.service`\n\n  ```INI\n  [Unit]\n  Description=Uvicorn instance serving Pycolytics\n  After=network.target\n\n  [Service]\n  Type=simple\n  DynamicUser=yes\n  User=pycolytics\n\n  WorkingDirectory=/srv/pycolytics\n  StateDirectory=pycolytics/databases\n\n  ExecStart=/srv/pycolytics/.venv/bin/uvicorn \\\n          --workers=4 \\\n          --host=0.0.0.0 \\\n          --port=8080 \\\n          app.main:app\n  ExecReload=/bin/kill -HUP ${MAINPID}\n  RestartSec=15\n  Restart=always\n\n  [Install]\n  WantedBy=multi-user.target\n\n  ```\n\n- Generate an API key:\n\n  ```sh\n  openssl rand -base64 24\n  ```\n\n  This will stop random people from logging events in your database, it will\n  not stop a script kiddie who can decompile the key from your app, or pluck it\n  from network traffic. I'd suggest creating a new one for every version of\n  your application, and retiring old ones after a while.\n\n- Set up the .env file:\n\n  - Replace `API_KEYS=[\"I-am-an-unsecure-dev-key-REPLACE_ME\"]` with the\n    newly generated key.\n  - Set the database path to the systemd state directory:\n    `SQLITE_FILE_NAME=\"/var/lib/pycolytics/databases/database.db\"`\n\n- Run read the new config:\n\n  `sudo systemctl daemon-reload`\n\n- Make the service start on boot:\n\n  `sudo systemctl enable pycolytics`\n\n- Start the service:\n\n  `sudo systemctl start pycolytics`\n\n- Check for errors:\n\n  `sudo systemctl status pycolytics`\n\n- In case you need to fix configurations and restart the service use:\n\n  ```sudo systemctl daemon-reload\n  sudo systemctl restart pycolytics\n  ```\n\nMost online guides also recommend setting up fastapi behind an nginx reverse\nproxy, in case somebody tries to DDOS your server. I've never been successful\nenough for this to happen, so I'll leave it to you to figure out the details.\n\n## How We Got Here\n\nWhen I was looking at Attolytics, I was too lazy to set up a rust compile\nenvironment and install postgresql for something so simple, so I spent two days\nwriting Pycolytics instead. To help you avoid my mistake, I made it so you can\njust clone it and move on with your life.\n\nTrue to its name, Pycolytics is probably 10⁶ times slower than Attolytics, but\nwho cares if it still serves my entire userbase from a raspberry-pi. It does\nasyncio and fancy multi-worker stuff to try and compensate.\n\nOpen an issue if you wish to contribute, or buy me a coffee if you find my work useful.\n\n\u003ca href='https://ko-fi.com/E1E712JJXK' target='_blank'\u003e\u003cimg height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /\u003e\u003c/a\u003e\n\n## Planned Features\n\n- HTTPS communication for you security nerds out there\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkerekesdavid%2Fpycolytics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkerekesdavid%2Fpycolytics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkerekesdavid%2Fpycolytics/lists"}