{"id":19696172,"url":"https://github.com/juanmanuelramallo/applogger","last_synced_at":"2026-06-25T08:31:08.404Z","repository":{"id":44597309,"uuid":"367737350","full_name":"juanmanuelramallo/applogger","owner":"juanmanuelramallo","description":"Web Analytics from application logs","archived":false,"fork":false,"pushed_at":"2022-02-05T22:11:14.000Z","size":175,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-26T04:22:55.087Z","etag":null,"topics":["heroku-ready","logs","syslog","web-analytics"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/juanmanuelramallo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-15T22:01:08.000Z","updated_at":"2022-10-14T12:59:09.000Z","dependencies_parsed_at":"2022-09-17T07:21:58.421Z","dependency_job_id":null,"html_url":"https://github.com/juanmanuelramallo/applogger","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/juanmanuelramallo/applogger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juanmanuelramallo%2Fapplogger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juanmanuelramallo%2Fapplogger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juanmanuelramallo%2Fapplogger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juanmanuelramallo%2Fapplogger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/juanmanuelramallo","download_url":"https://codeload.github.com/juanmanuelramallo/applogger/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juanmanuelramallo%2Fapplogger/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34767542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-25T02:00:05.521Z","response_time":101,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["heroku-ready","logs","syslog","web-analytics"],"created_at":"2024-11-11T19:34:10.881Z","updated_at":"2026-06-25T08:31:08.385Z","avatar_url":"https://github.com/juanmanuelramallo.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AppLogger\n\nMinimal Web Analytics provider.\n\nIn other words, syslog drainer (works with Heroku) that stores logs in a PostgreSQL DB, transforms them into aggregatable information and displays them in a friendly manner.\n\nThe goal is to build something simple yet useful.\n\n# How it works\n\n## Logging data\nIn your application, log the following line while processing your request:\n\nLOG_LINE_KEY=JSON_ENCODED_DATA\n\nLOG_LINE_KEY is a uniquely identifiable string that will be used to pick or ignore log lines from being processed. Configurable via the LOG_LINE_KEY env var.\n\nJSON_ENCODED_DATA is a JSON string that contains the following keys:\n\n|Key|Description|Example|Required|\n|---|-----------|-------|--------|\n|path|Requested path|/index|yes|\n|referrer|Value of the referrer header|https://myblog.com/|no|\n|query_string|Value of the query params|utm_source=Facebook\u0026igshid=1234ABCD|no|\n|http_method|HTTP Verb of the request|GET|yes|\n|format|MIME type of the request|text/html|yes|\n|ip|Remote IP|182.22.98.233|yes|\n|user_agent|User agent|Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36|yes|\n\n## Extraction\nThis app provides an HTTP POST endpoint to feed logs into the logger (POST /logs).\n\nHTTP header `Logplex-Msg-Count` must be set with the amount of messages sent in the body of the request, see [Heroku's docs](https://devcenter.heroku.com/articles/log-drains#https-drains).\n\nLogs are parsed  according to [RFC5424](https://datatracker.ietf.org/doc/html/rfc5424).\n\nOnce parsed they are stored in the logs table, and enqueued to be transformed in the background.\n\n## Transformers\n\nTransformation process is run in background jobs, via Sidekiq.\n\nCurrent transformations are:\n  - IP to country code\n  - ...\n\nLogs that do not contain transformable information are removed.\n\n## Visualization\n\nOnce the logs are transformed, they are displayed in the Dashboard page.\n\nIt uses chart.js for the line chart, and GeoChart from Google Charts for the country chart.\n\nBasic filters are in place, although they need to be improved.\n\n## Authorization\nHTTP Basic Auth is used to protect all endpoints.\n\nSet the env vars HTTP_BASIC_AUTH_NAME and HTTP_BASIC_AUTH_PASSWORD with your desired credentials.\n\n## Configuration\n\n#### Keep logs\nSet the `CONFIG_KEEP_LOGS` to `true` in order to make the app maintain all logs.\n\n# Contributing\n\n## Development\n1. Install ruby (2.7.x preferred), `gem install bundler` and `bundle install`.\n2. Create the database `bundle exec rails db:create db:migrate`.\n3. Populate some data with `bundle exec rails runner 'FactoryBot.create_list(:entry, 1000)'`.\n4. Copy the .env.dist file and update with your data `cp .env.dist .env`\n5. Download the GeoLite2 Country database from your MaxMind's account and set the following env vars:\n    - GEOIP_GEOLITE2_PATH the path to your downloaded MaxMind database\n    - GEOIP_GEOLITE2_COUNTRY_FILENAME the file name of the downloaded MaxMind database\n6. Run the server `bundle exec rails s`\n\n## Testing\n1. Follow first two steps from the Development section\n2. Copy the .env.test file `cp .env.test .env.test.local`\n3. Set the GeoLite2 env vars in the .env.test.local file the same way as in the previous Development section\n4. Run the specs `bundle exec rspec`\n\n## Deployment to Heroku\n1. Add the buildpack https://github.com/danstiner/heroku-buildpack-geoip-geolite2 to download MaxMind's database at deploy time.\n2. Get a MaxMind license key and set the env var `MAXMIND_LICENSE_KEY` with the license key.\n3. Set the env vars from .env.dist in your Heroku app — no need to set the GeoLite2 env vars as they are automatically set by the buildpack previously installed.\n4. The rest is a normal heroky deploy `git push heroku master`\n\nOnce the app is deployed, add it as a log drainer for the web app you want. In Heroku it's done with:\n\n```\nheroku drains:add https://HTTP_BASIC_AUTH_NAME:HTTP_BASIC_AUTH_PASSWORD@APPLOGGER_DOMAIN/logs -a NAME_OF_YOUR_APP\n```\n\nMake sure to replace HTTP_BASIC_AUTH_NAME, HTTP_BASIC_AUTH_PASSWORD and APPLOGGER_DOMAIN with the correct values.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuanmanuelramallo%2Fapplogger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuanmanuelramallo%2Fapplogger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuanmanuelramallo%2Fapplogger/lists"}