{"id":35017261,"url":"https://github.com/navapbc/its-log","last_synced_at":"2026-05-22T03:31:09.148Z","repository":{"id":329177521,"uuid":"1116353593","full_name":"navapbc/its-log","owner":"navapbc","description":"A fast, compliance-aware event logger. It's better than bad, it's good!","archived":false,"fork":false,"pushed_at":"2026-05-05T19:46:12.000Z","size":23355,"stargazers_count":8,"open_issues_count":8,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-05-05T21:28:50.571Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/navapbc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-14T17:37:24.000Z","updated_at":"2026-05-05T19:46:17.000Z","dependencies_parsed_at":"2026-01-19T04:01:05.047Z","dependency_job_id":null,"html_url":"https://github.com/navapbc/its-log","commit_stats":null,"previous_names":["navapbc/its-log"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/navapbc/its-log","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Fits-log","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Fits-log/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Fits-log/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Fits-log/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/navapbc","download_url":"https://codeload.github.com/navapbc/its-log/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Fits-log/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33327364,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-21T12:23:38.849Z","status":"online","status_checked_at":"2026-05-22T02:00:06.671Z","response_time":265,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-27T05:26:17.687Z","updated_at":"2026-05-22T03:31:09.141Z","avatar_url":"https://github.com/navapbc.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# its-log\n\n*It's better than bad, it's good!*\n\n`its-log` is a lightweight event logger and ETL environment for resource-constrained, compliance-burdened environments.\n\nOn a Mac M4, `its-log` can sustain logging 30K events/second to a local SQLite database.\n\n\n## testing\n\n`its-log` can test itself. It will run sequential correctness tests followed by parallel stress tests, exercising itself end-to-end and validating database contention correctness.\n\nIn one terminal, run the storage stack (a [ministack](https://ministack.org/) S3):\n\n```\nmake storage\n```\n\nand in another, run the E2E:\n\n```\nmake test\n```\n\n## debugging its-log\n\nA VS Code debugging bundle is prepared and part of this repository, with all of the envrionment variables required to run. Press `F5` to launch `its-log` under the debugger, and `COMMAND-F5` to halt the debugger (on a Mac).\n\n## testing with Postman\n\nA Postman bundle is included under the `docs` directory. It includes a set of scripts to test results, and can serve as a light E2E test. \n\n## running its-log\n\n`its-log` can be compiled for multiple architectures.\n\nTo run on a ARM processors (Mac):\n\n```\nmake itslog-arm\n```\n\nTo run on Intel:\n\n```\nmake itslog-amd\n```\n\nIn either case, you will need to make sure a suite of environment variables are present to run `its-log` natively.\n\n## model\n\n`its-log` is opinionated, and espouses ways of thinking about logging and subsequent ETL/analysis pipelines.\n\n### runtime events\n\nApplications generate events which are logged. Any given event:\n\n1. **could** belong to a *cluster* of logs\n2. **must** be *categorically tagged*\n3. **could** have an associated *unique* value\n\nSome events are logged in real-time, and others might be logged via periodic (hourly/daily/weekly) analysis of internal databases.\n\n### ETL\n\nAn explicit goal of `its-log` is to move analysis \"left\" in the pipeline, and reduce the number of moving parts (e.g. GitHub Actions, AWS Lambdas, etc.) involved in transforming raw events into end-user facing information and visualization. To this end, `its-log` encapsulates a light, SQLite-based ETL pipeline infrastructure, allowing data to be processed within the database and, in doing so:\n\n1. Simplify processing at the end of the pipeline (e.g. in Metabase, Superset, QuickSuite, or similar.)\n2. Keeping analytical code (and its results) with the data for archived compliance\n\n\n## the API\n\nThe logging endpoints are rooted at `/v1`. So, `/log/create` should be read as `/v1/log/create`.\n\n| HTTP | Endpoint         | Permissions | Desc                                                                                             |\n| ---- | ---------------- | ----------- | ------------------------------------------------------------------------------------------------ |\n| POST | /log/create      | logging     | Log an event with cluster (optional), tags, value (optional)                                     |\n| POST | /log/create/date | admin       | Log an event per above plus a *date*, for creating test scenarios by logging events in the past. |\n\n\nThe ETL and analysis endpoints are\n\n| HTTP | Endpoint                  | Permissions | Desc                                                                  |\n| ---- | ------------------------- | ----------- | --------------------------------------------------------------------- |\n| POST | /etl/create               | admin       | Create an ETL action                                                  |\n| POST | /etl/run/:date/:name      | admin       | Run an ETL action. Takes optional params in the JSON body.            |\n| POST | /sequence/create          | admin       | Create an ETL sequence                                                |\n| POST | /sequence/run/:date/:name | admin       | Run a sequence for a given date. Takes optional params like /etl/run. |\n\nFor working with the summary table directly\n\n| HTTP | Endpoint      | Permissions | Desc                                                                      |\n| ---- | ------------- | ----------- | ------------------------------------------------------------------------- |\n| POST | /summary/read | admin       | Read from the summary table; used for testing the results of ETL actions. |\n\n\nAdministrative endpoints include\n\n\n| HTTP | Endpoint | Permissions | Desc                             |\n| ---- | -------- | ----------- | -------------------------------- |\n| GET  | /health  | any         | A standard healthcheck endpoint  |\n| GET  | /status  | admin       | Get server stats (RAM, GC, etc.) |\n\n\n## extending its-log\n\nSee [DEVELOPMENT.md](docs/DEVELOMENT.md) in `docs`. \n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=navapbc/its-log\u0026type=date\u0026legend=top-left)](https://www.star-history.com/#navapbc/its-log\u0026type=date\u0026legend=top-left)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavapbc%2Fits-log","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnavapbc%2Fits-log","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavapbc%2Fits-log/lists"}