{"id":16930090,"url":"https://github.com/digorithm/go-finch","last_synced_at":"2026-01-11T21:04:09.392Z","repository":{"id":40958160,"uuid":"93667181","full_name":"digorithm/go-finch","owner":"digorithm","description":"Enabling configuration self-adaptation using machine learning. Highly experimental work from this thesis: https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0379346","archived":false,"fork":false,"pushed_at":"2023-07-06T21:21:18.000Z","size":21707,"stargazers_count":1,"open_issues_count":3,"forks_count":2,"subscribers_count":2,"default_branch":"research","last_synced_at":"2025-08-23T09:07:07.206Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/digorithm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-07T18:33:27.000Z","updated_at":"2023-07-17T18:56:45.000Z","dependencies_parsed_at":"2024-10-13T20:41:02.244Z","dependency_job_id":"0e613377-6e09-4e56-81f4-e9696491a618","html_url":"https://github.com/digorithm/go-finch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/digorithm/go-finch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digorithm%2Fgo-finch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digorithm%2Fgo-finch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digorithm%2Fgo-finch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digorithm%2Fgo-finch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/digorithm","download_url":"https://codeload.github.com/digorithm/go-finch/tar.gz/refs/heads/research","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/digorithm%2Fgo-finch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28323601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T18:42:50.174Z","status":"ssl_error","status_checked_at":"2026-01-11T18:39:13.842Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T20:40:52.423Z","updated_at":"2026-01-11T21:04:09.373Z","avatar_url":"https://github.com/digorithm.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Disclaimer\n\nThis is a highly experimental work. Shouldn't be used anywhere near a production environment. The thesis describing in detail what's going on here is hosted here: https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0379346\n\nThe core of this theoretical/experimental work lives in the `finch/` directory. The other directories in this repo is virtually a simple CRUD application that makes use of the ideas in `finch/`.\n\n**The code in this repo, and specially in `finch/` lives up to the meme about academics writing code that's hard to maintain.** Goes without saying that it doesn't reflect how I would build things that should be production-ready. Time was short, pressure was high, pay was low, and papers needed to be published at the time of the creation of `finch/` and this hasn't been updated in years. That said, proceed at your own caution. \n\n\n# Research and setup\n\nDue to advancements in distributed systems and the increasing industrial demands placed on these systems, distributed systems are comprised of multiple complex\ncomponents (e.g databases and their replication infrastructure, caching components, proxies, and load balancers) each of which have their own complex configuration parameters that enable them to be tuned for given runtime requirements.\n\nSoftware Engineers must manually tinker with many of these configuration parameters that change the behaviour and/or structure of the system in order to achieve their system requirements. In many cases, static configuration settings might not meet certain demands in a given context and ad hoc modifications of these configuration parameters can trigger unexpected behaviours, which can have negative effects on the quality of the overall system.\n\nIn this work, I show the design and analysis of Finch; a tool that injects a machine learning based MAPE-K feedback loop to existing systems to automate how these configuration parameters are set. Finch configures and optimizes the system to meet service-level agreements in uncertain workloads and usage patterns.\nRather than changing the core infrastructure of a system to fit the feedback loop, Finch asks the user to perform a small set of actions: instrumenting the code and configuration parameters, defining service-level objectives and agreements, and enabling programmatic changes to these configurations. As a result, Finch learns how to dynamically configure the system at runtime to self-adapt to its dynamic workloads.\n\nI show how Finch can replace the trial-and-error engineering effort that otherwise would be spent manually optimizing a system’s wide array of configuration\nparameters with an automated self-adaptive system.\n\n## Running app and pg on docker compose\n\n- Run `docker-compose up --build`\n\n**Important:** If running for the first time, make sure init.sql has been executed on the container. Configure grafana /datasources pointing a datasource to prometheus:9090 and import the dash.json to have a complete dashboard\n\n**Important**: If something goes wrong with the mapping of host to pgdata, try removing the db image with `docker rm db`.\n\n## Monitoring\n\nInstrumentation is being done with Prometheus + Granafa. After running docker compose up you can go to `localhost:3000` and login with admin:pass, you might have to setup the data source to Prometheus by passing the URI `http://prometheus:9090`.\n\nJust in case here's a very good tutorial on setting all this up: https://finestructure.co/blog/2016/5/16/monitoring-with-prometheus-grafana-docker-part-1\n\n## Big list of interesting metrics to monitor\n\nRemember that for latency and other similar metrics, mean means nothing, focus on 99th percentile.\n\n- System level:\n  - CPU:\n    - User\n    - IOwait (A high iowait means that you are disk or network bound, This query produces a familiar value which is iowait as a percentage of CPU time averaged across all cores in the system and once graphed provides an insight into IO performance.): `avg(irate(node_cpu{job=\"node-exporter\",mode=\"iowait\"}[1m])) * 100`\n    - Idle \n    - Note: Pass these functions to irate() to get the instant rate to see more details\n  - Memory:\n    - Mem useage in percentage: `((node_memory_MemTotal) - ((node_memory_MemFree+node_memory_Buffers+node_memory_Cached))) / node_memory_MemTotal * 100`\n  - Disk\n    - Still not sure about this\n\n---\n\n## Installation\n\n1. Install PostgreSQL 9.4.x\n\n2. Install Go 1.4.x, git, setup `$GOPATH`, and `PATH=$PATH:$GOPATH/bin`\n\n3. Create PostgreSQL database.\n    ```\n    cd $GOPATH/src/github.com/digorithm/meal_planner\n    go get github.com/rnubel/pgmgr\n    pgmgr db create\n    ```\n\n4. Run the PostgreSQL migration.\n    ```\n    pgmgr db migrate\n    ```\n\n5. Run the server\n    ```\n    cd $GOPATH/src/github.com/digorithm/meal_planner\n    go run main.go\n    ```\n\n\n## Environment Variables for Configuration\n\n* **HTTP_ADDR:** The host and port. Default: `\":8888\"`\n\n* **HTTP_CERT_FILE:** Path to cert file. Default: `\"\"`\n\n* **HTTP_KEY_FILE:** Path to key file. Default: `\"\"`\n\n* **HTTP_DRAIN_INTERVAL:** How long application will wait to drain old requests before restarting. Default: `\"1s\"`\n\n* **DSN:** RDBMS database path. Default: `postgres://$(whoami)@localhost:5432/meal_planner?sslmode=disable`\n\n* **COOKIE_SECRET:** Cookie secret for session. Default: Auto generated.\n\n\n## Running Migrations\n\nMigration is handled by a separate project: [github.com/rnubel/pgmgr](https://github.com/rnubel/pgmgr).\n\nHere's a quick tutorial on how to use it. For more details, read the tutorial [here](https://github.com/rnubel/pgmgr#usage).\n```\n# Installing the library\ngo get github.com/rnubel/pgmgr\n\n# Create a new migration file\npgmgr migration {filename}\n\n# Migrate all the way up\npgmgr db migrate\n\n# Reset to the latest dump\npgmgr db drop\npgmgr db create\npgmgr db load\n\n# Roll back the most recently applied migration, then run it again.\npgmgr db rollback\npgmgr db migrate\n\n# Show the latest migration version\npgmgr db version\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigorithm%2Fgo-finch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigorithm%2Fgo-finch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigorithm%2Fgo-finch/lists"}