{"id":20342326,"url":"https://github.com/spacether/taskrunner_taskui","last_synced_at":"2025-03-04T15:23:35.570Z","repository":{"id":243266884,"uuid":"811954622","full_name":"spacether/taskrunner_taskui","owner":"spacether","description":"a webserver that lets a user view and modify task definitions","archived":false,"fork":false,"pushed_at":"2024-06-08T01:33:45.000Z","size":131,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-14T19:11:45.566Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spacether.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-07T16:31:05.000Z","updated_at":"2024-06-08T01:40:17.000Z","dependencies_parsed_at":"2024-11-14T21:33:55.032Z","dependency_job_id":"64cf858a-1af8-4efd-9359-ab52f8fd1f70","html_url":"https://github.com/spacether/taskrunner_taskui","commit_stats":null,"previous_names":["spacether/taskrunner_taskui"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacether%2Ftaskrunner_taskui","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacether%2Ftaskrunner_taskui/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacether%2Ftaskrunner_taskui/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacether%2Ftaskrunner_taskui/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spacether","download_url":"https://codeload.github.com/spacether/taskrunner_taskui/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241869553,"owners_count":20034085,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T21:33:07.531Z","updated_at":"2025-03-04T15:23:35.540Z","avatar_url":"https://github.com/spacether.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# taskui\nA server that allows one to oversee tasks.\nSupported task operations are:\n- view all\n- add one\n- delete one\n\n## Future Improvements to Make\n- protect the taskui webpage with auth\n- add unit tests\n- turn println into logger.info\n- increase the max number of connections to rabbitmq from 20 to a higher number\n  - if that could not be done then: create more rabbitmq channels + hash the message id and modulo it to determine which queue to publish to\n  - or switch to kafka\n- use coroutines/threading in the runner so that each runner has 4 workers on it\n- records running task status in redis\n- have the server send the running task status to taskui webpack via polling or websockets + display that status on the page\n\n## Load Test Results\nI ran the below load test\nAll tasks are 5 second duration tasks\n\n| Time | Tasks/Min | Workers |\n|------|-----------|---------|\n| 0:42 | 60        | 5       |\n| 0:48 | 120       | 5       |\n| 0:50 | 120       | 10      |\n| 0:50 | 120       | 20      |\n| 1:00 | 120       | 19      |\n\n![alt text](load_test_images/5.51_pm.png \"Title\")\n![alt text](load_test_images/6.21_pm.png \"Title\")\n\nWith the initial 5 workers they consumed the 60 tasks per minute of work\n120 tasks per minute exceeded the capability of 5 workers\n10 workers consumed the work of 120 tasks per minute but could not do the remaining queue work which hab built up\n20  workers were able to do all of the queue work but crashed the task publisher because the number of max connections was exceeded\nThe lower plot shows 19 workers consuming 120 tasks/min and there is still extra capacity\n\nAfter the load test I set the number of workers to 5 for taskrunner\n\n## Usage\n- remote: https://taskui-741f289ec720.herokuapp.com/tasks\n- local: http://0.0.0.0:8080/tasks\n\nFor local usage:\n- postgresql must be running and must contain the below table\n- rabbitmq must be running with a queue named tasks\n\nLocal run:\n```\n./gradlew run\n```\n\n\n## Relational Db, Task Definition\nLocal postgresql creation\n```\npsql postgres\n# Note: user is only for local db\nCREATE USER taskrunner_readwriter WITH ENCRYPTED PASSWORD 'xfdz8t-mds-V';\nCREATE DATABASE taskrunner_db WITH OWNER taskrunner_readwriter;\n\\c taskrunner_db taskrunner_readwriter\ncreate table tasks (\n  name varchar(255) NOT NULL PRIMARY KEY,\n  filename varchar(255) NOT NULL,\n  minute SMALLINT,\n  hour SMALLINT\n);\n\nINSERT INTO tasks (name, filename, minute, hour) VALUES ('everyMin5sTask', 'FiveSecondTask.kt', null, null);\nINSERT INTO tasks (name, filename, minute, hour) VALUES ('twoPast10sTask', 'TenSecondTask.kt', 2, null);\n```\n\n## Other Repos\n- [taskpublisher](https://github.com/spacether/taskrunner_taskpublisher)\n- [taskrunner](https://github.com/spacether/taskrunner_taskrunner)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacether%2Ftaskrunner_taskui","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspacether%2Ftaskrunner_taskui","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacether%2Ftaskrunner_taskui/lists"}