{"id":15150404,"url":"https://github.com/dfandrich/testclutch-curl-web","last_synced_at":"2026-01-20T04:31:54.348Z","repository":{"id":231822946,"uuid":"782811217","full_name":"dfandrich/testclutch-curl-web","owner":"dfandrich","description":"Files needed to deploy an instance of Test Clutch customized for the curl project to a PaaS service","archived":false,"fork":false,"pushed_at":"2025-03-31T17:30:50.000Z","size":142,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-31T18:25:50.885Z","etag":null,"topics":["curl","deployment","testing","website"],"latest_commit_sha":null,"homepage":"https://testclutch.curl.se/","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dfandrich.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-06T05:18:26.000Z","updated_at":"2025-03-31T17:30:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"e13a1fa4-f788-4216-bd07-440fbfa86eb9","html_url":"https://github.com/dfandrich/testclutch-curl-web","commit_stats":{"total_commits":97,"total_committers":1,"mean_commits":97.0,"dds":0.0,"last_synced_commit":"b7d0278cb6dd405742a24fe93eb2fe2ec85d9898"},"previous_names":["dfandrich/testclutch-curl-web"],"tags_count":54,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfandrich%2Ftestclutch-curl-web","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfandrich%2Ftestclutch-curl-web/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfandrich%2Ftestclutch-curl-web/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfandrich%2Ftestclutch-curl-web/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dfandrich","download_url":"https://codeload.github.com/dfandrich/testclutch-curl-web/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247615481,"owners_count":20967182,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["curl","deployment","testing","website"],"created_at":"2024-09-26T14:03:49.077Z","updated_at":"2026-01-20T04:31:54.342Z","avatar_url":"https://github.com/dfandrich.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Test Clutch deployment for curl\n\nThis repository contains the files needed to deploy an instance of [Test\nClutch](https://github.com/dfandrich/testclutch/) customized for the curl\nproject to a host running Virtuozzo Application Platform cloud service.\n\n# Installation\n\nThe latest source can be obtained from\nhttps://github.com/dfandrich/testclutch-curl-web/\n\nThe build scripts require Python 3 (ver. 3.9 or higher) with the `pip` package,\nas well as `git` and `xz`.\n\nRun `./builddeployment` to create `testclutch_deploy.tar.xz`, a deployment\nbundle that contains the static web content, Python source code as well as all\nthe necessary Test Clutch source code and dependencies. The script assumes the\ntestclutch source is in the directory `../testclutch/` and the `master` branch\nis the version to deploy. To change these, run the script with two arguments of\nbranch or tag and git repository location (local or remote), respectively.\n\nThe exact Python dependency versions to use are found in the file\nrequirements-testclutch.txt.\n\nThe script writes the files `testclutch_deploy-commit.txt`,\n`testclutch_deploy-requirements.txt` and `testclutch_deploy-tag.txt` which\ncontain the commit ID of the testclutch source, the Python modules and their\nversions used, and the latest tag in the git repository, respectively.\n\n# Environment Setup\n\nThe deployment bundle is designed to be deployed on Virtuozzo Application\nPlatform (formerly Jelastic), a container orchestration service. Create a new\nenvironment from scratch by following these steps:\n\n0. Choose *NEW ENVIRONMENT*\n0. At the top of the environment topology diagram, click *SSL* and *enable wildcard SSL*\n0. in the *Application* block, choose *Apache Python*\n    - tested version: Apache 2.4.66\n    - tested version: Python 3.13.11\n    - tested version: Almalinux 9\n0. Under *Application Servers*:\n    - 1 reserved cloudlet, 1 scaling limit, 1 horizontal scaling\n    - horizontal scaling: stateless\n    - disk limit 5 GB\n    - access via SLB: ON\n    - Public IPv4: OFF\n    - Environment Name: \u0026lt;unique hostname\u003e\n0. In the *Storage Containers* block, choose *Shared Storage*\n    - tested version: NFS 2.0-10.5-almalinux-9\n    - 1 reserved cloudlet, 1 scaling limit, 1 horizontal scaling\n    - horizontal scaling: stateless\n    - auto-clustering: OFF\n    - access via SLB: OFF\n    - disk limit 60 GB\n0. In *Application Servers*, *Settings→Volumes*\n    - Choose Add, Data Container, Shared Storage node, click checkbox for\n      `/data /data`, then *Add*, *Apply*\n0. In *\u003cyour environment\u003e*, *Settings→Load Alerts*\n    - Change the *auto_alert_cpu* threshold to 99% and duration to 60 minutes\n      (the batch jobs are expected to run with 100% CPU for some time)\n    - Change the *auto_alert_disk* threshold to 95%.\n\n# Deployment\n\nIn the *Deployment Manager*, upload `testclutch_deploy.tar.xz` and choose the\n*Deploy to...* option to deploy to the application server just created. Set the\n*Post deploy* hook to:\n```sh\n#!/bin/bash\nexec /var/www/webroot/ROOT/application/bin/post-deploy\n```\n\n# Redeployment\n\nTo update Test Clutch, just upload a new bundle and deploy it as before. The\n*Post deploy* hook should already be set as above.\n\nWhen using the *Redeploy* option to update the application servers, ensure\nthat *Keep volume data* is ON. If redeployment happens with that off, the\napplication code will be gone and there will be a site outage until you deploy\na bundle again. Once the redeployment completes, log in via ssh and manually\nrun the post deploy script */var/www/webroot/ROOT/application/bin/post-deploy*.\nThis is necessary because the crontab file is not preserved over the\nredeployment. Instead of manually running the post deploy script like this, you\ncan instead deploy a new (or old) application version (under the *Deployment*\nsection above) which will run the post deploy script for you.\n\nAlso, remember to download any needed journal logs first as they will be lost\nduring redeployment.\n\n# Tokens\n\nGitHub requires an access token for Test Clutch to download GitHub Actions\nlogs and to comment on pull requests. Create one by going to\nhttps://github.com/settings/tokens and choosing a Classic token with the\n`public_repo` scope enabled. This can be done by any GitHub user, including the\n`testclutch` robot user.\n\nWhen running as a user who is an owner of the curl repository, a fine-grained\ntoken is preferred.  Create a token with these characteristics:\n\n  * Resource owner (select curl)\n  * Only select repositories (selecting the source repository or repositories)\n  * \"Metadata\" repository permissions (read)\n  * \"Pull requests\" repository permissions (read and write)\n\nEither way, copy the token contents from the web browser and store it in a file\ncalled `ghatoken` in a protected location on your local machine. Use the\n*Application Servers - Config* menu in Virtuozzo to upload that file to the\ncontainer in the location `/data/auth/ghatoken`. Once the file is there, the\nperiodic update job will start downloading GitHub Actions logs on its next run.\n\n# Logs\n\nApplication logs are stored in the journal and may be downloaded using the\n*Application Servers: Log* menu. Navigate to `/journal/X` (where `X` is a\nunique identifier) and download the journal files desired. The files are\nautomatically rotated.\n\nJob run times can be extracted from these journals with a command like:\n```sh\nscripts/collect-process-durations.py system*.journal | sort -n \u003e times.csv\n```\nThe extracted times can be plotted using gnuplot with:\n```sh\nscripts/plot-job-times times.csv\n```\nIf some interim partial log files were downloaded before the completed,\nrotated ones were ready, the partial ones can be deleted with the command:\n```sh\nscripts/delete-subset-logs system*.journal\n```\nto avoid duplicated data points.  The modification times of the log files must\nremain accurate to the times of the last log message or else an incorrect log\nfile could be deleted by this script by mistake (a full one could be deleted\ninstead of the matching partial one). The author's program `automtime`\n(available as part of\n[fileviewinfo](https://github.com/dfandrich/fileviewinfo)) can be used to set\nthe times correctly before the delete script is run, if in doubt. This can\nmost commonly be an issue if a partial log file is downloaded before an older,\nrotated one.\n\n## Author\n\nCopyright (c) 2024-2025 Dan Fandrich \u003cdan@coneharvesters.com\u003e\nLicensed under the MIT license (see the file [LICENSE](LICENSE) for details)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfandrich%2Ftestclutch-curl-web","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdfandrich%2Ftestclutch-curl-web","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfandrich%2Ftestclutch-curl-web/lists"}