{"id":20020684,"url":"https://github.com/netcentric/apply-server","last_synced_at":"2025-03-02T03:32:52.857Z","repository":{"id":48299362,"uuid":"169572324","full_name":"Netcentric/apply-server","owner":"Netcentric","description":"Deploy to servers/run scripts on servers via http","archived":false,"fork":false,"pushed_at":"2022-12-06T00:42:18.000Z","size":5109,"stargazers_count":3,"open_issues_count":6,"forks_count":0,"subscribers_count":33,"default_branch":"develop","last_synced_at":"2025-01-12T16:12:40.815Z","etag":null,"topics":["deployment","http","http-server"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Netcentric.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-07T13:00:20.000Z","updated_at":"2024-12-09T19:55:00.000Z","dependencies_parsed_at":"2023-01-24T08:00:11.028Z","dependency_job_id":null,"html_url":"https://github.com/Netcentric/apply-server","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcentric%2Fapply-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcentric%2Fapply-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcentric%2Fapply-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netcentric%2Fapply-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Netcentric","download_url":"https://codeload.github.com/Netcentric/apply-server/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241454527,"owners_count":19965405,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deployment","http","http-server"],"created_at":"2024-11-13T08:33:39.173Z","updated_at":"2025-03-02T03:32:52.836Z","avatar_url":"https://github.com/Netcentric.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Maven Central](https://maven-badges.herokuapp.com/maven-central/biz.netcentric.ops.applyserver/apply-server/badge.svg)](https://maven-badges.herokuapp.com/maven-central/biz.netcentric.ops.applyserver/apply-server)\n[![License](https://img.shields.io/badge/License-EPL%201.0-red.svg)](https://opensource.org/licenses/EPL-1.0)\n[![Build Status](https://github.com/Netcentric/apply-server/workflows/Build/badge.svg)](https://github.com/Netcentric/apply-server/actions)\n[![SonarCloud Status](https://sonarcloud.io/api/project_badges/measure?project=Netcentric_apply-server\u0026metric=alert_status)](https://sonarcloud.io/dashboard?id=Netcentric_apply-server)\n[![SonarCloud Coverage](https://sonarcloud.io/api/project_badges/measure?project=Netcentric_apply-server\u0026metric=coverage)](https://sonarcloud.io/component_measures/metric/coverage/list?id=Netcentric_apply-server)\n\nApply Server\n================================================\n\n# Overview\n\nThe idea behind \"Apply Server\" is to control a remote computer via http (instead of ssh). Use cases are \n\n* Deploy configuration files and restart services (e.g. httpd)\n* Run maintenance scripts \n\nApply Server was inspired by the config via http approach of nginx unit [1] and allows to turn apache, solr, or any other server into a http-configurable by just starting another process that acts as agent on target system. **Using the Apply Server, it is easy to deploy well-versioned configuration artifacts to target systems via a CI/CD pipeline** (like e.g. Jenkins). Obviously CI/CD tools also support ssh, but ssh access can be harder to obtain than opening an http port, also ssh often does not make it easy to transfer/unpack configuration tars (often a change of user is required, permissions can be a challenge). In contrary, an Apply Server can just be started with the same user as the service is running (hence all correct permissions are automatically in placed) and the start parameters ensure its actions are limited to what the Apply Server was started with.\n\n## Why not use Puppet/Chef/Ansible?\n\nIn general all these tools are very useful (e.g. even to install the apply server agent, see [Setup Apply Server via puppet](#setup-apply-server-via-puppet) below), however their strength is more in setting up a base configuration (OS level, files, directories, permissions, services etc.). For deployments of constantly changing code/configuration they lack support for a workflow where any people (including non-technical people) can make code/configuration travel across environments - here CI/CD servers like Jenkins have their strength. Many tools allow for deployment via http (e.g. Tomcat, Adobe AEM, etc.), Apply Server makes it easy to deploy Apache/Varnish/etc. configuration in the same way as software.\n\n## Why another http server? \n\nThere are many options for starting an http server already [2], but none that focus on\n\n* Running scripts on target system (and make the result history visible)\n* Upload/download files to/from a target directory in zip/tgz format (zipping/unzipping on the fly)\n* Have minimal dependencies (Apply Server is a lightweight fat jar and has only a JRE as dependency)\n\n\n[1] https://www.nginx.com/blog/nginx-unit-1-0-released/\nhttps://www.infoq.com/news/2018/05/nginx-unit-dynamic-web-server\n\n[2] https://gist.github.com/willurd/5720255\n\n# Security\n\nThere are two options to lock down access directly in apply server:\n\n* IP restriction: Only POST requests from given IP/IP range are accepted\n* API Key Secret: Only requests that carry the correct API key as header are accepted\n\nGenerally security restrictions are given as start parameters of the server. Additionally if needed, also the firewall of the production systems can be configured to allow only requests from certain IPs.\n\n# Size and dependencies\n\nApply server is very lightweight: It is just ~130 KB of size while all dependencies are included in the jar directly (e.g. commons-cli) and only the actual required classes are kept in the file (proguard-maven-plugin is used for shrinking). \n\nApply server is a simple jar that can be run via `java -jar` on JREs \u003e 1.8 and has no other dependencies.\n\n# Usage\n\n## Start Server on target system\n\n```\n$ java -jar apply-server-1.6.2.jar\nusage: apply-server\n -c,--command \u003carg\u003e                   allows to map URL paths to certain\n                                      scripts: -c myscript=myscript.sh\n                                      will run myscript.sh in destination\n                                      upon POST requests to /myscript.\n                                      Multiple -c options can be provided,\n                                      the script provided with -s is the\n                                      default if no command matches\n -d,--destination \u003carg\u003e               the target destination dir. This is\n                                      where payload is extracted to and\n                                      relative script paths are evaluated\n                                      against.\n -dl,--enable-download                enables download of the current\n                                      files at destination via URL\n                                      /download.tar.gz\n -du,--disable-upload                 disable upload (only allow script\n                                      execution and potentially download\n                                      if -ed is given)\n -ed,--exclude-from-download \u003carg\u003e    Regex for files to be excluded from\n                                      download\n -ef,--exclude-from-filtering \u003carg\u003e   to supply a regex of files to be\n                                      explicitly excluded from filtering\n -ip,--ip-range \u003carg\u003e                 when supplied, only upload/script\n                                      execution requests from the given\n                                      IP/IP range are accepted (can be\n                                      regular IP like '20.30.40.50' or\n                                      CIDR range like '20.30.40.50/24').\n                                      Useful to restrict clients that can\n                                      make changes to the system. Has no\n                                      effect for GET requests.\n -k,--api-key \u003carg\u003e                   when supplied, the given api key has\n                                      to be sent with every request as\n                                      header 'apikey'\n -nf,--no-filtering                   by default the incoming files are\n                                      filtered using the OS env and\n                                      '_apply.sh' - using this option will\n                                      disable that.\n -o,--optional-payload                allows to not send a payload to be\n                                      filtered/extracted with the request\n                                      but just to run the scripped as\n                                      mapped\n -p,--port \u003carg\u003e                      the port the server is listening to\n -P,--properties \u003carg\u003e                the properties file name to apply\n                                      the config  - if not given only env\n                                      variables will be taken into account\n -pid,--pid-file \u003carg\u003e                will write the the pid file\n -s,--script \u003carg\u003e                    the script name to run (relative to\n                                      destination dir after extracting) -\n                                      defaults to \"_apply.sh\". If a\n                                      command is matched the command takes\n                                      precedence. The script can already\n                                      exist in the destination or be part\n                                      of the uploaded package.\n```\n\n# Examples\n## Starting Apply Server Agent\n\n### start server to take and unpack zip, filter it and run an apply script afterwards:\n\n```\njava -jar apply-server-1.6.2.jar -p 448 -d /path/to/destination -s applyConfig.sh --api-key MT7HpOKnx5 # secured via api key secret\njava -jar apply-server-1.6.2.jar -p 448 -d /path/to/destination -s applyConfig.sh --ip-range 100.200.300.0/16 # secured via ip range protection\n\n```\n\n### start a server to run scripts remotely\n\n```\njava -jar apply-server-1.6.2.jar -p 448 -d /path/to/scripts --optional-payload -c /script1=myScript1.sh -c /script2=myScript2.sh --api-key MT7HpOKnx5  # secured via api key secret\n# alternatively (or in combination) use --ip-range 100.200.300.0/16 to secure via ip range protection\n\n```\n\n## Client Usage\nPush configurations to it from elsewhere and run a script to use the new files.\n\nThe full API documentation is at \u003chttps://netcentric.github.io/apply-server/index.html\u003e based on the OpenAPI v3 yaml file at [`docs/openapi.yaml`](./docs/openapi.yaml).\n\n\n### Upload and run default script\n\nTo push configurations to the target system and run the script as provided by server start:\n\n```\ncurl -X POST -H \"apikey: MT7HpOKnx5\" --data-binary \"@path/to/my-config-package.tar.gz\" http://myserver:448/package-name.tar.gz\n```\n\nalternatively the `format` parameter can be given to not include the filename in path:\n\n```\ncurl -X POST -H \"apikey: MT7HpOKnx5\" --data-binary \"@path/to/my-config-package.tar.gz\" http://myserver:448?format=tar.gz\n```\n\n### Run scripts only\nNo upload required, often used along with multiple -c parameters\n\nif the server was started with\n\n```\njava -jar apply-server-1.6.2.jar -d /path/to/scripts --optional-payload -c /script1=myScript1.sh -c /script2=myScript2.sh ...\n```\n\nthe following two commands will run myScript1.sh or myScript2.sh respectively:\n\n```\ncurl -X POST -H \"apikey: MT7HpOKnx5\" http://myserver:448/script1\n```\n```\ncurl -X POST -H \"apikey: MT7HpOKnx5\" http://myserver:448/script2\n```\n\n### Using Apply Server with Apache Httpd \n\nA simple script `restartApache.sh` looks as follows:\n\n```\n# fail immedately in case of invalid config\nset -e  \n# test the config\napachectl configtest\n# restart gracefully without downtime (will start server if not yet running)\napachectl graceful\n```\n\nThe following start command can be used to start the apply server:\n\n```\njava -jar apply-server-1.6.2.jar -d /etc/apache2 -p 8449 -s restartApache.sh -nf \n```\n\nTo perform a restart, the curl command\n\n```\n$ curl -X POST http://localhost:8449/?format=zip --data-binary @httpd-config.zip \n```\n\nproduces the following output for the case the configuration is correct:\n\n```\nRequest from /0:0:0:0:0:0:0:1:55878 at 2020-02-20 21:29:54\nProcessing entity /\n--- Placing files:\nFiltering is disabled\nExtracted httpd.conf                                         (not filtered)\n--- Executing apply script: restartSystemApache.sh\nSyntax OK\nRestarted apache.\n--- Apply script 'restartSystemApache.sh' returned 0\nFinished after 166ms\n```\n\nIf the config contains an error it looks as follows:\n\n```\nRequest from /0:0:0:0:0:0:0:1:56008 at 2020-02-20 21:33:17\nProcessing entity /\n--- Placing files:\nFiltering is disabled\nExtracted httpd.conf                                         (not filtered)\n--- Executing apply script: restartSystemApache.sh\nAH00526: Syntax error on line 1 of /usr/local/etc/httpd/httpd.conf:\nInvalid command 'InvalidDirective', perhaps misspelled or defined by a module not included in the server configuration\n--- Apply script 'restartSystemApache.sh' returned 1\nFinished after 34ms\n```\n\nFor the error case the http response code is `500`.\n\n### Listing past executions (via GET)\nJust calling `http://myserver:448` in browser will list all past executions and give links to see the logs of each execution.\n\n# Setup Apply Server via puppet\nThe following snippet will download and start the apply server with the given arguments:\n\n```\n### Parameters ###\n$apply_server_version = '1.6.2'\n$apply_server_target_dir = '/opt/files'\n$apply_server_port = 8089\n$apply_server_additional_arguments = ' -du -dl'\n$apply_server_dir = '/opt/apply-server'\n$nexus_base_url = 'https://repo.int.netcentric.biz/nexus/service/local/repositories'\n \n### Provision the jar ###\n$apply_server_path = \"${apply_server_dir}/apply-server-${apply_server_version}.jar\"\nnotice(\"Ensuring apply server exists: ${apply_server_version}\")\nfile { $apply_server_dir:\n  ensure  =\u003e directory,\n}\n \n$curl_command = \"/usr/bin/curl -u ${netcentric::maven::nexus_user}:${netcentric::maven::nexus_password} -o ${apply_server_path} ${nexus_base_url}/netcentric-releases/content/biz/netcentric/ops/applyserver/apply-server/${apply_server_version}/apply-server-${apply_server_version}.jar\"\nexec { 'retrieve_apply_server_jar':\n  command =\u003e \"/usr/bin/echo Downloading ${apply_server_path} \u0026\u0026 ${curl_command}\",\n  creates =\u003e $apply_server_path,\n  require =\u003e [ File[$apply_server_dir] ],\n}\n \nfile { $apply_server_path:\n  ensure  =\u003e file,\n  require =\u003e [ Exec[\"retrieve_apply_server_jar\"] ],\n}\n \n### Run the service ###\n# using the base command with port allows to run multiple servers\n$base_command = \"java -jar ${apply_server_path} -p ${apply_server_port}\"\nservice { \"apply-server-${apply_server_port}\":\n  ensure  =\u003e running,\n  start   =\u003e \"/usr/bin/nohup ${base_command} -d ${apply_server_target_dir} ${apply_server_additional_arguments} \u003c /dev/null \u003e ${apply_server_dir}/apply-server.log 2\u003e\u00261 \u0026\",\n  pattern =\u003e $base_command,\n  require =\u003e [ File[$apply_server_path] ],\n}  \n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetcentric%2Fapply-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetcentric%2Fapply-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetcentric%2Fapply-server/lists"}