{"id":16049015,"url":"https://github.com/sormy/aws-curl","last_synced_at":"2025-03-17T16:31:44.944Z","repository":{"id":38832882,"uuid":"289815867","full_name":"sormy/aws-curl","owner":"sormy","description":"AWS curl shell wrapper is like vanilla curl but it can automatically sign request using SIGV4 to call AWS services without requirement to have AWS CLI with Python installed","archived":false,"fork":false,"pushed_at":"2023-11-17T18:47:09.000Z","size":84,"stargazers_count":48,"open_issues_count":5,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-10T00:12:06.575Z","etag":null,"topics":["aws","aws-cli","aws-curl","curl","shell-script","sigv4"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sormy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-24T03:12:45.000Z","updated_at":"2024-08-28T01:02:48.000Z","dependencies_parsed_at":"2023-01-17T18:01:13.733Z","dependency_job_id":null,"html_url":"https://github.com/sormy/aws-curl","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sormy%2Faws-curl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sormy%2Faws-curl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sormy%2Faws-curl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sormy%2Faws-curl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sormy","download_url":"https://codeload.github.com/sormy/aws-curl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221696913,"owners_count":16865512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-cli","aws-curl","curl","shell-script","sigv4"],"created_at":"2024-10-09T00:12:19.923Z","updated_at":"2024-10-27T15:17:56.552Z","avatar_url":"https://github.com/sormy.png","language":"Shell","readme":"# aws-curl\n\n`aws-curl` is like `curl` but with automatic SIGV4 signing to simplify calling\nAWS services without requirement to have AWS CLI and Python to be installed.\n\nThe script is pure shell script designed for embedded and lightweight linux\ndistributions, docker images etc.\n\nThis utility also takes much less RAM than aws cli, so nano ec2 instances are\nnot dying with \"Out of memory\" when trying to download from s3 as aws cli does.\n\n## Prerequisites\n\nDependencies:\n\n- openssl (or libressl)\n- curl\n- GNU or similar coreutils (date/od/paste)\n- GNU or similar sed\n\nGNU coreutils/sed can be installed on macOS using Homebrew:\n`brew install coreutils gsed`\n\nOn OpenWRT you might need to install at least `coreutils-od` and\n`coreutils-paste` to get it working.\n\n## Installation\n\n```\ncurl -s https://raw.githubusercontent.com/sormy/aws-curl/master/aws-curl -o /usr/local/bin/aws-curl\nchmod +x /usr/local/bin/aws-curl\n```\n\n## AWS credentials\n\nBefore you can use the application, you need to set environment variables with\nAWS credentials.\n\nSet AWS credentials and region using standard AWS CLI environment variables:\n\n- `AWS_ACCESS_KEY_ID` - AWS public access key\n- `AWS_SECRET_ACCESS_KEY` - AWS private secret key\n- `AWS_SESSION_TOKEN` - temporary token received from STS or from EC2 metadata\n- `AWS_DEFAULT_REGION` - AWS default region, in case if region is not provided\n  in URL or as command line argument `--region`.\n\nYou can read more about AWS CLI environment variables here:\n\u003chttps://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html\u003e\n\nNOTE: Wrapper doesn't read configuration files generated by AWS CLI and reads\ncredentials only from environment variables.\n\n## Usage\n\nApplication has the same syntax of command line arguments as `curl`, it just\nautomatically adds mandatory headers to make AWS to accept the request.\n\nAWS provides documentation for all APIs, sometimes with explicit curl usage\nexamples. Find an API documentation you want to call and follow documentation to\nconstruct valid command line request.\n\n### Example 1: CloudWatchLogs.CreateLogGroup\n\nDocumentation:\n\u003chttps://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogGroup.html\u003e\n\nReference example http request:\n\n```\nPOST / HTTP/1.1\nHost: logs.\u003cregion\u003e.\u003cdomain\u003e\nX-Amz-Date: \u003cDATE\u003e\nAuthorization: AWS4-HMAC-SHA256 Credential=\u003cCredential\u003e, SignedHeaders=content-type;date;host;user-agent;x-amz-date;x-amz-target;x-amzn-requestid, Signature=\u003cSignature\u003e\nUser-Agent: \u003cUserAgentString\u003e\nAccept: application/json\nContent-Type: application/x-amz-json-1.1\nContent-Length: \u003cPayloadSizeBytes\u003e\nConnection: Keep-Alive\nX-Amz-Target: Logs_20140328.CreateLogGroup\n{\n  \"logGroupName\": \"my-log-group\"\n}\n```\n\nThe corresponding `aws-curl` request is:\n\n```sh\naws-curl --request POST \\\n  --header \"Content-Type: application/x-amz-json-1.1\" \\\n  --header \"x-amz-target: Logs_20140328.CreateLogGroup\" \\\n  --header \"Accept: application/json\" \\\n  --data '{\"logGroupName\": \"my-log-group\"}' \\\n  \"https://logs.us-east-1.amazonaws.com\"\n```\n\n### Example 2: STS.GetCallerIdentity\n\nDocumentation:\n\u003chttps://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html\u003e\n\nReference example http request:\n\n```\nPOST / HTTP/1.1\nHost: sts.amazonaws.com\nAccept-Encoding: identity\nContent-Length: 32\nContent-Type: application/x-www-form-urlencoded\nAuthorization: AWS4-HMAC-SHA256 Credential=AKIAI44QH8DHBEXAMPLE/20160126/us-east-1/sts/aws4_request,\n        SignedHeaders=host;user-agent;x-amz-date,\n        Signature=1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef\nX-Amz-Date: 20160126T215751Z\nUser-Agent: aws-cli/1.10.0 Python/2.7.3 Linux/3.13.0-76-generic botocore/1.3.22\n\nAction=GetCallerIdentity\u0026Version=2011-06-15\n```\n\nThe corresponding `aws-curl` request is:\n\n```sh\naws-curl --request POST \\\n  --header \"Content-Type: application/x-www-form-urlencoded\" \\\n  --data \"Action=GetCallerIdentity\" \\\n  --data \"Version=2011-06-15\" \\\n  --region \"us-east-1\" \\\n  \"https://sts.amazonaws.com\"\n```\n\nNOTE: Region can't be detected from URL, so it should be explicitly provided as\nargument or as `AWS_DEFAULT_REGION` env variable.\n\nNOTE: This API has xml response format by default, pass\n`Accept: application/json` header to change response format.\n\n### Example 3: S3\n\nDownload file from s3 to local file:\n\n```sh\naws-curl --request GET \\\n  --output \"my-file.txt\" \\\n  --region \"us-east-1\" \\\n  \"https://s3.amazonaws.com/my-bucket/my-file.txt\"\n```\n\nDownload file from s3 and print to stdout:\n\n```sh\naws-curl --request GET \\\n  --region \"us-east-1\" \\\n  \"https://s3.amazonaws.com/my-bucket/my-file.txt\"\n```\n\nUpload local file to s3.\n\n```sh\naws-curl --request PUT \\\n  --data \"@my-file.txt\" \\\n  --region \"us-east-1\" \\\n  \"https://s3.amazonaws.com/my-bucket/my-file.txt\"\n```\n\nUpload buffer to s3.\n\n```sh\naws-curl --request PUT \\\n  --data \"my content\" \\\n  --region \"us-east-1\" \\\n  \"https://s3.amazonaws.com/my-bucket/my-file.txt\"\n```\n\nDelete file from s3:\n\n```sh\naws-curl --request DELETE \\\n  --region \"us-east-1\" \\\n  \"https://s3.amazonaws.com/sormy/test.txt\"\n```\n\n## Example 4: EC2\n\nCreate AMI image:\n\n```sh\naws-curl --request POST \\\n  --header \"Content-Type: application/x-www-form-urlencoded\" \\\n  --data \"Action=CreateImage\" \\\n  --data \"Version=2016-11-15\" \\\n  --data \"InstanceId=i-something\" \\\n  --data \"Name=My Image Name\" \\\n  --data \"Description=My Image Description\" \\\n  --data \"NoReboot=true\" \\\n  --data \"BlockDeviceMapping.1.DeviceName=/dev/xvdb\" \\\n  --data \"BlockDeviceMapping.1.NoDevice=1\" \\\n  --data \"DryRun=true\" \\\n  --region \"us-east-1\" \\\n  \"https://ec2.amazonaws.com\"\n```\n\nTerminate EC2 instance:\n\n```sh\naws-curl --request POST \\\n  --header \"Content-Type: application/x-www-form-urlencoded\" \\\n  --data \"Action=TerminateInstances\" \\\n  --data \"Version=2016-11-15\" \\\n  --data \"InstanceId.1=i-something\" \\\n  --data \"DryRun=true\" \\\n  --region \"us-east-1\" \\\n  \"https://ec2.amazonaws.com\"\n```\n\n### Command line arguments\n\n`aws-curl` is very thin wrapper around curl. Most of options are passed as it is\nto `curl` with some exceptions (see below).\n\nWrapper recognizes these `curl` arguments:\n\n- `--url` as explicit request URL (but can be provided only once)\n- `https://*.amazonaws.com` somewhere in argument list as request URL\n- Last argument as `url` if not explicitly provided using `--url` and if there\n  is no argument that looks like AWS service endpoint `https://*.amazonaws.com`\n- `-X` | `--request` as request METHOD (`GET` is default)\n- `-H` | `--header` as request header\n- `-d` | `--data` as request body (but passed to curl as `--data-binary`)\n- `-V` | `--version` - shows version of `aws-curl` and `curl`\n- `-h` | `--help` - shows help for `aws-curl` and `curl`\n- `-v` | `--verbose` - dumps debug details (in addition to `curl` debug details)\n\nWrapper recognizes these non-curl arguments:\n\n- `--service` - AWS service name, if can't be automatically detected from host\n- `--region` - AWS region name, if can't be automatically detected from host or\n  if not explicitly provided in `AWS_DEFAULT_REGION` environment variable\n- `--ec2-creds` - use attached to EC2 credentials (instance role)\n\n### Response format\n\nAPIs for different services have different default response format. Sometimes it\nis json, sometimes xml. For most APIs you could enforce json output format by\nadding header `Accept: application/json` and xml output format by adding header\n`Accept: application/xml`.\n\n## Automatically computed headers\n\nThese headers are automatically handled by `aws-curl`:\n\n- `Host` - `curl` computes it automatically based on URL\n- `x-amz-date` - generated based on current date/time\n- `x-amz-content-sha256` - added automatically if request is to s3 service\n- `x-amz-security-token` - added automatically if corresponding env variable is\n  set\n- `Authorization` - automaticaly generated based on request and AWS credentials\n- `Content-Length` - `curl` computes it automatically\n- `Connection` - `curl` inserts it if needed\n\nSome headers provided by default by curl are unset by aws-curl:\n\n- `User-Agent` - doesn't have any impact on response\n- `Accept` - optional, but can be used to enforce response format (xml or json)\n- `Content-Type` - can be required or not depending on API\n\n## Service/region autodetection\n\nWrapper tries to auto detect AWS region name and AWS service name from service\nendpoint host.\n\nAssumed format for endpoint is `https://\u003cservice\u003e.\u003cregion\u003e.\u003cdomain\u003e` or\n`https://\u003cservice\u003e.\u003cdomain\u003e`, for example `https://logs.us-east-1.amazonaws.com`\nand `https://sts.amazonaws.com`.\n\nRead more about AWS service endpoints:\n\u003chttps://docs.aws.amazon.com/general/latest/gr/rande.html\u003e\n\nWrapper implicitly adds headers `authorization`, `x-amz-date`,\n`x-amz-content-sha256` and `x-amz-security-token` (if needed), so don't need to\npass them explicitly.\n\n## How to get know syntax for API\n\n### AWS documentation\n\nTake a look on AWS documentation, in most cases it has explicit http request\nexample.\n\nThe documentation is available here: \u003chttps://docs.aws.amazon.com\u003e\n\n### AWS CLI\n\nDownload and install AWS CLI: \u003chttps://aws.amazon.com/cli/\u003e\n\nRun the same command using aws cli with `--debug`. Take a look how AWS CLI is\nperforming the request, what body is used, what method is used, what url, what\nregion and what headers. Take a look on `Content-Type`, it impacts on how\nrequest body should be encoded.\n\nYou can get more details by exploring service json metadata file. AWS CLI with\n`--debug` dumps what service file is loaded for specific command. You can\nexplore it and find what body format is, what are all available options and what\nis the result.\n\n## EC2 attached role\n\nThis repo includes `ec2-import-creds` that can import attached credentials\nincluding access key, secret key, session token and region.\n\nJust import from the shell as `source ec2-import-creds`.\n\nOr you can use `--ec2-creds` options of `aws-cli` to get the same effect, but\nimporting credentials once in beginning is faster than importing for every\n`aws-curl` invocation.\n\n## Platforms\n\nThe script has been tested on bash in posix mode on macOS and Linux. It should\nwork on other shells and OS as well. If not, please cut a ticket.\n\n## License\n\nMIT\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsormy%2Faws-curl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsormy%2Faws-curl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsormy%2Faws-curl/lists"}