{"id":19629113,"url":"https://github.com/jiefei-wang/aws.ecx","last_synced_at":"2025-09-12T04:32:03.216Z","repository":{"id":56934172,"uuid":"339330237","full_name":"Jiefei-Wang/aws.ecx","owner":"Jiefei-Wang","description":"The package provides the functions to communicate with AWS EC2 and AWS ECS.","archived":false,"fork":false,"pushed_at":"2022-01-26T03:49:54.000Z","size":4443,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-10T04:45:28.186Z","etag":null,"topics":["ec2","ecs","ecs-functions","rpackage"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Jiefei-Wang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-16T08:25:12.000Z","updated_at":"2022-01-26T03:44:53.000Z","dependencies_parsed_at":"2022-08-21T06:50:37.234Z","dependency_job_id":null,"html_url":"https://github.com/Jiefei-Wang/aws.ecx","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jiefei-Wang%2Faws.ecx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jiefei-Wang%2Faws.ecx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jiefei-Wang%2Faws.ecx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jiefei-Wang%2Faws.ecx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Jiefei-Wang","download_url":"https://codeload.github.com/Jiefei-Wang/aws.ecx/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240927790,"owners_count":19879988,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ec2","ecs","ecs-functions","rpackage"],"created_at":"2024-11-11T11:57:55.512Z","updated_at":"2025-02-26T20:24:10.875Z","avatar_url":"https://github.com/Jiefei-Wang.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\ntitle: \"Quick-Start-Guide\"\noutput: rmarkdown::html_vignette\nvignette: \u003e\n  %\\VignetteIndexEntry{Quick-Start-Guide}\n  %\\VignetteEngine{knitr::rmarkdown}\n  %\\VignetteEncoding{UTF-8}\n---\n\n\n\n\n\n[![](https://www.r-pkg.org/badges/version/aws.ecx?color=green)](https://cran.r-project.org/package=aws.ecx)\n[![](https://img.shields.io/badge/devel%20version-1.0.4-orange.svg)](https://github.com/Jiefei-Wang/aws.ecx)\n[![](http://cranlogs.r-pkg.org/badges/last-month/aws.ecx?color=blue)](https://cran.r-project.org/package=aws.ecx)\n\n# Introduction\nThis package aims to provide the functions for communicating with Amazon Web Services(AWS) Elastic Container Service(ECS) using AWS REST APIs. The ECS functions start with the prefix `ecs_` and EC2 functions start with `ec2_`. The general-purpose functions have the prefix `aws_`.\n\nSince there are above 400 EC2 APIs, it is not possible for the unit test to cover all use cases. If you see any problems when using the package, please consider to submit the issue to [GitHub issue][GitHub issue].\n\n[GitHub issue]: https://github.com/Jiefei-Wang/aws.ecx/issues\n\n# Authentication\nCredentials must be provided for using the package. The package uses `access key id` and `secret access key` to authenticate with AWS. See [AWS user guide][AWS user guide] for the information about how to obtain them from AWS console. The credentials can be set via `aws_set_credentials()`.\n\n```r\naws_set_credentials()\n#\u003e $access_key_id\n#\u003e [1] \"AK**************OYX3\"\n#\u003e \n#\u003e $secret_access_key\n#\u003e [1] \"mL**********************************XGGH\"\n#\u003e \n#\u003e $region\n#\u003e [1] \"ap-southeast-1\"\n```\nThe function `aws_set_credentials` sets both the credentials and the region of the AWS service. You can either explicitly provide them by the function arguments or rely on the locator to automatically find your credentials. There are many ways to locate the credentials but the most important methods are as follow(sorted by the search order):\n\n1. environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`, and `AWS_SESSION_TOKEN`\n\n2. a profile in a global credentials dot file in a location set by `AWS_SHARED_CREDENTIALS_FILE` or defaulting typically to `\"~/.aws/credentials\"` (or another OS-specific location), using the profile specified by `AWS_PROFILE`\n\nUsers can find the details on how the credentials are located from `?aws.signature::locate_credentials`. A list of AWS regions can be found by calling the function `aws_list_regions`. \n\n```r\naws_list_regions()\n#\u003e  [1] \"us-east-1\"      \"us-east-2\"      \"us-west-1\"      \"us-west-2\"      \"us-gov-west-1\" \n#\u003e  [6] \"us-gov-east-1\"  \"ca-central-1\"   \"eu-north-1\"     \"eu-west-1\"      \"eu-west-2\"     \n#\u003e [11] \"eu-west-3\"      \"eu-central-1\"   \"eu-south-1\"     \"af-south-1\"     \"ap-northeast-1\"\n#\u003e [16] \"ap-northeast-2\" \"ap-northeast-3\" \"ap-southeast-1\" \"ap-southeast-2\" \"ap-east-1\"     \n#\u003e [21] \"ap-south-1\"     \"sa-east-1\"      \"me-south-1\"\n```\n\n\n[AWS user guide]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html\n\n# Call the functions\nCalling the EC2 or ECS function is simple, for example, you can list all ECS clusters via\n\n```r\necs_list_clusters()\n#\u003e REST request failed with the message:\n#\u003e  Timeout was reached: [ecs.ap-southeast-1.amazonaws.com] Send failure: Connection was aborted\n#\u003e [1] \"arn:aws:ecs:ap-southeast-1:020007817719:cluster/R-worker-cluster\"\n#\u003e [2] \"arn:aws:ecs:ap-southeast-1:020007817719:cluster/test\"\n```\nThe original EC2 and ECS APIs accept the request parameter via the query parameter or header and return the result by JSON or XML text in the REST request. The package provides a unified way to call both APIs. The request parameters can be given by the function arguments and the result is returned in a list format. If possible, the package will try to simplify the result and return a simple character vector. It will also handle the `nextToken` parameter in the REST APIs and collect the full result in a single function call. This default behavior can be turned off by providing the parameter `simplefy = FALSE`.\n\nEach EC2 or ECS API has its own document. For example, you can find the help page of `ecs_list_clusters` via `?ecs_list_clusters`. The full description of the APIs can be found at [AWS Documentation][AWS Documentation].\n\n[AWS Documentation]: https://docs.aws.amazon.com/index.html\n\n## A note for the AWS EC2 functions\n### Array\n\nWhile the AWS Documentation is very helpful in finding the API use cases. There are some inconsistencies between the AWS Documentation and the package functions. To be more specific, the array type parameter will get a special treatment in this package. For example, here is an example for `DescribeVpcs` from the [AWS Documentation][example] which describes the specified VPCs\n```\nhttps://ec2.amazonaws.com/?Action=DescribeVpcs\n\u0026VpcId.1=vpc-081ec835f3EXAMPLE\n\u0026vpcId.2=vpc-0ee975135dEXAMPLE\n\u0026VpcId.3=vpc-06e4ab6c6cEXAMPLE\n```\nThe `VpcId` is so-called array object in the AWS Documentation. The package uses a vector or a list to represent the array object. The dot `.` in the array can be explained as `[[` and anything followed by the dot can be explained as the index. Therefore, the corresponding R expression for `VpcId` is\n\n```r\n## VpcId can also be a character vector\nVpcId \u003c- list()\nVpcId[[1]] \u003c- \"vpc-081ec835f3EXAMPLE\"\nVpcId[[2]] \u003c- \"vpc-0ee975135dEXAMPLE\"\nVpcId[[3]] \u003c- \"vpc-06e4ab6c6cEXAMPLE\"\n\nVpcId\n#\u003e [[1]]\n#\u003e [1] \"vpc-081ec835f3EXAMPLE\"\n#\u003e \n#\u003e [[2]]\n#\u003e [1] \"vpc-0ee975135dEXAMPLE\"\n#\u003e \n#\u003e [[3]]\n#\u003e [1] \"vpc-06e4ab6c6cEXAMPLE\"\n```\nand the same request in R can be made by\n```\nec2_describe_vpcs(VpcId = VpcId)\n```\nInternally, `VpcId` will be converted to an array object using the function `list_to_array`, e.g.\n\n```r\n## The first argument is the parameter prefix\n## The second argument should be a (named) vector or list\nlist_to_array(\"VpcId\", VpcId)\n#\u003e $VpcId.1\n#\u003e [1] \"vpc-081ec835f3EXAMPLE\"\n#\u003e \n#\u003e $VpcId.2\n#\u003e [1] \"vpc-0ee975135dEXAMPLE\"\n#\u003e \n#\u003e $VpcId.3\n#\u003e [1] \"vpc-06e4ab6c6cEXAMPLE\"\n```\nJust like list can be nested, the array object can be nested as well. For example, if we have a `tags` array like\n```\ntags.1.name = name-example\ntags.1.value.1 = value-example1\ntags.1.value.2 = value-example2\n```\nUsing the array rule we mentioned above, the corresponding R expression is\n\n```r\ntags \u003c- list()\ntags[[1]] \u003c- list()\ntags[[1]][[\"name\"]] \u003c- \"name-example\"\ntags[[1]][[\"value\"]] \u003c- list()\ntags[[1]][[\"value\"]][[1]] \u003c- \"value-example1\"\ntags[[1]][[\"value\"]][[2]] \u003c- \"value-example2\"\n\ntags\n#\u003e [[1]]\n#\u003e [[1]]$name\n#\u003e [1] \"name-example\"\n#\u003e \n#\u003e [[1]]$value\n#\u003e [[1]]$value[[1]]\n#\u003e [1] \"value-example1\"\n#\u003e \n#\u003e [[1]]$value[[2]]\n#\u003e [1] \"value-example2\"\n```\nWe can verify the request parameter using `list_to_array`\n\n```r\nlist_to_array(\"tags\", tags)\n#\u003e $tags.1.name\n#\u003e [1] \"name-example\"\n#\u003e \n#\u003e $tags.1.value.1\n#\u003e [1] \"value-example1\"\n#\u003e \n#\u003e $tags.1.value.2\n#\u003e [1] \"value-example2\"\n```\n\n### Filter\nA even more aggressive change can be found on the `Filter` parameter. Here is an example of describing VPCs which satisfy some specific conditions from the [AWS Documentation][example]\n```\nhttps://ec2.amazonaws.com/?Action=DescribeVpcs\n\u0026Filter.1.Name=dhcp-options-id\n\u0026Filter.1.Value.1=dopt-7a8b9c2d\n\u0026Filter.1.Value.2=dopt-2b2a3d3c\n\u0026Filter.2.Name=state\n\u0026Filter.2.Value.1=available\n```\nThe `Filter` parameter is, of course, an array object in AWS documentation. However, it is clearly redundant to specify the filter's name and value separately. Therefore, the package allows users to provide the filter as a named list. The same request in R can be given by\n```\nec2_describe_vpcs(\n  Filter = list(\n    `dhcp-options-id` = c(\"dopt-7a8b9c2d\", \"dopt-2b2a3d3c\"),\n    state=\"available\"\n  )\n)\n```\nThe `Filter` parameter will be converted into a list object which meets the AWS filter requirement. If you are unsure about whether you has specified the correct filter, you can check the converted filter values via\n\n```r\nfilter \u003c- list(\n    `dhcp-options-id` = c(\"dopt-7a8b9c2d\", \"dopt-2b2a3d3c\"),\n    state=\"available\"\n  )\nlist_to_filter(filter)\n#\u003e $Filter.1.Name\n#\u003e [1] \"dhcp-options-id\"\n#\u003e \n#\u003e $Filter.1.Value.1\n#\u003e [1] \"dopt-7a8b9c2d\"\n#\u003e \n#\u003e $Filter.1.Value.2\n#\u003e [1] \"dopt-2b2a3d3c\"\n#\u003e \n#\u003e $Filter.2.Name\n#\u003e [1] \"state\"\n#\u003e \n#\u003e $Filter.2.Value.1\n#\u003e [1] \"available\"\n```\n\n[example]: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeVpcs.html\n\n\n## A note for the AWS ECS functions\nThe AWS ECS API uses JSON format to assemble the request parameter. Therefore, the ECS functions will call `rjson::toJSON` to convert the request parameters into JSON objects. If you are not sure if you use the parameter correctly, you can manually run `rjson::toJSON` and compare the result with the example provided in AWS documentation. For example, the request syntax for the `tags` parameter in `CreateCluster` API is \n```\n\"tags\": [ \n         { \n            \"key\": \"string\",\n            \"value\": \"string\"\n         }\n      ]\n```\nThe corresponding R format should be\n\n```r\ntags \u003c- list(\n  c(key = \"key\", value = \"value\")\n  )\n```\nYou can verify it by\n\n```r\ncat(rjson::toJSON(tags, indent = 1))\n#\u003e [\n#\u003e  {\n#\u003e \"key\":\"key\",\n#\u003e \"value\":\"value\"\n#\u003e  }\n#\u003e ]\n```\n\n# Package settings\nThe package handles the network issue via the parameter `retry_time`, `print_on_error` and `network_timeout`.\n\n\n```r\naws_get_retry_time()\n#\u003e [1] 5\naws_get_print_on_error()\n#\u003e [1] TRUE\naws_get_network_timeout()\n#\u003e [1] 2\n```\n`retry_time` determines the number of time the function will retry when network error occurs before throwing an error. If `print_on_error` is set to `False`, no message will be given when the network error has occurred and the package will silently resend the REST request. `network_timeout` decides how long the function will wait before it fails. They can be changed via the corresponding setters(e.g. `aws_set_retry_time`). You can also temporary alter the setting by providing the package setting as a parameter in the EC2 or ECS function.\n\n# Warning\nNot all AWS APIs are idempotent, especially for the functions that need to allocate resources on AWS(e.g. `ecs_start_task`). If you plan to use the AWS function in your package, special handle for the network issue is required for the non-idempotent API to avoid a double allocation. \n\n\n# Session info\n\n```r\nsessionInfo()\n#\u003e R version 4.0.4 (2021-02-15)\n#\u003e Platform: x86_64-w64-mingw32/x64 (64-bit)\n#\u003e Running under: Windows 10 x64 (build 19042)\n#\u003e \n#\u003e Matrix products: default\n#\u003e \n#\u003e locale:\n#\u003e [1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   \n#\u003e [3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          \n#\u003e [5] LC_TIME=English_United States.1252    \n#\u003e \n#\u003e attached base packages:\n#\u003e [1] stats     graphics  grDevices utils     datasets  methods   base     \n#\u003e \n#\u003e other attached packages:\n#\u003e [1] aws.ecx_1.0.4\n#\u003e \n#\u003e loaded via a namespace (and not attached):\n#\u003e  [1] tidyselect_1.1.0    xfun_0.19           remotes_2.2.0       purrr_0.3.4        \n#\u003e  [5] generics_0.1.0      vctrs_0.3.4         colorspace_2.0-0    testthat_3.0.2     \n#\u003e  [9] usethis_1.6.3       htmltools_0.5.0     stats4_4.0.4        yaml_2.2.1         \n#\u003e [13] base64enc_0.1-3     rlang_0.4.10        pkgbuild_1.1.0      pillar_1.4.6       \n#\u003e [17] rapiclient_0.1.3    glue_1.4.2          withr_2.3.0         BiocGenerics_0.36.0\n#\u003e [21] RColorBrewer_1.1-2  sessioninfo_1.1.1   rvcheck_0.1.8       lifecycle_0.2.0    \n#\u003e [25] stringr_1.4.0       dlstats_0.1.3       munsell_0.5.0       commonmark_1.7     \n#\u003e [29] gtable_0.3.0        devtools_2.3.2      memoise_1.1.0       evaluate_0.14      \n#\u003e [33] knitr_1.31          callr_3.5.1         ps_1.4.0            curl_4.3           \n#\u003e [37] parallel_4.0.4      Rcpp_1.0.5          BiocManager_1.30.10 scales_1.1.1       \n#\u003e [41] formatR_1.8         S4Vectors_0.28.0    desc_1.2.0          pkgload_1.1.0      \n#\u003e [45] jsonlite_1.7.2      fs_1.5.0            rjson_0.2.20        ggplot2_3.3.2      \n#\u003e [49] digest_0.6.27       stringi_1.5.3       dplyr_1.0.2         processx_3.4.4     \n#\u003e [53] rprojroot_2.0.2     grid_4.0.4          cli_2.3.1           tools_4.0.4        \n#\u003e [57] magrittr_1.5        tibble_3.0.4        pkgconfig_2.0.3     crayon_1.3.4       \n#\u003e [61] aws.signature_0.6.0 whisker_0.4         ellipsis_0.3.1      xml2_1.3.2         \n#\u003e [65] prettyunits_1.1.1   assertthat_0.2.1    rmarkdown_2.7       httr_1.4.2         \n#\u003e [69] roxygen2_7.1.1      rstudioapi_0.13     badger_0.0.9        R6_2.5.0           \n#\u003e [73] compiler_4.0.4\n```\n\n# future work\n1. Convert parameter type if it does not meet the AWS type requirement. Done\n\n2. add link to the function documentation\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiefei-wang%2Faws.ecx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjiefei-wang%2Faws.ecx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiefei-wang%2Faws.ecx/lists"}