{"id":20112848,"url":"https://github.com/cloudacademy/aws-xray-microservices-calc","last_synced_at":"2025-05-06T11:32:08.304Z","repository":{"id":20420821,"uuid":"89876385","full_name":"cloudacademy/aws-xray-microservices-calc","owner":"cloudacademy","description":"Dockerised Microservices Calculator - demonstrating AWS X-Ray instrumentation and telemetry","archived":false,"fork":false,"pushed_at":"2022-12-13T01:58:40.000Z","size":5529,"stargazers_count":35,"open_issues_count":14,"forks_count":16,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-04-14T21:49:29.900Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudacademy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-30T20:49:36.000Z","updated_at":"2024-03-30T14:18:12.000Z","dependencies_parsed_at":"2023-01-12T08:45:55.415Z","dependency_job_id":null,"html_url":"https://github.com/cloudacademy/aws-xray-microservices-calc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Faws-xray-microservices-calc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Faws-xray-microservices-calc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Faws-xray-microservices-calc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudacademy%2Faws-xray-microservices-calc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudacademy","download_url":"https://codeload.github.com/cloudacademy/aws-xray-microservices-calc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224501637,"owners_count":17321868,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T18:22:41.549Z","updated_at":"2024-11-13T18:22:42.149Z","avatar_url":"https://github.com/cloudacademy.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AWS X-Ray Microservices Calculator\nUpdated: June 2020 - Refreshed entire codebase. Retested to confirm it works with the latest updates available within both the X-Ray service, AWS SDK, and SQS service.\n\n![Alt text](documentation/ServiceMap.png?raw=true \"Amazon X-Ray Console Service Map\")\n\n## Architecture\n\n![Alt text](documentation/XRayDockerArch.png?raw=true \"AWS X-Ray Microservices Calculator\")\n\nDockerised Microservices Calculator - demonstrating AWS X-Ray instrumentation and telemetry workflows.\n\n## Background\n\nThis project implements a simple Node.js microservices based calculator for the purpose demonstrating Amazon's newly launched X-Ray service: `https://aws.amazon.com/xray/`\n\nAWS X-Ray helps developers analyze and debug production, distributed applications, such as those built using a microservices architecture. With X-Ray, you can understand how your application and its underlying services are performing to identify and troubleshoot the root cause of performance issues and errors. X-Ray provides an end-to-end view of requests as they travel through your application, and shows a map of your application’s underlying components. You can use X-Ray to analyze both applications in development and in production, from simple three-tier applications to complex microservices applications consisting of thousands of services.\n\nThe Node.js microservices based calculator has been instrumented with the Node.js `aws-xray-sdk` - allowing it to propagate telemetry into the Amazon X-Ray cloud hosted service.\n\nThe sample project has been designed to run locally on a workstation using Docker containers.\n\nA `docker-compose.yml` file has been provided to orchestrate the provisioning of the entire microservices docker container architecture.\n\n## Calculator\n\nThe dockerised microservices calculator has been designed to evaluate simple and complex mathematical expressions. The calculator is designed to evaluate any valid user provided expression using the **order of operations** (or **operator precedence**) rules.\n\n### Sample Expressions\n\n* `(5+3)/2`\n* `((5+3)/2)^3`\n* `3^2+((5*5-1)/2)`\n* `3^3+((5*5)-1)/2`\n* `(2*(9+22/5)-((9-1)/4)^2)`\n* `(2*(9+22/5)-((9-1)/4)^2)+(3^2+((5*5-1)/2))`\n\n### Usage\n\nThe calculator service is invoked from the command line using the `curl` utility:\n\n* `curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=(5+3)/2\" http://localhost:8080/api/calc\"`\n* `curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=((5+3)/2)^3\" http://localhost:8080/api/calc\"`\n* `curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=3^2+((5*5-1)/2)\" http://localhost:8080/api/calc\"`\n* `curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=3^3+((5*5)-1)/2\" http://localhost:8080/api/calc\"`\n* `curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=(2*(9+22/5)-((9-1)/4)^2)\" http://localhost:8080/api/calc\"`\n* `curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=(2*(9+22/5)-((9-1)/4)^2)+(3^2+((5*5-1)/2))\" http://localhost:8080/api/calc\"`\n\nNote: The optional `calcid` param will be promoted to an *Annotation* on the captured X-Ray trace. The X-Ray service will index the `calcid` - meaning you can filter on it. An example *Filter Expression* that leverages the `calcid` param follows:\n\n`service(\"CALCULATOR\") { fault = true } AND annotation.calcid = \"1234\"`\n\n### AWS X-Ray Traces Grouped by Annotation\n\n![Alt text](documentation/FilterExpression1.png?raw=true \"AWS X-Ray Traces Grouped by Annotation\")\n\n### AWS X-Ray Trace Annotation Based Filter Expression\n\n![Alt text](documentation/FilterExpression2.png?raw=true \"AWS X-Ray Trace Annotation Based Filter Expression\")\n\n## Prerequisites\n\nYou will need to have a Docker runtime installed locally. This project uses both `docker` and `docker-compose` utilities. There are generally 2 approaches to installing a workstation Docker runtime:\n* Download and install Docker Desktop from: `https://www.docker.com/products/docker-desktop`, or\n* Install and configure Vagrant - then download and setup using CoreOS box located at: `https://github.com/coreos/coreos-vagrant`\n\nThis project has been successfully tested on:\n\n![Alt text](documentation/DockerDesktop.png?raw=true \"Docker Desktop\")\n\n## Installation\n\n1. Create a new IAM credential for the AWS X-Ray and SQS service accesses. Ensure that the credential has API programmatic access - this will provision an ACCESS_KEY and SECRET_ACCESS_KEY - we will add these into the `.env` configuration file (step 4 below).\n2. Attach the following 2 IAM policies:\n    1. `AWSXrayWriteOnlyAccess`\n    2. `AmazonSQSFullAccess`\n\nNotes: \nAWSXrayWriteOnlyAccess\n```javascript\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"xray:PutTraceSegments\",\n                \"xray:PutTelemetryRecords\",\n                \"xray:GetSamplingRules\",\n                \"xray:GetSamplingTargets\",\n                \"xray:GetSamplingStatisticSummaries\"\n            ],\n            \"Resource\": [\n                \"*\"\n            ]\n        }\n    ]\n}\n```\n\nAmazonSQSFullAccess\n```javascript\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Action\": [\n                \"sqs:*\"\n            ],\n            \"Effect\": \"Allow\",\n            \"Resource\": \"*\"\n        }\n    ]\n}\n```\n\n3. Create a new Amazon **FIFO** SQS queue. Record the SQS URL -  we will add this into the `.env` configuration file (step 4 below).\n\n**Note**: The SQS queue must be of type **FIFO**.\n\n4. Create a `.env` file in the project root directory. Add the following environment variables:\n```bash\nAWS_ACCESS_KEY_ID=\u003cyour access key here\u003e\nAWS_SECRET_ACCESS_KEY=\u003cyour secret access key here\u003e\nAWS_REGION=\u003caws region\u003e\nXRAY_CONTAINER_TIMEZONE=\u003ctime zone that the X-Ray daemon runs in\u003e\nCALC_SQS_QUEUE_URL=\u003cyour SQS URL\u003e\n```\n\nexample `.env` file:\n\n```bash\nAWS_ACCESS_KEY_ID=ABCD1234ABCD1234ABCD\nAWS_SECRET_ACCESS_KEY=abcd1234ABCD1234abcd1234ABCD1234abcd1234\nAWS_REGION=ap-southeast-2\nXRAY_CONTAINER_TIMEZONE=Pacific/Auckland\nCALC_SQS_QUEUE_URL=https://sqs.ap-southeast-2.amazonaws.com/123456789012/calc.fifo\n```\n\n5. Run `docker-compose build` from within the project root directory - this step will take approx 5mins to complete as it downloads the base images over the Internet.\n6. Run `docker images` - this will list all of the container images that have just been built:\n\n```bash\ndocker images\nREPOSITORY          TAG                 IMAGE ID            CREATED              SIZE\nnode-subtract       latest              034ff79532fe        57 seconds ago       128 MB\nnode-calc           latest              1ba68a9c70b9        About a minute ago   166 MB\nnode-multiply       latest              234f14958e8f        2 minutes ago        128 MB\nnode-add            latest              02b62d251ffb        2 minutes ago        148 MB\nxray-daemon         latest              f73e98971b76        2 minutes ago        107 MB\nnode-postfix        latest              541aa36c5874        5 minutes ago        166 MB\nnode-power          latest              8d10cdae9009        6 minutes ago        128 MB\nnode-divide         latest              34b4bcc5ade0        7 minutes ago        128 MB\ndebian              stable-slim         40617bfb5493        6 days ago           80.3 MB\nnode                alpine              7fce0a61c1d6        10 days ago          59 MB\n```\n\n7. Run `docker-compose up` from within the project root directory:\n\n```bash\nCreating XRAY     ... done\nCreating SUBTRACT ... done\nCreating DIVIDE   ... done\nCreating POSTIX   ... done\nCreating ADD      ... done\nCreating POWER    ... done\nCreating MULTIPLY ... done\nCreating CALC     ... done\nAttaching to DIVIDE, MULTIPLY, ADD, POWER, SUBTRACT, XRAY, POSTIX, CALC\nADD         |\nADD         | \u003e node-api@ start /usr/src/app\nADD         | \u003e node server.js\nADD         |\nDIVIDE      |\nDIVIDE      | \u003e node-api@ start /usr/src/app\nDIVIDE      | \u003e node server.js\nDIVIDE      |\nMULTIPLY    |\nMULTIPLY    | \u003e node-api@ start /usr/src/app\nMULTIPLY    | \u003e node server.js\nMULTIPLY    |\nSUBTRACT    |\nSUBTRACT    | \u003e node-api@ start /usr/src/app\nSUBTRACT    | \u003e node server.js\nSUBTRACT    |\nPOWER       |\nPOWER       | \u003e node-api@ start /usr/src/app\nPOWER       | \u003e node server.js\nPOWER       |\nXRAY        | 2020-06-15T03:15:10Z [Info] Initializing AWS X-Ray daemon 3.2.0\nXRAY        | 2020-06-15T03:15:10Z [Info] Using buffer memory limit of 19 MB\nXRAY        | 2020-06-15T03:15:10Z [Info] 304 segment buffers allocated\nXRAY        | 2020-06-15T03:15:10Z [Info] Using region: ap-southeast-2\nXRAY        | 2020-06-15T03:15:10Z [Info] HTTP Proxy server using X-Ray Endpoint : https://xray.ap-southeast-2.amazonaws.com\nXRAY        | 2020-06-15T03:15:10Z [Info] Starting proxy http server on 127.0.0.1:2000\nPOSTIX      |\nPOSTIX      | \u003e node-api@ start /usr/src/app\nPOSTIX      | \u003e node server.js\nPOSTIX      |\nPOSTIX      | POSTFIX service listening on port: 9090\nCALC        |\nCALC        | \u003e node-api@ start /usr/src/app\nCALC        | \u003e node server.js\nCALC        |\nMULTIPLY    | MULTIPLY service listening on port: 8083\nDIVIDE      | DIVIDE service listening on port: 8084\nPOWER       | POWER service listening on port: 8085\nSUBTRACT    | SUBTRACT service listening on port: 8082\nADD         | ADD service listening on port: 8081\nCALC        | CALCULATOR service listening on port: 8080\nCALC        | ********************************************\nCALC        | ********************************************\nCALC        | sample calculator test commands:\nCALC        | curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=(5+3)/2\" http://localhost:8080/api/calc\nCALC        | curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=((5+3)/2)^3\" http://localhost:8080/api/calc\nCALC        | curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=3^2+((5*5-1)/2)\" http://localhost:8080/api/calc\nCALC        | curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=3^3+((5*5)-1)/2\" http://localhost:8080/api/calc\nCALC        | curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=(2*(9+22/5)-((9-1)/4)^2)\" http://localhost:8080/api/calc\nCALC        | curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=(2*(9+22/5)-((9-1)/4)^2)+(3^2+((5*5-1)/2))\" http://localhost:8080/api/calc\nCALC        | note: the optional calcid param will be added as an annotation to the xray trace\nCALC        | ********************************************\nCALC        | ********************************************\n```\n\n8. In another console window, fire a test calculation at it:\n\n`curl --data-urlencode \"calcid=1234\" --data-urlencode \"expression=((5+3)/2)^3\" http://localhost:8080/api/calc`\n\n9. Examine the console output - the response should be `((5+3)/2)^3=64`\n\n10. Examine the console output of the `docker-compose up` console:\n\n```bash\nCALC        | Calculator entry point...\nCALC        | calcid supplied: 1234\nCALC        | calcid: 1234, infix: ((5+3)/2)^3\nPOSTIX      | POSTFIX-\u003ecalcid: 1234, infix: ((5+3)/2)^3\nPOSTIX      | POSTFIX-\u003ecalcid: 1234, postfix: 5 3 + 2 / 3 ^\nCALC        | STATUS: 200\nCALC        | HEADERS: {\"x-powered-by\":\"Express\",\"date\":\"Mon, 15 Jun 2020 03:16:49 GMT\",\"connection\":\"close\",\"transfer-encoding\":\"chunked\"}\nCALC        | BODY: 5 3 + 2 / 3 ^\nCALC        | postfix:5 3 + 2 / 3 ^\nCALC        | http request host:port -\u003e 172.19.10.1:8081\nADD         | adding...\nADD         | 3+5=8\nCALC        | STATUS: 200\nCALC        | result=8\nCALC        | http request host:port -\u003e 172.19.10.4:8084\nDIVIDE      | dividing...\nDIVIDE      | 8/2=4\nCALC        | STATUS: 200\nCALC        | result=4\nCALC        | http request host:port -\u003e 172.19.10.5:8085\nPOWER       | powering...\nPOWER       | 4^3=64\nCALC        | STATUS: 200\nCALC        | result=64\nCALC        | CALC RESULT=64\nCALC        | add count 1\nCALC        | subtract count 0\nCALC        | multiply count 0\nCALC        | divide count 1\nCALC        | power count 1\nPOSTIX      | sqs success for POSTFIX service 72ba0156-06b0-4912-ae6f-154450c571bd\nADD         | sqs success for ADD service 72ba0156-06b0-4912-ae6f-154450c571bd\nDIVIDE      | sqs success for DIVIDE service 72ba0156-06b0-4912-ae6f-154450c571bd\nPOWER       | sqs success for POWER service 72ba0156-06b0-4912-ae6f-154450c571bd\nXRAY        | 2020-06-15T03:16:50Z [Info] Successfully sent batch of 5 segments (0.496 seconds)\n```\n\n11. Login into the AWS X-Ray console\n\n    1. Examine the Service Map:\n\n        ![Alt text](documentation/ServiceMap.png?raw=true \"Amazon X-Ray Console Service Map\")\n\n    2. Perform a filtered trace and examine response codes:\n\n        ![Alt text](documentation/Trace1.png?raw=true \"Amazon X-Ray Console Trace - filtered search\")\n\n        ![Alt text](documentation/Trace2.png?raw=true \"Amazon X-Ray Console Trace - examine response codes\")\n    3. Examine the configured SQS queue\n\n## Notes\n\n1. The X-Ray daemon may fail on its 1st attempt to publish batch results to the AWS X-Ray service - it appears to have some *warm-up* and/or initialisation to complete - after this all subsequent batch sends will work.\n\n2. The docker containers `POSTFIX`, `ADD`, `SUBTRACT`, `MULTIPLY`, `DIVIDE`, and `POWER` each publish a message to the configured SQS **FIFO** queue - this is done only to demonstrate how AWS services can be instrumented against, the messages on the SQS queue are not consumed by any external service. The SQS queue should be purged when the sample project is torn down.\n\n3. Each docker container in the solution performs a discrete function:\n    1. `CALC`: orchestrates the full calculation - consulting each of the other containers as and when required\n    2. `XRAY`: hosts the AWS X-Ray Daemon - this listens for incoming `UDP` traffic on port `2000`. All of the other docker containers are configured to send their X-Ray data to this container - it is then periodically batched up and delivered over `HTTPS` to the AWS X-Ray service via the Internet\n    3. `POSTFIX`: converts the calculation expression from INFIX form to POSTFIX form\n    4. `ADD`: returns result of the 1st number added to 2nd number\n    5. `SUBTRACT`: returns result of 1st number minus 2nd number\n    6. `MULTIPLY`: returns result of 1st number multiplied by 2nd number\n    7. `DIVIDE`: returns result of 1st number divided by second number\n    8. `POWER`: returns result of the 1st number (base) raised by the 2 number (exponent) \n\n4. Each docker container in the solution has been designed to return a percentage of HTTP error codes, e.g. *400s* and/or *500s*. The error codes returned have no effect on the outcome of the full calculation - this still calculates normally. The error codes are intentionally returned to demonstrate how they are rendered within the AWS X-Ray Service Map and Trace views. The following code snippet highlights this:\n\n```javascript\nvar responseCode = 200;\nvar random = Math.random();\n\nif (random \u003c 0.8) {\n    responseCode = 200;\n} else if (random \u003c 0.9) {\n    responseCode = 403;\n} else {\n    responseCode = 503;\n}\n\nres.statusCode = responseCode;\nres.end();\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudacademy%2Faws-xray-microservices-calc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudacademy%2Faws-xray-microservices-calc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudacademy%2Faws-xray-microservices-calc/lists"}