{"id":21888645,"url":"https://github.com/swisscom/dynstrg-howto","last_synced_at":"2025-04-15T10:20:17.181Z","repository":{"id":70508673,"uuid":"44685663","full_name":"swisscom/dynstrg-howto","owner":"swisscom","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-16T15:17:49.000Z","size":47,"stargazers_count":9,"open_issues_count":1,"forks_count":3,"subscribers_count":35,"default_branch":"master","last_synced_at":"2025-04-15T10:20:05.682Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/swisscom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-10-21T15:25:28.000Z","updated_at":"2024-10-16T15:17:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"be96b253-6779-4480-84f9-821fc16560dd","html_url":"https://github.com/swisscom/dynstrg-howto","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fdynstrg-howto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fdynstrg-howto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fdynstrg-howto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fdynstrg-howto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/swisscom","download_url":"https://codeload.github.com/swisscom/dynstrg-howto/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249048785,"owners_count":21204311,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-28T11:16:24.598Z","updated_at":"2025-04-15T10:20:17.156Z","avatar_url":"https://github.com/swisscom.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dynamic Storage How To\n\nThis tutorial illustrates how to use [Dynamic Storage](https://docs.developer.swisscom.com/service-offerings/dynamic-storage.html) on the Swisscom Application Cloud. It is a step by step guide that will help you understand how to connect to the service and how to upload files to it. In particular, this tutorial will show you how to create a simple Node.js application to upload images to Dynamic Storage, to visualize them and to download them. This tutorial assumes that you have the [Cloud Foundry CLI](https://docs.developer.swisscom.com/cf-cli/install-go-cli.html) installed.\n\n## Quick Start\n\nIf, instead of following the tutorial, you are looking for a quick start and deployment to Cloud Foundry, you can just `git clone` this repo. If you want to go through the tutorial, jump ahead to the [Step by step tutorial](#step-by-step-tutorial) section instead.\n\nYou'll have to create a dynamic storage service called `dynstrg-album` which can be done with the following command:\n\n```shell\n$ cf create-service dynstrg usage dynstrg-album\n```\n\nThen, you can create a bucket called `my-bucket` as described [here](#access-s3-storage). Once the bucket is created, you can push the app with.\n\n```shell\n$ cf push\n```\n\n## Step by step tutorial\n\n### Dynamic Storage S3 compatibility\n\nDynamic Storage is an S3 compatible service. This means that it is possible to use it with applications built to use the AWS S3 storage. Therefore, in this tutorial, we will use the [AWS SDK for Node.js](https://aws.amazon.com/it/sdk-for-node-js/).\n\n### Install Node.js\n\nIf you already have Node.js installed on your computer, you can skip this section and go to the next one.\nFirst of all, download [Node.js](https://nodejs.org/) and install it. At the end of the installation, to test that everything is set up properly, run the following command:\n\n```shell\n$ node --version\n```\n\nThe output should look something like this:\n\n```txt\nvx.x.x\n```\n\n### Create a simple Express application\n\nFor this tutorial, we will use [Express](https://expressjs.com/). The [Express docs](https://expressjs.com/en/starter/installing.html) offer a good starting point on how to set up our application. We follow the same steps for our tutorial.\nFirst of all, we create a directory for our application. Let's call it \"dynstrg-album\". Create this directory and navigate into it. Once in the directory, use the command:\n\n```shell\n$ npm init --yes\n```\n\nto create a `package.json` file.\n\nTo install express use this command:\n\n```shell\n$ npm install --save express\n```\n\nThis will install Express and add it to `package.json` as a dependency.\n\n### Create server\n\nOpen your favorite text editor or IDE and create a new folder `src` and, inside, a file called `index.js`.\n\nTo get started, let's create a local Node.js server using Express as shown in the [Express docs](https://expressjs.com/en/starter/hello-world.html). The app starts a server on your \"localhost\" and listens on port 3000 which simply responds \"Hello World\".\n\nTo create your app, add the following code to your `index.js`:\n\n```javascript\n\"use strict\";\n\nconst express = require(\"express\");\n\nconst app = express();\n\nconst port = 3000;\n\napp.get(\"/\", (req, res) =\u003e {\n  res.send(\"Hello World!\");\n});\n\napp.listen(port, () =\u003e {\n  console.log(\"Application listening on port\", port);\n});\n```\n\nThen, switch to the `package.json` file and change `\"scripts\"` to the following:\n\n```json\n\"scripts\": {\n  \"start\": \"node src\"\n},\n```\n\nYou can also remove the `\"main\"` property since we won't be needing it.\n\nThen you can run the following command to start your app:\n\n```shell\n$ npm start\n```\n\n### Deploy app to Cloud Foundry\n\nIn order to deploy the app to Cloud Foundry, a small modification must be made to `src/index.js` since we need to read the port from the environment:\n\nChange the line:\n\n```javascript\nconst port = 3000;\n```\n\nto\n\n```javascript\nconst port = process.env.PORT || 3000;\n```\n\nThe environment variable [PORT](https://docs.developer.swisscom.com/devguide/deploy-apps/environment-variable.html#PORT) is provided by Cloud Foundry.\n\nThen we need to create a `manifest.yml` file in the root directory of our app. This YAML file tells Cloud Foundry which parameters to use for the application. Create the file and add the following content:\n\n```yaml\n---\napplications:\n- name: dynstrg-album\n  memory: 64M\n  buildpack: https://github.com/cloudfoundry/nodejs-buildpack.git\n  random-route: true\n\n  env:\n    NODE_ENV: production\n```\n\nWith this manifest, an application called \"dynstrg-album\" will be deployed. Before we do that, we should add a `.cfignore` file at the root of our project. This works just like a `.gitignore` file but for pushing to Coud Foundry. Let's add `node_modules` to that file so we don't push the dependencies. These are installed by Cloud Foundry anyways.\n\nThen, push the app With the command:\n\n```shell\n$ cf push\n```\n\nThe application is deployed to Cloud Foundry. You can now visit your app's URI and get a \"Hello World\" message.\n\n### Create dynamic storage service instance\n\nYou can use the command\n\n```shell\n$ cf marketplace\n```\n\nto list the services available to your organization. You should see the service `dynstrg` with the plan `usage`. To create a new dynstrg service instance use the following command:\n\n```shell\n$ cf create-service dynstrg usage dynstrg-album\n```\n\nThis creates a new service instance called \"dynstrg-album\". At this point, the service can be bind to the application with the command:\n\n```shell\n$ cf bind-service dynstrg-album dynstrg-album\n```\n\nThen restage your app as suggested:\n\n```shell\n$ cf restage dynstrg-album\n```\n\nOnce restaged, the service can be accessed using the environment variable [VCAP_SERVICES](https://docs.developer.swisscom.com/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES). For our application, we can inspect this variable with the command\n\n```shell\n$ cf env dynstrg-album\n```\n\n### Access S3 storage\n\nIn S3, files are stored in so called buckets. In this case, we'll create the bucket with a CLI tool executed on our local machine.\n\nTo access the S3 storage from our local machine, we first create a service key:\n\n```bash\n$ cf create-service-key dynstrg-album manage\n```\n\nNow you can retrieve the credentials to your S3 service at any time using the command\n\n```bash\n$ cf service-key dynstrg-album manage\n```\n\nWith this key, we can now create a bucket to hold our data. I suggest to use the [s3cmd](https://github.com/s3tools/s3cmd) CLI tool for that. You can either download it from the [releases](https://github.com/s3tools/s3cmd/releases) page or by using [Homebrew](http://brew.sh/) if you're on macOS. As soon as you have it installed, configure it using the following command:\n\n```bash\n$ s3cmd --configure\n```\n\nYou will be prompted for your credentials. Most of them you can find by retrieving your service key from above or by using the defaults.\n\nSet the region to `nil` and note that `Secret Key` is your `sharedSecret` and that `S3 Endpoint` is your `accessHost`. For `DNS-style bucket+hostname:port template for accessing a bucket`, you can use `%(bucket)s.\u003cyour-accessHost\u003e` with your respective `accessHost`.\n\nNow you should be able to run\n\n```bash\n$ s3cmd ls\n```\n\nwhich should return empty if you don't have any buckets or list your existing ones. Now, we can finally create our bucket called \"my-bucket\" with the following command:\n\n```bash\n$ s3cmd mb s3://my-bucket\n```\n\n### Add service and bucket name to manifest\n\nSince we will modify and deploy the app many times, we should add this information to `manifest.yml`.\n\nThe new file should look like this:\n\n```yaml\n---\napplications:\n- name: dynstrg-album\n  memory: 64M\n  buildpack: https://github.com/cloudfoundry/nodejs-buildpack.git\n  random-route: true\n\n  services:\n  - dynstrg-album\n\n  env:\n    NODE_ENV: production\n    BUCKET_NAME: my-bucket\n```\n\n### Access service credentials\n\nIn Node.js, to access the credentials stored on the environment variable `VCAP_SERVICES` is simple. You just have to add these lines to the file `src/index.js` right after the port is set:\n\n```javascript\nconst vcapServices = JSON.parse(process.env.VCAP_SERVICES);\nconst credentials = vcapServices[\"dynstrg\"][0].credentials;\n```\n\n### Add AWS S3 SDK\n\nWe will use the AWS S3 SDK to interact with the dynamic storage service since it is compatible with Swisscom Dynamic Storage. The first thing to do is to import the `aws-sdk` package. This can be done by running the command\n\n```shell\n$ npm install --save aws-sdk\n```\n\nand then adding the following line to our `src/index.js` file:\n\n```javascript\nconst AWS = require(\"aws-sdk\");\n```\n\nThen, the imported package can be configured using the `credentials` variable we had created previously. Add these lines to `src/index.js` right after `credentials` is defined to create an S3 client:\n\n```javascript\nAWS.config.update({\n  accessKeyId: credentials.accessKey,\n  secretAccessKey: credentials.sharedSecret\n});\nconst s3 = new AWS.S3({ endpoint: new AWS.Endpoint(credentials.accessHost) });\n```\n\nThat's it. The client is configured to use the credentials from the `VCAP_SERVICES` and it is initialized.\n\n### Upload files\n\nIn order to be able to upload local files to the dynamic storage, we use the package [connect-multiparty](https://www.npmjs.com/package/connect-multiparty). This module allows to upload local files using a multipart approach. The native module `fs` must also be imported to be able to read the local file containing the image we want to upload. Let's install and add these packages by running the following command:\n\n```shell\n$ npm install --save connect-multiparty\n```\n\nThen add the two packages to the top of our `src/index.js` file:\n\n```javascript\nconst fs = require(\"fs\");\nconst multipart = require(\"connect-multiparty\");\n```\n\nNow we have to create an HTML form that allows us to choose a file to upload. To do that, we can simply use Node.js to send an HTML page containing a form. Add these lines to your file right after the root route is handled:\n\n```javascript\napp.get(\"/upload\", (req, res) =\u003e {\n  res.send(`\n    \u003c!DOCTYPE html\u003e\n    \u003chtml\u003e\n      \u003cbody\u003e\n        \u003cform\n          action=\"/api/upload\"\n          method=\"post\"\n          enctype=\"multipart/form-data\"\n        \u003e\n          \u003cinput type=\"file\" name=\"pic\"\u003e\n          \u003cinput type=\"submit\"\u003e\n        \u003c/form\u003e\n      \u003c/body\u003e\n    \u003c/html\u003e\n  `);\n});\n```\n\nTo handle the POST HTTP request coming from the form, we have to add another route that can receive HTTP POST requests. For this route, we will use the \"connect-multiparty\" module.\n\nThe bucket name can be retrieved through the environment variables. To do so, add this line to where you set the `port`:\n\n```javascript\nconst bucket = process.env.BUCKET_NAME;\n```\n\nOur new multipart middleware can be used directly on the new route. Add it to your route definitions:\n\n```javascript\napp.post(\"/api/upload\", multipart(), (req, res) =\u003e {\n  fs.readFile(req.files.pic.path, (err, data) =\u003e {\n    const key = req.files.pic.originalFilename;\n    const params = { Bucket: bucket, Key: key, Body: data };\n    s3.putObject(params, (err, data) =\u003e {\n      if (err) {\n        console.error(err);\n        res.status(500).json(err);\n        return;\n      }\n      res.redirect(\"/\");\n    });\n  });\n});\n```\n\nNow the file will be uploaded to the dynamic storage using the method `putObject` provided by the AWS S3 SDK. Let's redeploy our app to cloud foundry and try it out at `/upload`:\n\n```shell\n$ cf push\n```\n\n### Check if the file was uploaded correctly\n\nUsing `s3cmd`, we can verify that the picture was correctly uploaded to the bucket:\n\n```\n$ s3cmd ls s3://my-bucket\n```\n\n### Visualize uploaded images\n\nS3 allows to create a signed URL to share a specific file with another user. The same can be done with our dynamic storage using the AWS S3 SDK. We will use this method to visualize all the pictures we have stored on dynstrg.\n\nWe can get all the images we store in the dynamic storage using the method `listObjects`. For each image, we can get a signed URL and we can use this signed URL as a remote URL to visualize the image in a simple HTML `\u003cimg\u003e` source tag. By adding some CSS styles, we can achieve a tile look for our image viewers.\n\nThe code to visualize all the stored images is below. Replace the current definition of the root route with it.\n\n```javascript\napp.get(\"/\", (req, res) =\u003e {\n  s3.listObjects({ Bucket: bucket }, (err, data) =\u003e {\n    if (err) {\n      console.error(err);\n      res.status(500).json(err);\n      return;\n    }\n\n    let resString = `\n      \u003c!DOCTYPE html\u003e\n      \u003chtml\u003e\n        \u003chead\u003e\n          \u003cstyle\u003e\n            a {\n              text-decoration: none;\n            }\n            img {\n              padding: 6px;\n              width: 250px;\n              height: 250px;\n            }\n            .upload {\n              border: none;\n              padding: 10px 20px;\n              background-color: #1781e3;\n              color: white;\n              font-size: 14px;\n            }\n            .upload:hover{\n              background-color: #0851da;\n            }\n          \u003c/style\u003e\n        \u003c/head\u003e\n        \u003cbody\u003e\n    `;\n\n    data.Contents.forEach(item =\u003e {\n      const params = { Bucket: bucket, Key: item.Key, Expires: 600 };\n      const url = s3.getSignedUrl(\"getObject\", params);\n      resString += `\n        \u003ca href=\"${url}\"\u003e\n          \u003cimg src=\"${url}\" alt=\"${item.Key}\"\u003e\n        \u003c/a\u003e\n      `;\n    });\n\n    resString += `\n          \u003cbr\u003e\n          \u003cbutton class=\"upload\" onclick=\"uploadImage()\"\u003e\n            Upload image\n          \u003c/button\u003e\n          \u003cscript\u003e\n            function uploadImage() {\n              window.location.href = \"/upload\";\n            }\n          \u003c/script\u003e\n        \u003c/body\u003e\n      \u003c/html\u003e\n    `;\n\n    res.send(resString);\n  });\n});\n```\n\nAs before, we use Node.js to send dynamic HTML code. In this code we have embedded some CSS styles. We simply use an `\u003cimg\u003e` tag to visualize each stored image.\n\n## Code\n\nYou can find all the code used in this tutorial [on GitHub](https://github.com/swisscom/dynstrg-howto).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswisscom%2Fdynstrg-howto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fswisscom%2Fdynstrg-howto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswisscom%2Fdynstrg-howto/lists"}