{"id":20841613,"url":"https://github.com/iotexproject/pebble-backend","last_synced_at":"2025-08-10T13:12:52.075Z","repository":{"id":97187839,"uuid":"283058987","full_name":"iotexproject/pebble-backend","owner":"iotexproject","description":"Cloud backend of Pebble","archived":false,"fork":false,"pushed_at":"2021-03-18T03:53:07.000Z","size":3777,"stargazers_count":10,"open_issues_count":2,"forks_count":7,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-05-08T22:16:56.252Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iotexproject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-28T01:07:34.000Z","updated_at":"2024-10-13T21:36:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"c9deb267-9ece-4d8b-a5b2-e416ac6ff238","html_url":"https://github.com/iotexproject/pebble-backend","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/iotexproject/pebble-backend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iotexproject%2Fpebble-backend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iotexproject%2Fpebble-backend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iotexproject%2Fpebble-backend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iotexproject%2Fpebble-backend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iotexproject","download_url":"https://codeload.github.com/iotexproject/pebble-backend/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iotexproject%2Fpebble-backend/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269729252,"owners_count":24465795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-10T02:00:08.965Z","response_time":71,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-18T01:20:57.117Z","updated_at":"2025-08-10T13:12:52.067Z","avatar_url":"https://github.com/iotexproject.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pebble-backend\n\n## Quick Start\n### Prerequisites\n0. Find a Linux Machine\n\nTo run the backend services, you will need a machine with least 2CPU and 4GB memory and runs Debian or Ubuntu. You could grab one from AWS or GCP; if you want to use your local PC/laptop, make sure it has a public IP (e.g., using ngrok), where pebble can send data to.\n\n1. [Install Docker CE](https://docs.docker.com/engine/installation/)\n\n2. [Install Docker Compose](https://docs.docker.com/compose/install/)\n\n3. (Optional) Add Current User to `Docker` Group\n\nAdding the the current user group (e.g., `ubuntu`) to the `docker` group will allow you to use `docker` command without `sudo`.\n\n`sudo groupadd docker; sudo usermod -aG docker $USER`\n\nTo make sure it works\n```\ncat /etc/group | grep docker\n\n```\nshould return something like `docker:x:999:ubuntu`.\n\n4. Install SDK for MQTT\n\n```\nsudo apt-get update\nsudo apt install python3-pip\npip3 install AWSIoTPythonSDK\n```\n\n5. Make `git clone` Work\nConfig your `git`, e.g., with the correct SSH key, to make sure `git clone` will work properly.\n\n### Start Pebble Backend\n\n1. Run `./setup-dev.sh` to download all docker images and code repos first and then run `./start-dev.sh` to start the service. After that make sure everything is up and running by `docker ps` and you should see something like below:\n```\n:~/pebble-backend-master$ docker ps\nCONTAINER ID        IMAGE                            COMMAND                  CREATED             STATUS              PORTS                                  \n                                  NAMES\n116b6aa864c8        minio/minio:latest               \"/usr/bin/docker-ent…\"   14 minutes ago      Up 14 minutes       0.0.0.0:9000-\u003e9000/tcp                 \n                                  docker-compose_minio_1\na651095d850d        iotex-hmq:local                  \"/hmq -c /config/con…\"   14 minutes ago      Up 14 minutes       0.0.0.0:1884-\u003e1883/tcp                 \n                                  docker-compose_hmq_1\na67d65f71c76        thingsboard/tb-gateway:latest    \"/bin/sh ./start-gat…\"   14 minutes ago      Up 14 minutes                                              \n                                  docker-compose_thingsboard-gateway_1\n9ce61f993ca5        iotex-blockchain-data:local      \"pebble-data-contain…\"   14 minutes ago      Up 14 minutes                                              \n                                  docker-compose_api-server_1\ncbb292f45664        thingsboard/tb-postgres:latest   \"start-tb.sh\"            14 minutes ago      Up 14 minutes       0.0.0.0:1883-\u003e1883/tcp, 0.0.0.0:5683-\u003e5\n683/udp, 0.0.0.0:8080-\u003e9090/tcp   docker-compose_thingsboard_1\n```\n\n2. Make sure port `1884` is exposed for MQTT and port `8080` is exposed for Thingsboard, e.g., `telnet 1.2.3.4 1884` and `telnet 1.2.3.4 8080`.\n\n3. Login in Thingsboard via http://1.2.3.4:8080/ with the default username/password: tenant@thingsboard.org/tenant and change your password after login.\n\n4. Create Thingsboard Gateway as below\n![](images/create-gateway-1.jpg)\n\n![](images/create-gateway-2.jpg)\n\n![](images/create-gateway-3.jpg)\n\n5. Coopy the `Access Token` like below and modify the config of the gateway:\n\n![](images/create-gateway-4.jpg)\n\n```\nsudo vim ~/pebble-var/conf/tb-gateway/conf/tb_gateway.yaml\n```\nReplace accessToken: xxxxxxxxxxxxx with the newly copied token\n```\nthingsboard:\n  host: thingsboard\n  port: 1883\n  security:\n    accessToken: xxxxxxxxxxxxx\nstorage:\n  type: memory\n  read_records_count: 100\n  max_records_count: 100000\n\nconnectors:\n  - name: MQTT Broker Connector\n    type: mqtt\n    configuration: mqtt.json\n```\n\n6. Restart the gateway service\n```\ndocker restart docker-compose_thingsboard-gateway_1\n```\n\n7. Congrats! You already setup the pebble backend which is ready to receive pebble data!\n\n## Visualize Pebble Data on Thingsboard\nTo visualize the data, the easiest way is\n1. generate mock data and inject it to pebble backend\n2. use the predefined dashboard to see the data\n\n### 1. Inject Mock Data\nRun the following on the same machine that runs the backend:\n```\ncd scripts\n./mock-dev.sh\n```\nThis script continously produce data points according to pebble spec and inject it into the local `1884` port. If you run into an issue, e.g., not see data flow in, you can run directly `python3 ./run-dev.py -e localhost -p 1884 -id device/pebble-1/data -pf ../data/sample.txt` to debug.\n\n### 2. Import Predefined Widgets bundles\n![](images/import-widget-1.jpg)\n\ndrop example/widgets-bundles/custom.json to the box, and click \"import\"\n\n![](images/import-widget-2.jpg)\n\n### 3. Import Predefined Dashboard\n![](images/import-dashboard-1.jpg)\n\ndrop example/dashboard/pebble_template.json to the box, and click \"import\"\n\n![](images/import-dashboard-2.jpg)\n\n![](images/import-dashboard-3.jpg)\n\nModify\n![](images/import-dashboard-4.jpg)\n\nSet to pebble-1\n![](images/import-dashboard-5.jpg)\n\n## Other Tools\n### Mock tool\n- scripts/mock.sh: Start 50 clients to send messages to mqtt in the background.\n- scripts/stop-mock.sh: Stop all running clients.\n\n#### Run With More Options\nrun.py is a command to start a client, accepting parameters:\n```\n-e | --endpoint: Your AWS IoT custom endpoint\n-r | --rootCA: Root CA file path\n-c | --cert: Certificate file path\n-k | --key: Private key file path\n-p | --port: Port number override\n-w | --websocket: Use MQTT over WebSocket\n-id | --clientId: Targeted client id\n-pb | --publish: Publish payload\n-pf | --publish-file: Publish payload with file\n```\n\nrun.py will send a message every 30 seconds\n\nThe message will be stored in the 'pebble-store' bucket in s3, like:\n```\npebble-store/pebble-(1,2,3...50)/\u003ctimestamp\u003e\n```\n\n### Import Predifined RULE CHAIN\n![](images/import-rule-1.jpg)\n\ndrop example/thingsboard-rule/pebble.json to the box\n![](images/import-rule-2.jpg)\n\nApply\n![](images/import-rule-3.jpg)\n\nBack to here and make it to the root\n![](images/import-rule-4.jpg)\n\n## Setup Pebble Backend on AWS\nhttps://iotex.larksuite.com/docs/docuswsC2fyQNSH4fwdahKka5Rr#hbyvmo\n\n### Our data flow is\n```\nDevice(SDK) --\u003e aws iot --\u003e s3\n                       |\n                       + --\u003e thingsboard gateway --\u003e thingsboard\n                                                         |\n                                                         +--\u003e iotex blockchain\n```\n\n\n### Start\nMake directories for start thingsboard and thingsboard gateway\n```\nmkdir ~/{data,logs}\nmkdir -p ~/conf/keys\nmdkir -p ~/conf/tb-gateway/{conf,extensions,logs}\n```\n\nUse the file configs/docker-compose/docker-compose.yml\n```\ncd configs/docker-compose/\ndocker-compose up -d\n```\n\n### Configure gateway\nLogin Thingsboard as tenant and create a gateway\n\nCopy it \u0026lsquo;s token\n\nSet it to configs/tb-gateway/tb_gateway.yaml:\n```\nthingsboard.security.accessToken=\u003ctoken\u003e\n```\n\nRestart the thingsboard gateway\n```\ncd configs/docker-compose/\ndocker-compose restart\n```\n\n(Optional)\n\nAfter startup, some default configuration files will be generated\n\nIf you need to modify more, you can refer to the [official document](https://thingsboard.io/docs/iot-gateway/configuration/)\n\n## Integration with IoTeX blockchain\nIn Thingsboard, we use \"rule of thingsboard\" to send messages to the blockchain.\nAs shown\n\n![](images/rule.jpg)\n\n- First, switch with \"message type\":\n\nA message enters the chain from \"input\" and then into the node named \"message type CommonData\". If the \"message type\" is \"Post Telemetry\", it will be transmitted to the \"Script FilterGateway\" node. The \"post telemetry\" type includes device telemetry data.\n\n![](images/messageType.jpg)\n\n- Then, swicth with device type:\n\nA message enters the \"script FilterGateway\" from \"message type CommonData\". If the \"deviceType\" is not \"Gateway\"(Because the data entering thingsboard comes through thingsboard gateway, there is data of type \"Gateway\", which needs to be filtered out), it will be transmitted to the \"rest api call Push Blockchain\" node. The \"not Gateway\" and the \"post telemetry\" are equivalent to all devices.\n\n![](images/deviceType.jpg)\n\n- Last, Push data to the blockchain:\nA message enters the \"rest api call Push Blockchain\" from upstream. There are all of devices' post telemetries in the node.Then it will use the url to call the api server, and the variable \"deviceType\" is a metadata field. It comes from the configuration of Thingsboard Gateway.\n\n![](images/apiCurl.jpg)\n\nThe Api server is a proxy for rpc-call the IOTEX server. Finally put the data on the blockchain.\n\nThe request:\n\n```\nHTTP 1.1 POST /api/v1/topic/{topic}/data\nbody is json containing device telemetry data\n```\n\nwill be sent to a deployed contract address, and then executed by the contract to do more...\nThe important code is [iotexproject/pebble-data-container](https://github.com/iotexproject/pebble-data-container/blob/master/blockchain/put.go#L68)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiotexproject%2Fpebble-backend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiotexproject%2Fpebble-backend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiotexproject%2Fpebble-backend/lists"}