{"id":13994582,"url":"https://github.com/htm-community/skeleton-htmengine-app","last_synced_at":"2026-02-19T21:58:25.586Z","repository":{"id":33692488,"uuid":"37345381","full_name":"htm-community/skeleton-htmengine-app","owner":"htm-community","description":"Bare-bones example of an htmengine application","archived":false,"fork":false,"pushed_at":"2017-01-05T19:52:40.000Z","size":39,"stargazers_count":18,"open_issues_count":7,"forks_count":15,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-08-10T14:16:30.713Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/htm-community.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-06-12T21:27:38.000Z","updated_at":"2022-11-17T05:43:34.000Z","dependencies_parsed_at":"2022-09-13T05:20:51.916Z","dependency_job_id":null,"html_url":"https://github.com/htm-community/skeleton-htmengine-app","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/htm-community%2Fskeleton-htmengine-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/htm-community%2Fskeleton-htmengine-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/htm-community%2Fskeleton-htmengine-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/htm-community%2Fskeleton-htmengine-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/htm-community","download_url":"https://codeload.github.com/htm-community/skeleton-htmengine-app/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227166808,"owners_count":17740984,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-09T14:02:57.618Z","updated_at":"2026-02-19T21:58:25.580Z","avatar_url":"https://github.com/htm-community.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"So, you want to create an application with htmengine?!\n======================================================\n\nThis is a bare-bones, minimal example for building an application on the\n`htmengine` framework.  There are two important steps that must be done\nin order to _use_ htmengine: Configure the application, and setup the\ndatabase.  Once you have completed the following instructions, you will have\na running `htmengine` application.  You may send data to the\ngraphite-compatible custom metrics interface, or send data directly into the\nAMQP exchange, and receive real-time results from the model results exchange,\nor by polling the database.\n\nYou will see in several instances the use of the word \"skeleton\".  That is the\n_name_ of the app in this demonstration.  In all cases, you should replace\n\"skeleton\" with the name that you've chosen for your application.  This value\nis used in several places to segment resources by application, so that multiple\napplications may use the same mysql and rabbitmq instance.\n\n## Before you begin\n\nFirst, this assumes you have a fresh checkout of the `numenta-apps` repository\nat https://github.com/numenta/numenta-apps.  Clone that repository, and install\n`htmengine` in development mode:\n\n```\ncd numenta-apps/htmengine\npython setup.py develop --user\n```\n\nYou'll also need to install `nta.utils`, which is a dependency of `htmengine`:\n\n```\ncd numenta-apps/nta.utils\npython setup.py develop --user\n```\n\n## 1. Create a MySQL database\n\nFor example:\n\n```\nmysql -u root --execute=\"CREATE DATABASE skeleton\"\n```\n\n## 2. Edit configuration\n\n### conf/application.conf\n\nInside `conf/application.conf`, you will find a section called `[repository]`,\nedit the `db`, `host`, `user`, `passwd`, and `port` values to match the\ndatabase created in Step 1.\n\nElsewhere in `conf/application.conf`, replace \"skeleton\" with the name of your\napplication.  That includes `results_exchange_name` under `[metric_streamer]`,\nand `queue_name` in `[metric_listener]`.\n\n### conf/model-checkpoint.conf\n\nInside `conf/model-checkpoint.conf`, specify an absolute path on your\nfilesystem for permanent storage of model checkpoints in `root` of `[storage]`.\n\n### conf/model-swapper.conf\n\nInside `conf/model-swapper.conf`, replace \"skeleton\" with the name of your\napplication in `results_queue`, `model_input_queue_prefix`, and\n`scheduler_notification_queue` of the `[interface_bus]` section.\n\n### conf/supervisord.conf\n\nInside `conf/supervisord.conf`, specify the path to the configuration directory\nin the `environment` value of the `[supervisord]` section.  Specify the path to\nthe `supervisord-base.conf` bundled with the original `htmengine` installation.\nFor example, if you cloned the `numenta-apps` repository at\n`/Users/user/numenta-apps`, the value should be\n`/Users/user/numenta-apps/htmengine/conf/supervisord-base.conf`.\n\n## 3. Set APPLICATION_CONFIG_PATH in your environment\n\n`APPLICATION_CONFIG_PATH` must be set, and include the path to the config files\nmentioned in #2.  For example, if you're in the root of this project:\n\n```\nexport APPLICATION_CONFIG_PATH=`pwd`/conf\n```\n\n## 4. Apply database migrations\n\nAgain, from the root of this project:\n\n```\npython repository/migrate.py\n```\n\n## 5. Start services with supervisor\n\nAgain, from the root of this project:\n\n```\nmkdir -p logs\nsupervisord -c conf/supervisord.conf\n```\n\nAt this point, the core `htmengine` services are running.  You can see the\nsupervisor status at \u003chttp://localhost:9001/\u003e or by running\n`supervisorctl status`.\n\n## Usage\n\nA sample `send_cpu.py` script is included demonstrating how to send data into\n`htmengine` for processing using the `message_bus_connector` API.\n\n```\npython send_cpu.py\n```\n\n`send_cpu.py` will run indefinitely, sending cpu percent samples every 5\nseconds into the custom metrics queue for processing by htmengine.  In practice\nyou should use something like `nohup` to keep this process running in the\nbackground:\n\n```\nnohup python send_cpu.py \u003e send_cpu.stdout 2\u003e send_cpu.stderr \u003c /dev/null \u0026\n```\n\nIf you look in the `metric_data` MySQL table, you should start to see new\nrecords come in.\n\nFor example:\n\n```\n$ mysql -u root skeleton --execute=\"select * from metric_data order by rowid desc limit 5\"\n+----------------------------------+-------+---------------------+--------------+-------------------+---------------+---------------+\n| uid                              | rowid | timestamp           | metric_value | raw_anomaly_score | anomaly_score | display_value |\n+----------------------------------+-------+---------------------+--------------+-------------------+---------------+---------------+\n| 4258abfc6de947609f821095471dd0a2 |    94 | 2015-06-13 04:01:55 |          5.6 |              NULL |          NULL |          NULL |\n| 4258abfc6de947609f821095471dd0a2 |    93 | 2015-06-13 04:01:50 |          7.4 |              NULL |          NULL |          NULL |\n| 4258abfc6de947609f821095471dd0a2 |    92 | 2015-06-13 04:01:45 |          4.1 |              NULL |          NULL |          NULL |\n| 4258abfc6de947609f821095471dd0a2 |    91 | 2015-06-13 04:01:40 |          3.7 |              NULL |          NULL |          NULL |\n| 4258abfc6de947609f821095471dd0a2 |    90 | 2015-06-13 04:01:35 |          4.5 |              NULL |          NULL |          NULL |\n+----------------------------------+-------+---------------------+--------------+-------------------+---------------+---------------+\n```\n\nNotice that only `uid`, `rowid`, `timestamp`, and `metric_value` are populated.\nThe next step will be to explicitly create a model for the `cpu_percent` metric\nso that `raw_anomaly_score` and `anomaly_score` may be populated with anomaly\nscores.\n\n```\npython create_cpu_percent_model.py\n```\n\nThen, check the database again, and you should see anomaly scores:\n\n```\n$ mysql -u root skeleton --execute=\"select * from metric_data order by rowid desc limit 5\"\n+----------------------------------+-------+---------------------+--------------+-------------------+---------------+---------------+\n| uid                              | rowid | timestamp           | metric_value | raw_anomaly_score | anomaly_score | display_value |\n+----------------------------------+-------+---------------------+--------------+-------------------+---------------+---------------+\n| 4258abfc6de947609f821095471dd0a2 |   344 | 2015-06-13 04:23:11 |         10.5 |             0.075 |   0.344578258 |          1000 |\n| 4258abfc6de947609f821095471dd0a2 |   343 | 2015-06-13 04:23:06 |          9.6 |              0.05 |   0.184060125 |          1000 |\n| 4258abfc6de947609f821095471dd0a2 |   342 | 2015-06-13 04:23:01 |          8.4 |              0.05 |    0.11506967 |          1000 |\n| 4258abfc6de947609f821095471dd0a2 |   341 | 2015-06-13 04:22:56 |          9.4 |              0.05 |   0.080756659 |          1000 |\n| 4258abfc6de947609f821095471dd0a2 |   340 | 2015-06-13 04:22:51 |          7.6 |                 0 |   0.044565463 |          1000 |\n+----------------------------------+-------+---------------------+--------------+-------------------+---------------+---------------+\n```\n\nYou can query the `metric` table to get status on the model.  You'll know\neverything is working when `status` is `1`:\n\n```\n$ mysql -u root skeleton --execute=\"select uid, name, description, status from metric where name = 'cpu_percent'\"\n+----------------------------------+-------------+---------------------------+--------+\n| uid                              | name        | description               | status |\n+----------------------------------+-------------+---------------------------+--------+\n| 4258abfc6de947609f821095471dd0a2 | cpu_percent | Custom metric cpu_percent |      1 |\n+----------------------------------+-------------+---------------------------+--------+\n```\n\nAdditionally, you can consume the anomaly results in near realtime with the\nincluded `consume_realtime_results.py` script.\n\n```\n$ python consume_realtime_results.py\nHandling 1 model result(s) for 4258abfc6de947609f821095471dd0a2 - cpu_percent\n4258abfc6de947609f821095471dd0a2 [{u'rowid': 749, u'rawAnomaly': 0.0, u'anomaly': 0.04456546299999997, u'ts': 1434174392.0, u'value': 8.0}]\nHandling 1 model result(s) for 4258abfc6de947609f821095471dd0a2 - cpu_percent\n4258abfc6de947609f821095471dd0a2 [{u'rowid': 750, u'rawAnomaly': 0.0, u'anomaly': 0.04456546299999997, u'ts': 1434174397.0, u'value': 9.6}]\nHandling 1 model result(s) for 4258abfc6de947609f821095471dd0a2 - cpu_percent\n4258abfc6de947609f821095471dd0a2 [{u'rowid': 751, u'rawAnomaly': 0.0, u'anomaly': 0.04456546299999997, u'ts': 1434174402.0, u'value': 7.3}]\nHandling 1 model result(s) for 4258abfc6de947609f821095471dd0a2 - cpu_percent\n4258abfc6de947609f821095471dd0a2 [{u'rowid': 752, u'rawAnomaly': 0.0, u'anomaly': 0.04456546299999997, u'ts': 1434174407.0, u'value': 8.1}]\nHandling 1 model result(s) for 4258abfc6de947609f821095471dd0a2 - cpu_percent\n4258abfc6de947609f821095471dd0a2 [{u'rowid': 753, u'rawAnomaly': 0.0, u'anomaly': 0.04456546299999997, u'ts': 1434174412.0, u'value': 7.0}]\n...\n```\n\n## That's cool, and all, but I want an HTTP API!\n\nSee `webapp.py` for a minimal web service implementation that implements the\nabove described steps in the form of HTTP calls.\n\nFirst, run the web service:\n\n```\npython webapp.py\n```\n\nThen, to create a model, send a `PUT` request to the web service on port 8080.\nThe URI represents the metric name, and you may optionally specify model params\nin the request body.  Using curl on the command line:\n\n```\n$ curl http://localhost:8080/load_average -X PUT -d '{\"min\":0, \"max\":12}' -i\nHTTP/1.1 201 Created\nContent-Type: text/html\nDate: Sat, 13 Jun 2015 21:01:40 GMT\nTransfer-Encoding: chunked\n\nCreated 6e1f199a74274c5cbf9443f4ab4ad94e\n```\n\n**Note** you may create the model at any time during the process.  For example,\nyou may send data (described below) over an extended period of time, and then\ntrigger the creation of the model at a later time.  In the example above, the\n`min` and `max` is included, but that value may not be known.  Sometimes it is\nbetter to omit the min and max and let htmengine choose a value based on the\nhistory of a metric.\n\nTo send data, send a `POST` request to the same URL you used to create the\nmodel.  The body of the request must consist of two values: metric value and\ntimestamp separated by a single space.  The example below sends load average\ndata using curl in an infinite loop at 5 second intervals:\n\n```\n$ while true; do while true; do echo \"`(uptime | awk -F'[a-z]:' '{ print $2}' | awk '{print $1}' | tr -d ',')` `date +%s`\" | curl http://localhost:8080/load_average -X POST -d @-; sleep 5; done; done\nSaved /load_average 1.620000 @ 1434229721\nSaved /load_average 1.730000 @ 1434229726\nSaved /load_average 1.590000 @ 1434229731\nSaved /load_average 1.540000 @ 1434229736\nSaved /load_average 1.420000 @ 1434229741\n```\n\nTo retrieve data, `GET` data from the same original URL:\n\n```\n$ curl http://localhost:8080/load_average\n/load_average 1.7 1434229625 0.0\n/load_average 1.72 1434229628 0.0\n/load_average 1.72 1434229629 0.0\n/load_average 1.72 1434229630 0.0\n/load_average 1.71 1434229704 0.0\n/load_average 1.73 1434229709 0.0\n/load_average 1.62 1434229721 0.0\n/load_average 1.73 1434229726 0.0\n/load_average 1.59 1434229731 0.0\n/load_average 1.54 1434229736 0.0\n/load_average 1.42 1434229741 0.0\n```\n\nThe right-most column in the response is the anomaly score.  The first three\ncolumns are the original data.\n\n**Disclaimer** this web app is only a simple demonstration wrapping htmengine\nin a minimal web service and is not production-ready.\n\n## Final notes\n\nThis demonstration represents only the minimal amount of work to bring an\napplication online using `htmengine`.  Numenta have done a substantial amount\nof work in making much of the concepts demonstrated here robust and suitable\nfor a production workload in our showcase applications Grok, and Taurus.  If\nyou're interested in learning more, please see\n[Grok](https://github.com/numenta/numenta-apps/tree/master/grok),\n[Taurus Engine](https://github.com/numenta/numenta-apps/tree/master/taurus),\nand [Taurus Metric Collectors](https://github.com/numenta/numenta-apps/tree/master/taurus.metric_collectors)\n\nThanks!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhtm-community%2Fskeleton-htmengine-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhtm-community%2Fskeleton-htmengine-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhtm-community%2Fskeleton-htmengine-app/lists"}