{"id":20780739,"url":"https://github.com/farrajota/human_activity_torch","last_synced_at":"2025-07-10T17:32:37.158Z","repository":{"id":202488787,"uuid":"93498980","full_name":"farrajota/human_activity_torch","owner":"farrajota","description":"Human activity recognition using human pose information to aid classification.","archived":false,"fork":false,"pushed_at":"2017-10-01T21:31:08.000Z","size":1133,"stargazers_count":13,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-30T20:34:25.527Z","etag":null,"topics":["activity-recognition","human","human-activity-recognition","lstm","pose","torch7","ucf-sports"],"latest_commit_sha":null,"homepage":"","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/farrajota.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-06-06T09:11:21.000Z","updated_at":"2023-11-15T11:33:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"9384b4ce-f48b-4dce-a698-bcef78de1503","html_url":"https://github.com/farrajota/human_activity_torch","commit_stats":null,"previous_names":["farrajota/human_activity_torch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/farrajota/human_activity_torch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman_activity_torch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman_activity_torch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman_activity_torch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman_activity_torch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/farrajota","download_url":"https://codeload.github.com/farrajota/human_activity_torch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman_activity_torch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264619384,"owners_count":23638447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activity-recognition","human","human-activity-recognition","lstm","pose","torch7","ucf-sports"],"created_at":"2024-11-17T13:39:01.280Z","updated_at":"2025-07-10T17:32:32.149Z","avatar_url":"https://github.com/farrajota.png","language":"Lua","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Human Activity Recognition using Torch7\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE.md)\n\nRecognize human activities of individuals using human body joints on video sequences.\n\nThis method combines features from a ResNet 50 with human body joint prediction to improve performance in human action recognition applications. It uses a modifed [Stacked Hourglass Network](https://arxiv.org/abs/1603.06937) trained on the LSP+MPII datasets to predict human body joints for a sequence of images.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"img/pose_resnet_architecture.png\" alt=\"Network architecture\" height=\"50%\" width=\"50%\"\u003e\u003c/p\u003e\n\nThe proposed network takes as input a sequence of images with centered persons which are then processed by two parallel networks that produce image features and 2D heatmaps of human body joint which are then fed to a couple of LSTM layers to compute the classification score for a video sequence.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"img/intro_feats_v2.png\" alt=\"inputs, features and output\" height=\"80%\" width=\"80%\"\u003e\u003c/p\u003e\n\n\n\u003e Note: This was only tested on the UCF Sports dataset because it had bounding box annotations of humans available.\n\n\n## Installation\n\n### Requirements\n\n- NVIDIA GPU with compute capability 3.5+ (2GB+ ram)\n- [Torch7](http://torch.ch/docs/getting-started.html)\n- [dbcollection](https://github.com/dbcollection/dbcollection-torch7)\n\n### Packages/dependencies installation\n\nTo use this example code, some packages are required for it to work.\n\n```bash\nluarocks install loadcaffe\nluarocks install cudnn\nluarocks install display\n```\n\n### dbcollection\n\nTo install the dbcollection package do the following:\n\n- install the Python module (Python\u003e=2.7 and \u003e=3.5).\n\n    ```\n    pip install dbcollection==0.1.7\n    ```\n\n    or\n\n    ```\n    conda install -c farrajota dbcollection==0.1.7\n    ```\n\n- install the Lua/Torch7 dbcollection wrapper:\n\n    1. download the Lua/Torch7 git repo to disk.\n\n    ```\n    git clone https://github.com/dbcollection/dbcollection-torch7\n    ```\n\n    2. install the package.\n    ```\n    cd dbcollection-torch7 \u0026\u0026 luarocks make\n    ```\n\n\u003e For more information about the dbcollection package see [here](https://github.com/dbcollection/dbcollection-torch7).\n\n\n# Getting started\n\n## Download/setup this repo\n\nTo start using this repo you'll need to clone this repo into your home directory:\n\n```\ngit clone https://github.com/farrajota/human_activity_torch\n```\n\nBy default, this repo path points to `~/human_activity_torch/` when running the scripts. If you want to use this repo with another path/directory, you'll need to edit the `projectdir.lua` file in the repo's root dir and set the proper path to where it was cloned.\n\nNext, the necessary data for this code to run is needed to be set. Download the pre-trained models by ruining the script `download_data.lua` script in the `download/` dir:\n\n```\nth download/download_data.lua\n```\n\nThe pre-trained models uses the `cudnn` library, so make sure this package is installed on your system before proceeding any further.\n\n\n## Train\n\nTo train a network, simply run the `train.lua` script to start optimizing the proposed network (ResNet50 + Pose Hms + LSTM) on some default options. The default configurations are detailed in the the paper [TODO- insert paper link]().\n\nTo train a network, there are several input arguments to configure the training process. The most important ones are the following (the rest you can leave as defaults):\n\n- `-expID \u003cexp_name\u003e`: experiment id to store all the metadata, logs and model snapshots to disk under the `exp/` dir in the repo's main folder;\n- `-dataset \u003cdataset_name\u003e`: indicates which dataset to train on (default=`ucf_sports`); **(warning: for now, the ucf_sports is the only available dataset to train/test on)**\n- `-data_dir \u003cpath/to/dataset/files\u003e`: Path to store the dataset\\'s data files in case you haven't configured the `ucf_sports` previously in `dbcollection`. Specify a path if you want to store the data files into a specific folder.\n- `-expDir \u003cpath/to/folder\u003e`: specifies which folder to store the experiment directory. By default, it uses the `exp/` dir in the repo's main directory. (optional)\n- `-netType \u003cnet_type_name\u003e`: specifies which network to train. Options: vgg16-lstm | hms-lstm | vgg16-hms-lstm | vgg16-convnet | hms-convnet | vgg16-hms-convnet.\n- `-trainIters \u003cnum_iters\u003e`: Number of train iterations per epoch (default=300).\n- `-testIters \u003cnum_iters\u003e`: Number of test iterations per epoch (default=100).\n- `-nEpochs \u003cnum_epochs\u003e`: Total number of epochs to runh (default=15).\n- `-batchSize \u003csize\u003e`: Mini-batch size (default=4).\n- `-seq_length \u003csize\u003e`: Sequence length (number of frames per window) (default=10).\n\nFor more information about the available options, see the `options.lua` file or type `th train.lua -help`\n\n\u003e Note: In the `scripts/` dir there are several scripts to train + test other networks with different architectures and configurations. Check them out to see more training configurations specs.\n\n## Test\n\nEvaluating a network is done by running the `test.lua` script with some input arguments. To do so, you need to provide the following input arguments:\n\n- `-expID \u003cexp_name\u003e`: experiment id that contains the model's snapshots + logs;\n- `-loadModel \u003cpath/to/network.t7\u003e`: if this flag is used, it bypasses the `-expID` flag and loads a specific file;\n- `-dataset \u003cdataset_name\u003e`: indicates which dataset to test on (default=`ucf_sports`);\n- `-test_progressbar false`: displays text information per iteration instead of a progress bar (optional);\n- `-test_load_best false`: if true, it loads the best accuracy model (if exists, optional).\n\nThe results of the test, when finished, are displayed on screen and stored in the folder of the network file with the name `Evaluation_full.log`.\n\n\n## Results\n\n| Method | Accuracy (%) |\n| --- | --- |\n| Rodriguez et al. [1] | 69.2 |\n| Lan et al. [2] | 83.7 |\n| **Jones et al. [3]** | **93.5** |\n|  |  |\n| PoseNet + LSTM | 59.6 |\n| ResNet + LSTM | 80.9 |\n| **Resnet + PoseNet + LSTM** | **87.2** |\n\n\n[1] - Rodriguez, M.D., Ahmed, J., Shah, M.: Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: CVPR, pp. 1–8. IEEE (2008)\n\n[2] - Lan, T., Wang, Y., Mori, G.: Discriminative figure-centric models for joint action localization and recognition. In: ICCV, pp. 2003–2010. IEEE (2011)\n\n[3] - Jones, S., Shao, L., Zhang, J., Liu, Y.: Relevance feedback for real-world human action\nretrieval. Pattern Recognition Letters 33(4), 446–452 (2012)\n\n## License\n\nMIT license (see the [LICENSE](LICENSE.md) file)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarrajota%2Fhuman_activity_torch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffarrajota%2Fhuman_activity_torch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarrajota%2Fhuman_activity_torch/lists"}