{"id":18852564,"url":"https://github.com/quva-lab/timeception","last_synced_at":"2025-08-01T03:09:54.396Z","repository":{"id":69447551,"uuid":"182028686","full_name":"QUVA-Lab/timeception","owner":"QUVA-Lab","description":"Please check the updated repository https://github.com/noureldien/timeception","archived":false,"fork":false,"pushed_at":"2021-03-30T08:28:54.000Z","size":348,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-27T23:23:24.945Z","etag":null,"topics":["action-recognition","computer-vision","convolutional-neural-networks","temporal-models"],"latest_commit_sha":null,"homepage":"https://github.com/noureldien/timeception","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/QUVA-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-18T06:24:43.000Z","updated_at":"2021-03-30T08:28:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"3366f8ff-b7a8-4ed6-83b4-dd07f06b8477","html_url":"https://github.com/QUVA-Lab/timeception","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QUVA-Lab%2Ftimeception","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QUVA-Lab%2Ftimeception/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QUVA-Lab%2Ftimeception/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QUVA-Lab%2Ftimeception/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/QUVA-Lab","download_url":"https://codeload.github.com/QUVA-Lab/timeception/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248860218,"owners_count":21173342,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["action-recognition","computer-vision","convolutional-neural-networks","temporal-models"],"created_at":"2024-11-08T03:40:34.541Z","updated_at":"2025-04-14T10:10:22.998Z","avatar_url":"https://github.com/QUVA-Lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## New and Updated Repository\nhttps://github.com/noureldien/timeception\n\nhttps://noureldien.com/research/timeception/\n\n## Timeception for Complex Action Recognition\nThis code repository is the implementation for the paper [Timeception for Complex Action Recognition](https://arxiv.org/abs/1812.01289).\nWe provide the implementation for 3 different libraries: `keras`, `tensorflow` and `pytorch`.\n\n![Timeception for Complex Action Recognition](./data/assets/timeception_layer.jpg \"Timeception Block\")\n\n### Citation\nPlease consider citing this work using this BibTeX entry\n```bibtex\n@inproceedings{hussein2018timeception,\n  title     = {Timeception for Complex Action Recognition},\n  author    = {Hussein, Noureldien and Gavves, Efstratios and Smeulders, Arnold WM},\n  booktitle = {arxiv},\n  year      = {2018}\n}\n```\n\n### How to Use?\n\n###### Keras\n\nUsing `keras`, we can define `timeception` as a sub-model.\nThen we use it along with another model definition.\nFor example, here we define 4 `timeception` layers followed by a `dense` layer for classification.\n\n```python\nfrom keras import Model\nfrom keras.layers import Input, Dense\nfrom nets.layers_keras import MaxLayer\nfrom nets.timeception import Timeception\n\n# define the timeception layers\ntimeception = Timeception(1024, n_layers=4)\n\n# define network for classification\ninput = Input(shape=(128, 7, 7, 1024))\ntensor = timeception(input)\ntensor = MaxLayer(axis=(1, 2, 3))(tensor)\noutput = Dense(100, activation='softmax')(tensor)\nmodel = Model(inputs=input, outputs=output)\nmodel.summary()\n```\n\nThis results in the model defined as:\n\n```\nLayer (type)  Output Shape              Param #   \n================================================\n(InputLayer)  (None, 128, 7, 7, 1024)   0         \n(Timeception) (None, 8, 7, 7, 2480)     1494304   \n(MaxLayer)    (None, 2480)              0         \n(Dense)       (None, 100)               248100    \n================================================\nTotal params: 1,742,404\n```\n\n###### Tensorflow\n\nUsing `tensorflow`, we can define `timeception` as a list of nodes in the computational graph.\nThen we use it along with another model definition.\nFor example, here a functions defines 4 `timeception` layers.\nIt takes the input tensor, feedforward it to the `timeception` layers and return the output tensor `output`.\n\n```python\nimport tensorflow as tf\nfrom nets import timeception\n\n# define input tensor\ninput = tf.placeholder(tf.float32, shape=(None, 128, 7, 7, 1024))\n\n# feedforward the input to the timeception layers\ntensor = timeception.timeception_layers(input, n_layers=4)\n\n# the output is (?, 8, 7, 7, 2480)\nprint (tensor.get_shape())\n```\n\n###### PyTorch\n\nUsing `pytorch`, we can define `timeception` as a module.\nThen we use it along with another model definition.\nFor example, here we define 4 `timeception` layers followed by a `dense` layer for classification..\n\n```python\nimport numpy as np\nimport torch as T\nfrom nets import timeception_pytorch\n\n# define input tensor\ninput = T.tensor(np.zeros((32, 1024, 128, 7, 7)), dtype=T.float32)\n\n# define 4 layers of timeception\nmodule = timeception_pytorch.Timeception(input.size(), n_layers=4)\n\n# feedforward the input to the timeception layers \ntensor = module(input)\n\n# the output is (32, 2480, 8, 7, 7)\nprint (tensor.size())\n```\n\n### Installation\nWe use python 2.7.15, provided by Anaconda 4.6.2, and we depend on the following python packages.\n- Keras 2.2.4\n- Tensorflow 1.10.1\n- PyTorch 1.0.1\n\n### Training\n\n### Testing\n\n### Fine-tuning\n\n### Pretrained Models\n\n#### Charades\nWe will add all pretrained models for Charades by the end of April.\nFor testing, start with the script `./scripts/test_charades_timeception.sh`.\nIn order to change which baseline is uses for testing, set the `-- config-file` using on of the following options.\n\n###### 2D-ResNet-152\nTimeception on top of 2D-ResNet-152 as backnone.\n\n|  Config File | Backbone | TC Layers | Frames  | mAP (%)  | Model |\n|---|:---:|:---:|:---:|:---:|:---:|\n| [charades_r2d_tc3_f32.yaml](./configs/charades_r2d_tc3_f32.yaml)     | R2D   | 3 | 32  | 30.37  | [Link](./data/charades/charades_r2d_tc3_f32.pkl)   |\n| [charades_r2d_tc3_f64.yaml](./configs/charades_r2d_tc3_f64.yaml)     | R2D   | 3 | 64  | 31.25  | [Link](./data/charades/charades_r2d_tc3_f64.pkl)   |\n| [charades_r2d_tc4_f128.yaml](./configs/charades_r2d_tc4_f128.yaml)   | R2D   | 4 | 128 | 31.82  | [Link](./data/charades/charades_r2d_tc4_f128.pkl)  |\n\n###### I3D\nTimeception on top of ResNet-152 as backnone.\n\n|  Config File | Backbone | TC Layers | Frames  | mAP (%)  | Model |\n|---|:---:|:---:|:---:|:---:|:---:|\n| [charades_i3d_tc3_f256.yaml](./configs/charades_i3d_tc3_f256.yaml)    | I3D  | 3 | 256  | 33.89  | [Link](./data/charades/charades_i3d_tc3_f256.pkl)   |\n| [charades_i3d_tc3_f512.yaml](./configs/charades_i3d_tc3_f512.yaml)    | I3D  | 3 | 512  | 35.46  | [Link](./data/charades/charades_i3d_tc3_f512.pkl)   |\n| [charades_i3d_tc4_f1024.yaml](./configs/charades_i3d_tc4_f1024.yaml)  | I3D  | 4 | 1024 | 37.20  | [Link](./data/charades/charades_i3d_tc4_f1024.pkl)  |\n\n###### 3D-ResNet-100\nTimeception on top of 3D-ResNet-100 as backnone.\n\n\n|  Config File | Backbone | TC Layers | Frames  | mAP (%)  | Model |\n|---|:---:|:---:|:---:|:---:|:---:|\n| [charades_r3d_tc4_f1024.yaml](./configs/charades_r3d_tc4_f1024.yaml)  | R3D  | 4 | 1024 |  41.1  | [Link](./data/charades/charades_r3d_tc4_f1024.pkl)  |\n\n\n#### Kinetics 400\nWe will add all pretrained models for Kinetics 400 by the end of June.\n\n### License\nThe code and the models in this repo are released under the GNU 3.0 [LICENSE](LICENSE).\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquva-lab%2Ftimeception","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquva-lab%2Ftimeception","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquva-lab%2Ftimeception/lists"}