{"id":17994481,"url":"https://github.com/rafaelpadilla/tcf-lmo","last_synced_at":"2025-03-26T02:31:07.151Z","repository":{"id":47434190,"uuid":"369288113","full_name":"rafaelpadilla/TCF-LMO","owner":"rafaelpadilla","description":"TCF-LMO is a network made with dedicated modules to process videos and identify the presence of anomalies in frames. It is composed by: dissimilarity model; a differentiable morphology module; temporal consistency; and classification module.","archived":false,"fork":false,"pushed_at":"2024-08-28T01:06:59.000Z","size":28227,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-21T01:41:32.199Z","etag":null,"topics":["anomaly-detection","computer-vision","differentiable-morphology","signal-processing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rafaelpadilla.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-20T17:29:32.000Z","updated_at":"2024-08-28T01:07:03.000Z","dependencies_parsed_at":"2024-10-29T20:37:28.696Z","dependency_job_id":null,"html_url":"https://github.com/rafaelpadilla/TCF-LMO","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaelpadilla%2FTCF-LMO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaelpadilla%2FTCF-LMO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaelpadilla%2FTCF-LMO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaelpadilla%2FTCF-LMO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rafaelpadilla","download_url":"https://codeload.github.com/rafaelpadilla/TCF-LMO/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245576528,"owners_count":20638125,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anomaly-detection","computer-vision","differentiable-morphology","signal-processing"],"created_at":"2024-10-29T20:15:26.423Z","updated_at":"2025-03-26T02:31:02.137Z","avatar_url":"https://github.com/rafaelpadilla.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\nIf you use this code for your research, please consider citing:\n\n``` \n@article{PADILLA2023116969,\ntitle = {Change detection in moving-camera videos with limited samples using twin-CNN features and learnable morphological operations},\njournal = {Signal Processing: Image Communication},\nvolume = {115},\npages = {116969},\nyear = {2023},\nissn = {0923-5965},\ndoi = {https://doi.org/10.1016/j.image.2023.116969},\nurl = {https://www.sciencedirect.com/science/article/pii/S0923596523000516},\nauthor = {Rafael Padilla and Allan F. {da Silva} and Eduardo A.B. {da Silva} and Sergio L. Netto},\n}\n```\nDownload the paper [preprint version](https://github.com/rafaelpadilla/TCF-LMO/raw/main/paper_preprint.pdf) or [published version ](https://authors.elsevier.com/c/1gyvY3I06IgrB4) (available until June, 2023).\n\n\n---------\n\n# Lightweight Change Detection in Moving-Camera Videos Using Twin-CNN Features and Learnable Morphological Operations.\n\n* [Overview](#overview)  \n* [Requirements](#requirements)  \n* [Train the model from scratch](#training)  \n* [Testing](#testing)  \n* [Results](#results)  \n   * [Metrics](#metrics)  \n   * [Frame-level results](#fl-results)  \n   * [Object-level results](#ol-results)  \n* [Acknowledgement](#acknowledgment)   \n  \n\u003ca name=\"overview\"\u003e\u003c/a\u003e  \n## Overview\nThe anomaly detection network consists of five modules as illustrated below:\n\n\u003c!--- scheme ---\u003e\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/rafaelpadilla/dl-smo/blob/main/aux_imgs/pipeline_outputs.png?raw=true\" align=\"center\"/\u003e\u003c/p\u003e\n\nWhere: \n\n(a) aligned reference frame  \n(b) target frame (where the anomaly is present)  \n(c) output of the dissimilarity module (DM)  \n(d) output of the temporal consistency module (TCM)  \n(e) eroded version of (d) computed by differentiable morphology module (MM)  \n(f) eroded version of (e) computed by differentiable morphology module (MM)  \n\n\u003ca name=\"requirements\"\u003e\u003c/a\u003e  \n## Requirements\n\n### Installing packages and libraries:\n\nIf you use conda/anaconda, use the file environment.yml to install the needed packages to run the network:\n`conda env create -f environment.yml`\n\n## Replicating our results\n\nYou can train the model from scratch, or use the pretrained model to evaluate the VDAO testing videos.\n\nOptionally, you could download the full **not aligned** videos from the VDAO official web site: [training set](http://www02.smt.ufrj.br/~tvdigital/database/objects/page_01.html) and [testing set](http://www02.smt.ufrj.br/~tvdigital/database/research/page_01.html).Thus, you could apply alternative alignment/registration techniques and train the DN-LMO model.\n\n\u003ca name=\"training\"\u003e\u003c/a\u003e  \n### A. (Optional) Train the model from scratch:\n\nUse the commands below to download the datasets (training + validation) and train the network:\n\n**1. Download the aligned dataset for training:** `sh download_training_dataset.sh`    \n**2. Download the aligned dataset for validation:** `sh download_testing_dataset.sh`  \n**3. Train the network:** `python train.py --fold 1 --net DM_MM_TCM_CM`  \n*Notice:* Use the argument `--fold` to select which fold to train (options: `--fold 1`, `--fold 2`, `--fold 3`, `--fold 4`, `--fold 5`, `--fold 6`, `--fold 7`, `--fold 8`, `--fold 9`).\n\n\u003ca name=\"testing\"\u003e\u003c/a\u003e  \n### B. Testing\n\n**1. Download the aligned dataset for testing:** `sh download_testing_dataset.sh`  \n**2. Download the pretrained model:** `sh download_pretrained.sh`  \n**3. Evaluate the network:** `sh evaluate.sh`\n\n\u003ca name=\"results\"\u003e\u003c/a\u003e  \n## Results\n\nFor table of results, videos and frames in the testing set, access the folder [/results](https://github.com/rafaelpadilla/TCF-LMO/tree/main/results).  \n\n\u003ca name=\"metrics\"\u003e\u003c/a\u003e  \n### Metrics\n\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand\u003c/summary\u003e\n\nOur results are compared against previous works in the same database using the true positive rate (TPR), false positve rate (FPR) and DIS, which is the minimum distance of an operating point to the point of ideal behaviour of a ROC curve, as illustrated below\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/rafaelpadilla/dl-smo/blob/main/aux_imgs/ROC_curve.png?raw=true\" align=\"center\"/\u003e\u003c/p\u003e\n\nThe best value is obtained when TPR=1 and FPR=0 resulting in a DIS=0, which represents the best possible classification. And it is computed as:\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/rafaelpadilla/dl-smo/blob/main/aux_imgs/eq_DIS.png?raw=true\" align=\"center\"/\u003e\u003c/p\u003e\n\n \u003c/details\u003e\n\n\u003ca name=\"fl-results\"\u003e\u003c/a\u003e\n### Frame-level results \n\n\u003ctable\u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth rowspan=\"2\"\u003e\u003c/th\u003e\n    \u003cth colspan=\"3\"\u003eaverage\u003c/th\u003e\n    \u003cth colspan=\"3\"\u003eoverall\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003cth\u003eTPR\u003c/th\u003e\n    \u003cth\u003eFPR\u003c/th\u003e\n    \u003cth\u003eDIS\u003c/th\u003e\n    \u003cth\u003eTPR\u003c/th\u003e\n    \u003cth\u003eFPR\u003c/th\u003e\n    \u003cth\u003eDIS\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eDAOMC\u003c/td\u003e\n    \u003ctd\u003e0.88\u003c/td\u003e\n    \u003ctd\u003e0.39\u003c/td\u003e\n    \u003ctd\u003e0.49\u003c/td\u003e\n    \u003ctd\u003e0.89\u003c/td\u003e\n    \u003ctd\u003e0.42\u003c/td\u003e\n    \u003ctd\u003e0.43\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eADMULT\u003c/td\u003e\n    \u003ctd\u003e0.76\u003c/td\u003e\n    \u003ctd\u003e0.36\u003c/td\u003e\n    \u003ctd\u003e0.59\u003c/td\u003e\n    \u003ctd\u003e0.78\u003c/td\u003e\n    \u003ctd\u003e0.39\u003c/td\u003e\n    \u003ctd\u003e0.44\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eMCBS\u003c/td\u003e\n    \u003ctd\u003e1.00\u003c/td\u003e\n    \u003ctd\u003e0.83\u003c/td\u003e\n    \u003ctd\u003e0.83\u003c/td\u003e\n    \u003ctd\u003e1.00\u003c/td\u003e\n    \u003ctd\u003e0.98\u003c/td\u003e\n    \u003ctd\u003e0.98\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003emcDTSR\u003c/td\u003e\n    \u003ctd\u003e0.88\u003c/td\u003e\n    \u003ctd\u003e0.25\u003c/td\u003e\n    \u003ctd\u003e0.36\u003c/td\u003e\n    \u003ctd\u003e0.88\u003c/td\u003e\n    \u003ctd\u003e0.28\u003c/td\u003e\n    \u003ctd\u003e0.30\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eCNN+RF\u003c/td\u003e\n    \u003ctd\u003e0.74\u003c/td\u003e\n    \u003ctd\u003e0.25\u003c/td\u003e\n    \u003ctd\u003e0.48\u003c/td\u003e\n    \u003ctd\u003e0.75\u003c/td\u003e\n    \u003ctd\u003e0.27\u003c/td\u003e\n    \u003ctd\u003e0.37\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cb\u003eDL-DMO (ours)\u003c/td\u003e\n    \u003ctd\u003e0.85\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e0.18\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e0.33\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e0.86\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e0.21\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e0.25\u003c/b\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003ca name=\"ol-results\"\u003e\u003c/a\u003e  \n### Object-level results\n  \n\u003ctable\u003e\n\u003cthead\u003e\n  \u003ctr\u003e\n    \u003cth rowspan=\"2\"\u003e\u003c/th\u003e\n    \u003cth colspan=\"3\"\u003eaverage\u003c/th\u003e\n    \u003cth colspan=\"3\"\u003eoverall\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003cth\u003eTPR\u003c/th\u003e\n    \u003cth\u003eFPR\u003c/th\u003e\n    \u003cth\u003eDIS\u003c/th\u003e\n    \u003cth\u003eTPR\u003c/th\u003e\n    \u003cth\u003eFPR\u003c/th\u003e\n    \u003cth\u003eDIS\u003c/th\u003e\n  \u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eDAOMC\u003c/td\u003e\n    \u003ctd\u003e0.81\u003c/td\u003e\n    \u003ctd\u003e0.42\u003c/td\u003e\n    \u003ctd\u003e0.53\u003c/td\u003e\n    \u003ctd\u003e0.82\u003c/td\u003e\n    \u003ctd\u003e0.42\u003c/td\u003e\n    \u003ctd\u003e0.45\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eADMULT\u003c/td\u003e\n    \u003ctd\u003e0.70\u003c/td\u003e\n    \u003ctd\u003e0.29\u003c/td\u003e\n    \u003ctd\u003e0.54\u003c/td\u003e\n    \u003ctd\u003e0.72\u003c/td\u003e\n    \u003ctd\u003e0.29\u003c/td\u003e\n    \u003ctd\u003e0.40\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eMCBS\u003c/td\u003e\n    \u003ctd\u003e0.88\u003c/td\u003e\n    \u003ctd\u003e0.83\u003c/td\u003e\n    \u003ctd\u003e0.86\u003c/td\u003e\n    \u003ctd\u003e0.89\u003c/td\u003e\n    \u003ctd\u003e0.83\u003c/td\u003e\n    \u003ctd\u003e0.84\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003emcDTSR\u003c/td\u003e\n    \u003ctd\u003e0.86\u003c/td\u003e\n    \u003ctd\u003e0.29\u003c/td\u003e\n    \u003ctd\u003e0.39\u003c/td\u003e\n    \u003ctd\u003e0.86\u003c/td\u003e\n    \u003ctd\u003e0.29\u003c/td\u003e\n    \u003ctd\u003e0.32\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cb\u003eDL-DMO (ours)\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e0.85\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e0.22\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e0.35\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e0.86\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e0.22\u003c/b\u003e\u003c/td\u003e\n   \u003ctd\u003e\u003cb\u003e0.26\u003c/b\u003e\u003c/b\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003ca name=\"acknowledgment\"\u003e\u003c/a\u003e \n## Acknowledgement\n\nMost of the experimental results reported in this work were obtained with a Titan X Pascal board gently donated by the NVIDIA Corporation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frafaelpadilla%2Ftcf-lmo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frafaelpadilla%2Ftcf-lmo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frafaelpadilla%2Ftcf-lmo/lists"}