{"id":26509860,"url":"https://github.com/nna301023/stackconvprun","last_synced_at":"2025-03-21T01:37:43.795Z","repository":{"id":210477052,"uuid":"726653435","full_name":"NnA301023/StackConvPrun","owner":"NnA301023","description":"DenseNet \u0026 Resnet as Backbone to Perform Lahan Gambut Classification with Self-Attention Pruning Mechanism","archived":false,"fork":false,"pushed_at":"2024-01-22T04:36:18.000Z","size":1225,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-01-22T05:34:49.609Z","etag":null,"topics":["attention-mechanism","pruning-optimization","streamlit-webapp"],"latest_commit_sha":null,"homepage":"https://stackconvprun-erika.streamlit.app/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NnA301023.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-12-03T01:01:49.000Z","updated_at":"2023-12-04T08:30:30.000Z","dependencies_parsed_at":"2023-12-20T18:35:39.114Z","dependency_job_id":null,"html_url":"https://github.com/NnA301023/StackConvPrun","commit_stats":null,"previous_names":["nna301023/stackconvprun"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NnA301023%2FStackConvPrun","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NnA301023%2FStackConvPrun/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NnA301023%2FStackConvPrun/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NnA301023%2FStackConvPrun/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NnA301023","download_url":"https://codeload.github.com/NnA301023/StackConvPrun/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244722690,"owners_count":20499151,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention-mechanism","pruning-optimization","streamlit-webapp"],"created_at":"2025-03-21T01:37:43.408Z","updated_at":"2025-03-21T01:37:43.790Z","avatar_url":"https://github.com/NnA301023.png","language":"Python","readme":"# StackConvPrun\nDenseNet \u0026amp; Resnet as Backbone to Perform Lahan Gambut Classification with Self-Attention Pruning Mechanism\n\n# Initial Research Problem\n\nThe utilization of a concatenation layer based on two backbone architectures, such as DenseNet and ResNet, introduces additional computational overhead during both training and inference. In this study, we propose the implementation of pruning based on a self-attention mechanism scoring. This pruning mechanism aims to selectively remove layers in the backbone/head section, reducing computational requirements. The decision to prune a layer is determined by comparing its attention mechanism score to the standard deviation of the total attention mechanism scores.\n\n*note: This approach introduces variability in the pruned layers across different use cases, and we address this by initializing random variables for each layer.\n\n# Conclusion\n\nBased on the evaluation results, we successfully reduced the training time of the baseline model from 30 minutes per epoch to just 1 minute. While the evaluation results show a trade-off, the achieved improvements in computational efficiency are deemed acceptable.\n\n# Evaluation Result\n\n## Before Pruning\n![](images/Evaluation%20Metrics%20Before%20Pruning%20Attention.png)\n\n## After Pruning\n![](images/Evaluation%20Metrics%20After%20Pruning%20Attention.png)","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnna301023%2Fstackconvprun","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnna301023%2Fstackconvprun","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnna301023%2Fstackconvprun/lists"}