Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sjtu-tes/awesome-sjtu-tes
SJTU Technology Engage Square
https://github.com/sjtu-tes/awesome-sjtu-tes
List: awesome-sjtu-tes
aigc awesome huggingface ml4co motion paper security sjtu space stable-diffusion website
Last synced: 3 months ago
JSON representation
SJTU Technology Engage Square
- Host: GitHub
- URL: https://github.com/sjtu-tes/awesome-sjtu-tes
- Owner: SJTU-TES
- Created: 2024-03-06T02:15:21.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-04-09T13:41:50.000Z (9 months ago)
- Last Synced: 2024-09-24T12:59:16.269Z (3 months ago)
- Topics: aigc, awesome, huggingface, ml4co, motion, paper, security, sjtu, space, stable-diffusion, website
- Language: Python
- Homepage: https://heatingma.cn/sjtutes.html
- Size: 77.3 MB
- Stars: 6
- Watchers: 0
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- ultimate-awesome - awesome-sjtu-tes - SJTU Technology Engage Square. (Other Lists / Monkey C Lists)
README
## [Introduction](#introduction)
In recent years, technology has been increasingly subdivided into numerous fields, each of which has witnessed a proliferation of remarkable achievements. However, limited time and resources often confine individuals to focus on only a few domains, or even specific branches within a particular field.
In order to alleviate the impact of rising barriers in various domains on beginners, and to enable them to quickly experiment and set up application environments, we have developed the **``SJTU-TES``** (Shanghai Jiao Tong University Technology Engagement Square) platform.
Through this platform, users can gain insights into cutting-edge research across different domains and conveniently establish development or experimental environments using our interactive space , reproducible repositories , and testing datasets .
🔥 We mark work contributed by **SJTU-TES** with ⭐.
🔥 We have provided a demonstration video of the ``sjtu_tes space`` [here](https://github.com/SJTU-TES/awesome-sjtu-tes-demo).
🔥 We provide Chinese [Requirement Document](docs/文档/需求文档.md), [Design Document](docs/文档/分析文档.md), [Testing Document](docs/文档/测试文档.md) and [Deployment Document](docs/文档/部署文档.md)
🔥 We primarily use the following icons to indicate the organization of each repository.
**Paper Link** 
The corresponding published paper of the work, where "xxxx" refers to the name of the conference or journal in which it was published, and "arXiv" denotes the preprint version.
**Github Link** 
The corresponding github link of the work.
**Pretrained Link** 
The storage location of the pre-trained files for this repository (usually hosted on [Hugging Face](https://huggingface.co/) or [Google Drive](https://drive.google.com/)).
**Website Link** 
The webpage address for this work.
**⭐ Dataset Link** 
The dataset included with the work itself, as well as the datasets provided by the SJTU-TES team that are relevant to this work.
**⭐ Space Link** 
The reproduction of certain CPU-based work using the free space service provided by Hugging Face. You can visit the corresponding space to experience some practical applications of this work.
**⭐ Repro Link** 
There are some repositories that can only be run on GPUs (taking several hours or even days if run on CPUs), making it impractical to use the free space service provided by Hugging Face. Therefore, we provide reproducible repositories (including instructive README.md files) to address this limitation.
## [Content](#content)
2.1 Graph Matching (GM)
2.2 Graph Edit Distance (GED)
2.3 Travelling Salesman Problem (TSP)
2.4 Maximum Independent Set (MIS)
3.1 Online Chatting
3.2 Web Scraping
4.1 Motion Retargeting
4.2 Pose Estimation
5.1 DeepLearning Security
5.2 IOT Security
## [AIGC](#content)
### [Text2Img](#content)
#### 1.1.1 Stable Diffusion v1.4
[](https://openaccess.thecvf.com/content/CVPR2022/papers/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.pdf)
[](https://github.com/CompVis/stable-diffusion)
[](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original)
[](https://GitHub.com/CompVis/stable-diffusion/)
``Stable Diffusion``, **a latent text-to-image diffusion model capable of generating photo-realistic images given any text input.** ``stable-diffusion-v1-4`` is resumed from ``stable-diffusion-v1-2`` - 225,000 steps at resolution 512x512 on ``laion-aesthetics v2 5+`` and 10 % dropping of the text-conditioning to improve.
#### 1.1.2 Stable Diffusion v1.5
[](https://openaccess.thecvf.com/content/CVPR2022/papers/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.pdf)
[](https://github.com/CompVis/stable-diffusion)
[](https://huggingface.co/runwayml/stable-diffusion-v1-5)
[](https://github.com/SJTU-TES/repro-text2img)
[](https://GitHub.com/CompVis/stable-diffusion/)
The ``stable-diffusion-v1-5`` checkpoint was initialized with the weights of the ``stable-diffusion-v1-2`` checkpoint and subsequently fine-tuned on 595k steps at resolution 512x512 on ``laion-aesthetics v2 5+`` and 10% dropping of the text-conditioning to improve classifier-free guidance sampling.
Click to view examples we have implemented
- Scarlett, nature, (((beauty))), (((smooth)))，white，Highest quality
### [Text2Video](#content)
#### 1.2.1 Latte
[](https://arxiv.org/pdf/2401.03048v1.pdf)
[](https://github.com/Vchitect/Latte?tab=readme-ov-file)
[](https://huggingface.co/maxin-cn/Latte)
[](https://github.com/SJTU-TES/repro-latte)
[](https://GitHub.com/Vchitect/Latte/)
``Latte``, **a novel latent diffusion transformer for video generation**, utilizes spatio-temporal tokens extracted from input videos and employs a series of Transformer blocks to model the distribution of videos in the latent space. Latte achieves state-of-the-art performance on four standard video generation datasets ``FaceForensics``, ``SkyTimelapse``, ``UCF101``, and ``Taichi-HD``.
Click to view examples we have implemented
- Yellow and black tropical fish dart through the sea.
- An epic tornado attacking above aglowing city at night.
- Slow pan upward of blazing oak fire in an indoor fireplace.
- A cat wearing sunglasses and working as a lifeguard at pool.
- Sunset over the sea.
- A dog in astronaut suit and sunglasses floating in space.
### [Img2Text](#content)
#### 1.3.1 BLIP-2
[](https://arxiv.org/pdf/2301.12597.pdf)
[](https://github.com/salesforce/LAVIS/tree/main/projects/blip2)
[](https://huggingface.co/Salesforce/blip2-opt-2.7b/tree/main)
[](https://github.com/SJTU-TES/repro-Blip2)
[](https://github.com/salesforce/LAVIS/)
``BLIP-2``, **Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models**, BLIP-2 beats Flamingo on zero-shot VQAv2 (65.0 vs 56.3), establishing new state-of-the-art on zero-shot captioning (on NoCaps 121.6 CIDEr score vs previous best 113.2). Equipped with powerful LLMs (e.g. OPT, FlanT5), BLIP-2 also unlocks the new zero-shot instructed vision-to-language generation capabilities for various interesting applications!
Click to view examples we have implemented
- "Question: what is the main elements in the picture? "
- "Answer: the eiffel tower"
### [Img2Img](#content)
#### 1.4.1 Stable Diffusion v2
[](https://arxiv.org/pdf/2202.00512.pdf)
[](https://github.com/Stability-AI/stablediffusion)
[](https://huggingface.co/stabilityai/stable-diffusion-2/tree/main)
[](https://github.com/SJTU-TES/repro-stable-diffusion-v2)
[](https://github.com/Stability-AI/stablediffusion/)
``Stable Diffusion v2``, **high-resolution image synthesis with latent diffusion models**, This stable-diffusion-2 model is resumed from stable-diffusion-2-base (512-base-ema.ckpt) and trained for 150k steps using a v-objective on the same dataset.
Click to view examples we have implemented
- ((two)) ((dogs)) in the picture, ((nature)), (((beauty))), (((smooth)))，white，Highest quality
### [DeepFake](#content)
#### 1.5.1 FaceSwap
[](https://faceswap.dev/)
[](https://github.com/deepfakes/faceswap)
[](https://GitHub.com/deepfakes/faceswap/)
``FaceSwap``, **a tool that utilizes deep learning to recognize and swap faces in pictures and videos.** FaceSwap supports various operating systems(``windows``, ``linux``, ``macos``) and offers powerful face swapping capabilities, utilizing a modern GPU with CUDA support for optimal performance. With FaceSwap, users can gather photos and videos, extract faces from them, train a model based on the extracted faces, and then seamlessly swap faces in your sources using the trained model.
#### 1.5.2 Roop
[](https://github.com/s0md3v/roop)
[](https://huggingface.co/spaces/SJTU-TES/Roop)
[](https://huggingface.co/datasets/SJTU-TES/Roop)
[](https://GitHub.com/s0md3v/roop/)
``Roop``, **a fantastic tool of taking a video and replace the face in it with a face of users' choices.** Users only need one image of the desired face. No dataset, no training.
Click to view examples we have implemented
### [FakeDetect](#content)
#### 1.6.1 UniversalFakeDetect
[](https://utkarshojha.github.io/universal-fake-detection/)
[](https://openaccess.thecvf.com/content/CVPR2023/papers/Ojha_Towards_Universal_Fake_Image_Detectors_That_Generalize_Across_Generative_Models_CVPR_2023_paper.pdf)
[](https://github.com/Yuheng-Li/UniversalFakeDetect)
[](https://huggingface.co/spaces/SJTU-TES/Fake-Detect)
[](https://huggingface.co/datasets/SJTU-TES/Fake-Detect)
[](https://github.com/Yuheng-Li/UniversalFakeDetect/)
``UniversalFakeDetect``, **proposes to perform real-vs-fake classification without learning;** i.e., using a feature space not explicitly trained to distinguish real from fake images. The authors use nearest neighbor and linear probing as instantiations of this idea. When given access to the feature space of a large pretrained vision-language model, the very simple baseline of nearest neighbor classification has surprisingly good generalization ability in detecting fake images from a wide variety of generative models.
Click to view examples we have implemented
## [CO](#content)
### [Graph Matching](#content)
#### 2.1.1 ⭐Pygmtools
[](https://jmlr.org/papers/volume25/23-0572/23-0572.pdf)
[](https://github.com/Thinklab-SJTU/pygmtools)
[](https://huggingface.co/heatingma/pygmtools)
[](https://huggingface.co/spaces/SJTU-TES/Graph-Match)
[](https://huggingface.co/datasets/SJTU-TES/Graph-Match)
[](https://GitHub.com/Thinklab-SJTU/pygmtools/)
``pygmtools``, **Python Graph Matching Tools, provides graph matching solvers in Python.** To make researchers' lives easier, pygmtools support various solvers (``linear``, ``quadratic``, ``multi-graph``, ``neural``), various backends (``numpy``, ``pytorch``, ``jittor``, ``paddle``, ``tensorflow``, ``mindspore``). Also, pygmtools is deep-learning-friendly, whose operations are designed to best preserve the gradient during computation and batched operations support for the best performance.
Click to view examples we have implemented
### [Graph Edit Distance](#content)
#### 2.2.1 ⭐GENN-A*
[](https://openaccess.thecvf.com/content/CVPR2021/papers/Wang_Combinatorial_Learning_of_Graph_Edit_Distance_via_Dynamic_Embedding_CVPR_2021_paper.pdf)
[](https://github.com/Thinklab-SJTU/GENN-Astar)
[](https://drive.google.com/drive/folders/1mUpwHeW1RbMHaNxX_PZvD5HrWvyCQG8y)
[](https://huggingface.co/spaces/SJTU-TES/GED)
[](https://huggingface.co/datasets/SJTU-TES/GED)
[](https://GitHub.com/Thinklab-SJTU/GENN-Astar/)
``GENN-A*``, **Graph Edit Neural Network (GENN),** aims to accelerate the A* solver for graph edit distance problem based on Graph Neural Network. GENN-A* aided A* algorithm works by replacing the heuristic prediction module in A* by GNN. Since the accuracy of heuristic prediction is crucial for the performance of A*, this approach can significantly improve the efficiency of A*.
Click to view examples we have implemented
### [Travelling Salesman Problem](#content)
#### 2.3.1 ⭐T2T
[](https://openreview.net/pdf?id=JtF0ugNMv2)
[](https://github.com/Thinklab-SJTU/T2TCO)
[](https://drive.google.com/drive/folders/1IjaWtkqTAs7lwtFZ24lTRspE0h1N6sBH)
[](https://huggingface.co/spaces/SJTU-TES/TSP)
[](https://huggingface.co/datasets/SJTU-TES/TSP)
[](https://GitHub.com/Thinklab-SJTU/T2TCO/)
``T2T``, **Training to Testing.** T2TCO framework first leverages the generative modeling to estimate the high-quality solution distribution for each instance during training, and then conducts a gradient-based search within the solution space during testing.
Click to view examples we have implemented
### [Maximum Independent Set](#content)
#### 2.4.1 ⭐T2T
[](https://openreview.net/pdf?id=JtF0ugNMv2)
[](https://github.com/Thinklab-SJTU/T2TCO)
[](https://drive.google.com/drive/folders/1IjaWtkqTAs7lwtFZ24lTRspE0h1N6sBH)
[](https://huggingface.co/spaces/SJTU-TES/MIS)
[](https://huggingface.co/datasets/SJTU-TES/MIS)
[](https://GitHub.com/Thinklab-SJTU/T2TCO/)
``T2T``, **Training to Testing.** T2TCO framework first leverages the generative modeling to estimate the high-quality solution distribution for each instance during training, and then conducts a gradient-based search within the solution space during testing.
Click to view examples we have implemented
## [Website](#content)
### [Online Chatting](#content)
#### 3.1.1 ⭐GNetChat
[](https://gnetchat.cn/)
[](https://github.com/heatingma/GNetChat)
[](https://github.com/heatingma/Chat-Website-Tutorial)
[](https://GitHub.com/heatingma/GNetChat/)
``GNetChat``, **General Networking Chat Website designed by SJTUGN Group,** where students can easily form study groups, create posts, make friends, share essential resources, and collaborate on projects in real-time.
Click to view details
### [Web Scraping](#content)
#### 3.2.1 ⭐VidFetch
[](https://github.com/heatingma/VidFetch)
[](https://GitHub.com/heatingma/VidFetch/)
``VidFetch``, **an open-source dataset download tool to obtain copyright-free videos from various free video websites.**
Click to view details
### [Cpp Online](#content)
#### 3.3.1 ⭐web-cpp
[](https://github.com/zhouzhouzhouxiao/web-cpp)
[](https://GitHub.com/zhouzhouzhouxiao/web-cpp/)
``web-cpp``, **an online platform that enables users to write and execute C++ code directly within their browsers.**
Click to view details
## [Motion](#content)
### [Motion Retargeting](#content)
#### 4.1.1 Transmomo
[](https://arxiv.org/pdf/2003.14401.pdf)
[](https://github.com/yzhq97/transmomo.pytorch)
[](https://huggingface.co/faiimea/transmomo_enc)
[](https://huggingface.co/spaces/SJTU-TES/TMM)
[](https://GitHub.com/yzhq97/transmomo.pytorch)
``Transmomo``, **Invariance-Driven Unsupervised Video Motion Retargeting** A lightweight video motion retargeting approach that is capable of transferring motion in spite of structural and view-angle disparities between the source and the target.
Click to view details
#### 4.1.2 EDN
[](https://arxiv.org/pdf/1808.07371.pdf)
[](https://github.com/carolineec/EverybodyDanceNow)
[](https://huggingface.co/SJTU-TES/Everybody_Dance_Now)
[](https://github.com/carolineec/EverybodyDanceNow)
``EverybodyDanceNow``, **A simple method for "do as I do" motion transfer:** Given a source video of a person dancing, we can transfer that performance to a novel (amateur) target after only a few minutes of the target subject performing standard moves.
Click to view details
### [Pose Estimation](#content)
#### 4.2.1 Openpose
[](https://arxiv.org/pdf/1812.08008.pdf)
[](https://github.com/Hzzone/pytorch-openpose)
[](https://huggingface.co/spaces/SJTU-TES/OpenPose)
[](https://huggingface.co/faiimea/openpose)
[](https://github.com/Hzzone/pytorch-openpose)
``Openpose``, **Real-time multi-person keypoint detection library for pose estimation** 2D real-time multi-person keypoint detection.We provide pytorch implementation of openpose including Body and Hand Pose Estimation.
Click to view details
### [Video Matting](#content)
#### 4.3.1 RobustVideoMatting
[](https://arxiv.org/pdf/2108.11515.pdf)
[](https://github.com/PeterL1n/RobustVideoMatting)
[](https://huggingface.co/spaces/SJTU-TES/VideoMatting)
[](https://huggingface.co/SJTU-TES/RobustVideoMatting)
[](https://github.com/PeterL1n/RobustVideoMatting)
``RVM``, **Robust High-Resolution Video Matting with Temporal Guidance** RVM is specifically designed for robust human video matting. Unlike existing neural models that process frames as independent images, RVM uses a recurrent neural network to process videos with temporal memory.
Click to view details
## [Security](#content)
### [DeepLearning Security](#content)
#### 5.1.1 ⭐DLSec
[](https://github.com/faiimea/DLSec)
[](https://github.com/faiimea/DLSec)
``DLSec``, **Deep Learning model security evaluation platform** Taking attack paradigms and defense means such as anti-sample, data poisoning, backdoor attacks as examples, We studies and implements mainstream offensive and defensive algorithms for deep learning models, and builds a comprehensive and effective evaluation system for deep learning models from the perspectives of white box model and black box model.
#### 5.1.2 ⭐WDAD
[](https://github.com/faiimea/WDAD)
[](https://github.com/faiimea/WDAD)
``WDAD``, **Adversarial sample detection based on weak dark textures**
Click to view details
#### 5.1.3 ⭐UAP
[](https://openaccess.thecvf.com/content/CVPR2022/papers/Peng_Fingerprinting_Deep_Neural_Networks_Globally_via_Universal_Adversarial_Perturbations_CVPR_2022_paper.pdf)
[](https://github.com/faiimea/UAP)
[](https://huggingface.co/faiimea/uap)
[](https://github.com/faiimea/UAP)
``UAP``, **Fingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbations"** a novel and practical mechanism which enables the service provider to verify whether a suspect model is stolen from the victim model via model extraction attacks.
### [IOT Security](#content)
#### 5.2.1 ⭐WAV2COM
[](https://yan4meng.github.io/files/paper_security_22_arrayid.pdf)
[](https://github.com/faiimea/wav2com)
[](https://huggingface.co/faiimea/wav2com)
[](https://github.com/faiimea/wav2com)
``WAV2COM``, **Your Microphone Array Retains Your Identity: A Robust Voice Liveness Detection System for Smart Speakers**
### [Sandbox](#content)
#### 5.3.1 ⭐sandbox
[](https://github.com/zhouzhouzhouxiao/sandbox)
[](https://github.com/faiimea/wav2com)
``Sandbox``, **helps you compile and run your C++ code within an isolated Docker image. Using Docker ensures that your code runs consistently and predictably in any Docker-enabled environment. This makes it convenient for you to develop and test your C++ project across different systems or environments.**