Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tsinghua-ideal/TEMPER-Secure-MLaaS
A Secure MLaaS Framework based on Intel SGX
https://github.com/tsinghua-ideal/TEMPER-Secure-MLaaS
Last synced: about 1 month ago
JSON representation
A Secure MLaaS Framework based on Intel SGX
- Host: GitHub
- URL: https://github.com/tsinghua-ideal/TEMPER-Secure-MLaaS
- Owner: tsinghua-ideal
- Created: 2023-08-28T07:45:28.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2023-09-27T14:20:42.000Z (about 1 year ago)
- Last Synced: 2024-08-03T01:21:30.174Z (4 months ago)
- Language: Rust
- Size: 317 KB
- Stars: 4
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome-SGX-Open-Source - https://github.com/tsinghua-ideal/TEMPER-Secure-MLaaS
README
# Secure-MLaaS
A Secure MLaaS Framework based on Intel SGX.
The framework contains two parts: model partition and model inference.
## Preparation
Make sure you have SGX v1 with limited EPC. See https://github.com/intel/linux-sgx .
## Installation
1. Install the python packages
Install the python packages according to the `requirements.txt` on Python 3.6.9.
Use `while read requirement; do sudo pip3 install $requirement; done < requirements.txt` or `pip3 install -r requirements.txt` .
Note that the TVM packages should be installed by compiled packages.
2. Install TVM
Install TVM v0.7 from https://github.com/grief8/tvm.git or https://github.com/apache/incubator-tvm.git . You can use [TVM Docs][tvm_docs] to install TVM.
[tvm_docs]: https://tvm.apache.org/docs/install/index.html
You can also refer to the following commands:
```shell
git clone --recursive https://github.com/grief8/tvm.git tvm
sudo apt-get update
sudo apt-get install -y python3 python3-dev python3-setuptools gcc libtinfo-dev zlib1g-dev build-essential cmake libedit-dev libxml2-dev
mkdir build
cp cmake/config.cmake build
cd build
cmake ..
make -j4
cd ../python; python setup.py install --user; cd ..
```
After the compilation, install the python packages.
1. Prepare the Rust environment
Open a terminal and enter the following command:
```
sudo apt install -y build-essential
curl --proto '=https' --tlsv1.2 https://sh.rustup.rs -sSf | sh
```
This command will download a script and start installing the rustup tool, which will install the latest stable version of Rust. You may be prompted for an administrator password.
If the installation was successful, the following line will appear:
```
Rust is installed now. Great!
```
Then, you should switch the rustup toolchain to nightly.
Install the nightly version:
`rustup install nightly`
Switch to the nightly version of cargo (nightly-2021-04-15-x86_64-unknown-linux-gnu is recommended):
`rustup default nightly-2021-04-15-x86_64-unknown-linux-gnu`
4. Install Fortanix
Fortanix is a target for Intel SGX which automatically compiles the code into SGX SDK. Install it by its official [doc][doc]. Note that Intel SGX SDK is necessary here.
Then run `rustup component add llvm-tools-preview` to get llvm-ar and llvm-objcopy
[doc]: https://edp.fortanix.com/docs/installation/guide/
## Evaluation
To run the model partition, you should run `python auto_model_partition.py --model --input_size --build_dir `. The model will be partitioned into several TVM submodels and the submodels will be compiled into libraries and parameters. The enclave libraries will be stored in the `build_dir` directory.
To run the model inference, you should run the following commands:
```
cd cluster-inference
source environment.sh
# Generate instances
python worker_generator.py
# Build and Run
./clean.sh
./build.sh
./run.sh
```
## Debugging
1. Encounter the warning `Blocking waiting for file lock on package cache`.
> Run `rm ~/.cargo/.package-cache` and re-build the project to fix it. We could also disable the rust-analyzer to avoid it.
>
2. Cannot fetch crates
> change the crate sources.
```
mkdir ~/.cargo/config
cat << EOF >> ~/.cargo/config
[target.x86_64-fortanix-unknown-sgx]
runner = "ftxsgx-runner-cargo"
[source.crates-io]
registry = "https://github.com/rust-lang/crates.io-index"
replace-with = "tuna"
[source.tuna]
registry = "https://mirrors.tuna.tsinghua.edu.cn/git/crates.io-index.git"
[source.ustc]
registry = "git://mirrors.ustc.edu.cn/crates.io-index"
[source.sjtu]
registry = "https://mirrors.sjtug.sjtu.edu.cn/git/crates.io-index"
[source.rustcc]
registry = "https://code.aliyun.com/rustcc/crates.io-index.git"
EOF
```
3. Feature `edition2021` is required
> Manually add `edition = "2021"` to the `Cargo.toml` of the error packages.