https://github.com/feedzai/make-lightgbm
https://github.com/feedzai/make-lightgbm
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/feedzai/make-lightgbm
- Owner: feedzai
- Created: 2020-07-08T09:41:17.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2024-03-05T16:06:09.000Z (over 2 years ago)
- Last Synced: 2025-04-30T04:49:01.616Z (about 1 year ago)
- Language: Shell
- Size: 246 KB
- Stars: 2
- Watchers: 9
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# make-lightgbm
This repo serves to build all the needed LightGBM artifacts to create a Java library, including a copy of OpenMP's .so, from a *"replica"* of LightGBM's CI build environment that does not require Microsoft credentials.
# Requirements
- bash & sed
- docker
- git
# How to use
To generate the `build/` folder with all necessary artifacts just run:
```bash
bash make.sh [lightgbm_version] [package_version] # where lightgbm_version is any of (commit_id, tag, branch)
```
In case it fails compiling for ARM64 you may need to run:
```bash
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
```
If no `lightgbm_version` is specified, `master` is checked out.
If no `package_version` is specified:
- If `lightgbm_version` is a release (`vMAJOR.MINOR.PATCH`), `package_version=MAJOR.MINOR.PATCH`.
Otherwise, `package_version=0.0.0`.
Finally, in the output `pom.xml`, the package version is the one specified in `package_version`, except if it is `0.0.0`. In that case the build version becomes `package_version-lightgbm_version`.
### Building from another LightGBM repository
By defining the environment variable `LIGHTGBM_REPO_URL` which by default points to [LightGBM](https://github.com/microsoft/LightGBM), to another `http(s)` git LightGBM repo URL, you can build your own custom version of LightGBM. This can be useful to try building our own patched/custom versions of LightGBM. Ensure you use the _http(s)_ protocol instead of _git_.
### Single build for AMD64
In case you want to build only for AMD64, which doesn't have to use qemu emulation, you must define the environment variable `ARCH_BUILD=single` prior to running `make.sh`.
## Output artifacts
This is the output:
```bash
build
├── amd64
│ ├── glibc
│ │ ├── libgomp.so.1.0.0
│ │ ├── lib_lightgbm.so
│ │ └── lib_lightgbm_swig.so
│ ├── musl
│ │ ├── libgomp.so.1.0.0
│ │ ├── lib_lightgbm.so
│ │ └── lib_lightgbm_swig.so
├── arm64
│ ├── libgomp.so.1.0.0
│ ├── lib_lightgbm.so
│ └── lib_lightgbm_swig.so
├── __commit_id__
├── install_jar_locally.sh
├── libopenmp.licence
├── lightgbmlib.jar
├── __lightgbm_repo_url__
├── pom.xml
├── __timestamp__
└── __version__
```
Files with "__" are just single-line files containing meta-data for traceability so you don't lose track of build conditions.
You can now copy this folder into your project and either run `bash install_jar_locally.sh` or use maven's install plugin.
# Extra for developers: Building local patches (Debugging for developers)
This explains how to build local patches from source for LightGBM. This allows quicker iterations during LightGBM C++
development/debugging.
For instance, to perform debugging, one must build LightGBM using the compiler toolchain available
on the target machine running the debugger, otherwise there will be symbol compatibility issues.
Patching is done in a two-stage process:
1. Run at least once the base `make.sh` to have the base build.
2. Patch the base build by calling `make_patch.sh` as explained below.
## Running `make_patch.sh`
### Setup the LightGBM source for compilation
First, clone the LightGBM repo/fork to your computer and run CMake with the desired flags:
```bash
cd my_lgbm_repo
mkdir build
cd build
cmake .. -DUSE_DEBUG=ON -DUSE_SWIG=ON
```
### Create the patch
> **Note**
The patch only applies to AMD64 builds. It doesn't change other architectures' builds (e.g., ARM64)
After the CMake setup is complete, simply run `make_patch.sh` against that folder:
```bash
bash make_patch.sh my_lgbm_repo_build_folder
```
This will compile LightGBM from source with your settings and patch the current base LightGBM build in the [provider](https://github.com/feedzai/feedzai-openml-java/tree/master/openml-lightgbm/lightgbm-builder).
Run `make_patch.sh` every time you want to build a new patch.