Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ar-nowaczynski/spaceopt
Hyperparameter optimization via gradient boosting regression
https://github.com/ar-nowaczynski/spaceopt
automl hyperparameter-optimization machine-learning python
Last synced: 2 months ago
JSON representation
Hyperparameter optimization via gradient boosting regression
- Host: GitHub
- URL: https://github.com/ar-nowaczynski/spaceopt
- Owner: ar-nowaczynski
- License: mit
- Created: 2020-01-20T21:04:53.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2022-07-10T10:26:17.000Z (over 2 years ago)
- Last Synced: 2024-11-01T12:06:05.335Z (3 months ago)
- Topics: automl, hyperparameter-optimization, machine-learning, python
- Language: Python
- Homepage:
- Size: 33.2 KB
- Stars: 42
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# SpaceOpt: hyperparameter optimization via gradient boosting regression
[](https://www.python.org/downloads/)
[](https://pypi.org/project/spaceopt/)
[](https://github.com/ar-nowaczynski/spaceopt)
SpaceOpt is a hyperparameter optimization algorithm that uses gradient boosting regression to find the most promising candidates for the next trial by predicting their evaluation score.
## Installation
```bash
$ pip install spaceopt
```
## Usage
1. Define a discrete hyperparameter search space, for example:
```python
search_space = {
'a': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], # list of ordered numbers: ints
'b': [-3.5, -0.1, 0.0, 2.5, 10.0], # list of ordered numbers: floats
'c': [256, 512, 1024, 2048], # another list of ordered numbers
'd': ['ABC', 'IJK', 'XYZ'], # categorical variable
'e': [True, False], # boolean variable
# ... (add as many as you need)
}
```
2. Define your evaluation function:
```python
def evaluation_function(spoint: dict) -> float:
# your code (e.g. model fit)
return y # score (e.g. model accuracy)
spoint = {'a': 4, 'b': 0.0, 'c': 512, 'd': 'XYZ', 'e': False}
y = evaluation_function(spoint)
```
3. Use SpaceOpt for a hyperparameter optimization:
```python
from spaceopt import SpaceOpt
spaceopt = SpaceOpt(search_space=search_space,
target_name='y',
objective='maximize') # or 'minimize'
for iteration in range(200):
if iteration < 20:
spoint = spaceopt.get_random() # exploration
else:
spoint = spaceopt.fit_predict() # exploitation
spoint['y'] = evaluation_function(spoint)
spaceopt.append_evaluated_spoint(spoint)
```
More examples [here](https://github.com/ar-nowaczynski/spaceopt/tree/master/examples).
## Advanced
- get multiple points by setting `num_spoints`:
```python
spoints = spaceopt.get_random(num_spoints=2)
# or
spoints = spaceopt.fit_predict(num_spoints=5)
```
- control exploration vs. exploitation behaviour by adjusting `sample_size` (default=10000), which is the number of candidates sampled for ranking:
```python
spoint = spaceopt.fit_predict(sample_size=1000) # decreasing `sample_size` increses exploration
spoint = spaceopt.fit_predict(sample_size=100000) # increasing `sample_size` increses exploitation
```
- add manually selected evaluation points to SpaceOpt:
```python
my_spoint = {'a': 8, 'b': -3.5, 'c': 256, 'd': 'IJK', 'e': False}
my_spoint['y'] = evaluation_function(my_spoint)
spaceopt.append_evaluated_spoint(my_spoint)
```
- learn more by reading the code, there are only 3 classes: [SpaceOpt](https://github.com/ar-nowaczynski/spaceopt/blob/master/spaceopt/optimizer.py), [Space](https://github.com/ar-nowaczynski/spaceopt/blob/master/spaceopt/space.py) and [Variable](https://github.com/ar-nowaczynski/spaceopt/blob/master/spaceopt/variable.py).
## License
MIT License (see [LICENSE](https://github.com/ar-nowaczynski/spaceopt/blob/master/LICENSE)).