Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/simonblanke/search-data-explorer
Visualize search-data from your gradient-free-optimization run.
https://github.com/simonblanke/search-data-explorer
dashboard data-exploration data-science matplotlib pandas plotly python statistics streamlit tabular-data visualization
Last synced: 3 months ago
JSON representation
Visualize search-data from your gradient-free-optimization run.
- Host: GitHub
- URL: https://github.com/simonblanke/search-data-explorer
- Owner: SimonBlanke
- License: mit
- Created: 2021-02-04T09:43:08.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2024-04-03T06:50:38.000Z (10 months ago)
- Last Synced: 2024-10-13T16:13:09.099Z (3 months ago)
- Topics: dashboard, data-exploration, data-science, matplotlib, pandas, plotly, python, statistics, streamlit, tabular-data, visualization
- Language: Python
- Homepage:
- Size: 1.17 MB
- Stars: 3
- Watchers: 5
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Search Data Explorer
Visualize optimization search-data via plotly in a streamlit dashboard
The Search-Data-Explorer is a simple application specialized to visualize search-data generated from [Gradient-Free-Optimizers](https://github.com/SimonBlanke/Gradient-Free-Optimizers) or [Hyperactive](https://github.com/SimonBlanke/Hyperactive). It is designed as an easy-to-use tool to gain insights into multi-dimensional data, as commonly found in optimization.
I created this package, because I needed a convenient tool to visually analyse search-data during the development of gradient-free-optimization algorithms. My goal for this package is to help users get insight into the search-data and its corresponding objective-function and search-space. Building on this insight could help improve the selection of the search-space, compare models in the objective-function or explain the behaviour of the optimization algorithm.
## Disclaimer
This project is in an early development stage and is only tested manually. If you encounter bugs or have suggestions for improvements, then please open an issue.
## Installation
```console
pip install search-data-explorer
```
## How to use
The Search Data Explorer has a very simple API, that can be explained by the examples below or just execute the command "`search-data-explorer` [file]" to open the Search Data Explorer without executing a python script.
### search-data requirements
The Search Data Explorer is used by loading the search-data with a few lines of code. The search data that is loaded from file must follow the pattern below. The columns can have any name but must contain the `score`, which is always included in search-data from [Gradient-Free-Optimizers](https://github.com/SimonBlanke/Gradient-Free-Optimizers) or [Hyperactive](https://github.com/SimonBlanke/Hyperactive).
first column name
another column name
...
score
0.756
0.1
0.2
-3
0.823
0.3
0.1
-10
...
...
...
...
...
...
...
...
## Examples
### Load search-data by passing dataframe
You can pass the search-data directly, if you do not want to save your search-data to disk and just explore it one time after the optimization has finished.
```python
import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer
from search_data_explorer import SearchDataExplorer
def parabola_function(para):
loss = para["x"] * para["x"] + para["y"] * para["y"] + para["y"] * para["y"]
return -loss
search_space = {
"x": np.arange(-10, 10, 0.1),
"y": np.arange(-10, 10, 0.1),
"z": np.arange(-10, 10, 0.1),
}
# generate search-data for this example with gradient-free-optimizers
opt = RandomSearchOptimizer(search_space)
opt.search(parabola_function, n_iter=1000)
search_data = opt.search_data
# Open Search-Data-Explorer
sde = SearchDataExplorer()
sde.open(search_data) # pass search-data
```
### Load search-data by passing path to file
If you already have a search-data file on disk you can pass the path to the file to the search-data-explorer.
```python
import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer
from search_data_explorer import SearchDataExplorer
def parabola_function(para):
loss = para["x"] * para["x"] + para["y"] * para["y"] + para["y"] * para["y"]
return -loss
search_space = {
"x": np.arange(-10, 10, 0.1),
"y": np.arange(-10, 10, 0.1),
"z": np.arange(-10, 10, 0.1),
}
# generate search-data for this example with gradient-free-optimizers
opt = RandomSearchOptimizer(search_space)
opt.search(parabola_function, n_iter=1000)
search_data = opt.search_data
search_data.to_csv("search_data.csv", index=False)
# Open Search-Data-Explorer
sde = SearchDataExplorer()
sde.open("model1.csv") # pass path to file on disk
```
### Load search-data by browsing for file
You can just open the search-data-explorer without passing a file or path. In this case you can browse for the file via a menu inside the search-data-explorer.
```python
import numpy as np
from gradient_free_optimizers import RandomSearchOptimizer
from search_data_explorer import SearchDataExplorer
def parabola_function(para):
loss = para["x"] * para["x"] + para["y"] * para["y"] + para["y"] * para["y"]
return -loss
search_space = {
"x": np.arange(-10, 10, 0.1),
"y": np.arange(-10, 10, 0.1),
"z": np.arange(-10, 10, 0.1),
}
# generate search-data for this example with gradient-free-optimizers
opt = RandomSearchOptimizer(search_space)
opt.search(parabola_function, n_iter=1000)
search_data = opt.search_data
search_data.to_csv("search_data.csv", index=False)
# Open Search-Data-Explorer
sde = SearchDataExplorer()
sde.open() # start without passing anything and use the file explorer within the search-data-explorer
```