https://github.com/miskcoo/kscore
Nonparametric Score Estimators, ICML 2020
https://github.com/miskcoo/kscore
density-estimation gradient-estimator machine-learning statistics
Last synced: 12 months ago
JSON representation
Nonparametric Score Estimators, ICML 2020
- Host: GitHub
- URL: https://github.com/miskcoo/kscore
- Owner: miskcoo
- License: mit
- Created: 2020-04-12T14:37:05.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2021-06-25T21:59:20.000Z (almost 5 years ago)
- Last Synced: 2025-04-06T04:41:21.248Z (about 1 year ago)
- Topics: density-estimation, gradient-estimator, machine-learning, statistics
- Language: Python
- Homepage: https://arxiv.org/abs/2005.10099
- Size: 203 KB
- Stars: 36
- Watchers: 4
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Nonparametric Score Estimators
Yuhao Zhou, Jiaxin Shi, Jun Zhu. https://arxiv.org/abs/2005.10099
## Toy Example
```
python -m examples.spiral --lam=1.0e-5 --kernel=curlfree_imq --estimator=nu
```
## Dependencies
```
Tensorflow >= 1.14.0
```
## Usage
* Create a score estimator
```python
from kscore.estimators import *
from kscore.kernels import *
# Tikhonov regularization (Theorem 3.1), equivalent to KEF (Example 3.5)
kef_estimator = Tikhonov(lam=0.0001, use_cg=False, kernel=CurlFreeIMQ())
# Tikhonov regularization + Conjugate Gradient (KEF-CG, Example 3.8)
kefcg_estimator = Tikhonov(lam=0.0001, use_cg=True, kernel=CurlFreeIMQ())
# Tikhonov regularization + Nystrom approximation (Appendix C.1),
# equivalent to NKEF (Example C.1) using 60% samples
nkef_estimator = Tikhonov(lam=0.0001, use_cg=False, subsample_rate=0.6, kernel=CurlFreeIMQ())
# Tikhonov regularization + Nystrom approximation + Conjugate Gradient
nkefcg_estimator = Tikhonov(lam=0.0001, use_cg=True, subsample_rate=0.6, kernel=CurlFreeIMQ())
# Landweber iteration (Theorem 3.4)
landweber_estimator = Landweber(lam=0.00001, kernel=CurlFreeIMQ())
landweber_estimator = Landweber(iternum=100, kernel=CurlFreeIMQ())
# nu-method (Example C.4)
nu_estimator = NuMethod(lam=0.00001, kernel=CurlFreeIMQ())
nu_estimator = NuMethod(iternum=100, kernel=CurlFreeIMQ())
# Spectral cut-off regularization (Theorem 3.2),
# equivalent to SSGE (Example 3.6) using 90% eigenvalues
ssge_estimator = SpectralCutoff(keep_rate=0.9, kernel=DiagonalIMQ())
# Original Stein estimator
stein_estimator = Stein(lam=0.001)
```
* Fit the score estimator using samples
```python
# manually specify the hyperparameter
estimator.fit(samples, kernel_hyperparams=kernel_width)
# automatically choose the hyperparameter (using the median trick)
estimator.fit(samples)
```
* Predict the score
```python
gradient = estimator.compute_gradients(x)
```
* Predict the energy (unnormalized log-density)
```python
log_p = estimator.compute_energy(x) # only for curl-free kernels
```
* Construct other curl-free kernels (see `kscore/kernels/curlfree_gaussian.py`)