https://github.com/franckalbinet/soilspectfm
Provides Scikit-Learn compatible transforms for spectroscopic data preprocessing.
https://github.com/franckalbinet/soilspectfm
data-science soil-science soil-spectroscopy
Last synced: 5 months ago
JSON representation
Provides Scikit-Learn compatible transforms for spectroscopic data preprocessing.
- Host: GitHub
- URL: https://github.com/franckalbinet/soilspectfm
- Owner: franckalbinet
- License: apache-2.0
- Created: 2025-01-23T13:15:44.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-24T09:55:11.000Z (over 1 year ago)
- Last Synced: 2025-08-21T06:29:21.310Z (11 months ago)
- Topics: data-science, soil-science, soil-spectroscopy
- Language: Jupyter Notebook
- Homepage: http://fr.anckalbi.net/soilspectfm/
- Size: 8.61 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# SoilSpecTfm
> Spectral Processing Tools for Soil Spectroscopy
By translating specialized soil spectroscopy methods into the
[`scikit-learn`](https://scikit-learn.org/stable/) framework,
`SoilSpecTfm` and
[`SoilSpecData`](https://fr.anckalbi.net/soilspecdata/) connect this
niche domain with Python’s vast machine learning ecosystem, making
advanced ML/DL tools accessible to soil scientists.
Implemented transforms developed so far include:
- **Baseline corrections**:
- [x]
[`SNV`](https://franckalbinet.github.io/soilspectfm/core.html#snv):
Standard Normal Variate
- [x]
[`MSC`](https://franckalbinet.github.io/soilspectfm/core.html#msc):
Multiplicative Scatter Correction
- [ ] `Detrend`: Detrend the spectrum (planned)
- [ ] `ALS`: Asymmetric Least Squares detrend the spectrum (planned)
- **Derivatives**:
- [x]
[`TakeDerivative`](https://franckalbinet.github.io/soilspectfm/core.html#takederivative):
Take derivative (1st, 2nd, etc.) of the spectrum and apply
Savitzky-Golay smoothing
- [ ] `GapSegmentDerivative`: (planned)
- **Smoothing**:
- [x]
[`WaveletDenoise`](https://franckalbinet.github.io/soilspectfm/core.html#waveletdenoise):
Wavelet denoising
- [x]
[`SavGolSmooth`](https://franckalbinet.github.io/soilspectfm/core.html#savgolsmooth):
Savitzky-Golay smoothing
- **Other transformations**:
- [x]
[`ToAbsorbance`](https://franckalbinet.github.io/soilspectfm/core.html#toabsorbance):
Transform the spectrum to absorbance
- [x]
[`Resample`](https://franckalbinet.github.io/soilspectfm/core.html#resample):
Resample the spectrum to a new wavenumber range
- [x]
[`Trim`](https://franckalbinet.github.io/soilspectfm/core.html#trim):
Trim the spectrum to a specific wavenumber range
**Key Features**:
- Seamless integration with scikit-learn’s machine learning ecosystem
- Complement with [SoilSpecData](https://fr.anckalbi.net/soilspecdata/)
package for soil spectroscopy workflows
- Pipeline-ready transformers with consistent API
All transformers follow scikit-learn conventions:
- Implement fit/transform interface
- Support get_params/set_params for GridSearchCV
- Provide detailed documentation and examples
## Installation
``` bash
pip install soilspectfm
```
## Quick Start
``` python
from soilspectfm.core import (SNV,
TakeDerivative,
ToAbsorbance,
Resample,
WaveletDenoise)
from sklearn.pipeline import Pipeline
```
### Loading OSSL dataset
Let’s use OSSL dataset as an example using
[SoilSpecData](https://fr.anckalbi.net/soilspecdata/) package.
``` python
from soilspecdata.datasets.ossl import get_ossl
```
``` python
ossl = get_ossl()
mir_data = ossl.get_mir()
```
### Preprocessing pipeline
Transforms are fully compatible with
[scikit-learn](https://scikit-learn.org/stable/) and can be used in a
pipeline as follows:
``` python
pipe = Pipeline([
('snv', SNV()), # Standard Normal Variate transformation
('denoise', WaveletDenoise()), # Wavelet denoising
('deriv', TakeDerivative(window_length=11, polyorder=2, deriv=1)) # First derivative
])
X_tfm = pipe.fit_transform(mir_data.spectra)
```
### Quick visualization
``` python
from soilspectfm.visualization import plot_spectra
from matplotlib import pyplot as plt
```
``` python
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(15, 7))
ax1 = plot_spectra(
mir_data.spectra,
mir_data.wavenumbers,
ax=ax1,
ascending=False,
color='black',
alpha=0.6,
lw=0.5,
xlabel='Wavenumber (cm$^{-1}$)',
title='Raw Spectra'
)
ax2 = plot_spectra(
X_tfm,
mir_data.wavenumbers,
ax=ax2,
ascending=False,
color='steelblue',
alpha=0.6,
lw=0.5,
xlabel='Wavenumber (cm$^{-1}$)',
title='SNV + Derivative (1st order) Transformed Spectra'
)
plt.tight_layout()
```
## Dependencies
- fastcore
- numpy
- scipy
- scikit-learn
- matplotlib
## Further references
- https://orange-spectroscopy.readthedocs.io/en/latest/widgets/preprocess-spectra.html
## Contributing
### Developer guide
If you are new to using `nbdev` here are some useful pointers to get you
started.
Install spectfm in Development mode:
``` sh
# make sure spectfm package is installed in development mode
$ pip install -e .
# make changes under nbs/ directory
# ...
# compile to have changes apply to spectfm
$ nbdev_prepare
```
## License
This project is licensed under the Apache2 License - see the LICENSE
file for details.
## Support
For questions and support, please [open an
issue](https://github.com/franckalbinet/spectfm/issues) on GitHub.