https://github.com/ricardorobledo/ml_optimization

https://github.com/ricardorobledo/ml_optimization

matplotlib numpy python scikit-learn xgboost

Last synced: 2 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/ricardorobledo/ml_optimization
• Owner: RicardoRobledo
• Created: 2025-07-19T16:15:12.000Z (11 months ago)
• Default Branch: main
• Last Pushed: 2025-07-19T16:18:53.000Z (11 months ago)
• Last Synced: 2025-07-19T20:06:19.750Z (11 months ago)
• Topics: matplotlib, numpy, python, scikit-learn, xgboost
• Language: Jupyter Notebook
• Homepage:
• Size: 1.51 MB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Optimization for Machine Learning 📈

This repository contains my personal journey and hands-on implementations focused on **optimization in Machine Learning** — one of the fundamental skills for building efficient, accurate, and well-tuned models.

The concepts and structure are inspired by the book *Optimization for Machine Learning* available at [machinelearningmastery.com](https://machinelearningmastery.com/), which presents a clear and progressive path to mastering this essential area of ML.

---

## 🧠 What's included in this repository?

- From-scratch implementations of both local and global optimization algorithms.

- Practical applications of techniques such as:

  - **Hill Climbing**

  - **Simulated Annealing**

  - **Genetic Algorithms**

  - **Differential Evolution**

  - **Gradient Descent and variants**: Momentum, RMSProp, AdaGrad, Adadelta, Adam

- Manual hyperparameter optimization for:

  - **Perceptron**

  - **XGBoost**

  - **Simple Neural Networks**

- Function and optimization surface visualization.

- Feature selection using stochastic optimization techniques.

- Comparisons between methods applied to regression and classification tasks.

---

## 💡 Why is optimization important in Machine Learning?

Optimization is the engine behind model training: it helps us find the best parameters, improve generalization, and reduce error. It's critical for:

- Hyperparameter tuning

- Feature selection

- Training deep learning models

- Function analysis and sensitivity testing

---

## 🛠️ Tools & Libraries

- `scikit-learn`

- `xgboost`

- `matplotlib` for visualization

- `numpy` & `scipy`

---

## 🌐 Reference

Book: *Optimization for Machine Learning*  

Website: [https://machinelearningmastery.com](https://machinelearningmastery.com)