{"id":16391169,"url":"https://github.com/jeongyoonlee/kaggler","last_synced_at":"2025-05-15T14:08:07.277Z","repository":{"id":26534385,"uuid":"29987631","full_name":"jeongyoonlee/Kaggler","owner":"jeongyoonlee","description":"Code for Kaggle Data Science Competitions","archived":false,"fork":false,"pushed_at":"2024-04-25T03:17:01.000Z","size":2266,"stargazers_count":751,"open_issues_count":6,"forks_count":163,"subscribers_count":38,"default_branch":"master","last_synced_at":"2025-05-15T14:07:57.230Z","etag":null,"topics":["automl","feature-engineering","kaggle","kaggler","machine-learning","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jeongyoonlee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":["jeongyoonlee"]}},"created_at":"2015-01-28T20:51:13.000Z","updated_at":"2025-05-04T13:11:00.000Z","dependencies_parsed_at":"2024-04-25T04:37:42.638Z","dependency_job_id":null,"html_url":"https://github.com/jeongyoonlee/Kaggler","commit_stats":{"total_commits":226,"total_committers":13,"mean_commits":"17.384615384615383","dds":0.7212389380530974,"last_synced_commit":"736622e2a59461c69dc3dce87c2fdce8ea45a242"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeongyoonlee%2FKaggler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeongyoonlee%2FKaggler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeongyoonlee%2FKaggler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeongyoonlee%2FKaggler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jeongyoonlee","download_url":"https://codeload.github.com/jeongyoonlee/Kaggler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254355335,"owners_count":22057354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automl","feature-engineering","kaggle","kaggler","machine-learning","python"],"created_at":"2024-10-11T04:45:14.939Z","updated_at":"2025-05-15T14:08:02.262Z","avatar_url":"https://github.com/jeongyoonlee.png","language":"Python","funding_links":["https://github.com/sponsors/jeongyoonlee"],"categories":[],"sub_categories":[],"readme":"[![PyPI version](https://badge.fury.io/py/Kaggler.svg)](https://badge.fury.io/py/Kaggler)\n[![CI](https://github.com/jeongyoonlee/Kaggler/actions/workflows/test.yml/badge.svg)](https://github.com/jeongyoonlee/Kaggler/actions/workflows/test.yml)\n[![Downloads](https://pepy.tech/badge/kaggler)](https://pepy.tech/project/kaggler)\n[![codecov](https://codecov.io/gh/jeongyoonlee/Kaggler/branch/master/graph/badge.svg)](https://codecov.io/gh/jeongyoonlee/Kaggler)\n\n\n# Kaggler\nKaggler is a Python package for lightweight online machine learning algorithms and utility functions for ETL and data analysis. It is distributed under the MIT License.\n\nIts online learning algorithms are inspired by Kaggle user [tinrtgu's code](http://goo.gl/K8hQBx).  It uses the sparse input format that handles large sparse data efficiently.  Core code is optimized for speed by using Cython.\n\n\n## Installation\n\n### Dependencies\nPython packages required are listed in `requirements.txt`\n* cython\n* h5py\n* hyperopt\n* lightgbm\n* ml_metrics\n* numpy/scipy\n* pandas\n* scikit-learn\n\n### Using pip\nPython package is available at PyPi for pip installation:\n```\npip install -U Kaggler\n```\nIf installation fails because it cannot find `MurmurHash3.h`, please add `.` to\n`LD_LIBRARY_PATH` as described [here](https://github.com/jeongyoonlee/Kaggler/issues/32).\n\n### From source code\nIf you want to install it from source code:\n```\npython setup.py build_ext --inplace\npython setup.py install\n```\n\n\n## Feature Engineering\n\n### One-Hot, Label, Target, Frequency, and Embedding Encoders for Categorical Features\n```python\nimport pandas as pd\nfrom kaggler.preprocessing import OneHotEncoder, LabelEncoder, TargetEncoder, FrequencyEncoder, EmbeddingEncoder\n\ntrn = pd.read_csv('train.csv')\ntarget_col = trn.columns[-1]\ncat_cols = [col for col in trn.columns if trn[col].dtype == 'object']\n\nohe = OneHotEncoder(min_obs=100) # grouping all categories with less than 100 occurences\nlbe = LabelEncoder(min_obs=100)  # grouping all categories with less than 100 occurences\nte = TargetEncoder()\t\t\t # replacing each category with the average target value of the category\nfe = FrequencyEncoder()\t         # replacing each category with the frequency value of the category\nee = EmbeddingEncoder()          # mapping each category to a vector of real numbers\n\nX_ohe = ohe.fit_transform(trn[cat_cols])\t    # X_ohe is a scipy sparse matrix\ntrn[cat_cols] = lbe.fit_transform(trn[cat_cols])\ntrn[cat_cols] = te.fit_transform(trn[cat_cols])\ntrn[cat_cols] = fe.fit_transform(trn[cat_cols])\nX_ee = ee.fit_transform(trn[cat_cols], trn[target_col])          # X_ee is a numpy matrix\n\ntst = pd.read_csv('test.csv')\nX_ohe = ohe.transform(tst[cat_cols])\ntst[cat_cols] = lbe.transform(tst[cat_cols])\ntst[cat_cols] = te.transform(tst[cat_cols])\ntst[cat_cols] = fe.transform(tst[cat_cols])\nX_ee = ee.transform(tst[cat_cols])\n```\n\n### Denoising AutoEncoder (DAE)\nFor reference for DAE, please check out [Vincent et al. (2010), \"Stacked Denoising Autoencoders\"](https://www.jmlr.org/papers/volume11/vincent10a/vincent10a.pdf).\n```python\nimport pandas as pd\nfrom kaggler.preprocessing import DAE\n\ntrn = pd.read_csv('train.csv')\ntst = pd.read_csv('test.csv')\ntarget_col = trn.columns[-1]\ncat_cols = [col for col in trn.columns if trn[col].dtype == 'object']\nnum_cols = [col for col in trn.columns if col not in cat_cols + [target_col]]\n\n# Default DAE with only the swapping noise and a single encoder/decoder pair.\ndae = DAE(cat_cols=cat_cols, num_cols=num_cols, n_encoding=128)\nX = dae.fit_transform(pd.concat([trn, tst], axis=0))    # encoding input features into the encoding vectors with size of 128\n\n# Stacked DAE with the Gaussian noise, swapping noise and zero masking in 3 pairs of the encoder/decoder.\nsdae = DAE(cat_cols=cat_cols, num_cols=num_cols, n_encoding=128, n_layer=3,\n           noise_std=.05, swap_prob=.2, mask_prob=.1)\nX = sdae.fit_transform(pd.concat([trn, tst], axis=0))\n\n# Supervised DAE with the Gaussian noise, swapping noise and zero masking in 3 encoders in the encoder/decoder pair.\nsdae = SDAE(cat_cols=cat_cols, num_cols=num_cols, n_encoding=128, n_encoder=3,\n           noise_std=.05, swap_prob=.2, mask_prob=.1)\nX = sdae.fit_transform(trn, trn[target_col])\n\n```\n\n## AutoML\n\n### Feature Selection \u0026 Hyperparameter Tuning\n```python\nimport pandas as pd\nfrom sklearn.datasets import make_classification\nfrom sklearn.model_selection import train_test_split\nfrom kaggler.metrics import auc\nfrom kaggler.model import AutoLGB\n\n\nRANDOM_SEED = 42\nN_OBS = 10000\nN_FEATURE = 100\nN_IMP_FEATURE = 20\n\nX, y = make_classification(n_samples=N_OBS,\n                            n_features=N_FEATURE,\n                            n_informative=N_IMP_FEATURE,\n                            random_state=RANDOM_SEED)\nX = pd.DataFrame(X, columns=['x{}'.format(i) for i in range(X.shape[1])])\ny = pd.Series(y)\n\nX_trn, X_tst, y_trn, y_tst = train_test_split(X, y,\n                                                test_size=.2,\n                                                random_state=RANDOM_SEED)\n\nmodel = AutoLGB(objective='binary', metric='auc')\nmodel.tune(X_trn, y_trn)\nmodel.fit(X_trn, y_trn)\np = model.predict(X_tst)\nprint('AUC: {:.4f}'.format(auc(y_tst, p)))\n\n```\n\n## Ensemble\n\n### Netflix Blending\n```python\nimport numpy as np\nfrom kaggler.ensemble import netflix\nfrom kaggler.metrics import rmse\n\n# Load the predictions of input models for ensemble\np1 = np.loadtxt('model1_prediction.txt')\np2 = np.loadtxt('model2_prediction.txt')\np3 = np.loadtxt('model3_prediction.txt')\n\n# Calculate RMSEs of model predictions and all-zero prediction.\n# At a competition, RMSEs (or RMLSEs) of submissions can be used.\ny = np.loadtxt('target.txt')\ne0 = rmse(y, np.zeros_like(y))\ne1 = rmse(y, p1)\ne2 = rmse(y, p2)\ne3 = rmse(y, p3)\n\np, w = netflix([e1, e2, e3], [p1, p2, p3], e0, l=0.0001) # l is an optional regularization parameter.\n```\n\n\n## Algorithms\nCurrently algorithms available are as follows:\n\n### Online learning algorithms\n* Stochastic Gradient Descent (SGD)\n* Follow-the-Regularized-Leader (FTRL)\n* Factorization Machine (FM)\n* Neural Networks (NN) - with a single (NN) or two (NN_H2) ReLU hidden layers\n* Decision Tree\n\n### Batch learning algorithm\n* Neural Networks (NN) - with a single hidden layer and L-BFGS optimization\n\n### Examples\n```python\nfrom kaggler.online_model import SGD, FTRL, FM, NN\n\n# SGD\nclf = SGD(a=.01,                # learning rate\n          l1=1e-6,              # L1 regularization parameter\n          l2=1e-6,              # L2 regularization parameter\n          n=2**20,              # number of hashed features\n          epoch=10,             # number of epochs\n          interaction=True)     # use feature interaction or not\n\n# FTRL\nclf = FTRL(a=.1,                # alpha in the per-coordinate rate\n           b=1,                 # beta in the per-coordinate rate\n           l1=1.,               # L1 regularization parameter\n           l2=1.,               # L2 regularization parameter\n           n=2**20,             # number of hashed features\n           epoch=1,             # number of epochs\n           interaction=True)    # use feature interaction or not\n\n# FM\nclf = FM(n=1e5,                 # number of features\n         epoch=100,             # number of epochs\n         dim=4,                 # size of factors for interactions\n         a=.01)                 # learning rate\n\n# NN\nclf = NN(n=1e5,                 # number of features\n         epoch=10,              # number of epochs\n         h=16,                  # number of hidden units\n         a=.1,                  # learning rate\n         l2=1e-6)               # L2 regularization parameter\n\n# online training and prediction directly with a libsvm file\nfor x, y in clf.read_sparse('train.sparse'):\n    p = clf.predict_one(x)      # predict for an input\n    clf.update_one(x, p - y)    # update the model with the target using error\n\nfor x, _ in clf.read_sparse('test.sparse'):\n    p = clf.predict_one(x)\n\n# online training and prediction with a scipy sparse matrix\nfrom kaggler import load_data\n\nX, y = load_data('train.sps')\n\nclf.fit(X, y)\np = clf.predict(X)\n```\n\n## Data I/O\nKaggler supports CSV (`.csv`), LibSVM (`.sps`), and HDF5 (`.h5`) file formats:\n```\n# CSV format: target,feature1,feature2,...\n1,1,0,0,1,0.5\n0,0,1,0,0,5\n\n# LibSVM format: target feature-index1:feature-value1 feature-index2:feature-value2\n1 1:1 4:1 5:0.5\n0 2:1 5:1\n\n# HDF5\n- issparse: binary flag indicating whether it stores sparse data or not.\n- target: stores a target variable as a numpy.array\n- shape: available only if issparse == 1. shape of scipy.sparse.csr_matrix\n- indices: available only if issparse == 1. indices of scipy.sparse.csr_matrix\n- indptr: available only if issparse == 1. indptr of scipy.sparse.csr_matrix\n- data: dense feature matrix if issparse == 0 else data of scipy.sparse.csr_matrix\n```\n\n```python\nfrom kaggler.data_io import load_data, save_data\n\nX, y = load_data('train.csv')\t# use the first column as a target variable\nX, y = load_data('train.h5')\t# load the feature matrix and target vector from a HDF5 file.\nX, y = load_data('train.sps')\t# load the feature matrix and target vector from LibSVM file.\n\nsave_data(X, y, 'train.csv')\nsave_data(X, y, 'train.h5')\nsave_data(X, y, 'train.sps')\n```\n\n## Documentation\nPackage documentation is available at [here](https://kaggler.readthedocs.io/en/latest/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeongyoonlee%2Fkaggler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjeongyoonlee%2Fkaggler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeongyoonlee%2Fkaggler/lists"}