{"id":49882852,"url":"https://github.com/mljar/features_goldmine","last_synced_at":"2026-05-15T16:06:39.595Z","repository":{"id":353320223,"uuid":"1218913916","full_name":"mljar/features_goldmine","owner":"mljar","description":"Features Engineering Made Easy","archived":false,"fork":false,"pushed_at":"2026-04-23T11:18:02.000Z","size":296,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-23T12:25:45.198Z","etag":null,"topics":["data-science","features-engineering","machine-learning"],"latest_commit_sha":null,"homepage":"https://mljar.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mljar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-23T10:43:07.000Z","updated_at":"2026-04-23T11:18:06.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mljar/features_goldmine","commit_stats":null,"previous_names":["mljar/features_goldmine"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/mljar/features_goldmine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mljar%2Ffeatures_goldmine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mljar%2Ffeatures_goldmine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mljar%2Ffeatures_goldmine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mljar%2Ffeatures_goldmine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mljar","download_url":"https://codeload.github.com/mljar/features_goldmine/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mljar%2Ffeatures_goldmine/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33071623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-15T11:35:32.926Z","status":"ssl_error","status_checked_at":"2026-05-15T11:35:31.362Z","response_time":103,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","features-engineering","machine-learning"],"created_at":"2026-05-15T16:06:38.875Z","updated_at":"2026-05-15T16:06:39.583Z","avatar_url":"https://github.com/mljar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![features-goldmine banner](https://raw.githubusercontent.com/mljar/features_goldmine/main/assets/features_goldmine_banner.jpg)\n\n# Features Goldmine\n\nAre you stuck building your ML pipeline? Are you searching for creative ideas for new features? Looking for a quick, easy, and performant way to do feature engineering?\n\nWe would like to introduce `features_goldmine`, a Python package built exactly for that problem. It runs multiple feature engineering strategies on your raw tabular data, creates new candidate features, filters weak ideas, and proposes only the features that are worth checking.\n\nThe main goal of `features_goldmine` is simple: improve ML pipeline accuracy with minimal code changes.\n\n## California Housing Example\n\nUse your existing train/test split, generate golden features in one line, retrain your model.\n\n```python\nfrom features_goldmine import GoldenFeatures\n\n# 1) baseline model on X_train / X_test\n# 2) add golden features\n\ngf = GoldenFeatures(verbose=1, selectivity=\"balanced\")\nX_train_gold = gf.fit_transform(X_train, y_train)\nX_test_gold = gf.transform(X_test)\n\nX_train_aug = pd.concat([X_train, X_train_gold], axis=1)\nX_test_aug = pd.concat([X_test, X_test_gold], axis=1)\n\n# 3) train your model on augmented data and compare metric\n```\n\nReal run from this repo:\n\n```bash\nuv run python examples/california_housing.py\n```\n\n```text\nBaseline RMSE (single split): 0.450530\nGolden   RMSE (single split): 0.447588\nDelta RMSE (golden - baseline): -0.002942\nRMSE improvement vs baseline: +0.65%\n```\n\nSelected examples of created features:\n\n```text\nAveOccup_div_MedInc\nMedInc_div_AveOccup\nctx_raw_Longitude_z_k15\nHouseAge_mul_MedInc\nLatitude_mul_Longitude\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eFull California Housing training output\u003c/summary\u003e\n\n```text\nuv run python examples/california_housing.py\nDataset: california_housing\nRows=20640, Features=8\nMode: single train/test split (test_size=0.25), selectivity=balanced, gf_verbose=1\n[GoldenFeatures] fit: validating inputs\n[GoldenFeatures] fit: task=regression, rows=15480, total_features=8, numeric_features=8, categorical_features=0, selectivity=relaxed, max_selected_features=None, enabled_strategies=['categorical_frequency', 'categorical_group_deviation', 'categorical_hash_cross', 'categorical_oof_target', 'categorical_prototypes', 'context_knn', 'grouped_row_stats', 'path', 'projection_ica', 'projection_pca', 'residual_numeric']\n[GoldenFeatures] stage1: training fast LightGBM on raw features (3 repeats)\n[GoldenFeatures] stage1: repeat=1/3, seed=42, paths=1223\n[GoldenFeatures] stage1: repeat=2/3, seed=143, paths=1247\n[GoldenFeatures] stage1: repeat=3/3, seed=244, paths=1245\n[GoldenFeatures] stage1: trained with 8 raw features\n[GoldenFeatures] stage2: extracted total 3715 paths across repeats\n[GoldenFeatures] stage3: ranking feature interactions\n[GoldenFeatures] stage3: ranked 28 interaction pairs\n[GoldenFeatures] stage4: building candidate engineered features (enabled=['categorical_frequency', 'categorical_group_deviation', 'categorical_hash_cross', 'categorical_oof_target', 'categorical_prototypes', 'context_knn', 'grouped_row_stats', 'path', 'projection_ica', 'projection_pca', 'residual_numeric'])\n[GoldenFeatures] stage4: generated 160 path candidates + 3 projection candidates + 3 ica candidates + 20 grouped-stats candidates + 30 context-knn candidates + 50 residual candidates = 266 total\n[GoldenFeatures] stage5: quick filtering candidates\n[GoldenFeatures] stage5: kept 159 candidates, rejected 107\n[GoldenFeatures] stage6: survival competition with repeated LightGBM\n[GoldenFeatures] stage6: 16 candidates survived\n[GoldenFeatures] stage7: redundancy pruning\n[GoldenFeatures] stage7: final survivors after pruning = 15\n[GoldenFeatures] fit: completed (candidates=266, after_filter=159, survivors=16, final=15)\n[GoldenFeatures] transform: generating 15 golden features\n[GoldenFeatures] transform: generating 15 golden features\n[Split] created=15 features: ['AveOccup_div_MedInc', 'MedInc_div_AveOccup', 'MedInc_div_AveRooms', 'Latitude_div_Longitude', 'ctx_raw_Longitude_z_k15', 'HouseAge_mul_MedInc', 'grpstat_002_mean', 'AveOccup_absdiff_MedInc', 'AveRooms_div_MedInc', 'AveRooms_sub_MedInc', 'Latitude_mul_Longitude', 'AveBedrms_mul_MedInc', 'AveBedrms_sub_Longitude', 'Latitude_sub_Longitude', 'AveOccup_sub_MedInc']\nBaseline RMSE (single split): 0.450530\nGolden   RMSE (single split): 0.447588\nDelta RMSE (golden - baseline): -0.002942\nRMSE improvement vs baseline: +0.65%\n```\n\n\u003c/details\u003e\n\n## Breast Cancer Binary Example\n\n```bash\nuv run python examples/breast_cancer_binary.py\n```\n\n```text\nBaseline AUC (single split): 0.992872\nGolden   AUC (single split): 0.995178\nDelta AUC (golden - baseline): +0.002306\nAUC improvement vs baseline: +0.23%\n```\n\nSelected examples of created features:\n\n```text\nworst_area_mul_worst_concave_points\nworst_perimeter_mul_worst_smoothness\nrule_worst_perimeter_le_112p800_and_worst_concave_points_le_0p146_013\nworst_area_mul_worst_texture\narea_error_mul_mean_concave_points\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eFull Breast Cancer Binary training output\u003c/summary\u003e\n\n```text\nuv run python examples/breast_cancer_binary.py\nDataset: breast_cancer\nRows=569, Features=30\nMode: single train/test split (test_size=0.25), selectivity=balanced, gf_verbose=1\n[GoldenFeatures] fit: validating inputs\n[GoldenFeatures] fit: task=binary, rows=426, total_features=30, numeric_features=30, categorical_features=0, selectivity=balanced, max_selected_features=None, enabled_strategies=['categorical_frequency', 'categorical_group_deviation', 'categorical_hash_cross', 'categorical_oof_target', 'categorical_prototypes', 'context_knn', 'grouped_row_stats', 'path', 'projection_ica', 'projection_pca', 'residual_numeric']\n[GoldenFeatures] stage1: training fast LightGBM on raw features (3 repeats)\n[GoldenFeatures] stage1: repeat=1/3, seed=42, paths=572\n[GoldenFeatures] stage1: repeat=2/3, seed=143, paths=567\n[GoldenFeatures] stage1: repeat=3/3, seed=244, paths=572\n[GoldenFeatures] stage1: trained with 30 raw features\n[GoldenFeatures] stage2: extracted total 1711 paths across repeats\n[GoldenFeatures] stage3: ranking feature interactions\n[GoldenFeatures] stage3: ranked 235 interaction pairs\n[GoldenFeatures] stage4: building candidate engineered features (enabled=['categorical_frequency', 'categorical_group_deviation', 'categorical_hash_cross', 'categorical_oof_target', 'categorical_prototypes', 'context_knn', 'grouped_row_stats', 'path', 'projection_ica', 'projection_pca', 'residual_numeric'])\n[GoldenFeatures] stage4: generated 170 path candidates + 5 projection candidates + 5 ica candidates + 48 grouped-stats candidates + 30 context-knn candidates = 258 total\n[GoldenFeatures] stage5: quick filtering candidates\n[GoldenFeatures] stage5: kept 170 candidates, rejected 88\n[GoldenFeatures] stage6: survival competition with repeated LightGBM\n[GoldenFeatures] stage6: 13 candidates survived\n[GoldenFeatures] stage7: redundancy pruning\n[GoldenFeatures] stage7: final survivors after pruning = 13\n[GoldenFeatures] fit: completed (candidates=258, after_filter=170, survivors=13, final=13)\n[GoldenFeatures] transform: generating 13 golden features\n[GoldenFeatures] transform: generating 13 golden features\n[Split] created=13 features: ['worst_area_mul_worst_concave_points', 'worst_perimeter_mul_worst_smoothness', 'rule_worst_perimeter_le_112p800_and_worst_concave_points_le_0p146_013', 'worst_area_mul_worst_texture', 'area_error_mul_mean_concave_points', 'grpstat_004_mean', 'worst_perimeter_mul_worst_texture', 'area_error_mul_worst_concave_points', 'mean_texture_mul_worst_radius', 'worst_perimeter_mul_worst_symmetry', 'mean_concave_points_mul_worst_texture', 'ica_comp_003', 'mean_texture_mul_worst_area']\nBaseline AUC (single split): 0.992872\nGolden   AUC (single split): 0.995178\nDelta AUC (golden - baseline): +0.002306\nAUC improvement vs baseline: +0.23%\n```\n\n\u003c/details\u003e\n\n## Other Examples\n\nThese example scripts are included in this repository:\n\n```bash\nuv run python examples/breast_cancer_binary.py\nuv run python examples/credit_scoring.py\nuv run python examples/house_prices_rmse.py\n```\n\n\n## How It Works\n\n`features_goldmine` starts with your raw tabular data: a pandas `DataFrame` `X` and target `y`.\n\nFirst, the package generates many candidate features using several strategies. Some strategies look for interactions discovered by LightGBM tree paths. Others create numeric transformations, projection features, row-group statistics, categorical encodings, categorical-numeric deviation features, and context-style features.\n\nNext, `features_goldmine` runs a fast initial filtering step. This removes candidates that are obviously not useful: constant columns, near-constant columns, invalid values, too many missing values, duplicates, and features that are too similar to their parent columns.\n\nFinally, it trains several small LightGBM models and lets the candidates compete against the raw features. Candidate features are selected based on repeated feature importance: features that consistently receive useful gain and rank highly across runs are kept. Redundant survivors are pruned, and the final output is a clean DataFrame containing only the selected engineered features.\n\nIn short:\n\n```text\nraw X, y\n  -\u003e generate many candidate features\n  -\u003e remove obviously bad candidates\n  -\u003e train several small LightGBM models\n  -\u003e keep candidates with strong, stable importance\n  -\u003e return final golden features\n```\n\n![features engineering pipeline](https://raw.githubusercontent.com/mljar/features_goldmine/main/assets/features_engineering_pipeline.jpg)\n\n## Performance\n\nWe tested `features_goldmine` by comparing two models:\n\n- `LGBM_Baseline`: a simple LightGBM model trained on raw data.\n- `LGBM_GoldenFeatures`: the same LightGBM parameters, trained on raw data plus golden features.\n\nThe comparison was run on [TabArena Lite](https://github.com/autogluon/tabarena), across 51 datasets. Lower `metric_error` is better.\n\n```text\nLGBM_GoldenFeatures vs LGBM_Baseline\nBetter: 27 datasets\nWorse : 24 datasets\nWin rate: 52.9%\n```\n\nThe practical takeaway: golden features help often, but not always. Feature engineering is data-dependent, so `features_goldmine` is designed to make it fast and easy to check whether engineered features improve your pipeline.\n\n\n## Simple API\n\n```python\nfrom features_goldmine import GoldenFeatures\n\ngf = GoldenFeatures()\nX_gold = gf.fit_transform(X, y)\n```\n\nConstructor arguments:\n\n```python\ngf = GoldenFeatures(\n    random_state=42,\n    verbose=0,\n    selectivity=\"balanced\",\n    max_selected_features=None,\n    include_strategies=None,\n    exclude_strategies=None,\n)\n```\n\n- `random_state`\n  - Controls randomness for repeatable results.\n  - Use the same value if you want the same generated features across runs.\n- `verbose`\n  - Set to `1` to print detailed logs for every stage.\n  - Keep `0` for quiet mode.\n- `selectivity`\n  - Controls how strict the feature survival test is.\n  - Options: `relaxed`, `balanced`, `strict`.\n  - `balanced` is the default and a good starting point.\n- `max_selected_features`\n  - Limits the final number of selected golden features.\n  - Example: `max_selected_features=3` keeps only the top 3.\n- `include_strategies`\n  - Optional list of strategies to use.\n  - If `None`, all built-in strategies are enabled.\n- `exclude_strategies`\n  - Optional list of strategies to disable.\n  - Useful when you want to turn off a specific family, for example `categorical_frequency`.\n\nThat is the core API. Three methods:\n\n- `fit(X, y)`\n  - What it does: learns which engineered features are useful from your training data.\n  - Use it when: you want to fit once, then transform multiple datasets later.\n  - Returns: the same `GoldenFeatures` object (`self`).\n- `transform(X)`\n  - What it does: creates the selected engineered features for new data.\n  - Use it when: you already called `fit` (or loaded a fitted model) and now want features for validation/test/production data.\n  - Returns: a DataFrame with only engineered features.\n- `fit_transform(X, y)`\n  - What it does: `fit` + `transform` in one call.\n  - Use it when: you just want engineered features for your training split quickly.\n  - Returns: a DataFrame with only engineered features.\n\nBeginner rule of thumb:\n\n- training split: `fit_transform`\n- validation/test split: `transform`\n\nExample:\n\n```python\ngf = GoldenFeatures()\n\n# train split\nX_train_gold = gf.fit_transform(X_train, y_train)\n\n# validation/test split (same fitted feature logic)\nX_valid_gold = gf.transform(X_valid)\nX_test_gold = gf.transform(X_test)\n```\n\nOther useful methods:\n\n- `save(path)`\n  - What it does: saves a fitted `GoldenFeatures` object to disk.\n  - Use it when: you want to reuse the exact same feature logic later.\n- `GoldenFeatures.load(path)`\n  - What it does: loads a previously saved fitted object.\n  - Use it when: you want consistent features in another script or production job.\n\nUseful attributes:\n\n- `selected_feature_names_`\n- `golden_features_`\n- `report_`\n\n## Strategy Control and Available Strategies\n\nDefaults work well for first use. If needed:\n\n```python\ngf = GoldenFeatures(\n    include_strategies=[\"path\", \"context_knn\", \"categorical_group_deviation\"],\n    exclude_strategies=[\"categorical_frequency\"],\n)\n```\n\nCurrent strategy keys:\n\n- `path`\n  - finds feature interactions that tree models actually used, then creates formulas like multiply/divide/subtract and simple split-based rules.\n  - Usually helps when: non-linear numeric interactions matter.\n- `projection_pca`\n  - creates compact summary features (principal components) from many numeric columns.\n  - Usually helps when: numeric features are correlated and you want cleaner combined signals.\n- `projection_ica`\n  - creates independent numeric components that can reveal hidden patterns different from PCA.\n  - Usually helps when: signals are mixed and not well captured by simple linear combinations.\n- `grouped_row_stats`\n  - computes row-level stats (mean/std/min/max) over related feature groups.\n  - Usually helps when: relative scale inside a group of columns matters.\n- `context_knn`\n  - compares each row to nearby rows in numeric space (local context), generating deviation-style features.\n  - Usually helps when: local neighborhood behavior is informative.\n- `residual_numeric`\n  - for regression, creates numeric interactions that correlate with baseline model residual errors.\n  - Usually helps when: baseline model leaves structured numeric errors.\n- `categorical_frequency`\n  - encodes how common each category value is in the data.\n  - Usually helps when: rare vs common category values carry signal.\n- `categorical_oof_target`\n  - leak-safe target encoding using out-of-fold averages per category.\n  - Usually helps when: category values strongly relate to target.\n- `categorical_group_deviation`\n  - compares a numeric value to what is typical for its category (for example `value - category_mean`).\n  - Usually helps when: \"higher/lower than typical for this category\" matters.\n- `categorical_prototypes`\n  - measures distance between a row and category-specific numeric prototypes.\n  - Usually helps when: each category has a characteristic numeric profile.\n- `categorical_hash_cross`\n  - builds compact crossed-category signals via hashed category pairs.\n  - Usually helps when: interactions between two categorical columns matter but full one-hot crosses would explode.\n\n\n## Install\n\n```bash\n# from PyPI (recommended)\npip install features_goldmine\nuv add features_goldmine\n\n# local editable install (development)\nuv pip install -e .\npip install -e .\n```\n\n## Useful Tricks\n\nYou can experiment with different selection settings. In practice, it is often worth trying a few variants because the best setting depends on your data.\n\nIf you want to increase the number of proposed golden features, try `relaxed`:\n\n```python\ngf = GoldenFeatures(selectivity=\"relaxed\", verbose=1)\n```\n\nIf you want a smaller, more conservative set of features, try `strict`:\n\n```python\ngf = GoldenFeatures(selectivity=\"strict\", verbose=1)\n```\n\nIf you want only the top few golden features, use `max_selected_features`. For example, keep only the top 3:\n\n```python\ngf = GoldenFeatures(max_selected_features=3, verbose=1)\n```\n\n## Related Projects\n\nIf you find this library useful, you might also be interested in other tools from the MLJAR ecosystem:\n\n- **AutoML (mljar-supervised)** – automated model training and hyperparameter tuning  \n  https://github.com/mljar/mljar-supervised\n\n- **Mercury** – turn Jupyter Notebooks into interactive web apps  \n  https://github.com/mljar/mercury\n\n- **MLJAR Studio** – a desktop IDE for data science with built-in AutoML and AI assistant  \n  https://mljar.com/studio\n\n## License\n\nApache 2.0\n\n![features-goldmine footer](https://raw.githubusercontent.com/mljar/features_goldmine/main/assets/features_goldmine_footer.jpg)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmljar%2Ffeatures_goldmine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmljar%2Ffeatures_goldmine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmljar%2Ffeatures_goldmine/lists"}