{"id":13642603,"url":"https://github.com/splunk/mltk-algo-contrib","last_synced_at":"2025-10-11T09:57:02.488Z","repository":{"id":33955582,"uuid":"119612606","full_name":"splunk/mltk-algo-contrib","owner":"splunk","description":null,"archived":false,"fork":false,"pushed_at":"2023-07-06T21:07:49.000Z","size":110,"stargazers_count":89,"open_issues_count":8,"forks_count":26,"subscribers_count":47,"default_branch":"master","last_synced_at":"2025-04-20T16:38:47.849Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/splunk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-31T00:33:01.000Z","updated_at":"2025-03-27T01:46:55.000Z","dependencies_parsed_at":"2024-05-05T03:42:10.709Z","dependency_job_id":null,"html_url":"https://github.com/splunk/mltk-algo-contrib","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/splunk/mltk-algo-contrib","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/splunk%2Fmltk-algo-contrib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/splunk%2Fmltk-algo-contrib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/splunk%2Fmltk-algo-contrib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/splunk%2Fmltk-algo-contrib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/splunk","download_url":"https://codeload.github.com/splunk/mltk-algo-contrib/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/splunk%2Fmltk-algo-contrib/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006748,"owners_count":26084185,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:33.747Z","updated_at":"2025-10-11T09:57:02.440Z","avatar_url":"https://github.com/splunk.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# mltk-algo-contrib\n\nThis repo contains custom algorithms for use with the [Splunk Machine Learning Toolkit](https://splunkbase.splunk.com/app/2890/). The repo itself is also a Splunk app.\nCustom algorithms can be added to the Splunk Machine Learning toolkit by adhering to the [ML-SPL API](http://docs.splunk.com/Documentation/MLApp/latest/API/Introduction).\nThe API is a thin wrapper around machine learning estimators provided by libraries such as:\n* [scikit-learn](scikit-learn.org)\n* [statsmodels](http://www.statsmodels.org/).\n* [scipy](https://www.scipy.org)\n\nand custom algorithms.\n\nNote that this repo is a collection of custom *algorithms* only, and not any libraries. Any libraries required\nshould only be added to live environments manually and not to this repo.\n\nA comprehensive guide to using the ML-SPL API can be found [here](http://docs.splunk.com/Documentation/MLApp/latest/API/Introduction).\n\nA very simple example:\n\n```python\nfrom base import BaseAlgo\n\n\nclass CustomAlgorithm(BaseAlgo):\n    def __init__(self, options):\n        # Option checking \u0026 initializations here\n        pass\n\n    def fit(self, df, options):\n        # Fit an estimator to df, a pandas DataFrame of the search results\n        pass\n\n    def partial_fit(self, df, options):\n        # Incrementally fit a model\n        pass\n\n    def apply(self, df, options):\n        # Apply a saved model\n        # Modify df, a pandas DataFrame of the search results\n        return df\n\n    @staticmethod\n    def register_codecs():\n        # Add codecs to the codec manager\n        pass\n\n```\n\n# Dependencies\n\nTo use the custom algorithms contained in this app, you must also have installed:\n\n - [Splunk Machine Learning Toolkit](https://splunkbase.splunk.com/app/2890/) \n - Python for Scientific Computing Add-on\n    - [Linux64](https://splunkbase.splunk.com/app/2882/)\n    - [Linux32](https://splunkbase.splunk.com/app/2884/)\n    - [Windows64](https://splunkbase.splunk.com/app/2883/)\n    - [macOS](https://splunkbase.splunk.com/app/2881/)\n\n# Usage\nThis repository is contains public contributions and Splunk is not responsible for guaranteeing\nthe correctness or validity of the algorithms. Splunk is in no way responsible for the vetting of\nthe contents of contributed algorithms.\n\n# Deploying\n\nTo use the custom algorithms in this repository, you must deploy them as a Splunk app.\n\nThere are two ways to do this.\n\n### Manual copying\n\nYou can simple copy the following directories under src:\n  * bin\n  * default\n  * metadata\n\nto:\n  * ${SPLUNK_HOME}/etc/apps/SA_mltk_contrib_app (you will need to create the directory first):\n\nOR\n\n### Build and install\n\n#### 1. Build the app:\n\nYou will need to install tox. See [Test Prerequisites](#prereq)\n\n```bash\ntox -e package-macos        # if on Mac\ntox -e package-linux        # if on Linux\n```\n\n  * The resulting gzipped tarball will be in the `target` directory (e.g. target/SA_mltk_contrib_app.tgz).\n    * The location of the gzipped tarball can be overridden by `BUILD_DIR` environment variable.\n  * The default app name will be `SA_mltk_contrib_app`, but this can be overridden by the `APP_NAME` environment variable.\n\n* **NOTE**: You can run `tox -e clean` to remove the `target` directory.\n\n#### 2. Install the tarball:\n\n  * You can do one of the followings with the tarball from step 1:\n    * Manually untar it in `${SPLUNK_HOME}/etc/apps` directory\n    * Install it using the GUI:\n      * https://docs.splunk.com/Documentation/AddOns/released/Overview/Singleserverinstall\n\n# Contributing\n\nThis repository was specifically made for your contributions! See [Contributing](https://github.com/splunk/mltk-algo-contrib/blob/master/CONTRIBUTING.md) for more details.\n\n## Developing\n\nTo start developing, you will need to have Splunk installed. If you don't, read more [here](http://docs.splunk.com/Documentation/Splunk/latest/Installation/InstallonLinux).\n\n1. clone the repo and cd into the directory:\n\n```bash\ngit clone https://github.com/splunk/mltk-algo-contrib.git\ncd mltk-algo-contrib\n```\n\n2. symlink the `src` directory to the apps folder in Splunk and restart splunkd:\n\n```bash\nln -s \"$(pwd)/src\" $SPLUNK_HOME/etc/apps/SA_mltk_contrib_app\n$SPLUNK_HOME/bin/splunk restart\n```\n  * _This will eliminate the need to deploy the app to test changes._\n\n3. Add your new algorithm(s) to `src/bin/algos_contrib`.\n  (See SVR.py for an example.)\n  \n4. Add a new stanza to `src/default/algos.conf`\n\n```bash\n[\u003cyour_algo\u003e]\npackage=algos_contrib\n```\n\n  * **NOTE**: Due to the way configuration file layering works in Splunk,\n  the package name must be overridden in each section, and not\n  in the _default_ section.\n    \n5. Add your tests to `src/bin/algos_contrib/tests/test_\u003cyour_algo\u003e.py`\n  (See test_svr.py for an example.)\n\n## Running Tests\n\n\u003ca name=prereq\u003e\u003c/a\u003e\n### Prerequisites\n\n1. Install *tox*:\n   * http://tox.readthedocs.io/en/latest/install.html\n   ```bash\n   pip install tox\n   ```\n\n2. Install *tox-pip-extensions*:\n   * https://github.com/tox-dev/tox-pip-extensions\n   ```bash\n   pip install tox-pip-extensions\n   ```\n   * **NOTE**: You only need this if you do not want to\n   recreate the virtualenv(s) manually with `tox -r`\n   everytime you update requirements*.txt file, but\n   this is recommended for convenience.\n\n3. You must also have the following environment variable set to your\nSplunk installation directory (e.g. /opt/splunk):\n   * SPLUNK_HOME\n\n### Using tox\n\nTo run all tests, run the following command in the root source directory:\n\n```bash\ntox\n```\n\nTo run a single test, you can provide the directory or a file as a parameter:\n\n```bash\ntox src/bin/algos_contrib/tests/\ntox src/bin/algos_contrib/tests/test_example_algo.py\n...\n```\n\nBasically, any arguments passed to *tox* will be passed as an argument to the *pytest* command.\nTo pass in options, use double dashes (--):\n\n```bash\ntox -- -k \"example\"     # Run tests that has keyword 'example'\ntox -- -x               # Stop after the first failure\ntox -- -s               # Show stdout/stderr (i.e. disable capturing)\n...\n```\n\n### Using Python REPL (Interactive Interpreter)\n\n```python\n$ python   # from src/bin directory\n\u003e\u003e\u003e # Add the MLTK to our sys.path\n\u003e\u003e\u003e from link_mltk import add_mltk\n\u003e\u003e\u003e add_mltk()\n\u003e\u003e\u003e\n\u003e\u003e\u003e # Import our algorithm class\n\u003e\u003e\u003e from algos_contrib.ExampleAlgo import ExampleAlgo\n... (some warning from Splunk may show up)\n\u003e\u003e\u003e\n\u003e\u003e\u003e # Use utilities to catch common mistakes\n\u003e\u003e\u003e from test.contrib_util import AlgoTestUtils\n\u003e\u003e\u003e AlgoTestUtils.assert_algo_basic(ExampleAlgo, serializable=False)\n```\n\n### Package/File Naming\n\nFiles and packages under _test_ directory should avoid having names\nthat conflict with files or directories directly under:\n```bash\n$SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/bin\n```\n\n## Pull requests\n\nOnce you've finished what you're adding, make a pull request.\n\n## Bugs? Issues?\n\nPlease file issues with any information that might be needed to:\n - reproduce what you're experiencing\n - understand the problem fully\n\n# License\n\nThe algorithms hosted, as well as the app itself, is licensed under the permissive Apache 2.0 license.\n\n**Any additions to this repository must be under one of these licenses:**\n - MIT\n - BSD\n - Apache 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsplunk%2Fmltk-algo-contrib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsplunk%2Fmltk-algo-contrib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsplunk%2Fmltk-algo-contrib/lists"}