{"id":18123271,"url":"https://github.com/jay-johnson/spylunking","last_synced_at":"2026-03-17T15:27:47.689Z","repository":{"id":57470395,"uuid":"138234447","full_name":"jay-johnson/spylunking","owner":"jay-johnson","description":"Drill down into your python logs using JSON logs stored in Splunk - supports sending over TCP or the Splunk HEC REST API handlers (using threads or multiprocessing) - includes a pre-configured Splunk sandbox in a docker container","archived":false,"fork":false,"pushed_at":"2022-10-18T16:39:38.000Z","size":165,"stargazers_count":12,"open_issues_count":2,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-20T18:25:34.108Z","etag":null,"topics":["docker","python","python-logger","python-logging","splunk","splunk-hec","splunk-http","splunk-sdk","splunk-searches"],"latest_commit_sha":null,"homepage":"https://spylunking.readthedocs.io/en/latest/index.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jay-johnson.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-22T00:10:40.000Z","updated_at":"2024-03-22T16:55:39.000Z","dependencies_parsed_at":"2022-09-19T10:02:03.205Z","dependency_job_id":null,"html_url":"https://github.com/jay-johnson/spylunking","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jay-johnson/spylunking","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jay-johnson%2Fspylunking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jay-johnson%2Fspylunking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jay-johnson%2Fspylunking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jay-johnson%2Fspylunking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jay-johnson","download_url":"https://codeload.github.com/jay-johnson/spylunking/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jay-johnson%2Fspylunking/sbom","scorecard":{"id":508735,"data":{"date":"2025-08-11","repo":{"name":"github.com/jay-johnson/spylunking","commit":"bbc066c7490fea0bfa552383bbfdd774b2eadd3c"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-19T23:56:47.115Z","repository_id":57470395,"created_at":"2025-08-19T23:56:47.115Z","updated_at":"2025-08-19T23:56:47.115Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30626813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-17T14:16:03.965Z","status":"ssl_error","status_checked_at":"2026-03-17T14:16:03.380Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","python","python-logger","python-logging","splunk","splunk-hec","splunk-http","splunk-sdk","splunk-searches"],"created_at":"2024-11-01T07:08:55.642Z","updated_at":"2026-03-17T15:27:47.622Z","avatar_url":"https://github.com/jay-johnson.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Spylunking - Splunk + Python Logging\n------------------------------------\n\nDrill down into your logs with an integrated, colorized logger and search tools set up with the included Splunk docker sandbox.\n\nThis repository creates Splunk-ready, colorized Python loggers that work with a Splunk TCP Port or the Splunk HEC REST API. Both of these endpoints are automatically set up for use with the included docker container. \n\n.. image:: https://imgur.com/SUdcyWf.png\n    :alt: Splunk web app Python logs from the Spylunking test app\n\nSample Log Handlers\n===================\n\nDepending on your application's use case you can use one of the included Python logging handlers:\n\n- `TCP Splunk Publisher \u003chttps://github.com/jay-johnson/spylunking/blob/master/spylunking/tcp_splunk_publisher.py\u003e`__\n- `Threaded Splunk Publisher \u003chttps://github.com/jay-johnson/spylunking/blob/master/spylunking/splunk_publisher.py\u003e`__\n- `Multiprocessing Splunk Publisher \u003chttps://github.com/jay-johnson/spylunking/blob/master/spylunking/mp_splunk_publisher.py\u003e`__\n\nThe log publishing and search tools support using existing Splunk tokens or logging in using the configured user and password arguments or from environment variables. \n\nSample Log Config JSON Files\n============================\n\nHere are the sample logging config JSON files:\n\n- `TCP Splunk Publisher Log Config \u003chttps://github.com/jay-johnson/spylunking/blob/master/spylunking/log/shared-logging.json\u003e`__\n- `Threaded Splunk Publisher Log Config \u003chttps://github.com/jay-johnson/spylunking/blob/master/spylunking/log/threads-shared-logging.json\u003e`__\n- `Multiprocessing Splunk Publisher Log Config \u003chttps://github.com/jay-johnson/spylunking/blob/master/spylunking/log/mp-shared-logging.json\u003e`__\n\n.. list-table::\n   :header-rows: 1\n\n   * - Travis Build\n     - Read the Docs\n   * - .. image:: https://travis-ci.org/jay-johnson/spylunking.svg?branch=master\n           :alt: Travis Test Status\n           :target: https://travis-ci.org/jay-johnson/spylunking\n     - .. image:: https://readthedocs.org/projects/spylunking/badge/?version=latest\n           :alt: Read the Docs Status\n           :target: http://spylunking.readthedocs.io/en/latest/\n\nGetting Started\n===============\n\n#.  Clone the repo\n\n    ::\n\n        git clone https://github.com/jay-johnson/spylunking.git spylunking\n        cd spylunking\n\n#.  Install the pip \n\n    ::\n\n        pip install spylunking\n\n    If you want to develop use this command:\n\n    ::\n\n        pip install -e .\n\n#.  Start the Splunk docker container\n\n    ::\n\n       ./run-splunk-in-docker.sh \n\nGet a Splunk User Token\n-----------------------\n\nBy default the container creates a user with the credentials:\n\nusername: **trex**\npassword: **123321**\n\n::\n\n    get_splunk_token.py\n    955324da-742b-43d4-9746-bcbedf6ae7f4\n\nSet the Splunk Environment Variables\n\n::\n\n    export SPLUNK_INDEX=antinex\n    export SPLUNK_TOKEN=955324da-742b-43d4-9746-bcbedf6ae7f4\n\nPlease wait at least 30 seconds while the container is getting ready. You may see output like this when the ``splunk`` container is not ready yet or stops running:\n\n::\n\n    get_splunk_token.py \n    Traceback (most recent call last):\n    File \"\u003credacted path for doc\u003e\", line 171, in _new_conn\n        (self._dns_host, self.port), self.timeout, **extra_kw)\n    File \"\u003credacted path for doc\u003e\", line 79, in create_connection\n        raise err\n    File \"\u003credacted path for doc\u003e\", line 69, in create_connection\n        sock.connect(sa)\n    ConnectionRefusedError: [Errno 111] Connection refused\n\nPublishing Logs to Splunk using the Spylunking Logger\n-----------------------------------------------------\n\nBelow is a video showing how to tag your application's logs using the ``LOG_NAME`` environment variable. Doing this allows you to quickly find them in Splunk using the included ``sp`` command line tool.\n\n.. image:: https://asciinema.org/a/189711.png\n    :target: https://asciinema.org/a/189711?autoplay=1\n    :alt: Publishing Logs to Splunk using the Spylunking Logger\n\nCommands from the video:\n\n#.  Set an Application Log Name\n\n    ::\n\n        export LOG_NAME=payments\n\n#.  Search for Logs in Splunk\n\n    ::\n\n        sp -q 'index=\"antinex\" AND name=payments | head 5 | reverse'\n        No matches for search={\n            \"search\": \"search index=\\\"antinex\\\" AND name=payments | head 5 | reverse\"\n        } response={\n            \"init_offset\": 0,\n            \"messages\": [],\n            \"post_process_count\": 0,\n            \"preview\": false,\n            \"results\": []\n        }\n\n#.  Send Test Logs to Splunk\n\n    ::\n\n        test_logging.py \n        2018-07-02 09:18:22,197 - helloworld - INFO - testing INFO message_id=93e33f10-ebbf-49a1-a87a-a76858448c71\n        2018-07-02 09:18:22,199 - helloworld - ERROR - testing ERROR message_id=3b3f0362-f146-47b4-9fff-c6cc3b165279\n        2018-07-02 09:18:22,200 - helloworld - CRITICAL - testing CRITICAL message_id=8870f39e-82b5-4071-b19a-80ce6cfefbd6\n        2018-07-02 09:18:22,201 - helloworld - WARNING - testing WARNING message_id=6ab745cb-8a14-41ae-b16e-13c0c80c4963\n        2018-07-02 09:18:22,201 - helloworld - ERROR - Testing EXCEPTION with ex=Throw for testing exceptions message_id=26b3c421-46b7-49d2-960b-1ca2ed7b8e03\n\n#.  Search for Test Logs in Splunk\n\n    ::\n\n        sp -q 'index=\"antinex\" AND name=payments | head 5 | reverse'\n        2018-07-02 09:18:22,197 helloworld - INFO - testing INFO message_id=93e33f10-ebbf-49a1-a87a-a76858448c71 \n        2018-07-02 09:18:22,199 helloworld - ERROR - testing ERROR message_id=3b3f0362-f146-47b4-9fff-c6cc3b165279 \n        2018-07-02 09:18:22,200 helloworld - CRITICAL - testing CRITICAL message_id=8870f39e-82b5-4071-b19a-80ce6cfefbd6 \n        2018-07-02 09:18:22,201 helloworld - WARNING - testing WARNING message_id=6ab745cb-8a14-41ae-b16e-13c0c80c4963 \n        2018-07-02 09:18:22,201 helloworld - ERROR - Testing EXCEPTION with ex=Throw for testing exceptions message_id=26b3c421-46b7-49d2-960b-1ca2ed7b8e03 \n\nGet Splunk Logs from the Command Line Tool\n------------------------------------------\n\nUse the command line tool: **sp** to search for recent logs.\n\n#.  Set environment variables:\n\n    ::\n\n        export SPLUNK_ADDRESS=\"splunkenterprise:8088\"\n        export SPLUNK_API_ADDRESS=\"splunkenterprise:8089\"\n        export SPLUNK_PASSWORD=\"123321\"\n        export SPLUNK_USER=\"trex\"\n\n    .. note:: The remainder of this guide was recorded by running the splunk container on a remote vm and then setting the environment variables for the search tool ``sp`` and the spylunking logger to work. If you are running the container locally, either add ``splunkenterprise`` to ``/etc/hosts`` at the end of the ``127.0.0.1`` line or export these environment variables to work with the local splunk container: ``export SPLUNK_ADDRESS:localhost:8088`` and ``export SPLUNK_API_ADDRESS=localhost:8089``.\n\n#.  Run the tool:\n\n    ::\n\n        sp\n\n    Which will log something like:\n\n    ::\n\n        sp - INFO - No matches for search={\n            \"search\": \"search index=\\\"antinex\\\" | head 10\"\n        }\n        sp - INFO - done\n\nWrite Splunk Logs\n-----------------\n\nBy default, the container creates a Splunk index called: **antinex** with a user token for the user **trex** to search the index. Once the Splunk container is running, you can use the included **test_logging.py** script to create sample logs to verify the Splunk logging integration is working. The default logger will send logs over TCP using the `TCP Splunk Publisher \u003chttps://github.com/jay-johnson/spylunking/blob/master/spylunking/tcp_splunk_publisher.py\u003e`__. To change this, you can export the optional environment variable ``SHARED_LOG_CFG`` to the absolute path of another logging config JSON file like:\n\n::\n\n    export SHARED_LOG_CFG=\u003cabsolute path to logging config JSON file\u003e\n\nSend logs using the command: ``test_logging.py``\n\n::\n\n    test_logging.py \n    2018-06-24 01:07:36,378 - testingsplunk - INFO - testing INFO message_id=ce9c91dc-3af9-484d-aeb0-fc09194bb42e\n    2018-06-24 01:07:36,379 - testingsplunk - ERROR - testing ERROR message_id=9227cc2f-f734-4b99-8448-117776ef6bff\n    2018-06-24 01:07:36,379 - testingsplunk - CRITICAL - testing CRITICAL message_id=7271a65d-d563-4231-b24a-b17364044818\n    2018-06-24 01:07:36,379 - testingsplunk - WARNING - testing WARN message_id=54063058-dba1-47ee-a0ab-d654b3140e55\n    2018-06-24 01:07:36,379 - testingsplunk - ERROR - Testing EXCEPTION with ex=Throw for testing exceptions message_id=c1e100f4-202d-48ac-9803-91c4f02c9a92\n\nGet the Test Splunk Logs using the Command Line Tool\n----------------------------------------------------\n\nThe command line tool called ``sp`` is included with the pip on install. When you run it, it will return the most recent logs from the index (``antinex`` by default) and print them to stdout.\n\n::\n\n    sp\n\nIf you want to pull logs from splunk with user credentials (``SPLUNK_USER`` and ``SPLUNK_PASSWORD`` as environment variables works too):\n\n::\n\n    sp -u trex -p 123321 -a splunkenterprise:8089\n\nRunning ``sp`` should return something like these test logs:\n\n::\n\n    sp -u trex -p 123321 -a splunkenterprise:8089\n\n    sp - ERROR - testingsplunk.testingsplunk 2018-06-24 01:07:36,379 - Testing EXCEPTION with ex=Throw for testing exceptions message_id=c1e100f4-202d-48ac-9803-91c4f02c9a92 dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=41 ex=None\n    sp - CRITICAL - testingsplunk.testingsplunk 2018-06-24 01:07:36,379 - testing CRITICAL message_id=7271a65d-d563-4231-b24a-b17364044818 dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=31 ex=None\n    sp - ERROR - testingsplunk.testingsplunk 2018-06-24 01:07:36,379 - testing ERROR message_id=9227cc2f-f734-4b99-8448-117776ef6bff dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=29 ex=None\n    sp - INFO - testingsplunk.testingsplunk 2018-06-24 01:07:36,378 - testing INFO message_id=ce9c91dc-3af9-484d-aeb0-fc09194bb42e dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=27 ex=None\n    sp - INFO - done\n\nExamples\n--------\n\nPull Logs with a Query on the Command Line\n==========================================\n\n::\n\n    sp -q 'index=\"antinex\" AND levelname=INFO | head 10' \\\n        -u trex -p 123321 -a splunkenterprise:8089\n    sp - INFO - testingsplunk.testingsplunk 2018-06-24 01:40:18,313 - testing INFO message_id=74b8fe93-ce07-4b8f-a700-dcf4665416d3 dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=27 ex=None\n    sp - INFO - testingsplunk.testingsplunk 2018-06-24 01:25:19,162 - testing INFO message_id=766e1408-1252-47e2-99db-e3154f5b915a dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=27 ex=None\n    sp - INFO - testingsplunk.testingsplunk 2018-06-24 01:07:36,378 - testing INFO message_id=ce9c91dc-3af9-484d-aeb0-fc09194bb42e dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=27 ex=None\n    sp - INFO - done\n\nPull Logs with a Query on the Command Line\n==========================================\n\nGet CRITICAL logs\n=================\n\n::\n\n    sp -q 'index=\"antinex\" AND levelname=\"CRITICAL\"'\n\nGet First 10 ERROR logs\n=======================\n\n::\n\n    sp -q 'index=\"antinex\" AND levelname=\"ERROR\" | head 10' \\\n        -u trex -p 123321 -a splunkenterprise:8089\n\nRunning ``sp`` also works if you want to view the full json fields:\n\n::\n\n    sp -j -u trex -p 123321 -a splunkenterprise:8089\n\n    sp - ERROR - {\n        \"asctime\": \"2018-06-24 01:07:36,379\",\n        \"custom_key\": \"custom value\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"ERROR\",\n        \"lineno\": 41,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"Testing EXCEPTION with ex=Throw for testing exceptions message_id=c1e100f4-202d-48ac-9803-91c4f02c9a92\",\n        \"name\": \"testingsplunk\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529827656.3798487\n    }\n    sp - CRITICAL - {\n        \"asctime\": \"2018-06-24 01:07:36,379\",\n        \"custom_key\": \"custom value\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"CRITICAL\",\n        \"lineno\": 31,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"testing CRITICAL message_id=7271a65d-d563-4231-b24a-b17364044818\",\n        \"name\": \"testingsplunk\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529827656.3794894\n    }\n    sp - ERROR - {\n        \"asctime\": \"2018-06-24 01:07:36,379\",\n        \"custom_key\": \"custom value\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"ERROR\",\n        \"lineno\": 29,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"testing ERROR message_id=9227cc2f-f734-4b99-8448-117776ef6bff\",\n        \"name\": \"testingsplunk\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529827656.3792682\n    }\n    sp - INFO - {\n        \"asctime\": \"2018-06-24 01:07:36,378\",\n        \"custom_key\": \"custom value\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"INFO\",\n        \"lineno\": 27,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"testing INFO message_id=ce9c91dc-3af9-484d-aeb0-fc09194bb42e\",\n        \"name\": \"testingsplunk\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529827656.3789432\n    }\n    sp - INFO - done\n\nRunning Stats Commands like Counting Log Matches\n------------------------------------------------\n\nAfter running a few million logs through the Splunk container you can count the number of matches using ``sp``:\n\n::\n\n    sp -q 'index=\"antinex\" | stats count'\n    {\n        \"count\": \"9261227\"\n    }\n\nSplunk Client Load Testing\n--------------------------\n\nIf you are looking to tune your Splunk client logging performance, then please check out the `included load tester \u003chttps://github.com/jay-johnson/spylunking/blob/448d62e641f114104361bf380f37629cf57fe0c0/spylunking/scripts/start_logging_load_test.py#L5\u003e`__ to validate the deployed configuration will not fail to publish log messages (if that is required for your client).\n\nBefore using this in production, please note it is possible to overflow the current python queues during something like an extended Splunk maintenance window or if the client is publishing logs over an unreliable network connection. The default configuration is only going to queue up to 1 million log messages before starting to drop new logs. Another way to test this is if your application is writing logs faster than the Splunk REST API can keep up, then eventually it will overflow the queue's default depth. If you are concerned about not losing log messages, then the logger should set a `flush interval \u003chttps://github.com/jay-johnson/spylunking/blob/448d62e641f114104361bf380f37629cf57fe0c0/spylunking/log/shared-logging.json#L52\u003e`__ of ``0`` to disable the asynchronous, threaded queue support. This will put the client logger into a blocking mode and ensure there are no missed log messages. Please consider that this change will only create blocking log publishers where the ``retry_count`` and ``timeout`` values should be tuned to your application's needs to prevent slow application performance while waiting on the client's HTTP requests to acknowledge each log was received.\n\nHere is how to start a single process load tester:\n\n::\n\n    ./spylunking/scripts/start_logging_loader.py\n    2018-06-28 22:01:47,702 - load-test-2018_06_29_05_01_47 - INFO - INFO message_id=acdbfd0a-6349-4c2e-959c-f49572fc94ca\n    2018-06-28 22:01:47,702 - load-test-2018_06_29_05_01_47 - ERROR - ERROR message_id=7daf8a8e-0d8d-4aa8-9ed1-313cd5dfb421\n    2018-06-28 22:01:47,702 - load-test-2018_06_29_05_01_47 - CRITICAL - CRITICAL message_id=a27e7778-94be-4a35-9ce2-279403b7cf60\n    2018-06-28 22:01:47,703 - load-test-2018_06_29_05_01_47 - WARNING - WARN message_id=d4f39765-5812-4e2e-b7ce-857b231f79d4\n\nLogging to Splunk from a Python Shell\n-------------------------------------\n\nHere are python commands to build a colorized, splunk-ready python logger. On startup, the logger will authenticate with splunk using the provided credentials. Once authenticated you can use it like a normal logger.\n\n.. note:: The ``build_colorized_logger`` and ``search`` method also support authentication using a pre-existing ``splunk_token=\u003ctoken string\u003e`` or by setting a ``SPLUNK_TOKEN`` environment key\n\n.. code-block:: python\n\n    python -c '\\\n        import json;\\\n        from spylunking.log.setup_logging import build_colorized_logger;\\\n        import spylunking.search as sp;\\\n        from spylunking.ppj import ppj;\\\n        print(\"build the logger\");\\\n        log = build_colorized_logger(\\\n            name=\"spylunking-in-a-shell\",\\\n            splunk_user=\"trex\", \\\n            splunk_password=\"123321\");\\\n        print(\"import the search wrapper\");\\\n        res = sp.search(\\\n            user=\"trex\",\\\n            password=\"123321\",\\\n            address=\"splunkenterprise:8089\",\\\n            query_dict={\\\n                \"search\": \"search index=\\\"antinex\\\" | head 1\"\\\n            });\\\n        print(\"pretty print the first record in the result list\");\\\n        log.critical(\"found search results={}\".format(ppj(json.loads(res[\"record\"][\"results\"][0][\"_raw\"]))))'\n\nHere is sample output from running this command:\n\n::\n\n    build the logger\n    import the search wrapper\n    pretty print the first record in the result list\n    2018-06-21 22:38:38,475 - spylunking-in-a-shell - CRITICAL - found search results={\n        \"asctime\": \"2018-06-21 22:13:36,279\",\n        \"custom_key\": \"custom value\",\n        \"exc\": null,\n        \"filename\": \"\u003cstdin\u003e\",\n        \"levelname\": \"INFO\",\n        \"lineno\": 1,\n        \"logger_name\": \"spylunking-in-a-shell\",\n        \"message\": \"testing from a python shell\",\n        \"name\": \"spylunking-in-a-shell\",\n        \"path\": \"\u003cstdin\u003e\",\n        \"tags\": [],\n        \"timestamp\": 1529644416.2790444\n    }\n\nHere it is from a python shell:\n\n::\n\n    python\n    Python 3.6.5 (default, Apr  1 2018, 05:46:30) \n    [GCC 7.3.0] on linux\n    Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n    \u003e\u003e\u003e from spylunking.log.setup_logging import build_colorized_logger\n    \u003e\u003e\u003e log = build_colorized_logger(\n            name='spylunking-in-a-shell',\n            splunk_user='trex',\n            splunk_password='123321')\n    \u003e\u003e\u003e import spylunking.search as sp\n    \u003e\u003e\u003e res = sp.search(\n            user='trex',\n            password='123321',\n            address=\"splunkenterprise:8089\",\n            query_dict={\n                'search': 'search index=\"antinex\" | head 1'\n            })\n    \u003e\u003e\u003e from spylunking.ppj import ppj\n    \u003e\u003e\u003e log.critical('found search results={}'.format(ppj(json.loads(res['record']['results'][0]['_raw']))))\n    2018-06-21 22:31:04,231 - spylunking-in-a-shell - CRITICAL - found search results={\n        \"asctime\": \"2018-06-21 22:13:36,279\",\n        \"custom_key\": \"custom value\",\n        \"exc\": null,\n        \"filename\": \"\u003cstdin\u003e\",\n        \"levelname\": \"INFO\",\n        \"lineno\": 1,\n        \"logger_name\": \"spylunking-in-a-shell\",\n        \"message\": \"testing from a python shell\",\n        \"name\": \"spylunking-in-a-shell\",\n        \"path\": \"\u003cstdin\u003e\",\n        \"tags\": [],\n        \"timestamp\": 1529644416.2790444\n    }\n\nPublishing Logs to a Remote Splunk Server\n-----------------------------------------\n\nSet up the environment variables:\n\n::\n\n    export SPLUNK_API_ADDRESS=\"splunkenterprise:8089\"\n    export SPLUNK_ADDRESS=\"splunkenterprise:8088\"\n    export SPLUNK_USER=\"trex\"\n    export SPLUNK_PASSWORD=\"123321\"\n\nRun the test tool to verify logs are published:\n\n::\n\n    test_logging.py \n    2018-06-24 01:07:36,378 - testingsplunk - INFO - testing INFO message_id=ce9c91dc-3af9-484d-aeb0-fc09194bb42e\n    2018-06-24 01:07:36,379 - testingsplunk - ERROR - testing ERROR message_id=9227cc2f-f734-4b99-8448-117776ef6bff\n    2018-06-24 01:07:36,379 - testingsplunk - CRITICAL - testing CRITICAL message_id=7271a65d-d563-4231-b24a-b17364044818\n    2018-06-24 01:07:36,379 - testingsplunk - WARNING - testing WARN message_id=54063058-dba1-47ee-a0ab-d654b3140e55\n    2018-06-24 01:07:36,379 - testingsplunk - ERROR - Testing EXCEPTION with ex=Throw for testing exceptions message_id=c1e100f4-202d-48ac-9803-91c4f02c9a92\n\nGet the logs with ``sp``\n\n::\n\n    sp -a splunkenterprise:8089\n\nWhich should return the newly published logs:\n\n::\n\n    sp - ERROR - testingsplunk.testingsplunk 2018-06-24 01:07:36,379 - Testing EXCEPTION with ex=Throw for testing exceptions message_id=c1e100f4-202d-48ac-9803-91c4f02c9a92 dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=41 ex=None\n    sp - CRITICAL - testingsplunk.testingsplunk 2018-06-24 01:07:36,379 - testing CRITICAL message_id=7271a65d-d563-4231-b24a-b17364044818 dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=31 ex=None\n    sp - ERROR - testingsplunk.testingsplunk 2018-06-24 01:07:36,379 - testing ERROR message_id=9227cc2f-f734-4b99-8448-117776ef6bff dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=29 ex=None\n    sp - INFO - testingsplunk.testingsplunk 2018-06-24 01:07:36,378 - testing INFO message_id=ce9c91dc-3af9-484d-aeb0-fc09194bb42e dc= env= source=/opt/spylunking/spylunking/scripts/test_logging.py line=27 ex=None\n    sp - INFO - done\n\nSet up a Logger\n---------------\n\nThere are multiple loggers avaiable depending on the type of logger that is needed.\n\nSimple Logger\n-------------\n\nBuild a simple, no dates colorized logger that prints just the message in colors and does not publish logs to Splunk using:\n\n.. code-block:: python\n\n    from spylunking.log.setup_logging import simple_logger\n    log = simple_logger()\n    log.info('simple logger example')\n    simple logger example\n\nNo Date Colorized Logger\n------------------------\n\nBuild a colorized logger that preserves the parent application name and log level without a date field and does not publish logs to Splunk using:\n\n.. code-block:: python\n\n    from spylunking.log.setup_logging import no_date_colors_logger\n    log = no_date_colors_logger(name='app-name')\n    log.info('no date with colors logger example')\n    app-name - INFO - no date with colors logger example\n\nTest Logger\n-----------\n\nThe test logger is for unittests and does not publish to Splunk.\n\n.. code-block:: python\n\n    from spylunking.log.setup_logging import test_logger\n    log = test_logger(name='unittest logger')\n    log.info('unittest log line')\n    2018-06-25 16:01:50,118 - using-a-colorized-logger - INFO - colorized logger example\n\nConsole Logger\n--------------\n\nThe console logger is the same as the ``build_colorized_logger`` which can be created with authenticated Splunk-ready logging using:\n\n.. code-block:: python\n\n    from spylunking.log.setup_logging import build_colorized_logger\n    log = build_colorized_logger(name='using-a-colorized-logger')\n    log.info('colorized logger example')\n    2018-06-25 16:47:54,053 - unittest logger - INFO - unittest log line\n\nDefine Custom Fields for Splunk\n-------------------------------\n\nYou can export a custom JSON dictionary to send as JSON fields for helping drill down on log lines using this environment variable.\n\n::\n\n    export LOG_FIELDS_DICT='{\"name\":\"hello-world\",\"dc\":\"k8-splunk\",\"env\":\"development\"}'\n\nOr you can export the following environment variables if you just want a couple set in the logs:\n\n::\n\n    export LOG_NAME=\u003capplication log name\u003e\n    export DEPLOY_CONFIG=\u003cPaaS/CaaS deployment config name\u003e\n    export ENV_NAME\u003cdeployed environment name\u003e\n\nLog some new test messages to Splunk:\n\n::\n\n    test_logging.py \n    2018-06-25 20:48:51,367 - testingsplunk - INFO - testing INFO message_id=0c5e2a2c-9553-4c8a-8fff-8d77de2be78a\n    2018-06-25 20:48:51,368 - testingsplunk - ERROR - testing ERROR message_id=0dc1086d-4fe4-4062-9882-e822f9256d6f\n    2018-06-25 20:48:51,368 - testingsplunk - CRITICAL - testing CRITICAL message_id=0c0f56f2-e87f-41a0-babb-b71e2b9d5d5a\n    2018-06-25 20:48:51,368 - testingsplunk - WARNING - testing WARN message_id=59b099eb-8c0d-40d0-9d3a-7dfa13fefc90\n    2018-06-25 20:48:51,368 - testingsplunk - ERROR - Testing EXCEPTION with ex=Throw for testing exceptions message_id=70fc422d-d33b-4a9e-bb51-ed86aa0a02f9\n\nOnce published, you can search for these new logs using those new JSON fields with the ``sp`` search tool. Here is an example of searching for the logs with the application log name ``hello-world``:\n\n::\n\n    sp -q 'index=\"antinex\" AND name=hello-world'\n    2018-06-25 20:48:51,368 testingsplunk - ERROR - Testing EXCEPTION with ex=Throw for testing exceptions message_id=70fc422d-d33b-4a9e-bb51-ed86aa0a02f9 \n    2018-06-25 20:48:51,368 testingsplunk - CRITICAL - testing CRITICAL message_id=0c0f56f2-e87f-41a0-babb-b71e2b9d5d5a \n    2018-06-25 20:48:51,368 testingsplunk - ERROR - testing ERROR message_id=0dc1086d-4fe4-4062-9882-e822f9256d6f \n    2018-06-25 20:48:51,367 testingsplunk - INFO - testing INFO message_id=0c5e2a2c-9553-4c8a-8fff-8d77de2be78a \n    done\n\nAnd you can view log the full JSON dictionaries using the ``-j`` argument on the ``sp`` command:\n\n::\n\n    sp -q 'index=\"antinex\" AND name=hello-world' -j\n    {\n        \"asctime\": \"2018-06-25 20:48:51,368\",\n        \"custom_key\": \"custom value\",\n        \"dc\": \"k8-deploy\",\n        \"env\": \"development\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"ERROR\",\n        \"lineno\": 41,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"Testing EXCEPTION with ex=Throw for testing exceptions message_id=70fc422d-d33b-4a9e-bb51-ed86aa0a02f9\",\n        \"name\": \"hello-world\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529984931.3688767\n    }\n    {\n        \"asctime\": \"2018-06-25 20:48:51,368\",\n        \"custom_key\": \"custom value\",\n        \"dc\": \"k8-deploy\",\n        \"env\": \"development\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"CRITICAL\",\n        \"lineno\": 31,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"testing CRITICAL message_id=0c0f56f2-e87f-41a0-babb-b71e2b9d5d5a\",\n        \"name\": \"hello-world\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529984931.3684626\n    }\n    {\n        \"asctime\": \"2018-06-25 20:48:51,368\",\n        \"custom_key\": \"custom value\",\n        \"dc\": \"k8-deploy\",\n        \"env\": \"development\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"ERROR\",\n        \"lineno\": 29,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"testing ERROR message_id=0dc1086d-4fe4-4062-9882-e822f9256d6f\",\n        \"name\": \"hello-world\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529984931.3682773\n    }\n    {\n        \"asctime\": \"2018-06-25 20:48:51,367\",\n        \"custom_key\": \"custom value\",\n        \"dc\": \"k8-deploy\",\n        \"env\": \"development\",\n        \"exc\": null,\n        \"filename\": \"test_logging.py\",\n        \"levelname\": \"INFO\",\n        \"lineno\": 27,\n        \"logger_name\": \"testingsplunk\",\n        \"message\": \"testing INFO message_id=0c5e2a2c-9553-4c8a-8fff-8d77de2be78a\",\n        \"name\": \"hello-world\",\n        \"path\": \"/opt/spylunking/spylunking/scripts/test_logging.py\",\n        \"tags\": [],\n        \"timestamp\": 1529984931.3679354\n    }\n    done\n\nAvailable Environment Variables\n-------------------------------\n\nDrill down fields\n=================\n\nSplunk drill down fields with environment variables:\n\n::\n\n    export LOG_NAME=\"\u003capplication log name\u003e\"\n    export DEPLOY_CONFIG=\"\u003capplication deployed config like k8 filename\u003e\"\n    export ENV_NAME=\"\u003cenvironment name for this application\u003e\"\n\nCommon Environment Variables\n============================\n\n::\n\n    export SPLUNK_USER=\"\u003csplunk host\u003e\"\n    export SPLUNK_PASSWORD=\"\u003csplunk host\u003e\"\n    export SPLUNK_HOST=\"\u003csplunk host\u003e\"\n    export SPLUNK_PORT=\"\u003csplunk port: 8088\u003e\"\n    export SPLUNK_API_PORT=\"\u003csplunk port: 8089\u003e\"\n    export SPLUNK_ADDRESS=\"\u003csplunk address host:port\u003e\"\n    export SPLUNK_API_ADDRESS=\"\u003csplunk api address host:port\u003e\"\n    export SPLUNK_TOKEN=\"\u003csplunk token\u003e\"\n    export SPLUNK_INDEX=\"\u003csplunk index\u003e\"\n    export SPLUNK_SOURCE=\"\u003csplunk source\u003e\"\n    export SPLUNK_SOURCETYPE=\"\u003csplunk sourcetype\u003e\"\n    export SPLUNK_VERIFY=\"\u003cverify certs on HTTP POST\u003e\"\n    export SPLUNK_TIMEOUT=\"\u003ctimeout in seconds\u003e\"\n    export SPLUNK_QUEUE_SIZE=\"\u003cnum msgs allowed in queue - 0=infinite\u003e\"\n    export SPLUNK_SLEEP_INTERVAL=\"\u003csleep in seconds per batch\u003e\"\n    export SPLUNK_RETRY_COUNT=\"\u003cattempts per log to retry publishing\u003e\"\n    export SPLUNK_RETRY_BACKOFF=\"\u003ccooldown in seconds per failed POST\u003e\"\n    export SPLUNK_DEBUG=\"\u003cdebug the publisher - 1 enable debug|0 off\u003e\"\n    export SPLUNK_VERBOSE=\"\u003cdebug the sp command line tool - 1 enable|0 off\u003e\"\n\nDebug the Publishers\n====================\n\nExport this variable before creating a logger to see the publisher logs:\n\n::\n\n    export SPLUNK_DEBUG=1\n\nLogin to Splunk from a Browser\n------------------------------\n\nOpen this url in a browser to view the **splunk** container's web application:\n\nhttp://127.0.0.1:8000\n\nLogin with the credentials:\n\nusername: **trex**\npassword: **123321**\n\nTroubleshooting\n---------------\n\nSplunk Handler Dropping Logs\n============================\n\nIf the splunk handler is dropping log messages you can use these values to tune the handler's worker thread:\n\n::\n\n    export SPLUNK_RETRY_COUNT=\"\u003cnumber of attempts to send logs\u003e\"\n    export SPLUNK_TIMEOUT=\"\u003ctimeout in seconds per attempt\u003e\"\n    export SPLUNK_QUEUE_SIZE=\"\u003cinteger value or 0 for infinite\u003e\"\n    export SPLUNK_SLEEP_INTERVAL=\"\u003cseconds to sleep between publishes\u003e\"\n    export SPLUNK_DEBUG=\"\u003cdebug the Splunk Publisher by setting to 1\u003e\"\n\nTesting in a Python Shell\n=========================\n\nHere is a debugging python shell session for showing some common errors you can expect to see as you start to play around with ``spylunking``.\n\n::\n\n    python\n    Python 3.6.5 (default, Apr  1 2018, 05:46:30)\n    [GCC 7.3.0] on linux\n    Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n    \u003e\u003e\u003e from spylunking.log.setup_logging import build_colorized_logger\n    \u003e\u003e\u003e log = build_colorized_logger(\n            name='spylunking-in-a-shell',\n            splunk_user='trex',\n            splunk_password='123321')\n    \u003e\u003e\u003e log.info(\"testing from a python shell\")\n    2018-06-21 22:13:36,279 - spylunking-in-a-shell - INFO - testing from a python shell\n    \u003e\u003e\u003e import spylunking.search as sp\n    \u003e\u003e\u003e res = sp.search(\n            user='trex',\n            password='123321',\n            query_dict={\n                    'search': 'index=\"antinex\" | head 1'\n            },\n            verify=False)\n    \u003e\u003e\u003e log.info('job status={}'.format(res['status']))\n    2018-06-21 22:16:22,158 - spylunking-in-a-shell - INFO - job status=2\n    \u003e\u003e\u003e log.info('job err={}'.format(res['err']))\n    2018-06-21 22:16:28,945 - spylunking-in-a-shell - INFO - job err=Failed to get splunk token for user=trex url=https://None ex=HTTPSConnectionPool(host='none', port=443): Max retries exceeded with url: /services/auth/login (Caused by NewConnectionError('\u003curllib3.connection.VerifiedHTTPSConnection object at 0x7f869c2f2cc0\u003e: Failed to establish a new connection: [Errno -2] Name or service not known',))\n    \u003e\u003e\u003e print(\"now search with the url set\")\n    now search with the url set\n    \u003e\u003e\u003e res = sp.search(\n            user='trex',\n            password='123321',\n            query_dict={\n                    'search': 'index=\"antinex\" | head 1'\n            },\n            address=\"splunkenterprise:8089\")\n    2018-06-21 22:18:15,380 - spylunking.search - ERROR - Failed searching splunk response=\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n    \u003cresponse\u003e\n    \u003cmessages\u003e\n        \u003cmsg type=\"ERROR\"\u003eSearch Factory: Unknown search command 'index'.\u003c/msg\u003e\n    \u003c/messages\u003e\n    \u003c/response\u003e\n    for query={\n        \"search\": \"index=\\\"antinex\\\" | head 1\"\n    } url=https://splunkenterprise:8089/services/search/jobs ex=list index out of range\n    \u003e\u003e\u003e print(\"now nest the search correctly\")\n    now nest the search correctly\n    \u003e\u003e\u003e res = sp.search(\n            user='trex',\n            password='123321',\n            address=\"splunkenterprise:8089\",\n            query_dict={\n                    'search': 'search index=\"antinex\" | head 1'\n            })\n    \u003e\u003e\u003e log.info('job status={}'.format(res['status']))\n    2018-06-21 22:20:10,142 - spylunking-in-a-shell - INFO - job status=0\n    \u003e\u003e\u003e log.info('job err={}'.format(res['err']))\n    2018-06-21 22:20:14,667 - spylunking-in-a-shell - INFO - job err=\n    \u003e\u003e\u003e from spylunking.ppj import ppj\n    \u003e\u003e\u003e log.critical('found search results={}'.format(ppj(res['record'])))\n    2018-06-21 22:21:25,977 - spylunking-in-a-shell - CRITICAL - found search results={\n        \"fields\": [\n            {\n                \"name\": \"_bkt\"\n            },\n            {\n                \"name\": \"_cd\"\n            },\n            {\n                \"name\": \"_indextime\"\n            },\n            {\n                \"name\": \"_raw\"\n            },\n            {\n                \"name\": \"_serial\"\n            },\n            {\n                \"name\": \"_si\"\n            },\n            {\n                \"name\": \"_sourcetype\"\n            },\n            {\n                \"name\": \"_subsecond\"\n            },\n            {\n                \"name\": \"_time\"\n            },\n            {\n                \"name\": \"host\"\n            },\n            {\n                \"name\": \"index\"\n            },\n            {\n                \"name\": \"linecount\"\n            },\n            {\n                \"name\": \"source\"\n            },\n            {\n                \"name\": \"sourcetype\"\n            },\n            {\n                \"name\": \"splunk_server\"\n            }\n        ],\n        \"highlighted\": {},\n        \"init_offset\": 0,\n        \"messages\": [],\n        \"preview\": false,\n        \"results\": [\n            {\n                \"_bkt\": \"antinex~0~791398E7-6A0B-4640-B8D5-5D25E7EF3D02\",\n                \"_cd\": \"0:3\",\n                \"_indextime\": \"1529644419\",\n                \"_raw\": \"{\\\"asctime\\\": \\\"2018-06-21 22:13:36,279\\\", \\\"name\\\": \\\"spylunking-in-a-shell\\\", \\\"levelname\\\": \\\"INFO\\\", \\\"message\\\": \\\"testing from a python shell\\\", \\\"filename\\\": \\\"\u003cstdin\u003e\\\", \\\"lineno\\\": 1, \\\"timestamp\\\": 1529644416.2790444, \\\"path\\\": \\\"\u003cstdin\u003e\\\", \\\"custom_key\\\": \\\"custom value\\\", \\\"tags\\\": [], \\\"exc\\\": null, \\\"logger_name\\\": \\\"spylunking-in-a-shell\\\"}\",\n                \"_serial\": \"0\",\n                \"_si\": [\n                    \"splunkenterprise\",\n                    \"antinex\"\n                ],\n                \"_sourcetype\": \"json\",\n                \"_subsecond\": \".2792356\",\n                \"_time\": \"2018-06-22T05:13:36.279+00:00\",\n                \"host\": \"dev\",\n                \"index\": \"antinex\",\n                \"linecount\": \"1\",\n                \"source\": \"\u003cstdin\u003e\",\n                \"sourcetype\": \"json\",\n                \"splunk_server\": \"splunkenterprise\"\n            }\n        ]\n    }\n    \u003e\u003e\u003e exit()\n\nPlease refer to the command line tool's updated usage prompt for help searching for logs:\n\n::\n\n    usage: sp [-h] [-u USER] [-p PASSWORD] [-f DATAFILE] [-i INDEX_NAME]\n          [-a ADDRESS] [-e EARLIEST_TIME_MINUTES] [-l LATEST_TIME_MINUTES]\n          [-q [QUERY_ARGS [QUERY_ARGS ...]]] [-j] [-m] [-v] [-b]\n\n    Search Splunk\n\n    optional arguments:\n    -h, --help            show this help message and exit\n    -u USER               username\n    -p PASSWORD           user password\n    -f DATAFILE           splunk-ready request in a json file\n    -i INDEX_NAME         index to search\n    -a ADDRESS            host address: \u003cfqdn:port\u003e\n    -e EARLIEST_TIME_MINUTES\n                            (Optional) earliest_time minutes back\n    -l LATEST_TIME_MINUTES\n                            (Optional) latest_time minutes back\n    -q [QUERY_ARGS [QUERY_ARGS ...]], --queryargs [QUERY_ARGS [QUERY_ARGS ...]]\n                            query string for searching splunk: search\n                            index=\"antinex\" AND levelname=\"ERROR\"\n    -j                    (Optional) view as json dictionary logs\n    -m                    (Optional) verbose message when getting logs\n    -v                    (Optional) verify certs - disabled by default\n    -b                    verbose\n\nFor trying the host-only compose file, you may see errors like:\n\n``unable to resolve host splunkenterprise``\n\nPlease add ``splunkenterprise`` to the end of the line for ``127.0.0.1`` in your ``/etc/hosts``\n\nCleanup\n-------\n\nRemove the docker container with the commands:\n\n::\n\n    docker stop splunk\n    docker rm splunk\n\n\nManual Splunk Commands\n======================\n\nCreate Token\n\n::\n\n    curl -k -u admin:changeme https://splunkenterprise:8089/servicesNS/admin/splunk_httpinput/data/inputs/http -d name=antinex-token \n\nList Token\n\n::\n\n    curl -k -u admin:changeme https://splunkenterprise:8089/servicesNS/admin/splunk_httpinput/data/inputs/http\n\nUsing Splunk CLI\n================\n\nList Tokens\n\n::\n\n    ./bin/splunk http-event-collector list -uri 'https://splunkenterprise:8089' -auth 'admin:changeme'\n\nAdd Index\n\n::\n\n    ./bin/splunk add index antinex -auth 'admin:changeme'\n\nCreate Token\n\n::\n\n    ./bin/splunk \\\n        http-event-collector create  \\\n        antinex-token 'antinex logging token'  \\\n        -index antinex \\\n        -uri 'https://splunkenterprise:8089' \\\n        -auth 'admin:changeme'\n\nDevelopment\n-----------\n\nSetting up your development environment (right now this demo is using virtualenv):\n\n::\n\n    virtualenv -p python3 ~/.venvs/spylunk \u0026\u0026 source ~/.venvs/spylunk/bin/activate \u0026\u0026 pip install -e .\n\nTesting\n-------\n\nRun all\n\n::\n\n    py.test\n\nLinting\n-------\n\nflake8 .\n\npycodestyle .\n\nLicense\n-------\n\nApache 2.0 - Please refer to the LICENSE_ for more details\n\n.. _License: https://github.com/jay-johnson/spylunking/blob/master/LICENSE\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjay-johnson%2Fspylunking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjay-johnson%2Fspylunking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjay-johnson%2Fspylunking/lists"}