{"id":18678204,"url":"https://github.com/srl-labs/srl-elk-lab","last_synced_at":"2025-04-12T02:40:38.309Z","repository":{"id":110641565,"uuid":"576032509","full_name":"srl-labs/srl-elk-lab","owner":"srl-labs","description":"Integrating Nokia SR Linux with ELK stack","archived":false,"fork":false,"pushed_at":"2023-06-01T08:41:29.000Z","size":4407,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-05T22:08:19.470Z","etag":null,"topics":["clab-topo","elastic","elk","kibana","logstash","srlinux","syslog"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/srl-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-08T21:19:14.000Z","updated_at":"2025-01-07T01:11:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"810527a8-1a82-4936-be13-1f8d406630f6","html_url":"https://github.com/srl-labs/srl-elk-lab","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srl-labs%2Fsrl-elk-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srl-labs%2Fsrl-elk-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srl-labs%2Fsrl-elk-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srl-labs%2Fsrl-elk-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/srl-labs","download_url":"https://codeload.github.com/srl-labs/srl-elk-lab/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248507465,"owners_count":21115605,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clab-topo","elastic","elk","kibana","logstash","srlinux","syslog"],"created_at":"2024-11-07T09:36:21.592Z","updated_at":"2025-04-12T02:40:38.303Z","avatar_url":"https://github.com/srl-labs.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Intro\n\nThis repo contains containerlab-based labs demonstrating how logs from SR Linux network elements can be collected, parsed, and stored using Elasticsearch/Logstash/Kibana (ELK) stack.\n\nA series of blog posts go into the details of various ELK deployment models:\n\n1. [SR Linux logging with ELK][srk-with-elk-post] - an introduction to the modern logging infrastructure using ELK stack.\n\n## Lab Topology\n\nThe [srl-elk.clab.yml](srl-elk.clab.yml) topology represents a 2-tier Clos fabric with 2 clients participating in a single L2 EVPN domain.\n\n![ELK lab topology][topology]\n\nNaming conventions are straighforward:\n\n* leaf[1-3] - leaves\n* spine[1,2] - spines\n* client[1,2] - emulated clients\n\nclient1 connectivity uses a single interface attached to leaf1.\nclient2 is connected as A/S to leaf2 and leaf3 with standby link signalling using LACP.\n\nspine1 and spine2 are acting as BGP RR. This setup is sufficient to demonstrate a way to integrate a fabric with ELK stack.\n\n## Quick start\n\nIn order to bring up your lab follow the next simple steps:\n\n1. Clone repo\n\n```sh\ngit clone https://github.com/azyablov/srl-elk-lab.git\ncd srl-elk-lab\n```\n\n2. Deploy the lab\n\n```sh\ncd \u003clab folder\u003e\nsudo clab deploy -t srl-elk.clab.yml\n```\n\n3. For the fast and convenient start of demo, dashboard and discover search configuration [objects](./elk/kibana/kibana-dashboard.ndjson) are provided as part of the lab.\n\nRun `add-saved-objects.sh` in order to avoid manual import and creation.\n\n```sh\n./add-saved-objects.sh\n```\n\nDemo dashboard can be adjusted as necessary.\n\n4. Run simulation to quickly ingest data into elasticsearch as described in [Simulation](#simulation)\n\n\u003e Note! Index template is created automatically by logstash (to avoid automatic template creation by elastic).\n\u003e `manage_template` and `template*` configuration option stanzas are defining such logstash behavior.\n\n```r\noutput {\n    if \"srlinux\" in [tags] {\n        if \"_grokparsefailure\" in [tags] {\n            file {\n                path =\u003e \"/srl/fail_to_parse_srl.log\"\n                codec =\u003e rubydebug\n            }\n        } else {\n            elasticsearch {\n                hosts =\u003e [\"http://elastic\"]\n                ssl =\u003e false\n                index =\u003e \"fabric-logs-%{+YYYY.MM.dd}\"\n                manage_template =\u003e true\n                template =\u003e \"/tmp/index-template.json\"\n                template_name =\u003e \"fabric-template\"\n                template_overwrite =\u003e true\n                id =\u003e \"fabric-logs\"\n            }\n        }\n    }\n}\n```\n\n## Simulation\n\nIn order to help quickly enrich ELK stack with logs ```outage_simulation.sh``` script could be executed with the following parameters:\n\n```-S``` - to replace configuration for logstash remote server under ```/system/logging/remote-server[host=$LOGSTASHIP]\"``` with new one.\n\n```\u003cWAITTIMER\u003e``` - to adjust time interval between destructive actions applied (20 sec by default).\n\nBasic configuration can found [here](./sys_log_logstash.json.tmpl), which represent default lab configuration, and can be adjusted per your needs and requirements.\n\n```sh\n./outage_simulation.sh -S\n```\n\nBy default configuration for remote server using UDP:\n\n```json\n    {\n      \"host\": \"172.22.22.11\",\n      \"remote-port\": 1514,\n      \"subsystem\": [\n        {\n          \"priority\": {\n            \"match-above\": \"informational\"\n          },\n          \"subsystem-name\": \"aaa\"\n        },\n        {\n          \"priority\": {\n            \"match-above\": \"informational\"\n          },\n          \"subsystem-name\": \"acl\"\n        },\n\u003c...output omitted for brevity...\u003e\n    }\n```\n\n\u003e Note! In case TLS is a requirement, you can consider to put rsyslog in front, simple docker image with self-signed and custom certificate can be found on [github.com/azyablov/rsyslogbase](https://github.com/azyablov/rsyslogbase)\n\nTo run simulation just execute ```./outage_simulation.sh``` or ```./outage_simulation.sh 15``` in case machine is a bit slow or you have another labs running on the same compute.\n\n![Outage Simulation][outage_simulation]\n\n## Kibana\n\nYour pre-configured Kibana should available via [http://localhost:5601](http://localhost:5601).\nNow you can go to to Discovery and Dashboard under Analytics and see a demo dashboard.\n\n![kibana discovery][kibaba_dashboard]\n\n![kibana dashboard][kibaba_dashboard_2]\n\n[kibaba_dashboard]: ./pic/kibana_dashboard.png \"Kibana dashboard #1\"\n[kibaba_dashboard_2]: ./pic/kibana_dashboard_2.png \"Kibana dashboard #2\"\n[outage_simulation]: ./pic/outage_simulation.gif \"Simulation\"\n[srk-with-elk-post]: https://learn.srlinux.dev/blog/2023/sr-linux-logging-with-elk/\n[topology]: ./pic/topology.png\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrl-labs%2Fsrl-elk-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsrl-labs%2Fsrl-elk-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrl-labs%2Fsrl-elk-lab/lists"}