{"id":30029902,"url":"https://github.com/manticoresoftware/kibana-demo","last_synced_at":"2025-08-06T18:03:58.325Z","repository":{"id":270354210,"uuid":"908275010","full_name":"manticoresoftware/kibana-demo","owner":"manticoresoftware","description":"Demo to compare the performance of Manticore and Elastic when working with Kibana","archived":false,"fork":false,"pushed_at":"2025-02-25T16:53:29.000Z","size":3044,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-02-25T17:38:43.280Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/manticoresoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-25T16:46:28.000Z","updated_at":"2025-02-25T16:53:32.000Z","dependencies_parsed_at":"2025-02-05T07:27:40.208Z","dependency_job_id":"e2e5a76e-14ed-4498-835a-8a9943956d33","html_url":"https://github.com/manticoresoftware/kibana-demo","commit_stats":null,"previous_names":["manticoresoftware/kibana-demo"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/manticoresoftware/kibana-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fkibana-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fkibana-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fkibana-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fkibana-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/manticoresoftware","download_url":"https://codeload.github.com/manticoresoftware/kibana-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manticoresoftware%2Fkibana-demo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269128163,"owners_count":24364927,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-06T02:00:09.910Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-06T18:03:20.602Z","updated_at":"2025-08-06T18:03:58.227Z","avatar_url":"https://github.com/manticoresoftware.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Manticore Search Kibana Demo\n\nThis repository provides a ready-to-use setup to demonstrate the integration of [Manticore Search](https://manticoresearch.com/) with [Kibana](https://www.elastic.co/kibana). It also enables a performance comparison between two Kibana instances connected to Elasticsearch and Manticore, respectively. \n\nHere are our test results with approximately 40 million documents, showing that the same dashboard renders in 7 seconds with Manticore Search, compared to 18 seconds with Elasticsearch.\n\n\u003cimg src=\"manticore1.png\" alt=\"manticore\" width=\"350\" height=\"200\" /\u003e \u003cimg height=\"200\" hspace=\"20\"/\u003e \u003cimg src=\"elastic1.png\" alt=\"elastic\" width=\"350\" height=\"200\" /\u003e\n\nThe demo utilizes a set of Kibana visualizations inspired by [this popular blog post](https://ruan.dev/blog/2019/04/02/setup-kibana-dashboards-for-nginx-log-data-to-understand-the-behavior), which is based on data from an Nginx access log file. The [kscarlett-generator](https://github.com/kscarlett/nginx-log-generator) tool is used to generate fake data for the log file.\n\n## Prerequisites\n\nBefore getting started, ensure you have the following installed:\n\n- [Git](https://git-scm.com/)\n- [Docker Compose](https://docs.docker.com/compose/)\n- Zip and Unzip tools\n\nCommand to install these packages in Ubuntu / Debian:\n```\napt -y update \u0026\u0026 apt -y install git docker-compose zip unzip\n```\n\n## Getting Started\n\n### Clone the Repository\n\n```bash\ngit clone https://github.com/manticoresoftware/kibana-demo.git\ncd kibana-demo\n```\n\n### Configure the Environment\n\nGrant permissions for the Elasticsearch data folder:\n```bash\nchmod 777 es_data\n```\n\n\n#### Generate a New Log File\n\nEdit the `.env` file to set how many log entries to generate. By default, this is 50 million entries.\nNote that loading a large number of entries into Elasticsearch and Manticore can take time.\n\nSet the following in `.env`:\n\n```bash\nLOG_ENTRY_COUNT=50000000\n```\n\nYou can also make it add new log entries after the main volume is uploaded to mimic real-time logging. Use the `UPDATE_RATE` option to set how many log entries are added per second. To turn off this feature, set `UPDATE_RATE` to 0.  \n\n```bash\nUPDATE_RATE=1\n```\n\n#### Custom ports\n\nYou can configure custom ports for the Elasticsearch and Manticore Kibana instances by setting these options in `.env`:\n\n```bash\nES_KIBANA_PORT=5612\nMANTICORE_KIBANA_PORT=5613\n```\n\n### Launch the environment\n\nRun the startup script:\n\n```bash\n./start.sh\n```\n\nWait a few minutes for log generation and index loading to begin.\n\nVerify the services are running:\n- Elasticsearch Kibana: http://localhost:5612\n- Manticore Kibana : http://localhost:5613\n\nChoose `Explore on my own` on the Kibana start screen\n\n### Import Kibana objects\n\nImport the pre-built visualizations into Kibana to use for the generated data.\n\nFor the **Elasticsearch Kibana** instance:\n\n- Go to *Management \u003e Saved objects* in the Kibana menu\n- Click *Import* and select the [kibana_objects.ndjson](https://raw.githubusercontent.com/manticoresoftware/kibana-demo/refs/heads/main/kibana_objects.ndjson) file from the root of this repository.\n- Refresh the Kibana window to apply the changes.\n\nFor the **Manticore Kibana** instance:\n\n- Go to *Discover* in the Kibana menu\n- Set `test` as the value for the `Index pattern` input field\n- Click *Next step*\n- Select `@timestamp` in the `Time Filter` select field\n- Click *Create index pattern*\n\n\n### Explore the Dashboards\n\nOnce the setup is complete, explore the imported dashboards and visualizations based on the indexed log data. Navigate to *Dashboard* or *Visualize* in the Kibana menu.\n\nAfter all log entries are loaded, you'll notice differences in dashboard load times between Manticore and Elasticsearch.\nHere are our test results with about 40 million documents:  \n\n\u003cimg src=\"manticore1.png\" alt=\"manticore\" width=\"350\" height=\"200\" /\u003e \u003cimg height=\"200\" hspace=\"20\"/\u003e \u003cimg src=\"elastic1.png\" alt=\"elastic\" width=\"350\" height=\"200\" /\u003e\n\n### Stopping the Environment\n\nTo stop and remove all containers, run:\n\n```bash\ndocker-compose down\n```\n\n## Limitations\n\n- Currently, Kibana version 7.6.0 is tested and recommended. Other 7.x versions may work but could cause issues. Versions 8.x are not supported.\n- The following Elasticsearch-specific field types are not supported:\n  - [Spatial data types](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html#spatial_datatypes)\n  - [Structured data types](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html#structured-data-types)\n  - [Document ranking types](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html#document-ranking-types)\n  - [Text search types](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html#text-search-types) (except for plain 'text')\n  - [Relational data types](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html#object-types)\n- Metric aggregation functions are limited to [those supported by Manticore](../Searching/Grouping.md#Aggregation-functions).\n- The following Kibana tools are not supported:\n  - [Canvas](https://www.elastic.co/guide/en/kibana/7.6/canvas.html) – A visualization and presentation tool for combining data with colors and images.\n  - [Elastic Maps](https://www.elastic.co/guide/en/kibana/7.6/maps.html) – A tool for analyzing geographical data.\n  - [Metrics](https://www.elastic.co/guide/en/kibana/7.6/xpack-infra.html) – An app for monitoring infrastructure metrics.\n  - [Logs](https://www.elastic.co/guide/en/kibana/7.6/xpack-logs.html) – A console-like display for exploring logs from common services.\n  - Monitoring:\n    - [Uptime](https://www.elastic.co/guide/en/kibana/7.6/xpack-uptime.html) – Monitors the status of network endpoints via HTTP/S, TCP, and ICMP.\n    - [APM (Application Performance Monitoring)](https://www.elastic.co/guide/en/kibana/7.6/xpack-apm.html) – Collects in-depth performance metrics from applications.\n    - [SIEM (Security Information and Event Management)](https://www.elastic.co/guide/en/kibana/7.6/xpack-siem.html) – An interactive workspace for security teams to triage events and conduct initial investigations.\n    - [ILM (Index lifecycle management)](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/index-lifecycle-management.html) - Automatically manage indices according to performance, resiliency, and retention requirements.\n    - [Stack Monitoring](https://www.elastic.co/guide/en/kibana/7.6/xpack-monitoring.html) – Provides visualizations of monitoring data across the Elastic Stack.\n  - [Elasticsearch Management](https://www.elastic.co/guide/en/kibana/7.6/management.html) – A UI for managing Elastic Stack objects, including ILM (Index Lifecycle Management), etc.\n\n## Contributing\n\nContributions are welcome! Please open an issue or submit a pull request if you’d like to improve this project.\n\n## License\n\nThis project is licensed under the MIT License.\n\n## Support\n\nFor questions or issues, please visit the [Manticore Search community forums](https://forum.manticoresearch.com/) or open an issue in this repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanticoresoftware%2Fkibana-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanticoresoftware%2Fkibana-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanticoresoftware%2Fkibana-demo/lists"}