{"id":15349502,"url":"https://github.com/ravening/elasticsearch-cloudstack","last_synced_at":"2026-04-13T06:02:41.737Z","repository":{"id":117364381,"uuid":"214454547","full_name":"ravening/Elasticsearch-Cloudstack","owner":"ravening","description":"Java web application to search for cloudstack logs stored in Elasticsearch cluster","archived":false,"fork":false,"pushed_at":"2020-08-14T08:07:42.000Z","size":37,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-01T22:13:24.669Z","etag":null,"topics":["centralized-logging","cloudstack","docker","docker-compose","elasticsearch","jackson","jackson-databind","java-8","logging","maven","rest-client","restful-api","restful-webservices","spring-boot"],"latest_commit_sha":null,"homepage":"https://rakgenius.github.io/Elasticsearch-Cloudstack/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ravening.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-11T14:16:51.000Z","updated_at":"2020-08-14T08:07:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"294f2aa3-0c8a-4b81-bc51-aece5f7ba6de","html_url":"https://github.com/ravening/Elasticsearch-Cloudstack","commit_stats":{"total_commits":13,"total_committers":1,"mean_commits":13.0,"dds":0.0,"last_synced_commit":"bfc55753ea396717f8b0c648ff191ab532ca4d0d"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ravening%2FElasticsearch-Cloudstack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ravening%2FElasticsearch-Cloudstack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ravening%2FElasticsearch-Cloudstack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ravening%2FElasticsearch-Cloudstack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ravening","download_url":"https://codeload.github.com/ravening/Elasticsearch-Cloudstack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245907556,"owners_count":20691956,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["centralized-logging","cloudstack","docker","docker-compose","elasticsearch","jackson","jackson-databind","java-8","logging","maven","rest-client","restful-api","restful-webservices","spring-boot"],"created_at":"2024-10-01T11:54:50.096Z","updated_at":"2026-04-13T06:02:41.643Z","avatar_url":"https://github.com/ravening.png","language":"Java","readme":"# ElasticSearch plugin for Cloudstack\n\nA Java Web application to search for the cloudstack logs using REST API's stored in elasticsearch cluster.\n\n## Getting Started\n\nBelow instructions will tell you what are the prerequisites that\nneeds to be installed, configurations that has to be done to run the project.\n\n### Prerequisites\n\nBelow are the softwares that needs to be installed as a dependency\n\n```\nJava 8\nMaven\nRunning ElasticSearch Cluster\n```\n\n### Assumptions\n\nIt is assumed that elasticsearch is installed and configured on either remote or local machine\\\nfrom where we want to fetch the data.\n\nIf you want to run elasticsearch in docker then use the docker compose file from [here](https://github.com/ravening/dev_setup/blob/master/elasticsearch/docker-compose.yml)\n\nIf you want to run elasticsearch on a baremetal machine or VM then you follow the steps mentioned\\\n[here](https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-18-04)\n\nOnce everything is setup, make sure that you can reach the elasticsearch endpoint by making a get request to\n\n```bash\ncurl \u003cIP address of elasticsearch cluster\u003e:9200\n\n{\n  \"name\" : \"0DWGlCW\",\n  \"cluster_name\" : \"cloudstack\",\n  \"cluster_uuid\" : \"ZZ5NFlLBSFqOqVW3ONi5JA\",\n  \"version\" : {\n    \"number\" : \"6.8.3\",\n    \"build_flavor\" : \"default\",\n    \"build_type\" : \"deb\",\n    \"build_hash\" : \"0c48c0e\",\n    \"build_date\" : \"2019-08-29T19:05:24.312154Z\",\n    \"build_snapshot\" : false,\n    \"lucene_version\" : \"7.7.0\",\n    \"minimum_wire_compatibility_version\" : \"5.6.0\",\n    \"minimum_index_compatibility_version\" : \"5.0.0\"\n  },\n  \"tagline\" : \"You Know, for Search\"\n}\n```\n\nIf you dont see the above output then please configure it properly and then continue.\n\n## store some documents in elasticsearch\n```\ncurl -XPUT 'http://localhost:9200/twitter/_doc/1?pretty' -H 'Content-Type: application/json' -d '\n{\n    \"user\": \"kimchy\",\n    \"post_date\": \"2009-11-15T13:12:00\",\n    \"message\": \"Trying out Elasticsearch, so far so good?\"\n}'\n\ncurl -XPUT 'http://localhost:9200/twitter/_doc/2?pretty' -H 'Content-Type: application/json' -d '\n{\n    \"user\": \"kimchy\",\n    \"post_date\": \"2009-11-15T14:12:12\",\n    \"message\": \"Another tweet, will it be indexed?\"\n}'\n\ncurl -XPUT 'http://localhost:9200/twitter/_doc/3?pretty' -H 'Content-Type: application/json' -d '\n{\n    \"user\": \"elastic\",\n    \"post_date\": \"2010-01-15T01:46:38\",\n    \"message\": \"Building the site, should be kewl\"\n}'\n```\n\n### Installing\n\nA step by step series of examples that tell you how to get a development env running\n\n1 . Git clone the repo to any Linux machine.\n\n2 . Get the index name and the type of the document you want to fetch from the elasticsearch\n```bash\n$ curl \u003cIP of elasticsearch\u003e:9200/_cat/indices?pretty\nyellow open filebeat-2019.10.11             ZbB6mlWBTb6QfKfPRfX2lw 5 1 1737458   0 550.5mb 550.5mb\n```\n\n```bash\n$ curl \u003cIP of elasticsearch\u003e:9200/filebeat-2019.10.11/_mapping?pretty\n{\n  \"filebeat-2019.10.11\" : {\n    \"mappings\" : {\n      \"doc\"\n```\n\n3 . Now that we know the name of index and the type name, store those values in ```data.sql```\n\n4. If you want to customize the index name, query size, type of log then configure it in `data.sql`\n\n### Building the packages\n\nBelow are the steps to build the JAR file and to start the backend server\n\n```bash\nmvn package -DskipTests\n\njava -jar target/cloudstack-elasticsearch-0.0.1.jar\n```\n\nTo build a docker image, run the command\n```bash\ndocker build -t cloudstack-elasticsearch .\n```\n\nRun the docker image\n```bash\ndocker run -p 5000:9229 cloudstack-elasticsearch\n```\n\n### Accessing the end points\n\n1 . To search all the logs in the index navigate to\n```http request\nhttp://localhost:9229/api/v1/search/listall\n```\n\nIf running in docker then navigage to\n```http request\nhttp://localhost:9229/api/v1/search/listall\n```\n\n2 . To search for a log with particular ID, navigate to\n```http request\nhttp://localhost:9229/api/v1/search/\u003cID\u003e\n```\n\nIf running in docker then navigate to\n```http request\nhttp://localhost:9229/api/v1/search/\u003cID\u003e\n```\n\n3 . To search for a particular keyword in a message, navigate to\n```http request\nhttp://localhost:9229/api/v1/search/query/\u003cquery string\u003e\n```\n\n4 . To display all indices, navigate to\n```http request\nhttp://localhost:9229/api/v1/index\n```\n\n5. If you want to search for documents in different index then you can update the index dynamically using\n```\nhttp://localhost:9229/api/v1/config/updateindex/{name}\n```\n\n## Automating the generation of the docker images\nIf you want to build and push the docker images to the docker hub automatically\\\nthen add the below lines to the pom.xml\n\n```bash\n\u003cplugin\u003e\n\t\u003cgroupId\u003ecom.spotify\u003c/groupId\u003e\n\t\u003cartifactId\u003edockerfile-maven-plugin\u003c/artifactId\u003e\n\t\u003cversion\u003e1.4.0\u003c/version\u003e\n\t\u003cconfiguration\u003e\n\t\t\u003crepository\u003e\u003cyour dockerhub username\u003e/\u003crepo name\u003e\u003c/repository\u003e\n\t\t\u003ctag\u003e${project.version}\u003c/tag\u003e\n\t\t\u003cbuildArgs\u003e\n\t\t\t\u003cJAR_FILE\u003etarget/${project.build.finalName}.jar\u003c/JAR_FILE\u003e\n\t\t\u003c/buildArgs\u003e\n\t\u003c/configuration\u003e\n\t\u003cexecutions\u003e\n\t\t\u003cexecution\u003e\n\t\t\t\u003cid\u003edefault\u003c/id\u003e\n\t\t\t\u003cphase\u003einstall\u003c/phase\u003e\n\t\t\t\u003cgoals\u003e\n\t\t\t\t\u003cgoal\u003ebuild\u003c/goal\u003e\n\t\t\t\t\u003cgoal\u003epush\u003c/goal\u003e\n\t\t\t\u003c/goals\u003e\n\t\t\u003c/execution\u003e\n\t\u003c/executions\u003e\n\u003c/plugin\u003e\n```\n\nNow you can build the docker image using\n```bash\nmvn package dockerfile:build\n```\n\nPush the docker image using\n```bash\nmvn dockerfile:push\n```\n\nTo do the above two steps automatically, run\n```bash\nmvn install\n```\n\nThis will build the packages, creates docker image and uploads it to docker hub\n\n## Built With\n\n* [Maven](https://maven.apache.org/) - Dependency Management\n\n\n## Authors\nRakesh Venkatesh\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fravening%2Felasticsearch-cloudstack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fravening%2Felasticsearch-cloudstack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fravening%2Felasticsearch-cloudstack/lists"}