{"id":15862282,"url":"https://github.com/sydrawat01/info7255","last_synced_at":"2026-03-11T19:15:00.631Z","repository":{"id":210466770,"uuid":"691625678","full_name":"sydrawat01/INFO7255","owner":"sydrawat01","description":"RESTful API with strongly-typed JSON schema for distributed systems. Uses Elasticsearch-Kibana for indexed searching and RabbitMQ as a message broker.","archived":false,"fork":false,"pushed_at":"2023-12-02T22:46:08.000Z","size":272,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-12T22:43:44.792Z","etag":null,"topics":["docker","docker-compose","elasticsearch","kibana","rabbitmq","redis","rest-api","restful-api"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sydrawat01.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-14T14:54:58.000Z","updated_at":"2024-07-16T23:42:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"24644ac6-a2a8-457e-b20c-c4f74eed5f1a","html_url":"https://github.com/sydrawat01/INFO7255","commit_stats":null,"previous_names":["sydrawat01/info7255"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/sydrawat01/INFO7255","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sydrawat01%2FINFO7255","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sydrawat01%2FINFO7255/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sydrawat01%2FINFO7255/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sydrawat01%2FINFO7255/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sydrawat01","download_url":"https://codeload.github.com/sydrawat01/INFO7255/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sydrawat01%2FINFO7255/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30395156,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-11T18:46:22.935Z","status":"ssl_error","status_checked_at":"2026-03-11T18:46:17.045Z","response_time":84,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","elasticsearch","kibana","rabbitmq","redis","rest-api","restful-api"],"created_at":"2024-10-05T22:40:18.605Z","updated_at":"2026-03-11T19:15:00.564Z","avatar_url":"https://github.com/sydrawat01.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# INFO-7255: Adv Big Data Indexing Techniques\n\n[![Unit Test Suite](https://github.com/sydrawat01/INFO7255/actions/workflows/test-suite.yml/badge.svg)](https://github.com/sydrawat01/INFO7255/actions/workflows/test-suite.yml)\n\nA course on distributed systems and big data indexing techniques.\n\n## Automated Setup\n\nTo install all application dependencies [Elasticsearch, Kibana and RabbitMQ], use the `docker-compose.yml` file to deploy all application dependencies using `docker compose`.\n\n- To create the cluster, run:\n\n  ```bash\n  docker compose up\n  ```\n\n- To delete the cluster and the network:\n\n  ```bash\n  docker compose down\n  ```\n\n- The default username and password for RabbitMQ is defined in the `docker-compose.yml` file.\n  - default username: `guest`\n  - default password: `guest`\n\n- The following is a list of the endpoints for the dependencies being created in docker:\n  - Elasticsearch: [http://localhost:9200](http://localhost:9200)\n  - Kibana: [http://localhost:5610](http://localhost:5610)\n  - RabbitMQ: [http://localhost:15672](http://localhost:15672)\n\n## Manual Setup\n\n- Install redis\n\n  ```shell\n  # only for MacOs\n  brew install redis\n  ```\n\n- Install the dependencies\n\n   ```shell\n   yarn\n   # or if you are using npm: npm i\n   ```\n\n- Start the server locally\n\n   ```shell\n   yarn start:dev\n   # npm run start:dev\n   ```\n\n### [Elasticsearch Setup using Docker](https://www.elastic.co/guide/en/elasticsearch/reference/8.11/docker.html)\n\nWe'll use a docker container to connect to the elasticsearch service. So make sure you have docker installed.\n\nTo run the elasticsearch image in a docker container (single-node cluster):\n\n- Create a new docker network\n\n  ```bash\n  # docker network create elasticnet\n  docker network create \u003cnetwork-name\u003e\n  ```\n\n- Pull the elastic docker image\n\n  ```bash\n  # Running an elastic docker container only works with a specified tag, not with\n  # latest versions.\n  # docker pull docker.elastic.co/elasticsearch/elasticsearch:8.11.1\n  docker pull docker.elastic.co/elasticsearch/elasticsearch:[tag]\n  ```\n\n\u003e Use the -m flag to set a memory limit for the container. This removes the need to [manually set the JVM size](https://www.elastic.co/guide/en/elasticsearch/reference/8.11/docker.html#docker-set-heap-size).\n\n- Start an elasticsearch container\n\n  ```bash\n  # docker run --name es01 --net elasticnet -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.11.1\n  docker run --name \u003ccontainer-name\u003e --net \u003cnetwork-name\u003e -p 9200:9200 -it -m 1GB \\\n    docker.elastic.co/elasticsearch/elasticsearch:[tag]\n  ```\n\n\u003e The command prints the `elastic` user password and an enrollment token for Kibana.\n\n- Copy the generated `elastic` password and enrollment token. These credentials are only shown when you start Elasticsearch for the first time. You can regenerate the credentials using the following commands.\n\n  ```bash\n  docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic\n  docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana\n  ```\n\n- We recommend storing the elastic password as an environment variable in your shell\n\n  ```bash\n  export ELASTIC_PASSWORD=\"your_password\"\n  ```\n\n- Copy the `http_ca.crt` SSL certificate from the container to your local machine.\n\n  ```bash\n  # docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt /Users/sid/Developer/api/src/certs\n  docker cp \u003ccontainer-name\u003e:/usr/share/elasticsearch/config/certs/http_ca.crt /path/to/your/folder\n  ```\n\n- Make a REST API call to Elasticsearch to ensure the Elasticsearch container is running.\n\n  ```bash\n  curl --cacert /Users/sid/Developer/api/src/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200\n  ```\n\nThis should printout a success message that looks similar to this:\n\n```json\n{\n  \"name\" : \"686b14641876\",\n  \"cluster_name\" : \"docker-cluster\",\n  \"cluster_uuid\" : \"C4TdqRy3QKSQM9Cri2HBKA\",\n  \"version\" : {\n    \"number\" : \"8.11.1\",\n    \"build_flavor\" : \"default\",\n    \"build_type\" : \"docker\",\n    \"build_hash\" : \"6f9ff581fbcde658e6f69d6ce03050f060d1fd0c\",\n    \"build_date\" : \"2023-11-11T10:05:59.421038163Z\",\n    \"build_snapshot\" : false,\n    \"lucene_version\" : \"9.8.0\",\n    \"minimum_wire_compatibility_version\" : \"7.17.0\",\n    \"minimum_index_compatibility_version\" : \"7.0.0\"\n  },\n  \"tagline\" : \"You Know, for Search\"\n}\n```\n\n### Kibana\n\nTo install the Kibana docker image:\n\n- Pull the Kibana Docker image.\n\n  ```bash\n  # docker pull docker.elastic.co/kibana/kibana:8.11.1\n  docker pull docker.elastic.co/kibana/kibana:[tag]\n  ```\n\n- Start a Kibana container.\n\n  ```bash\n  # docker run --name kib01 --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.11.1\n  docker run --name \u003ccontainer-name\u003e --net \u003cnetwork-name\u003e -p 5601:5601 docker.elastic.co/kibana/kibana:[tag]\n  ```\n\n\u003e NOTE: When Kibana starts, it outputs a unique generated link to the terminal. To access Kibana, open this link in a web browser. In your browser, enter the enrollment token that was generated when you started Elasticsearch.\n\n- To regenerate the token\n\n  ```bash\n  docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana\n  ```\n\n- Log in to Kibana as the `elastic` user with the password that was generated when you started Elasticsearch\n\n  ```bash\n  # to regenerate the password, run:\n  docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic\n  ```\n\n### RabbitMQ\n\nTo install the RabbitMQ docker image:\n\n- Pull the RabbitMQ docker image\n\n  ```bash\n  # docker pull rabbitmq:3.12-management-alpine\n  docker pull rabbitmq:[tag]\n  ```\n\n- Create the RabbitMQ docker container\n\n  ```bash\n  # docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.12-management-alpine\n  docker run -it --rm --name \u003ccontainer-name\u003e -p 5672:5672 -p 15672:15672 rabbitmq:[tag]\n  ```\n\n## Manual setup teardown\n\nTo remove the containers and their network, run:\n\n```bash\n# Remove the Elastic network\n# docker network rm elastic\ndocker network rm \u003cnetwork-name\u003e\n\n# Remove Elasticsearch, Kibana and RabbitMQ containers\n# docker rm es01\ndocker rm \u003celastic-container-name\u003e\n# docker rm kib01\ndocker rm \u003ckibana-container-name\u003e\n# docker rm rabbitmq\ndocker rm \u003crabbitmq-container-name\u003e\n```\n\n## Author\n\n[Siddharth Rawat](https://sydrawat.live)\n\n## [License](./LICENSE)\n\nThis project is licensed under the MIT License. ([see license file for details](./LICENSE))\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsydrawat01%2Finfo7255","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsydrawat01%2Finfo7255","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsydrawat01%2Finfo7255/lists"}