Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/hiejulia/flight-booking

Flight booking services
https://github.com/hiejulia/flight-booking

databases distributedsystems docker microservice neo4j python spring-cloud spring-framework sql

Last synced: about 1 month ago
JSON representation

Flight booking services

Awesome Lists containing this project

README 

        
# flight-booking

### This is not production level code. I am no longer maintain for this project. 



Online flight reservation system 

+ User service : operations on User service 

    + Database : Neo4J

+ Booking service : Flight service and User service to perform operations on booking. It will use flight search and its associated table

    + Database :

+ Flight service : operations and searching based on criteria, providing association between Flight and flight trip 

    + Database : MySQL

    + Distributed Caching layer : Hazelcast

    + Flight search : ElasticSearch `localhost:9200/flights/external/_bulk`

    `localhost:9200/flights/_search?q=*&pretty`

+ Billing service : operations on billing 

    + Database : MongoDB

    + Messaging : RabbitMQ (queue: billingQueue)

+ Passenger service 

    + Database : Cassandra 

+ Organisation service 

    + Database : 



+ Account service 

    + Database : Cassandra 



+ Cash flow service

    + Database : PostgreSQL

    + Caching - Message queue : Redis

+ Bid service 

    + Database : PostgreSQL 

    + Caching : Redis 

    + Index tool : ElasticSearch 



+ Back up service : Back up microservice database Schedule distributed 

    + 

+ Booking-client 

    

+ Auth-server : User / Passenger need to register/ login - authorized 

+ Twitter Service : read twitter streams(from Airline branch twitter) and persist them on MongoDB and served them as a REST API(HATEOAS) to other service

    + Database : MongoDB 

+ Search twitter service : Read tweets from twitter service and push it on ElasticSearch as a search engine and produce userful search for these tweet

    + Database : 

    + Index engine : Solr

+ social-network-service : send flights and passengers data to Agency social network 

    + Database : Neo4J



+ Booking Request Service : passenger will confirm the flight booking request or cancel request 

    + Caching : Redis 

    + Database : Neo4J(BookingRequest node, Passenger node)



### Stack 

+ Applied to the principles of the 12 Factor App

+ Microservice architecture : Spring cloud, Netflix Eureka, Ribbon, Zuul, Hystrix, Service discovery, Load Balancing, API gateway, Circuit breaker (Hystrix)

+ Spring framework : Spring Boot, Spring cloud, Spring data, Spring Stream , Spring Reactor

+ CQRS - Event sourcing 

+ Database : PostgreSQL, MongoDB, Cassandra, MySQL - MariaDB, Neo4J , MySQL

+ Caching : Redis

+ Messaging system : RabbitMQ , Kafka(Zookeeper)

+ Batch process 

+ Apache Avro

+ ElasticSearch - Logstash - Kibana 

    + ES GUI plugin 

    + Install Logstash 

        + Config logstash 

    + Install Kibana 

        + localhost:5601

    + Docker image ELK stack from Docker hub 

        + Docker UI : Rancher - manage docker container by UI 

            + Install Rancher : `sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server`

            + Goto : `http://ip:8080`

            + Custom (EC2 - Azure, AWS )- connect server with rancher server 

+ Container: Docker  - Docker compose 

    + In each service - Docker image is built to production deploy - Docker maven plugin is added to pom.xml 

    + Update Spring profile with Docker

    + Config Docker-maven-plugin 

+ REST API testing using Postman

+ Testing : JUnit, E2E test with Cucumber

    + JUnit, Mockito, WireMock 

    + Unit test

    + Integration testing 

        + Docker - Fail safe plugin 

+ Event - driven system 

+ Security : OAuth/ JWT 

+ Log analysis : ELK stack - Logstash - ElasticSearch - Kibana to index logs 

    + ElasticSearch : distributed, JSON based search and analytics engine designed for horizontal scalability, maximum reliability , easy management 

    + Logstash : dynamic data collection pipeline with an extension plugin ecosystem and strong elasticsearch synergy

    + Kibana : visualization of data though UI 

    + ELK stack architecture 

        + ELK stack in Docker containers - RabbitMQ server has Logstash pipeline 

        + User view logs from Kibana which is the user interface of elasticsearch cluster -> logstash will listen the application logs and transform those to json format and send to elasticsearch 

    + Distributed tracing and centralized log management

        + Spring Cloud sleuth & Zipkin 

        + Config Kibanan and view the logs 

+ API documentation : Swagger - Curl -

+ API UI testing: Postman 

+ Service Logging / Monitoring 



+ Correlation ID for service call for all services 

    + REST call 

    + Zipkin and Sleuth



### Endpoint documentation 

+ Flight service

    + GET `v1/flights/id` : get one flight by id / info 

    + GET `v1/flights` : retrieve all the flights that matches the value of query param 

    + POST `v1/flights` : create new flight 

    + GET `v1/airports` : get a list of airports

    + GET `v1/airports/{airport-name}` : list of flights from this airport 

    + Search flight by name

    + Search flight by code 

        + POST `v1/airports/flights/code`



+ Booking service : User can book a flight ticket and fill the personal information - billing information 

    + POST `v1/flights/{flight-id}/booking ` : 

    + Get booking details 



+ Billing service : User can pay the flight order 

    + GET `v1/flights/{flight-id}/payment`

    + Make payment(handle payment errors: payment authorization timeout and invalid credit card info )



+ User can unsubscribe to the flight ticket info

    + GET `v1/flights/{flight-id}/payment/ubsubscribed`



+ Passenger service 

    + GET `v1/passengers`

    + GET `v1/passengers/id`

    + POST `v1/passengers`: create one passenger

    + GET `v1/passengers/organisations/id` : find passengers by organisation 

+ Organisation service 



### Run the project 



+ Run everything : 



+ `docker-compose up` : RabbitMQ port set up (in the docker folder) : it will start the RabbitMQ and MongoDB instance 



java -jar eureka-server/target/eureka-server.jar  : Start Eureke server  -  Eureka server port set up

java -jar dashboard-server/target/dashboard-server.jar : Start Hystrix dashboard - Hystrix port set up 

java -jar turbine-server/target/turbine-server.jar 



java -jar flight-service/target/restaurant-service.jar 

java -jar user-service/target/user-service.jar 

java -jar booking-service/target/booking-service.jar 

java -jar api-service/target/api-service.jar



+ Before start Zuul service, make sure that all of the services are up in the Eureka dashboad : `localhost:8761`

java -jar zuul-server/target/zuul-server.jar 



+ How to double check all components are setting up 

1. Browser : Eureka server 

`http://localhost:8761/`

Service instances are register with Eureka 



2. Browser : Hystrix monitor : `http://localhost:7979/ ` - Hystrix dashboard 

+ Substitude : [http://hostname:port/turbine/turbine.stream] to

                    URL:  http://localhost:9001/hystrix.stream

         --> Monitor Stream



3. Browser : RabbitMQ management : `http://localhost:15672/` (guest- guest )



+ Install and run 

    + MongoDB : `docker run --name mongo -p 27017:27017 -d mongo`

    + Redis : `dockewr run --name redis -p 6379:6379 -d redis`



### Microservice architecture 

+ Service discovery and registration - Eureka 

    + Run multiple instances of the same microservices

    + Look up the host name and the port number by checking discovery service 

    + Eureka server : `http://localhost:8761/`

    + Advanced discovery client config 

    + Enable secure communication between client and server

    + Config failover and peer - to - peer replication mechanism 

    + Register instances of client - side application in different zones 

    + Cluster env : zone mechanism - enable zone handling 

    + Replication and high availability - cluster mechanism - peer to peer replication model - replication mechanism 

    + Run instances of Eureka 

    + Enable secure communication between client and server 

        + Register a secure service 

        + Enable SSL by generating a self - signed certificate 

        + SSL is enabled for edge-service only 



+ Config service   

    + Application configuration to all the other microservices 

    + Port number, context paths 

    + Configuration - using GIT 

    + Vault - cmd - add new values to the server - run from docker container 



+ Edge or proxy server (API gateway) - Zuul - Gateway service 

    + Gateway service Zuul 

    + Proxy all calls to the target microservice 

    + Solve CORS request - Enable CORS headers on the proxy only

    + Zuul integrates with Eureka (discovery-service)

    + Gateway service that provides dynamic routing, monitor, resiliency, security 

+ Load balancing : Ribbon is used for load balancing . It is integrated with the Zuul and Eureka services to provide load balancing for both internal and external calls 

    + Server side load balancing : Zuul server as edge server 

    + Client side load balancing : Ribbon - FeignClient 

+ Circuit breaker : Netflix hystrix 

+ Distributed tracing : Zipkin, Spring Cloud Sleuth - distributed tracing via logs - distributed tracing system with request visualization 

    + tracing mechanism 

    + trace and span ID 

    + record time - latency analysis - statistics = zipkin = query and visualize data 

    + stream = producer of message sent to message broker 

+ Monitoring : Netflix Turbine  - and Dashboard 

    + Hystrix provides a dashboard UI `locahost:7979`

    + Turbine stream `http://localhost:8989/turbine.stream`

    + Hystrix uses RabbitMQ to send metrics data feed to Turbine 

    + Config and collect metrics and tracing from all services

+ UUA servvice 

    + User account & Authentication - security of the application 

    + /token endpoint to retrieve a token 

    + /user endpoint to validate a token and retrieve the user and its roles

    + Token in this case are long-lived 



+ Integration

+ Batch processing 

+ Security service 

    + Secure microservices architecture 

    + SSL enabled 



+ Dependency management : Maven 

+ Data Lake 

+ Testing microservices

    + Deploy all of them and test them in an end-to-end fashion 

    + Mock external dependencies in unit/integration test 

    

+ Event sourcing 

+ HTTP listener 

+ Containers / Virtual Machines 

+ Cluster Control and Provisioning 



+ Admin server 



+ Spring cloud DataFlow server 

    + `dataflow-service` is a Spring Boot app that loads the local DataFlow server. Port 9393

    + To interact with the dataflow server, you can donw 



+ Microservice service design : contract design and protocol selection 

    + Contract design 

        + Simplicity - consumer 

        + KISS 

        + Consumer driven contract 

    + Protocol selection 

        + HTTP/ SOAP 

        + Messaging 

    + Message oriented service 

        + JMS / AMQP protocol - JSON 

        + Messaging over HTTP 

        + Async REST 

    + HTTP and REST endpoint 

        + Protocol handling 

        + Traffic routing 

        + Load balancing 

        + Security systems 

        + HATEOAS

    + Optimize communication protocol - for communication between service 

        + Avro 

        + Protocol Buffers 

        + Apache Thrift 

        + Custom binary protocol 

        + RPC 

+ API documentation : Swagger, RAML, API blueprint 

+ API versioning 

    + URL segment

    + Accept header 

    + Custom Header 

+ Authentication - Authorization 

    + 2 legged authentication

    + Basic HTTP authentication

    + 3 legged authentication

+ ELK stack 

    + ELK config - Use Docker container to run the ELK stack 

    1. Run this command on Docker terminal `docker run -d -it --name es -p 9200:9200 -p 9300:9300 -e ES_JAVA_OPTS="-Xms1g -Xmx1g" -m 1500m elasticsearch` : start ElasticSearch container on 9200/9300 port 

    2. `docker run -d -it --name kibanak --link es:elasticsearch -p 5601:5601 kibana` : start Kibana on 5601 port and it will also link it with ElasticSearch container 

    3. `docker run -d -it --name logstash -p 5000:5000 logstash -e 'input { tcp { port => 5000 codec => "json" } } output { elasticsearch { hosts => ["192.168.99.100"] index => "micro-%{serviceName}"} }'` : start Logstash container on 5000 port and also create an index with name micro-*

    4. Checking with `docker ps` command, all the container should be running 

    + Default port used for docker container is 192.168.99.100 



    + Kibana : check logs on Kibana 

        + With `Log.info` statement and `logback.xml` configuration we can index and view log from Kibana 

        + Query with Kibana 

            + Key-value search 

            + Boolean operator - type:radar AND status:500

            + Request UUID tracking 

                + Query for a request ID header 

                + dynamic log verbosity 

+ Grafana - Graphite - StatsD 

+ Alert and Monitor tools : 



+ Zipkin server 

    + Check the log traces on zipkin server `localhost:9411`

    + Contain Spring Zipkin Stream server 



+ Docker 

    + Docker containers in this microservice group - Mongo - RabbitMQ - Config-service - Discovery - service , Gateway- service, Command-service, Query-service 

    + Using docker- compose, run : `docker-compose -f docker-compose.yml up`

    + To see the running containers `docker ps`



    + Run MongoDB and RabbitMQ 

        + `docker-compose up -d mongodb rabbitmq`



+ Jenkins 

    + Continuous deploy using Jenkins Pipeline 

        + Create docker image to have CD 

            + Image contains : build project, create docker images, deploy on AWS using ECS container

            + Using Jenkinsfile - config Job on Jenkins using Pipeline plugin and paste the content of Jenkins file in the Pipeline script box 



+ Deploy on AWS

    + Create credentials on AWS 

    + Create cluster on AWS 

    + Build deploy container 

    + Access Jenkins panel 

    + Create a pipeline job 

    + Run the job 



+ Scaling 

    + NGINX will be configured for browser caching of the static content and load balancer - scale App Gateway and update manually the ports in default.conf - upstream config section 



### Project architecture

+ Flight service    

    + Database : MongoDB 

    + User search for flight based on search query 

+ Catalog service 



+ User service - Account service 

    + User/ Customer register account

    + Database : MySQL 

+ Account service 

+ Booking service 

    + Database : MongoDB 

    + User book the flight and fill needed information for the flight 

+ Billing service 

    + Dabase : RDBMS 

    + User pay for flight ticket 



+ Subscription service 

    + Use

+ Route service 

    + User search for flight by flight route and flight city

+ Notification service 

    + Notification is send to user when user book a flight 

+ Ticker order service : before book for ticker, user can choose to order (reserve the flight ticker in a certain of time )

+ Payment service : After sending bill to user(customer), payment service is used for pay the fee of the flight ticket 

+ third party payment service 



+ Credit risk engine :  

    + Service to test for the validity of the bank account 

    



### Other 

#### How to deploy in AWS 

+ Create folder for each microservice(except flightsearchclient) project in AWS AMI root folder(e.g "eureka")

+ Copy the corresponding jar and the dockerfile in that folder (e.g "eurekaserver-0.0.1-SNAPSHOT.jar" and "Dockerfile" )

+ Go to that directory from root and run docker build command to create docker image("cd eureka","sudo docker build -t eureka .")

+ Go to root directory and run docker compose command("sudo docker-compose up")

+ To stop run "sudo docker-compose down"



### Microservices architecture 

#### Debug tools 

+ Curl and jq