Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Bwar/Nebula

Nebula is a powerful framwork for building highly concurrent, distributed, and resilient message-driven applications for C++.
https://github.com/Bwar/Nebula

cluster distributed distributed-systems high-performance libev nebula protobuf websockets

Last synced: about 1 month ago
JSON representation

Nebula is a powerful framwork for building highly concurrent, distributed, and resilient message-driven applications for C++.

Host: GitHub
URL: https://github.com/Bwar/Nebula
Owner: Bwar
License: apache-2.0
Created: 2016-09-03T02:39:48.000Z (over 8 years ago)
Default Branch: master
Last Pushed: 2023-09-02T13:40:05.000Z (over 1 year ago)
Last Synced: 2024-08-02T00:25:20.123Z (4 months ago)
Topics: cluster, distributed, distributed-systems, high-performance, libev, nebula, protobuf, websockets
Language: C++
Homepage:
Size: 72.4 MB
Stars: 549
Watchers: 34
Forks: 156
Open Issues: 2
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

awesome-distributed-system-projects - Nebula - powerful framework for building highly concurrent, distributed, and resilient message-driven applications

README

English | [中文](/README_cn.md) [Bwar's blog](https://www.bwar.tech).

```
_ __ __ __
/ | / /__ / /_ __ __/ /___ _
/ |/ / _ \/ __ \/ / / / / __ `/
/ /| / __/ /_/ / /_/ / / /_/ /
/_/ |_/\___/_.___/\__,_/_/\__,_/
one-click installation
```
# Nebula : a powerful framework for building highly concurrent, distributed, and resilient message-driven applications for C++.
[![](https://travis-ci.org/Bwar/Nebula.svg?branch=master)](https://travis-ci.org/Bwar/Nebula) [![Author](https://img.shields.io/badge/[email protected]?style=flat)]([email protected]) ![Platform](https://img.shields.io/badge/platform-Linux-green.svg?style=flat) [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](LICENSE)

1. [Overview](#Overview)
2. [Features](#Features)
3. [Getting Start](#GettingStart)
4. [Documentation](#Documentation)
5. [Depend on](#DependOn)
6. [Main Projects](#MainProjects)
7. [Todo list](#TODO)
8. [Change log](#ChangeLog)

## Overview

Nebula is a flexible, high-performance business-oriented IoC distributed network framework developed in C++ language, designed for production environments. It supports multiple application layer communication protocols including proto3, http, https, http2, grpc, cassandra, and websocket. Nebula makes it easy to deploy a fast and high-performance distributed service with C++, while keeping the same server architecture and APIs. NebulaBootstrap provides out-of-the-box integration with Nebula service, but can be easily extended to serve other types of application.

Nebula is a production level framework and distributed solution project for instant messaging, data collection, real-time computing, message push and other applications, as well as web api services. There were production applications for instant messaging, data acquisition and real-time analysis on line now, and a recommendation engine application for a large user base will be born soon. By the way, using Nebula for toy-level projects is also good for learning network communication. Bwar welcomes more developers to join the Nebula project.

Nebula can be used as a single high-performance TCP server, but building a cluster based on Nebula will be truly reflect its value. In order to build distributed service clusters quickly, Nebula Bootstrap cluster solutions including various types of services have been developed.

## Features
* Protocol communication such as http, protobuf, websocket, etc.
* Ssl connection encryption
* Microservices framework
* IoC container
* Dynamic service update
* Service registration and discovery
* Service monitoring
* Configuration management
* Routing
* Load balancing
* Circuit Breakers
* Leadership election and cluster state
* A convenient way to extend third-party protocols

## Getting Start
Nebula was developed with C++11/C++14 standard, requires a compiler capable of the C++11-standard and at least gcc4.8 (some C++14 features are replaced by C++11 standard when encountering a lower version of the compiler).
[NebulaBootstrap](https://github.com/Bwar/NebulaBootstrap) provides a quick way to build and deploy Nebula. A distributed solution based on NebulaBootstrap will make it easy to develop micro-service applications in C++.
All dependencies will be automatically resolved in the following build steps.

first install gcc and auto tools. centos:
```
sudo yum install -y gcc gcc-c++
sudo yum install -y autoconf automake libtool
```
build step：
1. wget https://github.com/Bwar/NebulaBootstrap/archive/master.zip
2. unzip master.zip; rm master.zip; mv NebulaBootstrap-master NebulaBootstrap; chmod u+x deploy.sh; chmod u+x deploy.sh
3. ./deploy.sh

Run deploy.sh, the NebulaBootstrap distributed services were build completed. The reliance of Nebula was also automatically downloaded and compiled by deploy from the Internet before the construction of Nebula. The deploy path as follows:
* NebulaBootstrap
+ bin         server bin location。
+ build        build path，created by deploy.sh, if you do not need to build again, just delete it.(optional)。
+ conf        configuration path.
+ data        application data path. e.g. [Nebio](https://github.com/Bwar/Nebio) which is a data collect and real-time analysis application, write it's data to this path (optional).
+ lib         library path.
+ log         program logs path.
+ plugins       plugins path.
- logic       plugins for logic server(optional).
+ script       script path. deploy.sh, startup.sh and shutdown.sh were depend on this path.
+ temp        temp file path (optional).
- configure.sh     run configure.sh for a simple configuration when deploy for the first time.
- deploy.sh      auto build and deploy.
- shutdown.sh      shutdown one or more server.
- startup.sh      startup one or more server.
- README_cn.md
- README.md

build completed, you can start the server:
```
./configure.sh
./startup.sh
```

The server should have started successfully now, startup.sh will print the server that had been started, If not, check logs for reason. Notice that the default configuration file of Nebula limits the number of connections per IP in a period. If you have a large amount of testing, you should check the configuration limit. If the server has been successfully started, testing with postman or curl.
```
# testing start with NebulaInterface only.
curl -H "Content-Type:application/json" -X POST -d '{"name": "Nebula", "address":"https://github.com/Bwar/Nebula"}' http://${your_ip}:16003/hello

# testing start with NebulaInterface,NebulaLogic and NebulaBeacon.
curl -H "Content-Type:application/json" -X POST -d '{"name": "Nebula", "address":"https://github.com/Bwar/Nebula"}' http://${your_ip}:16003/hello_nebula
```
A simple testing can be start with a NebulaInterface only, and also can be start with NebulaBootstrap. NebulaBootstrap provided the a cluster HelloWorld, the testing will launch NebulaBeacon, NebulaInterface and NebulaLogic. This is a diagram of the cluster architecture:

![nebula_cluster](https://github.com/Bwar/NebulaBootstrap/blob/master/image/nebula_cluster.png?raw=true)

## Documentation
The complete documentation for Nebula is available: [Nebula class reference](https://bwar.gitee.io/nebula)

## Depend on
* [protobuf](https://github.com/google/protobuf)
* [libev](http://software.schmorp.de/pkg/libev.html) or [libev](https://github.com/kindy/libev)
* [crypto++](https://github.com/weidai11/cryptopp)
* [http_parse](https://github.com/nodejs/http-parser) integrate into Nebula/src/util/http
* [CJsonObject](https://github.com/Bwar/CJsonObject) integrate into Nebula/src/util/json

## Main Projects
* [NebulaBootstrap](https://github.com/Bwar/NebulaBootstrap)
* [NebulaBeacon](https://github.com/Bwar/NebulaBeacon)
* [NebulaInterface](https://github.com/Bwar/NebulaInterface)
* [NebulaLogic](https://github.com/Bwar/NebulaLogic)
* [NebulaMydis](https://github.com/Bwar/NebulaMydis)
* [NebulaDbAgent](https://github.com/Bwar/NebulaDbAgent)
* [NebulaLogger](https://github.com/Bwar/NebulaLogger)
* [NebulaAccess](https://github.com/Bwar/NebulaAccess)
* [NebulaDynamic](https://github.com/Bwar/NebulaDynamic)
* [Nebcli](https://github.com/Bwar/Nebcli)

## Todo list
- Complete writing user guide

## Change log
#### v2.0.0
- add spec channel
- use SpecChannel instead of unix socket to transfer file descriptors
- add SocketChannel migration between threads
- add the client connection pool management
- increase the transceiver buffer size limit
- optimize the new connection processing
- remove process mode
#### v1.7.3
- error callback caused by fuse node detection bug fixed
- redis cluster returns when executing batch write commands and asking bug fixed
- the channel creation method and use CodecFactory instead
- Add connection backlog configuration
- Optimize worker mini load forwarding
- add dns cache
- add upstream and downstren monitor
- add console log
#### v1.7.2
- codec bind channel circular reference bug fixed
- CodecProto no packet communication bug fixed
- enable connection protection configuration
#### v1.7.1
- async logger optimization
- add selfchannel sequence
- StepRedisCluster circuit break and recovery
- Nodes recovery bug fixed
#### v1.7.0
- IO optimization, codec plug-in.
- remove shared_from_this of Channel and Actor and use ChannelWatcher and ActorWatcher instead to improve performance.
- removed the unload feature of dynamically loaded plugins (which reduces performance).
- raw data codec is independent into codec.
- add cassandra client codec.
- add support for password verification of redis cluster.
- improve log performance; add asynchronous file logs.
#### v1.6.2
- circuit breaker optimization
- http chunk decode bug fixed; resp bulk string bug fixed
#### v1.6.1
- http2 chunk notice for stream
- CodecResp bug string decode bug fixed
#### v1.6.0
- add SelfChannel within the Worker, improve the actor model
- add service status monitoring and metrics for prometheus
- add Manager to Manager communication feature
- add initial connection timeout configuration
- replace va_list with variadic templates in logger
- add multi-protocol port support
- thread id optimization
- chain optimization
- http2 dynamic table and stream priority bug fix
#### v1.5.0
- add native http2 server and client
- add native grpc server and client
- channel read and write optimization
- add absolute log path
#### v1.4
- replace hiredis client with native CodecResp
- add redis cluster client
- add raw data transmission
- add log real-time flush configurability
- add support for IP address combination node type
- dispatcher optimization
- bug fixed
#### v1.3
- add non-pipeline mode support for redis connection
- the worker thread to start before the Loader thread in thread mode, and bring the worker thread ID to the Loader
- merge CodecResp and CodecHttp2 written by @nebim
- add third-party cityhash
- update CJsonObject
- bug fixed
#### v1.2
- delay start when loading a large amount of data
#### v1.1
- add worker thread mode
#### v1.0
- separate the network dispatch and actor management from Labor (Manager and Worker) to Dispatcher and ActorBuilder
- add supports of Actor classes to Manager process, and move the manager system management features to Cmd and Step
- add Loader
- optimize Actor dynamic creator
- optimize HTTP connection and HTTP data sending and receiving
- RedisChannel bug fixed
- add bind ip
- add load stress
- add codec auto switch
#### v0.10
- the plugin version control dynamically unloads and loads the instant validation feature.
- optimize reflection dynamic creation of Actor.
#### v0.9
- add Model
- add Chain
- simplify Context
- optimize Actor
#### v0.8
- compatible with gcc4.8 compiler.
- add cpu affinity inorder to support cpu binding.
- add dynamic library unload.
#### v0.7
- add configuration management (check [Nebcli](https://github.com/Bwar/Nebcli) for detail).
#### v0.6
- NebulaBeacon adds node status information query, registration center leader-fllower election.
- NebulaInterface adds hello demo.
#### v0.5
- add node info to worker the worker process terminated unexpectedly and restarted by the Manager.
- ipv6 support.
#### v0.4
- distributed log service test passing.
- add https support.
- add keep alive settings to http channel.
- replace repeated http headers with proto3 map.
- provides a symmetric encryption key setup interface for channel.
- bug fix.
#### v0.3
- rewrite with C++14
- create actors by reflection (using template)
- add distributed trace log
#### v0.2
- the first runable version