Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/yyzybb537/libgo

Go-style concurrency in C++11
https://github.com/yyzybb537/libgo

c-plus-plus coroutine libgo

Last synced: 25 days ago
JSON representation

Go-style concurrency in C++11

Awesome Lists containing this project

README 

        
# libgo



[![Build Status](https://travis-ci.org/yyzybb537/libgo.svg?branch=master)](https://travis-ci.org/yyzybb537/libgo)



### libgo -- a coroutine library and a parallel Programming Library



Libgo is a stackful coroutine library for collaborative scheduling written in C++ 11, and it is also a powerful and easy-to-use parallel programming library.



Three platforms are currently supported:



    Linux



    MacOSX

    

    Windows (Win7 or above,x86 or x64,complie with VS2015/2017)



Using libgo to write multi-threaded programs, it can be developed as fast and logical as golang and Erlang concurrent languages, and has the performance advantages of C++ native.It make it happen that one can serve God and Mammon.



Libgo has the following characteristics:



*   1.Provide golang's General powerful protocol, write code based on coroutine, can write simple code in a synchronous manner, while achieving asynchronous performance.



*   2.Supporting massive coroutines, creating 1 million coroutines requires only 4.5 GB of physical memory. (data from real test, in no deliberately compressed stack situation.)



*   3.Supporting multi-threaded scheduling protocols, providing efficient load balancing strategy and synchronization mechanism, it is easy to write efficient multi-threaded programs.



*   4.The number of scheduled threads supports dynamic scaling, and there is no head blocking caused by slow scheduling.



*   5.Use hook technology to make synchronous third-party libraries of linking processes become asynchronous calls, which greatly improves their performance. There's no need to worry that some DB authorities don't provide asynchronous drivers, such as hiredis and mysqlclient, which are client drivers that can be used directly and can achieve performance comparable to that of asynchronous drivers.



*   6.Both dynamic links and full static links are supported, which makes it easy to generate executable files using C++ 11 static links and deploy them to low-level Linux systems.



*   7.Provide Channel, Co_mutex, Co_rwmutex, timer and other features to help users write programs more easily.



*   8.Supports local variables (CLS) of the process, and completely covers all scenarios of TLS (read the tutorial code sample13_cls.cpp for details).



* From user feedback in the past two years, many developers have a project with an asynchronous non-blocking model (probably based on epoll, libuv or ASIO network libraries) and then need access to DBs such as MySQL that do not provide asynchronous driver. Conventional connection pool and thread pool schemes are intensive in high concurrency scenarios (each connection have to correspond to a thread for Best performance. Thousands of instruction cycles of thread context switching  are intensive and too many active threads will lead to a sharp decline performance in OS scheduling capacity, which is unacceptable to many develops.



* In this situation, there is no need to reconstruct the existing code if we want to use libgo to solve the problem of blocking operation in non-blocking model. The new libgo 3.0 has created three special tools for this scenario, which can solve this problem without intrusion: multi-scheduler with isolated running environment and easy interaction (read the tutorial code sample1_go.cpp for details), libggo can instead of the traditional thread pool scheme. (read tutorial code sample10_co_pool.cpp and sample11_connection_pool.cpp for details)



* ** tutorial directory contains many tutorial codes, including detailed instructions, so that develop can learn libgo library step by step. **



* If you find any bugs, good suggestions, or use ambiguities, you can submit a issue or contact the author directly:

Email: [email protected]



 

### compile and use libgo :



 *    Vcpkg:



        

If you have installed vcpkg, you can install it directly using vcpkg:

      $ vcpkg install libgo



 *    Linux: 



        1.Use cmake to compile and install：



            $ mkdir build

            $ cd build

            $ cmake ..

	    $ make debug     #Skip it if you don`t want a debuggable versions.

            $ sudo make uninstall

            $ sudo make install



 

        2.Dynamic link to glibc: (put libgo at the front of link list)

        

            g++ -std=c++11 test.cpp -llibgo -ldl [-lother_libs]

            

        3.Full static link: (put libgo at the front of link list)



            g++ -std=c++11 test.cpp -llibgo -Wl,--whole-archive -lstatic_hook -lc -lpthread -Wl,--no-whole-archive [-lother_libs] -static



 *    Windows: (3.0 is compatible with windows, just use master branch directly!)

 

        0.When using GitHub to download code on windows, we must pay attention to the problem of newline characters. Please install git correctly (using default options) and use git clone to download source code. (Do not download compressed packages)

 

        1.Use CMake to build project. 

			

			#For example vs2015(x64)：

			$ cmake .. -G"Visual Studio 14 2015 Win64"



			#For example vs2015(x86)：

			$ cmake .. -G"Visual Studio 14 2015"

        

        2.If you want to execute the test code, please link the boost library. And set BOOST_ROOT in the cmake parameter:

        

        		For example：

        		$ cmake .. -G"Visual Studio 14 2015 Win64" -DBOOST_ROOT="e:\\boost_1_69_0"



### performance



Like golang, libgo implements a complete scheduler (users only need to create a coroutine without concern for the execution, suspension and resource recovery of the coroutine). Therefore, libgo is qualified to compare the performance of single-threaded with golang (It is not qualified to do performance comparison in different ability).






Test environment: 

2018 13-inch MAC notebook (CPU minimum)

Operating System: Mac OSX

CPU: 2.3 GHz Intel Core i5 (4 Core 8 Threads)

Test script: $test/golang/test.sh thread_number






### Matters needing attention(WARNING)：

 

TLS or non-reentrant library functions that depend on TLS implementation should be avoided as far as possible.

If it is unavoidable to use, we should pay attention to stop accessing the TLS data generated before handover after the process handover.



### There are several kinds of behaviors that may cause the process switching:



* The user calls co_yield to actively give up the cpu span.

* Competitive Cooperative Lock, Channel Reading and Writing.

* System Call of Sleep Series.

* System calls waiting for events to trigger, such as poll, select, epoll_wait.

* DNS-related system calls (gethostbyname series).

* Connect, accept, data read-write operations on blocking sockets.

* Data Read-Write Operation on Pipe.



### System Call List of Hook on Linux System:



		connect   

		read      

		readv     

		recv      

		recvfrom  

		recvmsg   

		write     

		writev    

		send      

		sendto    

		sendmsg   

		poll      

		__poll

		select    

		accept    

		sleep     

		usleep    

		nanosleep

		gethostbyname                                                               

		gethostbyname2                                                              

		gethostbyname_r                                                             

		gethostbyname2_r                                                            

		gethostbyaddr                                                               

		gethostbyaddr_r



	The above system calls are all possible blocking system calls. The whole thread is no longer blocked in the process. During the blocking waiting period, the CPU can switch to other processes to execute.System calls executed in native threads by HOOK are 100% consistent with the behavior of the original system calls without any change.

  

		socket

		socketpair

		pipe

		pipe2

		close     

		__close

		fcntl     

		ioctl     

		getsockopt

		setsockopt

		dup       

		dup2      

		dup3      



    The above system calls will not cause blocking, although they are also Hook, but will not completely change their behavior, only for tracking socket options and status.



### System Call List of Hook on Windows System:



		ioctlsocket                                                                        

		WSAIoctl                                                                           

		select                                                                             

		connect                                                                            

		WSAConnect                                                                         

		accept                                                                             

		WSAAccept                                                                          

		WSARecv                                                                            

		recv                                                                               

		recvfrom                                                                           

		WSARecvFrom                                                                        

		WSARecvMsg                                                                         

		WSASend                                                                            

		send                                                                               

		sendto                                                                             

		WSASendTo                                                                          

		WSASendMsg