https://github.com/didi/levin
A Quick Way to Bulk Loading
https://github.com/didi/levin
Last synced: 7 months ago
JSON representation
A Quick Way to Bulk Loading
- Host: GitHub
- URL: https://github.com/didi/levin
- Owner: didi
- License: apache-2.0
- Created: 2019-07-01T02:44:25.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2021-05-28T03:23:15.000Z (over 4 years ago)
- Last Synced: 2024-04-14T13:44:08.438Z (over 1 year ago)
- Language: C++
- Homepage:
- Size: 479 KB
- Stars: 105
- Watchers: 12
- Forks: 26
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
Levin - A Quick Way to Bulk Loading
=======================
Description
-------
Levin provides a highly efficient solution in bulk loading scenario.
A collection of STL-like containers are implemented with high performance and memory efficiency.
Levin containers arrange memory layout SEQUENTIALLY, allocated at share memory region for inter-process reuse.
In which case, we can extend the lifetime of data resources, and prevent unnecessary time-consuming reload.
Loading with levin is fast, let's get started.
Getting Started
-------
* Brief Intro for Levin Container
| STD Container | Levin Container | Notes | Suggestion |
| --------------------------------- | ------------------------------ | --------------- | ------------------------------------- |
| vector\ | SharedVector\ | T is POD type | |
| set\ | SharedSet\ | K is POD type | use SharedHashSet if no comparison |
| map\ | SharedMap\ | K/V is POD type | use SharedHashMap if no comparison |
| unordered_set\ | SharedHashSet\ | K is POD type | |
| unordered_map\ | SharedHashMap\ | K/V is POD type | |
| vector\ \> | SharedNestedVector\ | T is POD type; SizeType is unsigned integral type | specified SizeType for memory space efficiency |
| unordered_map\ \> | SharedNestedHashMap\ | K/V is POD type | |
| vector\ \> | SharedNestedMap\ | K/V is POD type; SizeType is unsigned integral type | |
* How to Dump Container Data to A File
```c++
// vector dump demo
std::vector vec_data = {1, 2, 3, 4, 5};
levin::SharedVector::Dump("./vec_demo.dat", vec_data);
```
```c++
// map/hashmap dump demo
std::unordered_map map_data = { {1, 100}, {2, 200}, {3, 300} };
// or std::map map_data = { {1, 100}, {2, 200}, {3, 300} };
levin::SharedHashMap::Dump("./map_demo.dat", map_data);
```
* How to Use Container
Tips: Levin container SHOULD be immutable, NOT suggest to modify or reallocate.
```c++
// shared vector use demo
levin::SharedVector vec("./vec_demo.dat");
if (vec.Init() == levin::SC_RET_OK && vec.Load() == levin::SC_RET_OK) {
...
}
vec.Destroy();
```
```c++
// shared hashmap use demo
auto map_ptr = new levin::SharedHashMap("./map_demo.dat");
if (map_ptr->Init() == levin::SC_RET_OK && map_ptr->Load() == levin::SC_RET_OK) {
...
}
map_ptr->Destroy();
delete map_ptr;
```
* How to Manage a set of Containers
[example code](example/shared_manager_demo.cpp) of container manager usage
Dependencies
-------
Levin depends on following packages, required deps:
* gcc >= 4.8.5
* cmake >= 2.6.0
* boost >= 1.56.0
* openssl
* gtest
Compile
-------
* create directory build, cd build
* compile: cmake .. && make
Documentation
-------
About the details, please see [Wiki](../../wiki).
Contributing
-------
Welcome to contribute by creating issues or sending pull requests. See [Contributing Guide](CONTRIBUTING.md) for guidelines.
Communication
-------
License
-------
Levin is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file.