Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/cnosdb/cnosdb

A cloud-native open source distributed time series database with high performance, high compression ratio and high availability. http://www.cnosdb.cloud
https://github.com/cnosdb/cnosdb

database distributed-database rust rust-lang sql time-series time-series-database timeseries

Last synced: 25 days ago
JSON representation

A cloud-native open source distributed time series database with high performance, high compression ratio and high availability. http://www.cnosdb.cloud

Host: GitHub
URL: https://github.com/cnosdb/cnosdb
Owner: cnosdb
License: agpl-3.0
Created: 2021-10-25T09:47:14.000Z (about 3 years ago)
Default Branch: main
Last Pushed: 2024-04-12T11:10:32.000Z (7 months ago)
Last Synced: 2024-04-12T17:57:25.628Z (7 months ago)
Topics: database, distributed-database, rust, rust-lang, sql, time-series, time-series-database, timeseries
Language: Rust
Homepage: https://www.cnosdb.com
Size: 15.4 MB
Stars: 1,565
Watchers: 129
Forks: 311
Open Issues: 168
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.md
- Code of conduct: CODE_OF_CONDUCT.md
- Roadmap: docs/roadmap/ROADMAP.md

Awesome Lists containing this project

awesome-distributed-system-projects - CnosDB is a high-performance, high-compression, and easy-to-use open-source distributed time-series database. Used in fields such as IoT, industrial internet, connected cars, and IT operations

README

        


  





  

  

  

  

  

  

  

  

  

  

  

  

  

  

  

  





  

    

  





    CnosDB Cloud

    •

    Website

    •

    Documentation

    •

    Quick Start

    •

    Eco-Integration



English | [简体中文](./README_CN.md)

CnosDB is a high-performance, high-compression, and easy-to-use open-source distributed time-series database. It is primarily used in fields such as IoT, industrial internet, connected cars, and IT operations. All of the code is open-sourced and available on GitHub.

In its design, we fully utilize the characteristics of time-series data, including structured data, non-transactions, fewer deletions and updates, more writes and less reads, etc. As a result, CnosDB has a number of advantages that set it apart from other time-series databases:

- **High performance**: CnosDB addresses the issue of time-series data expansion and theoretically supports unlimited time-series data. It supports aggregate queries along the timeline, including queries divided by equal intervals, queries divided by enumeration values of a column, and queries divided by the length of the time interval between adjacent time-series records. It also has caching capabilities for the latest data and the cache space can be configured for fast access to the latest data.

- **Easy to use**: CnosDB provides clear and simple interfaces, easy configuration options, standard SQL support, seamless integration with third-party tools, and convenient data access functions. It supports schema-less writing mode and supports historical data supplement(including out of order writing).

- **Cloud native**: CnosDB has a native distributed design, data sharding and partitioning, separation of storage and computing, Quorum mechanism, Kubernetes deployment and complete observability, ensuring final consistency. It can be deployed in public clouds, private clouds, and hybrid clouds. t also supports multi-tenancy and has role-based permission control. The computing and storage nodes support horizontal scaling.

  **CnosDB Cloud is now live, [click here](https://www.cnosdb.cloud/) to get started now.**

# Architecture

![arch](./docs/source/_static/img/cnosdb_arch.png)

# Quick Start

## Build&Run from source

### **Support Platform**

We support the following platforms, if found to work on a platform not listed,

Please [report](https://github.com/cnosdb/cnosdb/issues) to us.

- Linux x86(`x86_64-unknown-linux-gnu`)

- Darwin arm(`aarch64-apple-darwin`)

### **Requirements**

1. Install `Rust`, You can check [official website](https://www.rust-lang.org/learn/get-started) to download and install

2. Install Cmake

```shell

# Debian or Ubuntu

apt-get install cmake

# Arch Linux

pacman -S cmake

# CentOS

yum install cmake

# Fedora

dnf install cmake

# macOS

brew install cmake

```

3. Install FlatBuffers

```shell

# Arch Linux

pacman -S flatbuffers

# Fedora

dnf install flatbuffers

# Ubuntu

snap install flatbuffers

# macOS

brew install flatbuffers

```

If your system is not listed, you can install FlatBuffers as follows

```shell

$ git clone -b v22.9.29 --depth 1 https://github.com/google/flatbuffers.git && cd flatbuffers

# Choose one of the following commands depending on your operating system

$ cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release

$ cmake -G "Visual Studio 10" -DCMAKE_BUILD_TYPE=Release

$ cmake -G "Xcode" -DCMAKE_BUILD_TYPE=Release

$ sudo make install

```

4. Install Protobuf

```shell

# Arch Linux

pacman -S protobuf

# Fedora

dnf install protobuf

# Ubuntu

snap install protobuf

# macOS

brew install protobuf

```

### **Compile**

```shell

git clone https://github.com/cnosdb/cnosdb.git && cd cnosdb

make build

```

### **Run**

#### Run CnosDB

The following is a single node startup. If you need to start a cluster, see the [Deploy](https://docs.cnosdb.com/en/docs/deploy/) section.

```bash

./target/debug/cnosdb run -M singleton --config ./config/config.toml

```

#### **Run CLI**

```shell

cargo run --package client --bin cnosdb-cli

```

## Run with Docker

1. Install [Docker](https://www.docker.com/products/docker-desktop/)

2. Start container

```shell

docker run --name cnosdb -d cnosdb/cnosdb:community-latest cnosdb run -M singleton --config /etc/cnosdb/cnosdb.conf

```

3. Run a command in the running container

```shell

docker exec -it cnosdb bash

```

4. Run `cnosdb-cli`

```shell

cnosdb-cli

```

> Quit `\q`

> Help `\?`

> For more details, check [quick start](https://docs.cnosdb.com/en/latest/start/quick_start.html)

## Write data

- [SQL](https://docs.cnosdb.com/en/latest/reference/sql.html#insert)

- [influxdb line-protocol](https://docs.influxdata.com/influxdb/v2.6/reference/syntax/line-protocol/)

- [bulk loading](https://docs.cnosdb.com/en/latest/develop/write.html#load-data)

- [telegraf](https://docs.cnosdb.com/en/eco-integration/telegraf)

The following will show an example of using cli to write data by SQL

1. CREATE TABLE

```sql

CREATE TABLE air (

    visibility DOUBLE,

    temperature DOUBLE,

    pressure DOUBLE,

    TAGS(station)

);

```

```bash

public ❯ CREATE TABLE air (

    visibility DOUBLE,

    temperature DOUBLE,

    pressure DOUBLE,

    TAGS(station)

);

Query took 0.063 seconds.

```

2. Insert a row

```sql

INSERT INTO air (TIME, station, visibility, temperature, pressure) VALUES

                (1673591597000000000, 'XiaoMaiDao', 56, 69, 77);

```

```bash

public ❯ INSERT INTO air (TIME, station, visibility, temperature, pressure) VALUES

                (1673591597000000000, 'XiaoMaiDao', 56, 69, 77);

+------+

| rows |

+------+

| 1    |

+------+

Query took 0.032 seconds.

```

3. insert multiple rows

```sql

INSERT INTO air (TIME, station, visibility, temperature, pressure) VALUES

                ('2023-01-11 06:40:00', 'XiaoMaiDao', 55, 68, 76),

                ('2023-01-11 07:40:00', 'DaMaiDao', 65, 68, 76);

```

```bash

public ❯ INSERT INTO air (TIME, station, visibility, temperature, pressure) VALUES

                ('2023-01-11 06:40:00', 'XiaoMaiDao', 55, 68, 76),

                ('2023-01-11 07:40:00', 'DaMaiDao', 65, 68, 76);

+------+

| rows |

+------+

| 2    |

+------+

Query took 0.038 seconds.

```

## Query data

- [SQL](https://docs.cnosdb.com/en/latest/reference/sql.html), compatible with SQL standard.

- [Prometheus remote read](https://docs.cnosdb.com/en/eco-integration/prometheus#remote-read).

The following will show an example of SQL query using cli

```sql

-- query table data

SELECT * FROM air;

```

```bash

public ❯ -- query table data

SELECT * FROM air;

+---------------------+------------+------------+-------------+----------+

| time                | station    | visibility | temperature | pressure |

+---------------------+------------+------------+-------------+----------+

| 2023-01-11T06:40:00 | XiaoMaiDao | 55         | 68          | 76       |

| 2023-01-13T06:33:17 | XiaoMaiDao | 56         | 69          | 77       |

| 2023-01-11T07:40:00 | DaMaiDao   | 65         | 68          | 76       |

+---------------------+------------+------------+-------------+----------+

Query took 0.036 seconds.

```

# Connector

CnosDB supports connections from various clients:

- C/C++

- Go

- Java

- Rust

- Python

- JDBC

- ODBC

- Arrow Filght SQL

Please refer to the "Connector" section in the documentation for the above examples. You can access it [here](https://docs.cnosdb.com/en/latest/reference/connector/).

# Roadmap

- Click to view [Roadmap](./docs/roadmap/ROADMAP.md)

# Join the community

Welcome to join our developer community at discord: 

  

  https://discord.com/invite/D8cB4WGpP4



  

    

  



## Contributing

Please refer to [Contribution Guide](./CONTRIBUTING_EN.md) to contribute to CnosDB.

# Acknowledgement

- CnosDB 2.0 uses [Apache Arrow](https://github.com/apache/arrow) as the memory model.

- CnosDB 2.0's query engine is powered by [Apache Arrow DataFusion](https://github.com/apache/arrow-datafusion).

- CnosDB 2.0's bug detection is powered by [SQLancer](https://github.com/sqlancer/sqlancer).

- CnosDB 2.0's integration test framework is powered by [sqllogictest-rs](https://github.com/risinglightdb/sqllogictest-rs).

- CnosDB 2.0 combining [LangChain](https://github.com/hwchase17/langchain) to realize the natural language to communicate with the database。

Ecosyste.ms: Awesome

https://github.com/cnosdb/cnosdb

Awesome Lists containing this project

README

CnosDB Cloud • Website • Documentation • Quick Start • Eco-Integration

CnosDB Cloud
•
Website
•
Documentation
•
Quick Start
•
Eco-Integration