https://github.com/chillfish8/jocky

Giddy up! Things are going to be wild.
https://github.com/chillfish8/jocky

Last synced: 3 months ago
JSON representation

Giddy up! Things are going to be wild.

• Host: GitHub
• URL: https://github.com/chillfish8/jocky
• Owner: ChillFish8
• Created: 2022-12-11T17:19:48.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-04-10T23:12:17.000Z (about 2 years ago)
• Last Synced: 2025-02-01T14:32:17.670Z (5 months ago)
• Language: Rust
• Homepage:
• Size: 113 KB
• Stars: 3
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Jocky - For controlling your horses

### Current Idea

The idea is essentially to have a custom directory that writes immutable segments, this means only `2` file descriptors (one for writing and one for reading) are required per-index, which reduces the amount used in multi-tenant situations.

The writers have more stable performance since they depend less on the file cache to buffer writes and reads, the AIO implementation also leverages asynchronous background writing which makes CPU usage a bit more stable:

- 8 Threads, 2GB RAM, 2GB Swap, 5GB Produced index compressed.

- Green: AIO Writer

- Blue: MMapDirectory

![image](https://user-images.githubusercontent.com/57491488/207173717-b64bae6b-51e5-4e1b-a293-b8b16b29ec57.png)

![image](https://user-images.githubusercontent.com/57491488/207174299-d75e4215-f865-4494-bd86-e7d9b2a08a4e.png)

![image](https://user-images.githubusercontent.com/57491488/207178010-76b469f4-7044-4868-afd7-654eda6031b2.png)

*Writes are messured in negative MiB/s for some reason, but just ignore the `-`*

These unfinished segments as such can then be exported to a new file as the finished 'segment' as such, with each file re-organised to be contiguous. 

### Testing Screenshots

#### Mmap No Advise

![image](https://user-images.githubusercontent.com/57491488/207358796-bcbd2db0-f4af-4bfd-9d52-a95741d4437f.png)

![image](https://user-images.githubusercontent.com/57491488/207358856-35d7d838-1e82-44c4-a9ee-87b8a8700c6c.png)

#### Mmap MADV_SEQUENTIAL

![image](https://user-images.githubusercontent.com/57491488/207357681-ed8b16e8-9770-49be-8f2c-2b0871f57a23.png)

![image](https://user-images.githubusercontent.com/57491488/207358047-b22fb85c-ea4c-468f-8be9-8724dd6e8d2a.png)

#### Mmap MADV_RANDOM

![image](https://user-images.githubusercontent.com/57491488/207358409-650f8064-bad0-4a41-b2bb-cc66309a838f.png)

![image](https://user-images.githubusercontent.com/57491488/207358477-74185865-11ab-47d1-bce6-7631955efc6f.png)

#### Mmap No Advise - Constrained File Cache (6GB MAX)

![image](https://user-images.githubusercontent.com/57491488/207358796-bcbd2db0-f4af-4bfd-9d52-a95741d4437f.png)

![image](https://user-images.githubusercontent.com/57491488/207358856-35d7d838-1e82-44c4-a9ee-87b8a8700c6c.png)

#### Mmap MADV_SEQUENTIAL - Constrained File Cache (6GB MAX)

![image](https://user-images.githubusercontent.com/57491488/207357681-ed8b16e8-9770-49be-8f2c-2b0871f57a23.png)

![image](https://user-images.githubusercontent.com/57491488/207358047-b22fb85c-ea4c-468f-8be9-8724dd6e8d2a.png)

#### Mmap MADV_RANDOM - Constrained File Cache (6GB MAX)

![image](https://user-images.githubusercontent.com/57491488/207358409-650f8064-bad0-4a41-b2bb-cc66309a838f.png)

![image](https://user-images.githubusercontent.com/57491488/207358477-74185865-11ab-47d1-bce6-7631955efc6f.png)