https://github.com/xxlabaza/log-file

A Java library for working with append-only files
https://github.com/xxlabaza/log-file

write-ahead-log write-to-file

Last synced: 14 days ago
JSON representation

A Java library for working with append-only files

• Host: GitHub
• URL: https://github.com/xxlabaza/log-file
• Owner: xxlabaza
• License: apache-2.0
• Created: 2019-11-30T17:54:30.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2020-01-31T18:26:54.000Z (almost 5 years ago)
• Last Synced: 2023-12-19T11:46:14.531Z (11 months ago)
• Topics: write-ahead-log, write-to-file
• Language: Java
• Size: 68.4 KB
• Stars: 2
• Watchers: 2
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Overview log-file

[![build_status](https://travis-ci.org/xxlabaza/log-file.svg?branch=master)](https://travis-ci.org/xxlabaza/log-file)

[![maven_central](https://maven-badges.herokuapp.com/maven-central/com.xxlabaza.utils/log-file/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.xxlabaza.utils/log-file)

[![License](http://img.shields.io/:license-apache-brightgreen.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)

A Java library for working with append-only files (also known as the log files). The main methods of such files are:

- **append** - for appending arbitary data;

- **load** - for fully file read from the begining till the end.

## Usage example

```java

import java.nio.file.Paths;

import com.xxlabaza.utils.log.file.Config;

import com.xxlabaza.utils.log.file.LogFile;

import io.appulse.utils.Bytes;

import io.appulse.utils.HexUtil;

// let's instantiate a log file:

Config config = Config.builder()

    .path(Paths.get("./my.log"))

    .blockBufferSizeBytes(32)

    .forceFlush(false)

    .build();

LogFile logFile = new LogFile(config);

// than, write some data to log file

Bytes record = Bytes.resizableArray()

    .write4B(42);

logFile.append(record);

// now, read all records from the file and print them to STDOUT

logFile.load((buffer, position) -> {

  String dump = HexUtil.prettyHexDump(buffer);

  System.out.println(dump);

  return true; // true - continue reading, false - stop reading

});

// flush and close the file

logFile.close();

```

## Under the hood

A log file has the next structure:

  

    

      log file structure

    

  

  

    

      file header

      blocks

    

    

      version

      block size

      block 0

      ...

      block n

    

    

      1 byte

      4 bytes

      blocks count * block size

    

  

### File header

File header's description:



  
version

  
The file's format version, which tells the features set is used in the file.

  
block size

  
The size of a block buffer, in bytes, which is used in the current file.



### Block

Each block consist of the records set:

  

    

      block

    

  

  

    

      record 1

      ...

      record n

    

  

The records have the following structure:

  

    

      record

    

  

  

    

      checksum

      type

      body length

      body

    

    

      4 bytes

      1 byte

      2 bytes

      body length

    

  



  
checksum

  
The checksum value, which is calculated from type, body length and body values.

  
type

  
One of the record's types:

    


      


FULL - the data is presented entirely in this record;

      


FIRST - it is the first chunk of data;

      


MIDDLE - one of the middle parts of data;

      


LAST - the last piece of data.

    

  

  
body length

  
The length of the next body section, in bytes.

  
body

  
the record's payload.