{"id":19867065,"url":"https://github.com/azurespheredev/fifolog","last_synced_at":"2026-05-30T23:31:39.966Z","repository":{"id":244027679,"uuid":"814093209","full_name":"azurespheredev/fifolog","owner":"azurespheredev","description":"[WIP - NOT READY FOR PRODUCTION] Portable fifolog: FreeBSD's libfifolog and utilities for POSIX systems","archived":false,"fork":false,"pushed_at":"2024-06-12T10:25:29.000Z","size":169,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-05-19T15:56:20.619Z","etag":null,"topics":["circular","fifolog","libfifolog","logfile"],"latest_commit_sha":null,"homepage":"https://cgit.freebsd.org/src/tree/usr.sbin/fifolog","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/azurespheredev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-12T10:23:26.000Z","updated_at":"2025-11-24T06:13:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"72ef0959-3ddf-4dfe-9e14-840fda478cf9","html_url":"https://github.com/azurespheredev/fifolog","commit_stats":null,"previous_names":["azuresphere7/fifolog","blitzsprinter/fifolog","azurespheredev/fifolog"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/azurespheredev/fifolog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azurespheredev%2Ffifolog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azurespheredev%2Ffifolog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azurespheredev%2Ffifolog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azurespheredev%2Ffifolog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/azurespheredev","download_url":"https://codeload.github.com/azurespheredev/fifolog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azurespheredev%2Ffifolog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33714033,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["circular","fifolog","libfifolog","logfile"],"created_at":"2024-11-12T15:28:14.183Z","updated_at":"2026-05-30T23:31:39.951Z","avatar_url":"https://github.com/azurespheredev.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- SPDX-License-Identifier: BSD-2-Clause --\u003e\n\u003c!-- Copyright (c) 2005-2008 Poul-Henning Kamp --\u003e\n\u003c!-- Copyright (c) 2023 Jeffrey H. Johnson \u003ctrnsz@pobox.com\u003e --\u003e\n# Portable fifolog\n\n\u0026nbsp;Portable [*fifolog*](https://cgit.freebsd.org/src/tree/usr.sbin/fifolog) — compact round-robin circular storage\n\n---\n\n\u003c!-- toc --\u003e\n\n- [Name](#name)\n- [Synopsis](#synopsis)\n- [Description](#description)\n- [Implementation Notes](#implementation-notes)\n- [Examples](#examples)\n- [History](#history)\n- [Build](#build)\n- [Test](#test)\n- [Authors](#authors)\n\n\u003c!-- tocstop --\u003e\n\n## NAME\n\n`fifolog_create`, `fifolog_writer`, `fifolog_reader` — \u003cbr\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; initialize, write, seek\nand extract data from a *fifolog*\n\n## SYNOPSIS\n\n* `fifolog_create` [`−l record-size`] [`−r record-count`]\n  [`−s size`] `file`\n\n* `fifolog_reader` [`−t`] [`−b tstart]` [`−B Tstart`] [`−e tend`]\n  [`−E Tend`] [`−o ofile]` [`−R regexp`] [`−T timefmt`] `file`\n\n* `fifolog_writer` [`−w write-rate`] [`−s sync-rate`]\n  [`−z compression`] `file`\n\n## DESCRIPTION\n\n*Fifologs* provide a compact round-robin circular storage for recording\ntext and binary information to permanent storage in a bounded and\npredictable fashion, time and space wise.\n\nA *fifolog* can be stored either directly on a disk partition or in a\nregular file.\n\nThe input data stream is encoded, compressed and marked up with\ntimestamps before it is written to storage, such that it is possible\nto seek out a particular time interval in the stored data, without\nhaving to decompress the entire logfile.\n\nThe `fifolog_create` utility is used to initialize the first sector of\na disk device or file system file to make it a *fifolog* and should be\ncalled only once.\n\nRunning `fifolog_create` on an existing *fifolog* will reset it so\nthat `fifolog_reader` and `fifolog_writer` will not see the previous\ncontents. (The previous contents are not physically erased, and with\na bit of hand-work all but the first record can be easily recovered.)\n\nIf the file does not already exist, `fifolog_create` will attempt\nto create and `ftruncate`(`2`) it to the specified size, defaulting to\n86400 records of 512 bytes if the `−r`, `−l` or `−s` options do not\nspecify otherwise.\n\nThe `fifolog_writer` utility will read standard input and write it to\nthe end of the *fifolog* according to the parameters given.\n\nWrites happen whenever the output buffer is filled with compressed\ndata or when either of two timers expire, forcing a partially filled\nbuffer to be written.\n\nThe first and faster timer, `−w write-rate`, forces available\ndata to be written but does not flush and reset the compression\ndictionary. This timer is intended to minimize the amount of logdata\nlost in RAM in case of a crash and by default it fires 10 seconds\nafter the previous write.\n\nThe second and slower timer, `−s sync-rate`, forces a full flush and\nreset of the compression engine and causes the next record written\nto be a synchronization point with an uncompressed timestamp, making\nit possible to start reading the logfile from that record. By default\nthis timer fires a minute after the previous sync.\n\nThe `−z compression` option controls the `zlib`(`3`) compression level;\nlegal values are zero to nine which is the default.\n\nThe `fifolog_reader` utility will retrieve records from the *fifolog*\naccording to the specified parameters and write them either to\nstandard output or the file specified with `−o`.\n\nIt is possible to specify a start and end time to limit the amount\nof data `fifolog_reader` will report.  The lower-case variants `−b`\nand `−e` take a `time_t` value, whereas the upper-case variants `−B`\nand `−E` take human-readable specifications such as “1 hour ago”.\n\nThe `−t` option forces timestamps to be formatted as\n“`YYYYMMDDhhmmss`” instead of as `time_t`, and `−T` allows the\nspecification of an `strftime`(`3`) formatting string.\n\nFinally, records can be filtered such that only records matching the\n(REG_BASIC) regular expression specified with `−R` are output.\n\n## IMPLEMENTATION NOTES\n\nThe data stored in the *fifolog* consists of three layers, an\n*outer layer* that allows searches to synchronization points based\non timestamps without having to decompress and decode the actual\ncontents, a *compression layer* implemented with `zlib`(`3`), and an\n*inner serialization and timestamping layer*.\n\nThe exact encoding is described in the `fifolog.h` file.\n\n*Fifolog* is particularly well suited for use on Flash based media,\nwhere it results in much lower write-wear, than a file system with\nregular log files rotated with `newsyslog`(`8`) etc.\n\n## EXAMPLES\n\n* Create a *fifolog* with 1024*1024 records of 512 bytes:\u003cbr\u003e\n  `fifolog_create -r 10m /tmp/fifolog`\n\n* Write a single record to this file:\u003cbr\u003e\n  `date | fifolog_writer /tmp/fifolog`\n\n* Read it back with human readable timestamps:\u003cbr\u003e\n  `fifolog_reader -t /tmp/fifolog`\n\n* One particular useful use of `fifolog_writer` is with `syslogd`(`8`)\n  using a line such as this in `syslog.conf`(`5`):\u003cbr\u003e\n  `*.* |fifolog_writer /var/log/syslog_fifolog`\n\n## HISTORY\n\nThe *fifolog* tools have been liberated from an open source SCADA\napplication called “*measured*”, which monitors and controls\nremote radio navigation transmitters for the Danish Air Traffic\nControl system.\n\n## BUILD\n\nPortable *fifolog* is known to build and work on AIX, FreeBSD, macOS,\nSolaris, illumos, Haiku, Cygwin, and Linux (glibc, musl, uClibc-ng,\nBionic) systems using GCC, Clang, NVIDIA HPC SDK C, IBM XL C, IBM\nOpen XL C, and CompCert-C.\n\nGNU Bison (or Berkeley Yacc) is required.  Review the\n[GNUmakefile](GNUmakefile) for details.\n\n* Build using *GNU Make* and system zlib:\u003cbr\u003e\n  `make`\n\n* Build using *GNU Make* and bundled miniz:\u003cbr\u003e\n  `make USE_MINIZ=1`\n\n## TEST\n\n* Run tests:\u003cbr\u003e\n  `make test`\n\n## AUTHORS\n\nThe *fifolog* tools were written by *Poul-Henning Kamp*.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazurespheredev%2Ffifolog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fazurespheredev%2Ffifolog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazurespheredev%2Ffifolog/lists"}