https://github.com/fluent/fluent-logger-forward-node
A fluent forward protocol implementation for Node.js
https://github.com/fluent/fluent-logger-forward-node
fluent fluent-bit fluentd forward logger node
Last synced: 8 months ago
JSON representation
A fluent forward protocol implementation for Node.js
- Host: GitHub
- URL: https://github.com/fluent/fluent-logger-forward-node
- Owner: fluent
- License: apache-2.0
- Created: 2021-06-22T13:43:19.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-05-09T15:32:56.000Z (almost 3 years ago)
- Last Synced: 2025-06-25T21:40:52.119Z (8 months ago)
- Topics: fluent, fluent-bit, fluentd, forward, logger, node
- Language: TypeScript
- Homepage: https://fluent.github.io/fluent-logger-forward-node/
- Size: 547 KB
- Stars: 12
- Watchers: 3
- Forks: 9
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# @fluent-org/logger
[](https://github.com/fluent/fluent-logger-forward-node/actions)
[](https://opensource.org/licenses/Apache-2.0)
[](https://fluent.github.io/fluent-logger-forward-node/)
[Fluent Forward Protocol](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1) implementation for Node.js.
Built upon [fluent-logger-node](https://github.com/fluent/fluent-logger-node).
[](https://npmjs.com/package/@fluent-org/logger)
## Install
$ npm install @fluent-org/logger
## Client
`@fluent-org/logger` provides a fully functional client that implements the Forward protocol. It supports reconnection, acknowledgements, timeouts, event retries, and more, and exposes its functionality via a simple typed Promise interface.
For a full list of the client options and methods, see the [FluentClient docs](https://fluent.github.io/fluent-logger-forward-node/classes/FluentClient.html)
### Prerequisites
The fluent daemon should be listening in forward mode.
A simple starting configuration for Fluentd is the following:
```aconf
@type forward
port 24224
@type stdout
```
See the [FluentD docs](https://docs.fluentd.org/input/forward) for more info.
A similar starting configuration for Fluent Bit is the following:
```ini
[INPUT]
Name forward
Listen 0.0.0.0
Port 24224
Buffer_Chunk_Size 1M
Buffer_Max_Size 6M
[OUTPUT]
Name stdout
Match *
```
See the [Fluent Bit docs](https://docs.fluentbit.io/manual/pipeline/inputs/forward) for more info.
### Sending an event record to an upstream Fluent server
```js
const FluentClient = require("@fluent-org/logger").FluentClient;
const logger = new FluentClient("tag_prefix", {
socket: {
host: "localhost",
port: 24224,
timeout: 3000, // 3 seconds
}
});
```
The emit method has following signature
```typescript
emit(data: Record): Promise;
emit(data: Record, timestamp: number | Date | EventTime): Promise;
emit(label: string, data: Record): Promise;
emit(label: string, data: Record, timestamp: number | Date | EventTime): Promise;
```
The returned Promise is resolved once the event is written to the socket, or rejected if an error occurs.
### Acknowledgements
The [Fluent forward protocol](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1) provides explicit support for acknowledgements, which allow the client to be sure that the event reached its destination.
Enabling acknowledgements means that the promise returned by `emit` will be resolved once the client receives an explicit acknowledgement from the server.
```js
const FluentClient = require("@fluent-org/logger").FluentClient;
const logger = new FluentClient("tag_prefix", {
ack: {}
});
```
### Event modes
The [Fluent forward protocol](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1) provides multiple message modes, `Message`, `Forward`, `PackedForward`(default), `CompressedPackedForward`. The Fluent client supports all of them.
```js
const FluentClient = require("@fluent-org/logger").FluentClient;
const logger = new FluentClient("tag_prefix", {
eventMode: "Message" | "Forward" | "PackedForward" | "CompressedPackedForward"
});
```
### Disable automatic reconnect
```js
const logger = new FluentClient("tag_prefix", {
socket: {
host: "localhost",
port: 24224,
timeout: 3000, // 3 seconds
disableReconnect: true
}
});
// If you disable reconnections, the socket has to be manually connected,
// connect() returns a promise, which rejects on connection errors.
logger.connect();
```
### Shared key authentication
Logger configuration:
```js
const logger = new FluentClient("tag_prefix", {
socket: {
host: "localhost",
port: 24224,
timeout: 3000, // 3 seconds
},
security: {
clientHostname: "client.localdomain",
sharedKey: "secure_communication_is_awesome"
}
});
```
Fluentd configuration:
```aconf
@type forward
port 24224
self_hostname input.testing.local
shared_key secure_communication_is_awesome
@type stdout
```
See also the [Fluentd](https://github.com/fluent/fluentd) examples.
### TLS/SSL encryption
Logger configuration:
```js
const logger = new FluentClient("tag_prefix", {
socket: {
host: "localhost",
port: 24224,
timeout: 3000, // 3 seconds
tls: {
ca: fs.readFileSync("/path/to/ca_cert.pem")
},
},
security: {
clientHostname: "client.localdomain",
sharedKey: "secure_communication_is_awesome"
},
});
```
Fluentd configuration:
```aconf
@type forward
port 24224
ca_cert_path /path/to/ca_cert.pem
ca_private_key_path /path/to/ca_key.pem
ca_private_key_passphrase very_secret_passphrase
self_hostname input.testing.local
shared_key secure_communication_is_awesome
@type stdout
```
FYI: You can generate certificates using the `fluent-ca-generate` command since Fluentd 1.1.0.
See also [How to enable TLS/SSL encryption](https://docs.fluentd.org/input/forward#how-to-enable-tls-encryption).
### Mutual TLS Authentication
Logger configuration:
```js
const logger = new FluentClient("tag_prefix", {
socket: {
host: "localhost",
port: 24224,
timeout: 3000, // 3 seconds
tls: {
ca: fs.readFileSync("/path/to/ca_cert.pem"),
cert: fs.readFileSync("/path/to/client-cert.pem"),
key: fs.readFileSync("/path/to/client-key.pem"),
passphrase: "very-secret"
},
},
security: {
clientHostname: "client.localdomain",
sharedKey: "secure_communication_is_awesome"
}
});
```
Fluentd configuration:
```aconf
@type forward
port 24224
ca_path /path/to/ca-cert.pem
cert_path /path/to/server-cert.pem
private_key_path /path/to/server-key.pem
private_key_passphrase very_secret_passphrase
client_cert_auth true
self_hostname input.testing.local
shared_key secure_communication_is_awesome
@type stdout
```
### EventTime support
We can also specify an [EventTime](https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1#eventtime-ext-format) as a timestamp. See the [EventTime docs](https://fluent.github.io/fluent-logger-forward-node/classes/eventtime.html)
```js
const FluentClient = require("@fluent-org/logger").FluentClient;
const EventTime = require("@fluent-org/logger").EventTime;
const eventTime = new EventTime(1489547207, 745003500); // 2017-03-15 12:06:47 +0900
const logger = new FluentClient("tag_prefix", {
socket: {
host: "localhost",
port: 24224,
timeout: 3000, // 3 seconds
}
});
logger.emit("tag", { message: "This is a message" }, eventTime);
```
### Handling errors
The Fluent client will manage errors internally, and reject promises on errors. If you"d like to access the non-user facing internal errors, you can do so by passing `errorHandler`
```js
const FluentClient = require("@fluent-org/logger").FluentClient;
const logger = new FluentClient("tag_prefix", {
onSocketError: (err: Error) => {
console.error("error!", err)
}
});
```
### Retrying events
Sometimes it makes sense to resubmit events if their initial submission failed. You can do this by specifying `eventRetry`.
```js
const FluentClient = require("@fluent-org/logger").FluentClient;
const logger = new FluentClient("tag_prefix", {
eventRetry: {}
});
```
## Server
`@fluent-org/logger` includes a fully functional forward server which can be used as a downstream Fluent sink.
```js
const FluentServer = require("@fluent-org/logger").FluentServer;
const server = new FluentServer({ listenOptions: { port: 24224 }});
await server.listen();
```
Fluentd config:
```aconf
@type forward
send_timeout 60s
recover_wait 10s
hard_timeout 60s
name fluent_node
host 127.0.0.1
port 24224
weight 60
@type file
path /var/log/fluent/forward-failed
```
See the [FluentD docs](https://docs.fluentd.org/output/forward) for more info.
Alternatively, see the [Fluent Bit docs](https://docs.fluentbit.io/manual/pipeline/outputs/forward) for info on setting up Fluent Bit.
For a full list of the server options and methods, see the [FluentServer docs](https://fluent.github.io/fluent-logger-forward-node/classes/fluentserver.html)
## License
Apache License, Version 2.0.
[fluent-logger-python]: https://github.com/fluent/fluent-logger-python
[fluent-logger-node]: https://github.com/fluent/fluent-logger-node
## About NodeJS versions
This package is compatible with NodeJS versions >= 12.