{"id":13581627,"url":"https://github.com/papertrail/remote_syslog2","last_synced_at":"2025-12-16T21:17:02.109Z","repository":{"id":16295646,"uuid":"19044299","full_name":"papertrail/remote_syslog2","owner":"papertrail","description":"To install, see \"Releases\" tab. Self-contained daemon for reading local files and emitting remote syslog (without using local syslogd).","archived":false,"fork":false,"pushed_at":"2024-01-04T13:02:57.000Z","size":2040,"stargazers_count":637,"open_issues_count":58,"forks_count":157,"subscribers_count":54,"default_branch":"master","last_synced_at":"2024-11-05T21:44:51.287Z","etag":null,"topics":["daemons","go","papertrail","syslog"],"latest_commit_sha":null,"homepage":"http://help.papertrailapp.com/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/papertrail.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-04-22T20:04:03.000Z","updated_at":"2024-10-25T16:17:56.000Z","dependencies_parsed_at":"2024-11-05T21:43:09.709Z","dependency_job_id":null,"html_url":"https://github.com/papertrail/remote_syslog2","commit_stats":{"total_commits":238,"total_committers":30,"mean_commits":7.933333333333334,"dds":0.7647058823529411,"last_synced_commit":"3efcaf211ef49b095dec776ee3ec2d84fb8d9799"},"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/papertrail%2Fremote_syslog2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/papertrail%2Fremote_syslog2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/papertrail%2Fremote_syslog2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/papertrail%2Fremote_syslog2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/papertrail","download_url":"https://codeload.github.com/papertrail/remote_syslog2/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247470387,"owners_count":20944146,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["daemons","go","papertrail","syslog"],"created_at":"2024-08-01T15:02:08.434Z","updated_at":"2025-12-16T21:16:57.031Z","avatar_url":"https://github.com/papertrail.png","language":"Go","readme":"# remote_syslog2\n\n[![Download remote_syslog2](https://papertrail.github.io/remote_syslog2/images/download.png)][releases]\n\nremote_syslog tails one or more log files and sends syslog messages to a\nremote central syslog server. It generates packets itself, ignoring the system\nsyslog daemon, so its configuration doesn't affect system-wide logging.\n\nUses:\n\n * Collecting logs from servers \u0026 daemons which don't natively support syslog\n * When reconfiguring the system logger is less convenient than a\n   purpose-built daemon (e.g., automated app deployments)\n * Aggregating files not generated by daemons (e.g., package manager logs)\n\nThis code is tested with the hosted log management service [Papertrail]\nand should work for transmitting to any syslog server.\n\n## Migrating from remote_syslog 1\n\nremote_syslog2 is a rewrite of the ruby [remote_syslog] package. Not all\nfeatures of the ruby version are supported, and there are some backwards\nincompatible changes.\n\n### Which should I use?\n\nUse remote_syslog2 (this README and application) unless you have a\nspecific reason to use remote_syslog1.\n\n### Changes from remote_syslog 1\n\n* The syntax of some command-line arguments have changed slightly,\nthough most are identical.\n* Default hostname has been removed. Either the `host` config file\noption or the `-d` invocation flag are required.\n\n\n\n## Installing\n\nPrecompiled binaries for Mac (Darwin), Linux and Windows are available on the\n[remote_syslog2 releases page][releases].\n\nUntar the package, copy the \"remote_syslog\" executable into your $PATH,\nand then customize the included example_config.yml with the log file paths\nto read and the host/port to log to.\n\nOptionally, move and rename the configuration file to `/etc/log_files.yml` so\nthat remote_syslog picks it up automatically. For example:\n\n    sudo cp ./remote_syslog /usr/local/bin\n    sudo cp example_config.yml /etc/log_files.yml\n    sudo vi /etc/log_files.yml\n\nConfiguration directives can also be specified as command-line arguments (below).\n\n## Usage\n\n    Usage of remote_syslog2:\n      -c, --configfile string             Path to config (default \"/etc/log_files.yml\")\n          --debug-log-cfg string          The debug log file; overridden by -D/--no-detach\n      -d, --dest-host string              Destination syslog hostname or IP\n      -p, --dest-port int                 Destination syslog port (default 514)\n      -t, --dest-token string             Destination ingestion token\n          --eventmachine-tail             No action, provided for backwards compatibility\n      -f, --facility string               Facility (default \"user\")\n      -h, --help                          Display this help message\n          --hostname string               Local hostname to send from (default: OS hostname)\n          --log string                    Set loggo config, like: --log=\"\u003croot\u003e=DEBUG\" (default \"\u003croot\u003e=INFO\")\n          --new-file-check-interval int   How often to check for new files (seconds) (default 10)\n      -D, --no-detach                     Don't daemonize and detach from the terminal; overrides --debug-log-cfg\n          --no-eventmachine-tail          No action, provided for backwards compatibility\n          --pid-file string               Location of the PID file\n          --poll                          Detect changes by polling instead of inotify\n      -s, --severity string               Severity (default \"notice\")\n          --tcp                           Connect via TCP (no TLS)\n          --tls                           Connect via TCP with TLS\n      -V, --version                       Display version and exit\n\n## Example\n\nDaemonize and collect messages from files listed in `./example_config.yml` as\nwell as the file `/var/log/mysqld.log`. Write PID to `/tmp/remote_syslog.pid`\nand send to port `logs.papertrailapp.com:12345`:\n\n    $ remote_syslog -c example_config.yml -p 12345 --pid-file=/tmp/remote_syslog.pid /var/log/mysqld.log\n\nStay attached to the terminal, look for and use `/etc/log_files.yml` if it\nexists, and send with facility local0 to `a.example.com:514`:\n\n    $ remote_syslog -D -d a.example.com -f local0 /var/log/mysqld.log\n\n\n## Auto-starting at boot\n\nSample init files can be found [in the examples directory](examples/). You may be able to:\n\n    $ cp examples/remote_syslog.init.d /etc/init.d/remote_syslog\n    $ chmod 755 /etc/init.d/remote_syslog\n\nAnd then ensure it's started at boot, either by using:\n\n    $ sudo update-rc.d remote_syslog defaults\n\nor by creating a link manually:\n\n    $ sudo ln -s /etc/init.d/remote_syslog /etc/rc3.d/S30remote_syslog\n\nremote_syslog will daemonize by default.\n\nAdditional information about init files (`init.d`, `supervisor`, `systemd` and `upstart`) are\navailable [in the examples directory](examples/).\n\n\n## Sending messages securely ##\n\nIf the receiving system supports sending syslog over TCP with TLS, you can\npass the `--tls` option when running `remote_syslog`:\n\n    $ remote_syslog -D --tls -p 1234 /var/log/mysqld.log\n\nor add `protocol: tls` to your configuration file.\n\n\n## Configuration\n\nBy default, remote_syslog looks for a configuration in `/etc/log_files.yml`.\n\nThe archive comes with a [sample config](https://github.com/papertrail/remote_syslog2/blob/master/example_config.yml). Optionally:\n\n    $ cp example_config.yml.example /etc/log_files.yml\n\n`log_files.yml` has filenames to log from (as an array) and hostname and port\nto log to (as a hash). Wildcards are supported using * and standard shell\nglobbing. Filenames given on the command line are additive to those in\nthe config file.\n\nOnly 1 destination server is supported; the command-line argument wins.\n\n    files:\n     - /var/log/httpd/access_log\n     - /var/log/httpd/error_log\n     - /var/log/mysqld.log\n     - /var/run/mysqld/mysqld-slow.log\n    destination:\n      host: logs.papertrailapp.com\n      port: 12345\n      protocol: tls\n\nremote_syslog sends the name of the file without a path (\"mysqld.log\") as\nthe syslog tag (program name).\n\nAfter changing the configuration file, restart `remote_syslog` using the\ninit script or by manually killing and restarting the process. For example:\n\n    /etc/init.d/remote_syslog restart\n\n\n## Advanced Configuration (Optional)\n\nHere's an [advanced config](https://github.com/papertrail/remote_syslog2/blob/master/examples/log_files.yml.example.advanced) which uses all options.\n\n### Override hostname\n\nProvide `--hostname somehostname` or use the `hostname` configuration option:\n\n    hostname: somehostname\n\n\n### Detecting new files\n\nremote_syslog automatically detects and activates new log files that match\nits file specifiers. For example, `*.log` may be provided as a file specifier,\nand remote_syslog will detect a `some.log` file created after it was started.\n\nBy default, globs are re-checked every 10 seconds. To check for new files more\nfrequently, use the `--new-file-check-interval` argument. For example, to\nrecheck globs every 1 second, use:\n\n    --new-file-check-interval 1\n\nNote: messages may be written to new files in the period between when the\nfile is created and when the periodic glob check detects it. This data is not\ntransmitted.\n\nIf globs are specified on the command-line, enclose each one in single-quotes\n(`'*.log'`) so the shell passes the raw glob string to remote_syslog (rather\nthan the current set of matches). This is not necessary for globs defined in\nthe config file.\n\n\n### Log rotation and the behavior of remote_syslog\n\nExternal log rotation scripts often move or remove an existing log file\nand replace it with a new one (at a new inode). The Linux standard script\n[logrotate](https://iain.cx/src/logrotate/) supports a `copytruncate` config\noption.  With that option, `logrotate` will copy files, operate on the copies,\nand truncate the original so that the inode remains the same.\n\n`remote_syslog` will handle both approaches seamlessly, so it should be no\nconcern as to which method is used. If a log file is moved or renamed, \nand a new file is created (at a new inode), `remote_syslog` will follow that\nnew file at the new inode (assuming it has the same absolute path name). If\na file is copied then truncated, `remote_syslog` will seek to the beginning of\nthe truncated file and continue to read it.\n\n#### Log rotation edge cases to be aware of\n\nSome logging programs such as Java's gclog (`-XX:+PrintGC` or `-verbose:gc`)\ndo not log in append mode, so if another program such as `logrotate` (set to\n`copytruncate`) truncates the file, on the next write of the Java logger, the\nOS will fill the file with NUL bytes upto the current offset of the file descriptor.\nMore info on that [here](https://stackoverflow.com/questions/8353401/garbage-collector-log-loggc-file-rotation-with-logrotate-does-not-work-properl).\n`remote_syslog` will detect those leading NUL bytes, discard them, and log the discard count.\n\n\n### Excluding files from being sent\n\nProvide one or more regular expressions to prevent certain files from being\nmatched.\n\n    exclude_files:\n      - \\.\\d$\n      - .bz2\n      - .gz\n\n\n### Excluding lines matching a pattern\n\nThere may be certain log messages that you do not want to be sent.  These may be\nrepetitive log lines that are \"noise\" that you might not be able to filter out\neasily from the respective application.  To filter these lines, use the\nexclude_patterns with an array or regexes:\n\n    exclude_patterns:\n     - exclude this\n     - \\d+ things\n\n\n### Multiple instances\n\nRun multiple instances to specify unique syslog hostnames.\n\nTo do that, provide an alternate PID path as a command-line option to the\nadditional instance(s). For example:\n\n    --pid-file=/var/run/remote_syslog_2.pid\n\nNote: Daemonized programs use PID files to identify whether the program is already\nrunning ([more](https://unix.stackexchange.com/questions/12815/what-are-pid-and-lock-files-for/12818#12818)). Like other daemons, remote_syslog will refuse to run as a\ndaemon (the default mode) when a PID file is present. If a .pid file is\npresent but the daemon is not actually running, remove the PID file.\n\n### Choosing app name\n\nremote_syslog uses the log file name (like \"access_log\") as the syslog\nprogram name, or what the syslog RFCs call the \"tag.\" This is ideal unless\nremote_syslog watches many files that have the same name.\n\nIn that case, tell remote_syslog to set another program name using the\n`tag` attribute in the configuration file:\n```\nfiles:\n  - path: /var/log/httpd/access_log\n    tag: apache\ndestination:\n  host: logs.papertrailapp.com\n  port: 12345\n  protocol: tls\n```\n... or on the command line:\n`remote_syslog apache=/var/log/httpd/access_log`\n\nThis functionality was introduced in version 0.17\n\n## Troubleshooting\n\n### Generate debug log\n\nTo output debugging events with maximum verbosity, run:\n\n```\nremote_syslog --debug-log-cfg=logfile.txt --log=\"\u003croot\u003e=DEBUG\"\n```\n\n.. as well as any other arguments which are used in normal operation. This\nwill set [loggo](https://github.com/juju/loggo#func-parseconfigurationstring)'s\nroot logger to the `DEBUG` level and output to `logfile.txt`.\n\n### Truncated messages\n\nTo send messages longer than 1024 characters, use TCP (either TLS or cleartext\nTCP) instead of UDP. See \"[Sending messages securely](#sending-messages-securely)\" to\nuse TCP with TLS for messages of any length.\n\n[Here's why](https://help.papertrailapp.com/kb/configuration/troubleshooting-remote-syslog-reachability/#message-length) longer UDP messages are impossible to send over\nthe Internet.\n\n### inotify\n\nWhen running remote_syslog in the foreground using the `-D` switch, if you\nreceive the error:\n\n    Error creating fsnotify watcher: inotify_init: too many open files\n\ndetermine the maximum number of inotify instances that can be created using:\n\n    cat /proc/sys/fs/inotify/max_user_instances\n\nand then increase this limit using:\n\n    echo VALUE \u003e\u003e /proc/sys/fs/inotify/max_user_instances\n\nwhere VALUE is greater than the present setting. Confirm that remote_syslog starts\nup and then apply this new value permanently by adding the following to\n`/etc/sysctl.conf:`:\n\n    fs.inotify.max_user_instances = VALUE\n\n### \"No space left on device\"\n\nWhen monitoring a large number of files, this error may occur:\n\n    FATAL -- Error watching /path/here : no space left on device\n\nTo solve this, determine the maximum number of user watches that can be\ncreated using:\n\n    cat /proc/sys/fs/inotify/max_user_watches\n\nand then increase them using:\n\n    echo VALUE \u003e\u003e /proc/sys/fs/inotify/max_user_watches\n\nOnce again, confirm that remote_syslog starts and then apply this value permanently by adding the following to `/etc/sysctl.conf:`:\n\n    fs.inotify.max_user_watches = VALUE\n\n## Credits\n\n* [Paul Morton](https://twitter.com/mortonpe)\n* [Papertrail](https://papertrailapp.com/) staff\n* [Paul Hammond](https://paulhammond.org/)\n\n## Reporting bugs\n\n1. See whether the issue has already been reported: \u003chttps://github.com/papertrail/remote_syslog2/issues/\u003e\n2. If you don't find one, create an issue with a repro case.\n\n\n## Development\n\nremote_syslog2 is written in go. To get everything set up,\n[install go][goinstall] then run:\n\n    go install github.com/mitchellh/gox\n    git clone git@github.com:papertrail/remote_syslog2.git\n\nTo run tests:\n\n    # run all tests\n    go test ./...\n    # run all tests except the slower syslog reconnection tests\n    go test -short ./...\n\n\n## Building\n\n    make\n\n### ARM support\n\nAs of 0.18, we introduced ARM support for remote_syslog2. Current ARM builds\nsupport all ARM platforms with hardware floating point instruction sets. This\nincludes All Raspberry PI devices, most ARMv6 chips (Cortex), and ARMv7 and\nbeyond. \n\n## Contributing\n\nOnce you've made your great commits:\n\n1. [Fork][fk] remote_syslog\n2. Create a topic branch - `git checkout -b my_branch`\n3. Commit the changes without changing the Rakefile or other files unrelated to your enhancement.\n4. Push to your branch - `git push origin my_branch`\n5. Create a Pull Request or an [Issue][is] with a link to your branch\n6. That's it!\n\n\n[Papertrail]: https://papertrailapp.com/\n[remote_syslog]: https://github.com/papertrail/remote_syslog\n[releases]: https://github.com/papertrail/remote_syslog2/releases\n\n[goinstall]: https://golang.org/doc/install\n\n[fk]: https://help.github.com/forking/\n[is]: https://github.com/papertrail/remote_syslog/issues/\n","funding_links":[],"categories":["Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpapertrail%2Fremote_syslog2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpapertrail%2Fremote_syslog2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpapertrail%2Fremote_syslog2/lists"}