{"id":21322041,"url":"https://github.com/advantageous/systemd-cloud-watch","last_synced_at":"2025-07-12T05:30:45.032Z","repository":{"id":47184016,"uuid":"75040709","full_name":"advantageous/systemd-cloud-watch","owner":"advantageous","description":"Alt util to AWS cloudwatch agent that works w/ systemd journal and sends the data in batches to AWS cloudwatch.","archived":false,"fork":false,"pushed_at":"2021-09-09T12:38:03.000Z","size":191,"stargazers_count":92,"open_issues_count":11,"forks_count":36,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-06T14:21:33.615Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://advantageous.github.io/systemd-cloud-watch/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/advantageous.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-11-29T03:48:07.000Z","updated_at":"2025-01-28T02:05:40.000Z","dependencies_parsed_at":"2022-08-28T17:12:16.529Z","dependency_job_id":null,"html_url":"https://github.com/advantageous/systemd-cloud-watch","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/advantageous/systemd-cloud-watch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advantageous%2Fsystemd-cloud-watch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advantageous%2Fsystemd-cloud-watch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advantageous%2Fsystemd-cloud-watch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advantageous%2Fsystemd-cloud-watch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/advantageous","download_url":"https://codeload.github.com/advantageous/systemd-cloud-watch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/advantageous%2Fsystemd-cloud-watch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264941544,"owners_count":23686509,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-21T20:12:54.322Z","updated_at":"2025-07-12T05:30:44.754Z","avatar_url":"https://github.com/advantageous.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Systemd Journal CloudWatch Writer\n\nThis utility reads from the [systemd journal](https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html),\n and sends the data in batches to [Cloudwatch](https://aws.amazon.com/cloudwatch/).\n \nThis is an alternative process to the AWS-provided logs agent.\nThe AWS logs agent copies data from on-disk text log files into [Cloudwatch](https://aws.amazon.com/cloudwatch/).\nThis utility `systemd-cloud-watch` reads the `systemd journal` and writes that data in batches to CloudWatch.\n\nThere are other ways to do this using various techniques. But depending on the size of log messages and size of the core parts\nthese other methods are fragile as AWS CloudWatch limits the size of the messages. \nThis utility allows you cap the log field size, include only the fields that you want, or\nexclude the fields you don't want. We find that this is not only useful but essential. \n\n\n## Log format\n\nThe journal event data is written to ***CloudWatch*** Logs in JSON format, making it amenable to filtering using the JSON filter syntax.\nLog records are translated to ***CloudWatch*** JSON events using a structure like the following:\n\n#### Sample log\n```json\n{\n    \"instanceId\" : \"i-xxxxxxxx\",\n    \"pid\" : 12354,\n    \"uid\" : 0,\n    \"gid\" : 0,\n    \"cmdName\" : \"cron\",\n    \"exe\" : \"/usr/sbin/cron\",\n    \"cmdLine\" : \"/usr/sbin/CRON -f\",\n    \"systemdUnit\" : \"cron.service\",\n    \"bootId\" : \"fa58079c7a6d12345678b6ebf1234567\",\n    \"hostname\" : \"ip-10-1-0-15\",\n    \"transport\" : \"syslog\",\n    \"priority\" : \"INFO\",\n    \"message\" : \"pam_unix(cron:session): session opened for user root by (uid=0)\",\n    \"syslogFacility\" : 10,\n    \"syslogIdent\" : \"CRON\"\n}\n```\n\nThe JSON-formatted log events could also be exported into an AWS ElasticSearch instance using the ***CloudWatch***\nsync mechanism. Once in ElasticSearch, you can use an ELK stack to obtain more elaborate filtering and query capabilities.\n\n\n## Installation\n\nIf you have a binary distribution, you just need to drop the executable file somewhere.\n\nThis tool assumes that it is running on an EC2 instance.\n\nThis tool uses `libsystemd` to access the journal. systemd-based distributions generally ship\nwith this already installed, but if yours doesn't you must manually install the library somehow before\nthis tool will work.\n\nThere are instructions on how to install the Linux requirements for development below see - \n[Setting up a Linux env for testing/developing (CentOS7)](#setting-up-a-linux-env-for-testingdeveloping-centos7).\n\nWe also have two excellent examples of setting up a dev environment using [bin.packer](https://www.packer.io/) for both \n[AWS EC2](#building-the-ec2-image-with-packer-to-build-the-linux-instance-to-build-this-project) and \n[Docker](#building-the-docker-image-to-build-the-linux-instance-to-build-this-project). We setup CentoOS 7.\nThe EC2 instance bin.packer build uses the ***aws command line*** to create and connect to a running image. \nThese should be instructive for how to setup this utility in your environment to run with ***systemd*** as we provide\nall of the systemd scripts in the bin.packer provision scripts for EC2.  An example is good. A running example is better.\n\n## Configuration\n\nThis tool uses a small configuration file to set some values that are required for its operation.\nMost of the configuration values are optional and have default settings, but a couple are required.\n\nThe configuration file uses a syntax like this:\n\n```js\nlog_group = \"my-awesome-app\"\n\n```\n\nThe following configuration settings are supported:\n\n* `aws_region`: (Optional) The AWS region whose CloudWatch Logs API will be written to. If not provided,\n  this defaults to the region where the host EC2 instance is running.\n\n* `ec2_instance_id`: (Optional) The id of the EC2 instance on which the tool is running. There is very\n  little reason to set this, since it will be automatically set to the id of the host EC2 instance.\n\n* `journal_dir`: (Optional) Override the directory where the systemd journal can be found. This is\n  useful in conjunction with remote log aggregation, to work with journals synced from other systems.\n  The default is to use the local system's journal.\n\n* `log_group`: (Required) The name of the cloudwatch log group to write logs into. This log group must\n  be created before running the program.\n\n* `log_priority`: (Optional) The highest priority of the log messages to read (on a 0-7 scale). This defaults\n    to DEBUG (all messages). This has a behaviour similar to `journalctl -p \u003cpriority\u003e`. At the moment, only\n    a single value can be specified, not a range. Possible values are: `0,1,2,3,4,5,6,7` or one of the corresponding\n    `\"emerg\", \"alert\", \"crit\", \"err\", \"warning\", \"notice\", \"info\", \"debug\"`.\n    When a single log level is specified, all messages with this log level or a lower (hence more important)\n    log level are read and pushed to CloudWatch. For more information about priority levels, look at\n    https://www.freedesktop.org/software/systemd/man/journalctl.html\n\n* `log_stream`: (Optional) The name of the cloudwatch log stream to write logs into. This defaults to\n  the EC2 instance id. Each running instance of this application (along with any other applications\n  writing logs into the same log group) must have a unique `log_stream` value. If the given log stream\n  doesn't exist then it will be created before writing the first set of journal events.\n\n* `buffer_size`: (Optional) The size of the event buffer to send to CloudWatch Logs API. The default is 50.\n This means that cloud watch will send 50 logs at a time. \n\n* `fields`: (Optional) Specifies which fields should be included in the JSON map that is sent to CloudWatch.\n\n* `omit_fields`: (Optional) Specifies which fields should NOT be included in the JSON map that is sent to CloudWatch.\n\n* `field_length`: (Optional) Specifies how long string fileds can be in the JSON  map that is sent to CloudWatch.\n   The default is 255 characters.\n   \n*  `queue_batch_size` : (Optional) Internal. Default to 10,000 entries, how large the queue buffer is. This is chunks of log entries\nthat can be sent to the cloud watch repeater.\n   \n*  `queue_channel_size`: (Optional) Internal.  Default to 3 entries, how large the queue buffer is. This is how many `queue_batch_size`\ncan be around to send before the journald reader waits for the cloudwatch repeater. \n\n*  `queue_poll_duration_ms` : (Optional) Internal. Default to 10 ms, how long the queue manager will wait if there are no log entries to send\nto check again to see if there are log entries to send. \n\n*  `queue_flush_log_ms` : (Optional) If `queue_batch_size` has not been met because there are no more journald entries to \nread, how long to flush the buffer to cloud watch receiver. Defaults to 100 ms.\n\n* `debug`: (Optional) Turns on debug logging.\n\n* `local`: (Optional) Used for unit testing. Will not try to create an AWS meta-data client to read region and AWS credentials.\n\n* `tail`: (Optional) Start from the tail of log. Only send new log entries. This is good for reboot so you don't send all of the\nlogs in the system, which is the default behavior. \n\n* `rewind`: (Optional) Used to rewind X number of entries from the tail of the log. Must be used in conjunction with the \n`tail` setting.\n\n* `mock-cloud-watch` : (Optional) Used to send logs to a Journal Repeater that just spits out message and priority to the console.\nThis is used for development only. \n\n\nIf your average log message was 500 bytes, and your used the default setting then assuming the server was generating \njournald messages rapidly you could use a heap of up to `queue_channel_size` (3) * `queue_batch_size`(10,000) * 500 bytes\n(15,000,000). If you had a very resource constrained env, reduce the `queue_batch_size` and/or the `queue_channel_size`.\n\n\n\n### AWS API access\n\nThis program requires access to call some of the Cloudwatch API functions. The recommended way to\nachieve this is to create an\n[IAM Instance Profile](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html)\nthat grants your EC2 instance a role that has Cloudwatch API access. The program will automatically\ndiscover and make use of instance profile credentials.\n\nThe following IAM policy grants the required access across all log groups in all regions:\n\n#### IAM file\n```json\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"logs:CreateLogStream\",\n                \"logs:PutLogEvents\",\n                \"logs:DescribeLogStreams\"\n            ],\n            \"Resource\": [\n                \"arn:aws:logs:*:*:log-group:*\",\n                \"arn:aws:logs:*:*:log-group:*:log-stream:*\"\n            ]\n        }\n    ]\n}\n```\n\nIn more complex environments you may want to restrict further which regions, groups and streams\nthe instance can write to. You can do this by adjusting the two ARN strings in the `\"Resource\"` section:\n\n* The first `*` in each string can be replaced with an AWS region name like `us-east-1`\n  to grant access only within the given region.\n* The `*` after `log-group` in each string can be replaced with a Cloudwatch Logs log group name\n  to grant access only to the named group.\n* The `*` after `log-stream` in the second string can be replaced with a Cloudwatch Logs log stream\n  name to grant access only to the named stream.\n\nOther combinations are possible too. For more information, see\n[the reference on ARNs and namespaces](http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html#arn-syntax-cloudwatch-logs).\n\n\n\n### Coexisting with the official Cloudwatch Logs agent\n\nThis application can run on the same host as the official Cloudwatch Logs agent but care must be taken\nto ensure that they each use a different log stream name. Only one process may write into each log\nstream.\n\n## Running on System Boot\n\nThis program is best used as a persistent service that starts on boot and keeps running until the\nsystem is shut down. If you're using `journald` then you're presumably using systemd; you can create\na systemd unit for this service. For example:\n\n```\n[Unit]\nDescription=journald-cloudwatch-logs\nWants=basic.target\nAfter=basic.target network.target\n\n[Service]\nUser=nobody\nGroup=nobody\nExecStart=/usr/local/bin/journald-cloudwatch-logs /usr/local/etc/journald-cloudwatch-logs.conf\nKillMode=process\nRestart=on-failure\nRestartSec=42s\n```\n\nThis program is designed under the assumption that it will run constantly from some point during\nsystem boot until the system shuts down.\n\nIf the service is stopped while the system is running and then later started again, it will\n\"lose\" any journal entries that were written while it wasn't running. However, on the initial\nrun after each boot it will clear the backlog of logs created during the boot process, so it\nis not necessary to run the program particularly early in the boot process unless you wish\nto *promptly* capture startup messages.\n\n## Building\n\n#### Test cloud-watch package\n```sh\ngo test -v  github.com/advantageous/systemd-cloud-watch/cloud-watch\n```\n\n\n#### Build and Test on Linux (Centos7)\n```sh\n ./run_build_linux.sh\n```\n\nThe above starts up a docker container, runs `go get`, `go build`, `go test` and then copies the binary to\n`systemd-cloud-watch_linux`.\n\n#### Debug process running Linux\n```sh\n ./run_test_container.sh\n```\n\n\nThe above starts up a docker container that you can develop with that has all the prerequisites needed to\ncompile and test this project.\n\n#### Sample debug session\n```sh\n$ ./run_test_container.sh\nlatest: Pulling from advantageous/golang-cloud-watch\nDigest: sha256:eaf5c0a387aee8cc2d690e1c5e18763e12beb7940ca0960ce1b9742229413e71\nStatus: Image is up to date for advantageous/golang-cloud-watch:latest\n[root@6e0d1f984c03 /]# cd gopath/src/github.com/advantageous/systemd-cloud-watch/\n.git/                      README.md                  cloud-watch/               bin.packer/                    sample.conf                \n.gitignore                 build_linux.sh             main.go                    run_build_linux.sh         systemd-cloud-watch.iml    \n.idea/                     cgroup/                    output.json                run_test_container.sh      systemd-cloud-watch_linux  \n\n[root@6e0d1f984c03 /]# cd gopath/src/github.com/advantageous/systemd-cloud-watch/\n\n[root@6e0d1f984c03 systemd-cloud-watch]# ls\nREADME.md  build_linux.sh  cgroup  cloud-watch  main.go  output.json  bin.packer  run_build_linux.sh  \nrun_test_container.sh  sample.conf  systemd-cloud-watch.iml  systemd-cloud-watch_linux\n\n[root@6e0d1f984c03 systemd-cloud-watch]# source ~/.bash_profile\n\n[root@6e0d1f984c03 systemd-cloud-watch]# export GOPATH=/gopath\n\n[root@6e0d1f984c03 systemd-cloud-watch]# /usr/lib/systemd/systemd-journald \u0026\n[1] 24\n\n[root@6e0d1f984c03 systemd-cloud-watch]# systemd-cat echo \"RUNNING JAVA BATCH JOB - ADF BATCH from `pwd`\"\n\n[root@6e0d1f984c03 systemd-cloud-watch]# echo \"Running go clean\"\nRunning go clean\n\n[root@6e0d1f984c03 systemd-cloud-watch]# go clean\n\n[root@6e0d1f984c03 systemd-cloud-watch]# echo \"Running go get\"\nRunning go get\n\n[root@6e0d1f984c03 systemd-cloud-watch]# go get\n\n[root@6e0d1f984c03 systemd-cloud-watch]# echo \"Running go build\"\nRunning go build\n[root@6e0d1f984c03 systemd-cloud-watch]# go build\n\n[root@6e0d1f984c03 systemd-cloud-watch]# echo \"Running go test\"\nRunning go test\n\n[root@6e0d1f984c03 systemd-cloud-watch]# go test -v github.com/advantageous/systemd-cloud-watch/cloud-watch\n=== RUN   TestRepeater\nconfig DEBUG: 2016/11/30 08:53:34 config.go:66: Loading log...\naws INFO: 2016/11/30 08:53:34 aws.go:42: Config set to local\naws INFO: 2016/11/30 08:53:34 aws.go:72: Client missing credentials not looked up\naws INFO: 2016/11/30 08:53:34 aws.go:50: Client missing using config to set region\naws INFO: 2016/11/30 08:53:34 aws.go:52: AWSRegion missing using default region us-west-2\nrepeater ERROR: 2016/11/30 08:53:44 cloudwatch_journal_repeater.go:141: Error from putEvents NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors\n--- SKIP: TestRepeater (10.01s)\n\tcloudwatch_journal_repeater_test.go:43: Skipping WriteBatch, you need to setup AWS credentials for this to work\n=== RUN   TestConfig\ntest DEBUG: 2016/11/30 08:53:44 config.go:66: Loading log...\ntest INFO: 2016/11/30 08:53:44 config_test.go:33: [Foo Bar]\n--- PASS: TestConfig (0.00s)\n=== RUN   TestLogOmitField\ntest DEBUG: 2016/11/30 08:53:44 config.go:66: Loading log...\n--- PASS: TestLogOmitField (0.00s)\n=== RUN   TestNewJournal\n--- PASS: TestNewJournal (0.00s)\n=== RUN   TestSdJournal_Operations\n--- PASS: TestSdJournal_Operations (0.00s)\n\tjournal_linux_test.go:41: Read value=Runtime journal is using 8.0M (max allowed 4.0G, trying to leave 4.0G free of 55.1G available → current limit 4.0G).\n=== RUN   TestNewRecord\ntest DEBUG: 2016/11/30 08:53:44 config.go:66: Loading log...\n--- PASS: TestNewRecord (0.00s)\n=== RUN   TestLimitFields\ntest DEBUG: 2016/11/30 08:53:44 config.go:66: Loading log...\n--- PASS: TestLimitFields (0.00s)\n=== RUN   TestOmitFields\ntest DEBUG: 2016/11/30 08:53:44 config.go:66: Loading log...\n--- PASS: TestOmitFields (0.00s)\nPASS\nok  \tgithub.com/advantageous/systemd-cloud-watch/cloud-watch\t10.017s\n```\n\n\n\n\n#### Building the docker image to build the linux instance to build this project\n\n```sh\n# from project root\ncd bin.packer\nbin.packer build packer_docker.json\n```\n\n\n#### To run docker dev image\n```sh\n# from project root\ncd bin.packer\n./run.sh\n\n```\n\n#### Building the ec2 image with bin.packer to build the linux instance to build this project\n\n```sh\n# from project root\ncd bin.packer\nbin.packer build packer_ec2.json\n```\n\nWe use the [docker](https://www.packer.io/docs/builders/docker.html) support for [bin.packer](https://www.packer.io/).\n(\"Packer is a tool for creating machine and container images for multiple platforms from a single source configuration.\")\n\nUse `ec2_env.sh_example` to create a `ec2_env.sh` with the instance id that was just created. \n\n#### ec2_env.sh_example\n```\n#!/usr/bin/env bash\nexport ami=ami-YOURAMI\nexport subnet=subnet-YOURSUBNET\nexport security_group=sg-YOURSG\nexport iam_profile=YOUR_IAM_ROLE\nexport key_name=MY_PEM_FILE_KEY_NAME\n\n```\n\n##### Using EC2 image (assumes you have ~/.ssh config setup)\n```sh\n# from project root\ncd bin.packer\n\n# Run and log into dev env running in EC2\n./runEc2Dev.sh\n\n# Log into running server\n./loginIntoEc2Dev.sh\n\n```\n\n\n\n\n\n## Setting up a Linux env for testing/developing (CentOS7).\n```sh\nyum -y install wget\nyum install -y git\nyum install -y gcc\nyum install -y systemd-devel\n\n\necho \"installing go\"\ncd /tmp\nwget https://storage.googleapis.com/golang/go1.7.3.linux-amd64.tar.gz\ntar -C /usr/local/ -xzf go1.7.3.linux-amd64.tar.gz\nrm go1.7.3.linux-amd64.tar.gz\necho 'export PATH=$PATH:/usr/local/go/bin' \u003e\u003e ~/.bash_profile\n```\n\n## Setting up Java to write to systemd journal\n\n#### gradle build\n```\ncompile 'org.gnieh:logback-journal:0.2.0'\n\n```\n\n#### logback.xml\n```xml\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cconfiguration\u003e\n\n    \u003cappender name=\"journal\" class=\"org.gnieh.logback.SystemdJournalAppender\" /\u003e\n\n    \u003croot level=\"INFO\"\u003e\n        \u003cappender-ref ref=\"journal\" /\u003e\n        \u003ccustomFields\u003e{\"serviceName\":\"adfCalcBatch\",\"serviceHost\":\"${HOST}\"}\u003c/customFields\u003e\n    \u003c/root\u003e\n\n\n    \u003clogger name=\"com.mycompany\" level=\"INFO\"/\u003e\n\n\u003c/configuration\u003e\n```\n\n## Commands for controlling systemd service EC2 dev env\n\n```sh\n# Get status\nsudo systemctl status journald-cloudwatch\n# Stop Service\nsudo systemctl stop journald-cloudwatch\n# Find the service\nps -ef | grep cloud\n# Run service manually\n/usr/bin/systemd-cloud-watch_linux /etc/journald-cloudwatch.conf\n\n```\n\n\n\n## Derived\nThis is based on [advantageous journald-cloudwatch-logs](https://github.com/advantageous/journald-cloudwatch-logs)\nwhich was forked from [saymedia journald-cloudwatch-logs](https://github.com/saymedia/journald-cloudwatch-logs).\n\n\n## Status\nDone and released. \n\n\n\n### Using as a lib.\n\nYou can use this project as a lib and you can pass in your own *JournalRepeater* and your own *Journal*.\n\n\n#### Interface for JournalRepeater\n```go\npackage cloud_watch\n\n\ntype Record struct {...} //see source code\n\ntype JournalRepeater interface {\n\t// Close closes a journal opened with NewJournal.\n\tClose() error;\n\tWriteBatch(records []Record) error;\n}\n```\n\n#### Interface for Journal\n```go\ntype Journal interface {\n\t// Close closes a journal opened with NewJournal.\n\tClose() error;\n\n\t// Next advances the read pointer into the journal by one entry.\n\tNext() (uint64, error);\n\n\t// NextSkip advances the read pointer by multiple entries at once,\n\t// as specified by the skip parameter.\n\tNextSkip(skip uint64) (uint64, error);\n\n\t// Previous sets the read pointer into the journal back by one entry.\n\tPrevious() (uint64, error);\n\n\t// PreviousSkip sets back the read pointer by multiple entries at once,\n\t// as specified by the skip parameter.\n\tPreviousSkip(skip uint64) (uint64, error);\n\n\t// GetDataValue gets the data object associated with a specific field from the\n\t// current journal entry, returning only the value of the object.\n\tGetDataValue(field string) (string, error);\n\n\n\t// GetRealtimeUsec gets the realtime (wallclock) timestamp of the current\n\t// journal entry.\n\tGetRealtimeUsec() (uint64, error);\n\n \tAddLogFilters(config *Config)\n\n\t// GetMonotonicUsec gets the monotonic timestamp of the current journal entry.\n\tGetMonotonicUsec() (uint64, error);\n\n\t// GetCursor gets the cursor of the current journal entry.\n\tGetCursor() (string, error);\n\n\n\t// SeekHead seeks to the beginning of the journal, i.e. the oldest available\n\t// entry.\n\tSeekHead() error;\n\n\t// SeekTail may be used to seek to the end of the journal, i.e. the most recent\n\t// available entry.\n\tSeekTail() error;\n\n\t// SeekCursor seeks to a concrete journal cursor.\n\tSeekCursor(cursor string) error;\n\n\t// Wait will synchronously wait until the journal gets changed. The maximum time\n\t// this call sleeps may be controlled with the timeout parameter.  If\n\t// sdjournal.IndefiniteWait is passed as the timeout parameter, Wait will\n\t// wait indefinitely for a journal change.\n\tWait(timeout time.Duration) int;\n}\n\n```\n\n#### Using as a lib\n```go\n\npackage main\n\nimport (\n\tjcw  \"github.com/advantageous/systemd-cloud-watch/cloud-watch\"\n\t\"flag\"\n\t\"os\"\n)\n\nvar help = flag.Bool(\"help\", false, \"set to true to show this help\")\n\nfunc main() {\n\n\tlogger := jcw.NewSimpleLogger(\"main\", nil)\n\n\tflag.Parse()\n\n\tif *help {\n\t\tusage(logger)\n\t\tos.Exit(0)\n\t}\n\n\tconfigFilename := flag.Arg(0)\n\tif configFilename == \"\" {\n\t\tusage(logger)\n\t\tpanic(\"config file name must be set!\")\n\t}\n\n\tconfig := jcw.CreateConfig(configFilename, logger)\n\tlogger = jcw.NewSimpleLogger(\"main\", config)  \n\tjournal := jcw.CreateJournal(config, logger) //Instead of this, load your own journal\n\trepeater := jcw.CreateRepeater(config, logger) //Instead of this, load your own repeater\n\n\tjcw.RunWorkers(journal, repeater, logger, config )\n}\n\nfunc usage(logger *jcw.Logger) {\n\tlogger.Error.Println(\"Usage: systemd-cloud-watch  \u003cconfig-file\u003e\")\n\tflag.PrintDefaults()\n}\n\n```\n\nYou could for example create a *JournalRepeater* that writes to *InfluxDB* instead of *CloudWatch*.\n\n\n\n\nImprovements:\n\n* Added unit tests (there were none).\n* Heavily reduced locking by using [qbit](https://github.com/advantageous/go-qbit) instead of original implementation.\n* Added cross compile so I can develop/test on my laptop (MacOS).\n* Made logging stateless. No more need for a state file.\n* No more getting out of sync with CloudWatch.\n* Detects being out of sync and recovers.\n* Fixed error with log messages being too big.\n* Added ability to include or omit logging fields.\n* Created docker image and scripts to test on Linux (CentOS7).\n* Created EC2 image and scripts to test on Linux running in AWS EC2 (CentOS7).\n* Code organization (we use a package).\n* Added comprehensive logging which includes debug logging by config.\n* Uses actual timestamp from journal log record instead of just current time\n* Auto-creates CloudWatch log group if it does not exist\n* Allow this to be used as a library by providing interface for Journal and JournalWriter.\n\n\n## License\n\nThe original work was from Say Media Inc. We had issues with it and did about a 90% rewrite.\n\nAll additional work is covered under Apache 2.0 license.\nCopyright (c) 2016 Geoff Chandler, Rick Hightower\n\n\nCopyright (c) 2015 Say Media Inc\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadvantageous%2Fsystemd-cloud-watch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadvantageous%2Fsystemd-cloud-watch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadvantageous%2Fsystemd-cloud-watch/lists"}