{"id":19040652,"url":"https://github.com/cloudfoundry/log-cache-release","last_synced_at":"2026-05-03T01:03:03.972Z","repository":{"id":27598594,"uuid":"110728341","full_name":"cloudfoundry/log-cache-release","owner":"cloudfoundry","description":"The BOSH release for Log Cache","archived":false,"fork":false,"pushed_at":"2025-04-14T10:11:36.000Z","size":20219,"stargazers_count":15,"open_issues_count":5,"forks_count":22,"subscribers_count":26,"default_branch":"main","last_synced_at":"2025-04-16T01:16:24.240Z","etag":null,"topics":["cff-wg-app-runtime-platform","cloud-foundry"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudfoundry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-14T18:25:45.000Z","updated_at":"2025-04-14T10:11:41.000Z","dependencies_parsed_at":"2023-12-20T07:34:18.406Z","dependency_job_id":"c65d5662-fbdc-4d62-ad45-0d397cb0bfe2","html_url":"https://github.com/cloudfoundry/log-cache-release","commit_stats":null,"previous_names":[],"tags_count":141,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Flog-cache-release","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Flog-cache-release/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Flog-cache-release/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Flog-cache-release/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudfoundry","download_url":"https://codeload.github.com/cloudfoundry/log-cache-release/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249178225,"owners_count":21225350,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cff-wg-app-runtime-platform","cloud-foundry"],"created_at":"2024-11-08T22:24:09.434Z","updated_at":"2026-05-03T01:03:03.958Z","avatar_url":"https://github.com/cloudfoundry.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"Log Cache Release\n=================\n\nIf you have any questions, or want to get attention for a PR or issue please reach out on the [#logging-and-metrics channel in the cloudfoundry slack](https://cloudfoundry.slack.com/archives/CUW93AF3M)\n\n## What is Log Cache?\n\nLog Cache provides an in memory caching layer for logs and metrics which are emitted by\napplications and system components in Cloud Foundry. \n\nLog Cache is deployed as a group of nodes which communicate between themselves\nover GRPC. Source IDs (such as the CF application ID or a unique string for a\nsystem component) are hashed to determine which Log Cache\ninstance will cache the data. Envelopes from a source ID can be sent to any log cache instance and\nare then forwarded to the assigned node. Similarly queries for Log Cache can be\nsent to any node and the query will be forwarded to the appropriate node which\nis caching the data for the requested source ID.\n\n## How does Log Cache fit into Cloud Foundry?\n\nLog Cache is included by default in \n[Cloud Foundry's cf-deployment](https://github.com/cloudfoundry/cf-deployment).\n\nBy default, Log Cache receives data from syslog agents which is an add on which runs on all instance groups by default.\n\nLog Cache is queried by Cloud Controller for app instance metrics such as CPU usage and memory when retrieving details for applications and \nby the cf cli directly to retrieve recent logs. It can also be queried using the Log Cache CLI plugin to retrieve system component metrics.\n\n\n## How do I configure it?\n\n### Scaling\n\nNumerous variables affect the retention of Log Cache:\n\nNumber of instances - Increasing adds more storage space, allows higher throughput and reduces contention between sources\n\nMax Per Source ID - Increasing allows a higher max storage allowance, but may decrease the storage of less noisy apps on the same node\n\nMemory per instance - Increasing allows more storage in general, but any given instance may not be able to take advantage of that increase due to max per source id\n\nMemory limit - Increasing memory limit allows for more storage, but may cause out of memory errors and crashing if set too high for the total throughput of the system\n\nLarger CPUs - Increasing the CPU budget per instance should allow higher throughput\n\nSometimes, depending on the scaling of Log Cache, the number of sources (CF applications and platform components) and the log load, ingress drops may occur when Log Cache nodes send items between each other. The Log Cache `ingress_dropped` metric should be monitored, to make sure that there are no drops. For such cases, the following three parameters can be adjusted until the log loss is gone.\n\n- Ingress Buffer Size - The ingress buffer (diode) size in number of items used when LogCache nodes send items between each other. The default size is 10000. Can be increased when ingress drops occur.\n- Ingress Buffer Read Batch Size - The ingress buffer read batch size in number of items. The size of the ingress buffer read batch used when LogCache nodes send items between each other.  The default size is 100. Can be increased when ingress drops occur.\n- Ingress Buffer Read Batch Interval - The ingress buffer read interval in milliseconds. The default value is 250. Can be increased when ingress drops occur.\n\n\nLog Cache is known to exceed memory limits under high throughput/stress. If you see your log-cache reaching higher memory\nthen you have set, you might want to scale your log-cache up. Either solely in terms of CPU per instance, or more instances.\n\nYou can monitor the performance of log cache per source id (app or platform component) using the Log Cache CLI. The command `cf log-meta` allows viewing\nthe amount of logs and metrics as well as the period of time for those logs and metrics for each source on the system. This can be used in conjunction with scaling\nto target your use cases. For simple pushes, a low retention period may be adequate. For running analysis on metrics for debugging and scaling, higher retention\nperiods may be desired; although one should remember all logs and metrics will always be lost upon crashes or re-deploys of log-cache.\n\n### Log Cache Syslog Server TLS and mutual TLS configuration\n\nIf someone runs Cloud Foundry with a hardened setup in terms of security, they might want to activate TLS or even mutual TLS(mTLS) for the incoming connections to the Log Cache Syslog Server. The activation of TLS and mTLS is optional and is configured by the presence of the needed certificates. For TLS a syslog certificate or syslog key should be present in the BPM configuration and for mTLS a syslog client CA certificate should be present in the BPM configuration. Check the BOSH [BPM template](jobs/log-cache-syslog-server/templates/bpm.yml.erb) and the [spec](jobs/log-cache-syslog-server/spec) for details.\n\n### Reliability\n\nLog Cache is an in memory cache and as such will drop envelopes when it restarts. Users should not expect 100% availability of\nlogs in Log Cache and should plan accordingly. For example, Cloud Controller can function without Log Cache though users are\ninformed that Log Cache is unavailable.\n\n## How do I use it?\n\n### From the `cf` CLI\n\nApplication developers using Cloud Foundry will use Log Cache automatically. Build logs while running\n`cf push` are streamed through Log Cache. Application logs when running `cf logs` are retrieved from Log Cache.\nApplication metrics when running `cf app APP_NAME` are retrieved from Log Cache.\n\n### Using the Log Cache CLI plugin\nTo query Log Cache directly users or operators can install the [Log Cache CLI plugin](https://github.com/cloudfoundry/log-cache-cli)\nby running `cf install-plugin -r CF-Community \"log-cache\"` which provides additional commands in the CLI for querying logs and metrics\nstored in log cache. This is useful for querying system component metrics which are not exposed otherwise. See the CLI plugin README for details. \n\n### Log Cache API\nDocumentation about the internals of Log Cache and its API can be found [here](https://github.com/cloudfoundry/log-cache-release/blob/main/src/README.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry%2Flog-cache-release","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudfoundry%2Flog-cache-release","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry%2Flog-cache-release/lists"}