{"id":19347173,"url":"https://github.com/dcos/dcos-log","last_synced_at":"2025-04-23T05:31:44.185Z","repository":{"id":57480670,"uuid":"69283878","full_name":"dcos/dcos-log","owner":"dcos","description":"dcos-log: is a HTTP server for systemd journal events","archived":false,"fork":false,"pushed_at":"2020-06-02T08:19:11.000Z","size":5837,"stargazers_count":18,"open_issues_count":2,"forks_count":8,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-06-20T14:50:47.534Z","etag":null,"topics":["dcos","dcos-ux-guild"],"latest_commit_sha":null,"homepage":"https://dcos.io","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dcos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-09-26T19:11:21.000Z","updated_at":"2020-07-01T15:24:04.000Z","dependencies_parsed_at":"2022-09-26T17:41:19.303Z","dependency_job_id":null,"html_url":"https://github.com/dcos/dcos-log","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcos%2Fdcos-log","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcos%2Fdcos-log/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcos%2Fdcos-log/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcos%2Fdcos-log/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dcos","download_url":"https://codeload.github.com/dcos/dcos-log/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223909905,"owners_count":17223592,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dcos","dcos-ux-guild"],"created_at":"2024-11-10T04:14:37.952Z","updated_at":"2024-11-10T04:14:38.555Z","avatar_url":"https://github.com/dcos.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dcos-go/dcos-log - HTTP server for journal events\n\n# REST API\n#### Endpoints:\n- `/range/` get a range of logs matching the request query.\n- `/range/framework/{framework_id}/executor/{executor_id}/container/{container_id}` get a range of application logs.\n- `/stream/` tail logs keeping the connection opened, implements Server Sent Events.\n- `/stream/framework/{framework_id}/executor/{executor_id}/container/{container_id}` tail application logs.\n- `/fields/\u003cfield\u003e` returns all possible unique values for a specific `\u003cfield\u003e`.\n\n#### Request Header Accept\n- `text/plain`, `text/html`, `*/*` request logs in text format, ending with `\\n`.\n- `application/json` request logs in JSON format.\n- `text/event-stream` request logs in Server-Sent-Events format.\n\n#### Request Header Last-Event-ID\nIf `Last-Event-ID` is set dcos-log will use it as a cursor position. `Last-Event-ID` header works with `/stream/` endpoints only.\n\nNOTE: Accept header `text/event-stream` cannot be used with `/fields/\u003cfield\u003e` endpoint.\n\n#### Response Header X-Journal-Skip-Prev\n`X-Journal-Skip-Prev` indicates how many entries we actually skipped backwards from the current position.\n\nExample:\n```\nrequest: `?cursor=\u003ccursor string\u003e\u0026skip_prev=1000`\nresponse includes the headers: `X-Journal-Skip-Prev: 100`, `X-Journal-Skip-Next: 0`\n\nThis means that there are only 100 entries before the given cursor position and we reached the top of the journal.\n\n```\n\n#### Response Header X-Journal-Skip-Next\n\n`Header X-Journal-Skip-Next` indicates how many entries we actually skipped forward from the current position.\n\nsee `X-Journal-Skip-Prev` example above.\n\n#### GET parameters:\n- `?filter=FIELD:value` add match.\n- `?limit=N` limit number of entries.\n- `?skip_next=N` skip forward number of entries from the current cursor position.\n- `?skip_prev=N` skip backwards number of entries from the current cursor position.\n- `?cursor=CURSOR` set cursor position. (Special characters must be escaped).\n- `?read_reverse=true` read the journal in opposite direction (bottom to top).\n\nwhere\n- `FIELD`, `value` and `CURSOR` are strings.\n- `N` is uint64.\n\nNOTE:\n- It is possbile to move to the tail of the journal. If the `?cursor` parameter is not used then we consider the cursor\n  is pointing to a head of the journal. `?skip_prev=1` can be used to move to the tail of the journal (very last entry). If you need to read last 10 entries you should use `?skip_prev=10`.\n- Parameter `?limit` cannot be used with `/stream/` endpoint.\n\n#### Response codes:\n- `200` OK.\n- `204` Content not found, returned if no entries matching requesting filters.\n- `400` Bad request, returned if request is incorrect.\n- `500` Internal server error.\n\n# CLI flags\n```\nUsage of dcos-log:\n  -config string\n       \tUse config file.\n  -config-json-schema string\n       \tUse a custom json schema.\n  -port int\n       \tSet TCP port. (default 8080)\n  -verbose\n       \tPrint out verbose output.\n```\n\n# Examples:\n#### GET parameters\n- `/stream/?skip_prev=10` get the last 10 entires from the journal and follow new events.\n- `/range/?skip_next=100\u0026limit=10` skip 100 entries from the beggining of the journal and return 10 following entries.\n- `/stream/?cursor=s%3Dcea8150abb0543deaab113ed2f39b014%3Bi%3D1%3Bb%3D2c357020b6e54863a5ac9dee71d5872c%3Bm%3D33ae8a1%3Bt%3D53e52ec99a798%3Bx%3Db3fe26128f768a49` get all logs after the specific cursor and follow new events.\n- `/range/?cursor=s%3Dcea8150abb0543deaab113ed2f39b014%3Bi%3D1%3Bb%3D2c357020b6e54863a5ac9dee71d5872c%3Bm%3D33ae8a1%3Bt%3D53e52ec99a798%3Bx%3Db3fe26128f768a49\u0026skip_prev=2\u0026limit=2` get 2 entries. The first one is the one before the cursor position and the second one is the entry with given cursor position.\n\n#### Accept: text/plain\n```\ncurl -H 'Accept: text/plain' '127.0.0.1:8080/range/?skip_prev=200\u0026limit=1'\nWed Oct 12 06:28:20 2016 a60c1d059aea systemd [1] Starting Daily Cleanup of Temporary Directories.\n```\n\n#### Accept: application/json\n```\ncurl -H 'Accept: application/json' '127.0.0.1:8080/range/?skip_prev=200\u0026limit=1' | jq '.'\n{\n  \"fields\": {\n    \"CODE_FILE\": \"../src/core/unit.c\",\n    \"CODE_FUNCTION\": \"unit_status_log_starting_stopping_reloading\",\n    \"CODE_LINE\": \"1272\",\n    \"MESSAGE\": \"Starting Daily Cleanup of Temporary Directories.\",\n    \"MESSAGE_ID\": \"7d4958e842da4a758f6c1cdc7b36dcc5\",\n    \"PRIORITY\": \"6\",\n    \"SYSLOG_FACILITY\": \"3\",\n    \"SYSLOG_IDENTIFIER\": \"systemd\",\n    \"UNIT\": \"systemd-tmpfiles-clean.timer\",\n    \"_BOOT_ID\": \"637573ba91ae4008b58eaa9505a11f86\",\n    \"_CAP_EFFECTIVE\": \"3fffffffff\",\n    \"_CMDLINE\": \"/sbin/init\",\n    \"_COMM\": \"systemd\",\n    \"_EXE\": \"/lib/systemd/systemd\",\n    \"_GID\": \"0\",\n    \"_HOSTNAME\": \"a60c1d059aea\",\n    \"_MACHINE_ID\": \"48230110dd084e91b7b6885728b98295\",\n    \"_PID\": \"1\",\n    \"_SOURCE_REALTIME_TIMESTAMP\": \"1476253700204523\",\n    \"_SYSTEMD_CGROUP\": \"e\",\n    \"_TRANSPORT\": \"journal\",\n    \"_UID\": \"0\"\n  },\n  \"cursor\": \"s=f78aeb5184144e2a94963a42b0cac49e;i=262;b=637573ba91ae4008b58eaa9505a11f86;m=6fbb8f76b;t=53ea51966297e;x=69cba0539a7e4576\",\n  \"monotonic_timestamp\": 29993006955,\n  \"realtime_timestamp\": 1476253700204926\n}\n```\n\n#### Accept: text/event-stream\n```\ncurl -H 'Accept: text/event-stream' '127.0.0.1:8080/range/?skip_prev=200\u0026limit=1'\nid: s=f78aeb5184144e2a94963a42b0cac49e;i=262;b=637573ba91ae4008b58eaa9505a11f86;m=6fbb8f76b;t=53ea51966297e\ndata: {\"fields\":{\"CODE_FILE\":\"../src/core/unit.c\",\"CODE_FUNCTION\":\"unit_status_log_starting_stopping_reloading\",\"CODE_LINE\":\"1272\",\"MESSAGE\":\"Starting Daily Cleanup of Temporary Directories.\",\"MESSAGE_ID\":\"7d4958e842da4a758f6c1cdc7b36dcc5\",\"PRIORITY\":\"6\",\"SYSLOG_FACILITY\":\"3\",\"SYSLOG_IDENTIFIER\":\"systemd\",\"UNIT\":\"systemd-tmpfiles-clean.timer\",\"_BOOT_ID\":\"637573ba91ae4008b58eaa9505a11f86\",\"_CAP_EFFECTIVE\":\"3fffffffff\",\"_CMDLINE\":\"/sbin/init\",\"_COMM\":\"systemd\",\"_EXE\":\"/lib/systemd/systemd\",\"_GID\":\"0\",\"_HOSTNAME\":\"a60c1d059aea\",\"_MACHINE_ID\":\"48230110dd084e91b7b6885728b98295\",\"_PID\":\"1\",\"_SOURCE_REALTIME_TIMESTAMP\":\"1476253700204523\",\"_SYSTEMD_CGROUP\":\"e\",\"_TRANSPORT\":\"journal\",\"_UID\":\"0\"},\"cursor\":\"s=f78aeb5184144e2a94963a42b0cac49e;i=262;b=637573ba91ae4008b58eaa9505a11f86;m=6fbb8f76b;t=53ea51966297e;x=69cba0539a7e4576\",\"monotonic_timestamp\":29993006955,\"realtime_timestamp\":1476253700204926}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcos%2Fdcos-log","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdcos%2Fdcos-log","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcos%2Fdcos-log/lists"}