{"id":16114871,"url":"https://github.com/asaaki/cloudfront-logs","last_synced_at":"2026-02-28T01:09:24.011Z","repository":{"id":222007550,"uuid":"756014956","full_name":"asaaki/cloudfront-logs","owner":"asaaki","description":"AWS CloudFront log line parser","archived":false,"fork":false,"pushed_at":"2026-02-03T19:49:35.000Z","size":356,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-04T02:46:17.002Z","etag":null,"topics":["amazon","aws","cloudfront","cloudfront-logs","logs","parser","rust","rust-lang"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/cloudfront-logs","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/asaaki.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-02-11T18:30:05.000Z","updated_at":"2026-02-03T13:27:42.000Z","dependencies_parsed_at":"2024-03-28T06:40:02.209Z","dependency_job_id":"e5fa2f98-b63c-4196-9784-c3935280b3dd","html_url":"https://github.com/asaaki/cloudfront-logs","commit_stats":null,"previous_names":["asaaki/cloudfront-logs"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/asaaki/cloudfront-logs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asaaki%2Fcloudfront-logs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asaaki%2Fcloudfront-logs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asaaki%2Fcloudfront-logs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asaaki%2Fcloudfront-logs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/asaaki","download_url":"https://codeload.github.com/asaaki/cloudfront-logs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asaaki%2Fcloudfront-logs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29922071,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-27T19:37:42.220Z","status":"ssl_error","status_checked_at":"2026-02-27T19:37:41.463Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon","aws","cloudfront","cloudfront-logs","logs","parser","rust","rust-lang"],"created_at":"2024-10-09T20:16:15.228Z","updated_at":"2026-02-28T01:09:23.992Z","avatar_url":"https://github.com/asaaki.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cloudfront-logs\n\nA Rust-based AWS CloudFront log line parser\n\n## Log format\n\nThe AWS CloudFront log file format is described here:\n\n\u003chttps://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html#LogFileFormat\u003e\n\n## Purpose and Design\n\nThis parser currently focuses on parsing a single line of a log file only.\nIt provides are structured view to those tab-separated field items and avoids fiddling with numeric indices.\n\nIt's up to the user of this library to pass the individual lines to the parser.\nThis makes it very flexible to use in different scenarios as there are no assumptions about where those log lines come from and how they pass through the program.\n\nIt's possible that in the future more utilities get added, but as of now it's more important to deliver a fast and reliable parsing functionality.\n\nThe library therefore serves different parser implementation, so you can pick the one for your use cases and needs.\n\nConsult the benchmarks (run `./benches.sh`) for a synthetic overview.\n\n## Example\n\nGiven the following log line:\n\n```log\n2019-12-04\t21:02:31\tLAX1\t392\t192.0.2.100\tGET\td111111abcdef8.cloudfront.net\t/index.html\t200\t-\tMozilla/5.0%20(Windows%20NT%2010.0;%20Win64;%20x64)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/78.0.3904.108%20Safari/537.36\t-\t-\tHit\tSOX4xwn4XV6Q4rgb7XiVGOHms_BGlTAC4KyHmureZmBNrjGdRLiNIQ==\td111111abcdef8.cloudfront.net\thttps\t23\t0.001\t-\tTLSv1.2\tECDHE-RSA-AES128-GCM-SHA256\tHit\tHTTP/2.0\t-\t-\t11040\t0.001\tHit\ttext/html\t78\t-\t-\n```\n\nYou have different options to proces this line:\n\n```rust\nuse cloudfront_logs::*;\n\nlet logline: \u0026str = \"2019-12-04\t21:02:31\tLAX1\t392\t192.0.2.100\tGET\td111111abcdef8.cloudfront.net\t/index.html\t200\t-\tMozilla/5.0%20(Windows%20NT%2010.0;%20Win64;%20x64)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/78.0.3904.108%20Safari/537.36\t-\t-\tHit\tSOX4xwn4XV6Q4rgb7XiVGOHms_BGlTAC4KyHmureZmBNrjGdRLiNIQ==\td111111abcdef8.cloudfront.net\thttps\t23\t0.001\t-\tTLSv1.2\tECDHE-RSA-AES128-GCM-SHA256\tHit\tHTTP/2.0\t-\t-\t11040\t0.001\tHit\ttext/html\t78\t-\t-\";\n\n// -- borrowing the input --\n\n// reasonable default parser\nlet item = ValidatedRawLogline::try_from(logline).unwrap();\n\n// fields are only sub-slices from the input and therefore all return \u0026str\nassert_eq!(item.date, \"2019-12-04\");\nassert_eq!(item.sc_bytes, \"392\");\nassert_eq!(item.c_ip, \"192.0.2.100\");\n\n// -- get an owned version --\n\n// parser which only uses types accessible without external dependencies,\n// only Rust's core and std library is allowed\nlet item = ValidatedSimpleLogline::try_from(logline).unwrap();\n\nassert_eq!(item.date, \"2019-12-04\");\nassert_eq!(item.sc_content_len, 78);\nassert_eq!(item.c_ip, IpAddr::V4(Ipv4Addr::new(192, 0, 2, 100)));\n\n// -- get an owned and typed version --\n\n// parser which also converts some fields via external dependencies,\nlet item = ValidatedTimeLogline::try_from(logline).unwrap();\n\n// here: date and time from the `time` crate\nassert_eq!(item.date, time_macros::date!(2019-12-04));\nassert_eq!(item.time, time_macros::time!(21:02:31));\nassert_eq!(item.time_taken, Duration::from_millis(1));\n```\n\n## Benchmark example\n\nThe following was run under WSL Ubuntu, on a AMD Ryzen 9 7950X3D 16-Core Processor, 64 GiB RAM machine.\n\nYour own numbers may vary. What's more important are the relative differences of the parser implementations.\n\n```shell\n# code under benches/comparison-real-world.rs\nRUSTFLAGS=\"-Ctarget-cpu=native\" cargo bench -q --all-features --bench real-world\n```\n\n```txt\n*** Comparing different parsers for AWS CloudFront logs ***\n\nParses lines and extracts a few fields, slightly unordered,\nthis should simulate close to real-world usages.\nTimer precision: 10 ns\nreal_world                   fastest       │ slowest       │ median        │ mean          │ samples │ iters\n├─ 00 CheckedRawLogLine                    │               │               │               │         │\n│  ├─ Line A                 162.3 ns      │ 210.6 ns      │ 167.1 ns      │ 167.8 ns      │ 1000    │ 1000000\n│  ├─ Line B                 164.2 ns      │ 275.6 ns      │ 171.8 ns      │ 175.8 ns      │ 1000    │ 1000000\n│  ├─ Lines A+B              325.1 ns      │ 398.2 ns      │ 337.4 ns      │ 337.5 ns      │ 1000    │ 1000000\n│  ╰─ Sample File            994 ns        │ 1.1 µs        │ 1.024 µs      │ 1.029 µs      │ 1000    │ 1000000\n├─ 10 CheckedRawLogLineView                │               │               │               │         │\n│  ├─ Line A                 366.6 ns      │ 422.9 ns      │ 376.8 ns      │ 378 ns        │ 1000    │ 1000000\n│  ├─ Line B                 358.8 ns      │ 412.5 ns      │ 369 ns        │ 370 ns        │ 1000    │ 1000000\n│  ├─ Lines A+B              716.5 ns      │ 888.4 ns      │ 748.2 ns      │ 749.7 ns      │ 1000    │ 1000000\n│  ╰─ Sample File            2.178 µs      │ 2.784 µs      │ 2.279 µs      │ 2.279 µs      │ 1000    │ 1000000\n├─ 11 SmartRawLogLineView                  │               │               │               │         │\n│  ├─ Line A                 287.5 ns      │ 385 ns        │ 298.9 ns      │ 301.3 ns      │ 1000    │ 1000000\n│  ├─ Line B                 285.2 ns      │ 401.5 ns      │ 301.5 ns      │ 303 ns        │ 1000    │ 1000000\n│  ├─ Lines A+B              556.7 ns      │ 680.3 ns      │ 594.7 ns      │ 595.8 ns      │ 1000    │ 1000000\n│  ╰─ Sample File            1.694 µs      │ 2.671 µs      │ 1.789 µs      │ 1.796 µs      │ 1000    │ 1000000\n├─ 20 SimpleLogLine                        │               │               │               │         │\n│  ├─ Line A                 355.2 ns      │ 432 ns        │ 370.8 ns      │ 372.9 ns      │ 1000    │ 1000000\n│  ├─ Line B                 347.8 ns      │ 533.7 ns      │ 370.7 ns      │ 373.5 ns      │ 1000    │ 1000000\n│  ├─ Lines A+B              715.5 ns      │ 883.4 ns      │ 752 ns        │ 753.9 ns      │ 1000    │ 1000000\n│  ╰─ Sample File            2.136 µs      │ 3.085 µs      │ 2.236 µs      │ 2.247 µs      │ 1000    │ 1000000\n╰─ 21 TypedLogLine                         │               │               │               │         │\n   ├─ Line A                 395.5 ns      │ 467.9 ns      │ 407.9 ns      │ 409.6 ns      │ 1000    │ 1000000\n   ├─ Line B                 387.8 ns      │ 512.7 ns      │ 397.1 ns      │ 399.5 ns      │ 1000    │ 1000000\n   ├─ Lines A+B              781 ns        │ 1.164 µs      │ 812.2 ns      │ 813.6 ns      │ 1000    │ 1000000\n   ╰─ Sample File            2.317 µs      │ 3.551 µs      │ 2.384 µs      │ 2.409 µs      │ 1000    │ 1000000\n```\n\nThere are more benches you can run, like `single-field` and `two-fields` which should highlight where the \"View\" parsers shine.\n\n## Safety\n\nThis crate uses ``#![forbid(unsafe_code)]`` to ensure everything is implemented in 100% Safe Rust.\n\n## License\n\n\u003csup\u003e\nLicensed under either of\n  \u003ca href=\"https://raw.githubusercontent.com/asaaki/cloudfront-logs/main/LICENSE-APACHE\"\u003eApache License, Version 2.0\u003c/a\u003e or\n  \u003ca href=\"https://raw.githubusercontent.com/asaaki/cloudfront-logs/main/LICENSE-MIT\"\u003eMIT license\u003c/a\u003e\nat your option.\n\u003c/sup\u003e\n\n\u003cbr/\u003e\n\n\u003csub\u003e\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in this crate by you, as defined in the Apache-2.0 license, shall\nbe dual licensed as above, without any additional terms or conditions.\n\u003c/sub\u003e\n\n\u003c!-- links --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasaaki%2Fcloudfront-logs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fasaaki%2Fcloudfront-logs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasaaki%2Fcloudfront-logs/lists"}