{"id":19499701,"url":"https://github.com/apsl/go-aws-elb-log-analyzer","last_synced_at":"2025-04-25T22:34:24.369Z","repository":{"id":57546838,"uuid":"66954088","full_name":"APSL/go-aws-elb-log-analyzer","owner":"APSL","description":"Download logs from ELB and join in a sorted file","archived":false,"fork":false,"pushed_at":"2017-05-03T15:19:02.000Z","size":12477,"stargazers_count":30,"open_issues_count":1,"forks_count":5,"subscribers_count":23,"default_branch":"master","last_synced_at":"2024-06-21T03:06:04.117Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/APSL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-08-30T15:35:51.000Z","updated_at":"2024-04-13T01:56:57.000Z","dependencies_parsed_at":"2022-09-16T23:21:14.391Z","dependency_job_id":null,"html_url":"https://github.com/APSL/go-aws-elb-log-analyzer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/APSL%2Fgo-aws-elb-log-analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/APSL%2Fgo-aws-elb-log-analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/APSL%2Fgo-aws-elb-log-analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/APSL%2Fgo-aws-elb-log-analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/APSL","download_url":"https://codeload.github.com/APSL/go-aws-elb-log-analyzer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224019605,"owners_count":17242177,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T22:05:52.648Z","updated_at":"2024-11-10T22:05:53.581Z","avatar_url":"https://github.com/APSL.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Download logs from ELB and concat all of the in one file in order #\n\nThis project was created to analyze ELB logs stored in S3. Logs are pared and processed in memory. Don't need to download the log files to the local hard drive or send by a pipe.\n\nWhy develop new tool in go? We need something simple and fast, easy like execute a binary with the possibility to deploy this binary anywhere without any dependency issues. Just copy and execute.\n\nIt's compatible with credentials of AWS-CLI http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-multiple-profiles\n\n## How to use ##\n\nIn case your ELB has a dedicated bucket\n\n```bash\ngoelbanalyzer \\\n  --profile=\"my_project\" \\\n  --bucket=\"my-elb-bucket-logs\" \\\n  --start=\"2016-08-17 01:00:00 +0000\" \\\n  --end=\"2h\"\n```\n\n## Concat and sort file ##\n\nAfter locate and download all the files whitch are in the time frame, this software will join in one file and sort it by date. So you will have a uniq file with all the records in order.\n\nFinale filename **complete.log**\n\nTo change the name of the file you can use \"-filename\", example:\n\n```bash\ngoelbanalyzer \\\n  --profile=\"my_project\" \\\n  --bucket=\"my-elb-bucket-logs\" \\\n  --start=\"2016-08-17 01:00:00 +0000\" \\\n  --end=\"2h\" \\\n  --filename=\"myfull.log\"\n```\n\nTo disable the contact, use '--join 0' in the parameters.\n\n\n## Example report ##\n\n### Command ###\n\nIn case your ELB has a dedicated bucket\n\n```bash\ngoelbanalyzer \\\n  --profile=\"my_project\" \\\n  --bucket=\"my-elb-bucket-logs\" \\\n  --start=\"2016-08-17 01:00:00 +0000\" \\\n  --end=\"2h\"\n  --analyze\n```\n\n```\n2016/08/30 16:23:41 Time Range: 2016-08-17 01:00:00 +0000 +0000 - 2016-08-17 07:08:00 +0000 +0000\n2016/08/30 16:23:41 Bucket: ?.....?\n2016/08/30 16:23:43 Loading..\n2016/08/30 16:34:10 81464 lines where processed\n2016/08/30 16:34:10 Top clients by hits\n***** TOP by hists\n              IP |   Hits |  Median latency |  Percentile 90 latency | Average latency\n    52.18.999.11 |  24438 |    0.1749155000 |          79.1077260000 | 13.1908529439\n    47.89.999.11 |   4542 |    1.1352130000 |           8.6127860000 | 2.5885444800\n  203.183.999.11 |   2157 |    0.1801270000 |          78.2074270000 | 13.3125795345\n    52.68.999.11 |   1848 |    0.1780620000 |           1.4558380000 | 0.5151559892\n  120.132.999.11 |    954 |    0.1222715000 |           1.1246040000 | 0.2924876447\n    13.95.999.11 |    936 |    4.7280725000 |           9.3240620000 | 4.1887639006\n  120.132.999.11 |    897 |    0.1253540000 |           1.1191700000 | 0.3108006778\n  120.132.999.11 |    889 |    0.1248950000 |           1.1063860000 | 0.2765490641\n  120.132.999.11 |    782 |    0.1207545000 |           1.1104930000 | 0.2850222621\n   169.57.999.11 |    704 |   68.1784910000 |         104.9732110000 | 65.1308670170\n  218.244.999.11 |    628 |    5.6638660000 |          10.0451020000 | 5.6309517452\n   169.57.999.11 |    438 |   70.5174620000 |         102.7895760000 | 66.3744711644\n  222.122.999.11 |    382 |    0.1758160000 |          74.2409350000 | 15.1374491204\n  222.122.999.11 |    303 |    2.0829630000 |           8.6622070000 | 2.7980553102\n  213.182.999.11 |    300 |   79.8319910000 |         107.8641890000 | 72.3946263033\n  222.122.999.11 |    266 |    0.5016970000 |          75.3529030000 | 16.5369312406\n  133.242.999.11 |    255 |    5.9971280000 |          10.6584430000 | 6.3336784039\n  153.120.999.11 |    247 |    5.1963570000 |           8.8504320000 | 4.9291020162\n   175.99.999.11 |    198 |    2.5931665000 |           8.9435810000 | 3.8328212879\n  222.122.999.11 |    177 |    0.0503840000 |          51.6966140000 | 6.0209316102\n\n***** TOP by median latency\n              IP  |  Hits |  Median latency |  Percentile 90 latency | Average latency\n   182.213.999.11 |   300 |   79.8319910000 |         107.8641890000 | 72.3946263033\n   213.182.999.11 |    82 |   71.9786335000 |         103.3641420000 | 65.9088648171\n    169.57.999.11 |   438 |   70.5174620000 |         102.7895760000 | 66.3744711644\n    169.57.999.11 |   704 |   68.1784910000 |         104.9732110000 | 65.1308670170\n   213.182.999.11 |    71 |   57.0327860000 |         109.0471830000 | 55.9872595070\n   213.182.999.11 |    31 |   54.0556680000 |         101.3834090000 | 52.8765958710\n    121.78.999.11 |     2 |   38.3978040000 |          69.1167380000 | 38.3978040000\n   213.182.999.11 |    53 |   37.4184290000 |          94.6283570000 | 42.5396707547\n   222.122.999.11 |     3 |   36.7208290000 |          47.4126790000 | 28.1037163333\n   125.141.999.11 |    42 |   15.6014080000 |          81.4945850000 | 26.3813173810\n   125.141.999.11 |    76 |   13.3390590000 |         103.4595620000 | 28.4127705921\n   221.139.999.11 |     1 |   11.0201320000 |          11.0201320000 | 11.0201320000\n     65.52.999.11 |     1 |    8.5374680000 |           8.5374680000 | 8.5374680000\n   218.213.999.11 |     1 |    8.1529430000 |           8.1529430000 | 8.1529430000\n    119.81.999.11 |     3 |    7.5322690000 |           8.5617910000 | 7.1656163333\n   182.162.999.11 |    22 |    6.2606350000 |           9.7995030000 | 6.6234874091\n   133.242.999.11 |   255 |    5.9971280000 |          10.6584430000 | 6.3336784039\n   218.153.999.11 |     4 |    5.6652730000 |           8.5056560000 | 5.0062797500\n   218.244.999.11 |   628 |    5.6638660000 |          10.0451020000 | 5.6309517452\n   153.120.999.11 |   247 |    5.1963570000 |           8.8504320000 | 4.9291020162\n```\n\n# How to install #\n\nDownload the bin/goelbanalyzer and copy in /usr/local/bin\n```\nchmod +x /usr/local/bin/goelbanalyzer\n```\n\n# How to compile #\nThe final binary name will be \"goelbanalyzer\"\n```bash\ngo build -o goelbanalyzer *.go\n```\n\n## Help args ##\n\n```\nUsage of goelbanalyzer:\n  -analyze\n    \tAnalyze the logs to find top requests and top slow requests\n  -bucket string\n    \tName of the S3 bucket\n  -end string\n    \tTime after start, example of 30 minutes: 30m (default \"1h\")\n  -filename string\n    \tName of the final log file (default \"complete.log\")\n  -join\n    \tContact and sort all logs in one file (default true)\n  -prefix string\n    \tPrefix or folder used in S3 bucket\n  -profile string\n    \tProfile credentials used by aws cli\n  -region string\n    \tName of the region in AWS (default \"eu-west-1\")\n  -start string\n    \tDate and time to start the download. Example: 2016-09-16 05:00:00 +0000\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapsl%2Fgo-aws-elb-log-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapsl%2Fgo-aws-elb-log-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapsl%2Fgo-aws-elb-log-analyzer/lists"}