{"id":27201086,"url":"https://github.com/gregl83/parlog","last_synced_at":"2025-08-18T05:11:24.828Z","repository":{"id":29297237,"uuid":"32830235","full_name":"gregl83/parlog","owner":"gregl83","description":"Log Parser for ExpressJS/Morgan Middleware","archived":false,"fork":false,"pushed_at":"2015-04-18T02:17:14.000Z","size":636,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-02T19:21:33.186Z","etag":null,"topics":["log-parser","morgan-middleware"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gregl83.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-03-24T23:05:39.000Z","updated_at":"2023-03-01T12:18:50.000Z","dependencies_parsed_at":"2022-08-25T22:41:51.484Z","dependency_job_id":null,"html_url":"https://github.com/gregl83/parlog","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/gregl83/parlog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregl83%2Fparlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregl83%2Fparlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregl83%2Fparlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregl83%2Fparlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gregl83","download_url":"https://codeload.github.com/gregl83/parlog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregl83%2Fparlog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270946068,"owners_count":24672890,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["log-parser","morgan-middleware"],"created_at":"2025-04-09T21:55:06.788Z","updated_at":"2025-08-18T05:11:24.759Z","avatar_url":"https://github.com/gregl83.png","language":"JavaScript","readme":"# parlog\nLog Parser for ExpressJS/Morgan Middleware\n\nA tool for querying and parsing Morgan log file directories and outputting results in CSV format.\n\n**CAUTION:**\n\nThis package does NOT have automated tests. While the package has served its intended purpose it should be considered\nan Alpha product until testing has been added. Hopefully tests will be added shortly.\n\n## Requirements\n\n- NodeJS\n- NPM\n\nSee `./package.json`\n\n## Installation\n\n    $ npm install parlog\n\n## Usage\n\n    $ ./bin/parlog [options]\n\nThis will output a CSV file of the parsed log file(s) to the working directory.\n\n## Options\n\n```\n$ ./bin/parlog --help\n\n  Usage: parlog [options]\n\n  Log Parser for ExpressJS/Morgan Middleware\n\n  Options:\n\n    -h, --help                  output usage information\n    -V, --version               output the version number\n    -d, --debug                 debug mode\n    -s, --start [MM/DD/YYYY]    start date for log match\n    -e, --end [MM/DD/YYYY]      end date for log match\n    -q, --query [regex]         query regular expression match\n    -f, --format [format]       format of logs\n    -i, --directory [path]      directory of log files\n    -t, --transform [filepath]  filepath of transform function\n    -o, --output [filepath]     filepath of output\n```\n\nAdditional details of *some options* provided below:\n\n### -s, --start | -e, --end\n\nInclusive date range of logs to query.\n\n### -q, --query\n\nQuery in regular expression (regex) format that matches rows to be parsed.\n\nThis option must be a regex string using **single quotes** in order to avoid any character issues.\n\nA JavaScript RegExp object is created from this option and tested against each log line. Log lines without a match are returned in results.\n\n### -f, --format\n\nThe format of the log file(s) to parse. The default is set to the *combined* log format.\n\nThe following are the parlog supported formats. Documentation copied directly from the [expressjs/morgan](https://github.com/expressjs/morgan) README.\n\nThere are various pre-defined formats provided:\n\n#### combined\n\nStandard Apache combined log output.\n\n```\n:remote-addr - :remote-user [:date[clf]] \":method :url HTTP/:http-version\" :status :res[content-length] \":referrer\" \":user-agent\"\n```\n\n#### common\n\nStandard Apache common log output.\n\n```\n:remote-addr - :remote-user [:date[clf]] \":method :url HTTP/:http-version\" :status :res[content-length]\n```\n\n#### dev\n\nConcise output colored by response status for development use. The `:status`\ntoken will be colored red for server error codes, yellow for client error\ncodes, cyan for redirection codes, and uncolored for all other codes.\n\n```\n:method :url :status :response-time ms - :res[content-length]\n```\n\n#### short\n\nShorter than default, also including response time.\n\n```\n:remote-addr :remote-user :method :url HTTP/:http-version :status :res[content-length] - :response-time ms\n```\n\n#### tiny\n\nThe minimal output.\n\n```\n:method :url :status :res[content-length] - :response-time ms\n```\n\n### -d, --directory\n\nDirectory of log files. Log files are found using a regular expression match on filename. The regular expression is located in the config.default.json file.\n\nNote: The filename regular expression can be overridden by creating a config.local.json file.\n\n### -t, --transform\n\nFile path of an exported transform function. This function is called during the transform process with a log line object argument that is passed by reference.\n\nThe transform function can be used to make changes to parsed log lines prior to outputting them.\n\nExample of the expected transform file format can be found in the `./src/transform.js` file.\n\n```js\nmodule.exports = function(logLine) {\n  // todo put logLine.data transformation code here\n};\n```\n\n#### logLine\n\nAn object that is accessible within the transform function. It is passed by reference.\n\nImportant properties:\n\n```js\nlogLine.data // contains the log line params that are in output results\n```\n\n```js\nlogLine.logOut // boolean that if false does NOT include log line to output results\n```\n\n### -o, --output\n\nFile path to write all parsing output.\n\nCurrently only CSV format is supported for output files.\n\n## Configuration\n\nA default configuration is provided with parlog: `./config.default.json`.\n\nConfig params can be overwritten using a local config file. To create the local config:\n\n```\n$ cp ./config.default.json ./config.local.json\n```\n\nMake configuration changes to the `./config.local.json` file. \n\n## Examples\n\nA few examples of running parlog.\n\n### Default\n\nThis will parse all log files in the default directory `./log`.\n\n```\n$ ./bin/parlog\n```\n\n### Directory\n\nThis will parse all log files in the user defined directory `/home/ubuntu/log`.\n\n```\n$ ./bin/parlog --directory /home/ubuntu/log\n```\n\n### Start Date\n\nThis will parse all logs that took place at or after 04/11/1970 at 0000.\n\n```\n$ ./bin/parlog --start \"04/11/1970\"\n```\n\n### Date Range\n\nThis will parse all logs that took place at or between 04/11/1970 at 0000 to 04/12/1970 0000.\n\n```\n$ ./bin/parlog --start \"04/11/1970\" --end \"04/12/1970\"\n```\n\n### Query with Start Date\n\nThis will parse all logs that took place at or after 04/11/1970 at 0000 and contain the word *houston* and *problem*.\n\n```\n$ ./bin/parlog --start \"04/11/1970\" --query '^(?=.*houston)(?=.*problem).*$'\n```\n\n### Debug Mode\n\nThis will run parser in debug mode which outputs more runtime details.\n\n```\n$ ./bin/parlog --debug\n```\n\n## License\n\nMIT","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgregl83%2Fparlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgregl83%2Fparlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgregl83%2Fparlog/lists"}