{"id":19621390,"url":"https://github.com/commenthol/ua-parser-tools","last_synced_at":"2025-09-07T14:44:41.845Z","repository":{"id":11736200,"uuid":"14262357","full_name":"commenthol/ua-parser-tools","owner":"commenthol","description":"Development Tools for ua-parser","archived":false,"fork":false,"pushed_at":"2015-02-11T19:20:30.000Z","size":464,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-26T18:53:36.291Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/commenthol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-11-09T18:20:28.000Z","updated_at":"2015-02-11T19:20:30.000Z","dependencies_parsed_at":"2022-09-06T13:52:30.958Z","dependency_job_id":null,"html_url":"https://github.com/commenthol/ua-parser-tools","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/commenthol/ua-parser-tools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commenthol%2Fua-parser-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commenthol%2Fua-parser-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commenthol%2Fua-parser-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commenthol%2Fua-parser-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/commenthol","download_url":"https://codeload.github.com/commenthol/ua-parser-tools/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commenthol%2Fua-parser-tools/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274051154,"owners_count":25214023,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T11:22:44.044Z","updated_at":"2025-09-07T14:44:41.755Z","avatar_url":"https://github.com/commenthol.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ua-parser-tools\n\n\u003e Development Tools for Development Tools for uap-core (ua-parser)\n\nThis project contains development tools which may help to add new regular expressions to the `regexes.yaml` file of the [uap-core](https://github.com/ua-parser/uap-core) Project.\n\nFrom a file containing user-agent strings sorted csv-tables for the different parsers can be generated.\nWith the csv-tables the parsing results for the given user-agent strings can be compared.\n\nTo detect the matching parsing line in the `regexes.yaml`, debug information can be added to the file.\n\n## Table of Contents\n\n\u003c!-- !toc (minlevel=2 omit=\"Table of Contents\") --\u003e\n\n* [General Files](#general-files)\n* [Files to generate lists and add test cases](#files-to-generate-lists-and-add-test-cases)\n* [Quick guide to get this project up and running](#quick-guide-to-get-this-project-up-and-running)\n* [Development Process](#development-process)\n* [Advanced Settings](#advanced-settings)\n  * [Conversions](#conversions)\n  * [Add unmatched entries to the testcases file](#add-unmatched-entries-to-the-testcases-file)\n  * [Generate a new testcases file](#generate-a-new-testcases-file)\n  * [Run tests against you testcases file](#run-tests-against-you-testcases-file)\n* [Special Tools](#special-tools)\n\n\u003c!-- toc! --\u003e\n\n## General Files\n\n**useragents.txt**\nList of user-agents which are used as input. Each line shall contain one user-agent string.\n\n**debuginfo.js**\nAdd or remove debug information to the `regexes.yaml` file. Run the file with `node debuginfo.js`.\u003cbr\u003e\nPlease *remove* the debug infomation before commiting the changed `regexes.yaml` file.\n\n**config.js**\nConfiguration settings to locate the `test_resources` and the `regexes.yaml` file.\n\n\n## Files to generate lists and add test cases\n\n**ua.js**\nParse the user-agents with the ua-parser.\n\n**os.js**\nParse the user-agents with the os-parser.\n\n**device.js**\nParse the user-agents with the device-parser.\n\nAll files can be used with the following arguments:\n\n* *-u PATH* : Instead of \"useragents.txt\" the file with \"PATH\" is used as input.\n* *-s*  : Change the column for showing the regex matcher number from column one. The sorting of the resulting cvs-table will be different. This option allows to check different matchers for same model, brand or family.\n* *-t*  : Generate testcases file. All user-agents encountered in the testcases file will be appended\n\n\n## Quick guide to get this project up and running\n\n1. Install node from [nodeJS](https://nodejs.org/download/)\n\n2. Clone this project and run\n\n   ````\n   git clone https://github.com/commenthol/ua-parser-tools.git\n   npm install\n   ````\n\n## Development Process\n\nAs an example the development process to add and change regular expressions\nis depicted with adding new devices to the \"device_parsers\". For any other\nparser you can follow the same steps with replacing `device.js` by either\n`os.js` or `ua.js` .\n\n1. Clone (or fork) the `ua-parser` project within this directory.\n\n   ````\n   git clone https://github.com/ua-parser/uap-core.git\n   ````\n\n   *Note:* If you have forked `uap-core` into a different dir adapt the setting `config.ua_parser.dir` in `config.js` accordingly.\n\n2. Add the debug information to the `regexes.yaml` file. For each\n   \"regex\" a debug info in the form \"#0001\" will be added and counted up.\n\n   ````\n   node debuginfo.js\n   ````\n\n3. Add your user-agents to the file `useragents.txt`.\n4. Parse the user-agents with the parser you like to change.\n   E.g. here \"device_parsers\"\n\n   ````\n   node device.js\n   ````\n\n5. Open the csv-output file in a spreadsheet or with\n\n   ````\n   less -Six12 report/device.csv\n   ````\n\n6. Check the csv-table if the user-agents were parsed the way they should.\n   In the first column the debug number will be displayed. If this is\n   missing either no match was found (default should be \"Other\") or the\n   debug information is missing in the `regexes.yaml`.\n7. Change one or more \"regex\" expressions in the `regexes.yaml` file.\n   Parse the user-agents as in Step 3.\n8. Recheck list again. To get a different view by changing the sorting\n   order with family or brand model first use:\n\n   ````\n   node device.js -s\n   ````\n\n9. If everything is as expected then re-run parsing with involving the\n   testcases\n\n   ````\n   node device.js -t\n   ````\n\n10. This run writes the file `report/test_device.yaml` and maybe\n    `report/device.log`. In `device.log` all broken tests are reported.\n    The file `test_device.yaml` writes a new testcases file which contains\n    the results for the changed `regexes.yaml` file.\n    All testcases (even for \"broken\" ones) get updated.\n    You can check the differences with:\n\n    ````\n    diff report/test_device.yaml ua-parser/test_resources/test_device.yaml\n    ````\n    I recommend [diffuse](http://diffuse.sourceforge.net/index_de.html)\n    in case you should prefer a GUI-based difftool.\n\n11. If you are really sure that your changes do not corrupt the previous\n    testcases and contain the right changes or corrections, remove the\n    debuginfo from the `regexes.yaml` file with:\n\n    ````\n    node debuginfo.js\n    ````\n\n    Then you can copy the the generated `test_device.yaml` file to your\n    fork of the **ua-parser** project.\n\n12. Within your fork of the [ua-parser](https://github.com/tobie/ua-parser)\n    project run the mocha tests with:\n\n    ````\n    npm test\n    ````\n\n13. If these tests did run without any problems then commit your changes\n    and issue a pull-request.\n\n\n## Advanced Settings\n\n````\nOptions:\n  -u, --ua PATH          Read user-agent strings from file\n  -o, --out PATH         Write output files .cvs, .log\n  -t, --tc               Process testcases.\n      --tcin PATH        Read testcases from file\n      --tcout PATH       Write testcases to file\n  -c, --console          Output to console\n      --other            Add unmatched user-agents to testcases output file\n  -s, --swapdebug        Swap debug field in .csv output\n  -d, --nodebug          Do not show debug field in .cvs output\n  -f, --nofamily         Do not show family field in .csv output (device.js only)\n  -a, --noappend         Do not append user-agent strings from -u\n  -h, --help             Display help and usage details\n````\n\n* *-u PATH* : Instead of \"useragents.txt\" the file with \"PATH\" is used as input.\n* *-s*  : Change the column for showing the regex matcher number from column one. The sorting of the resulting cvs-table will be different. This option allows to check different matchers for same model, brand or family.\n* *-t*  : Generate testcases file. All user-agents encountered in the testcases file will be appended\n* *--tcin PATH* : Use \"PATH\" as testcases input file instead of the default one of the selected tool. Both YAML and JSON files can be used. Requires setting *testcases:true*. If \"#\" is used as input, no file is used, thus allowing generation of new testcase files.\n* *--tcout PATH* : Use \"PATH\" as testcases output file instead of the default one of the selected tool. Both YAML and JSON files can be used. Requires setting *testcases:true*.\n* *-o* : Add also unmatched user-agents to testcases. Requires setting *testcases:true*.\n* *-a* : Usually the User-Agents of the testcases input file get appended to check for broken tests. If this is not desired, then use this setting. The User-Agents will be missing in the resulting testcases output file then.\n\n### Conversions\n\nIn case that you just want to convert testcases from YAML to JSON you can do the following:\n\n````\nnode os.js -u no -t --tcout mytests.json\n````\n\nThis is in particuar usefull, if you are processing very large testcases with more than 500,000 User-Agents. Parsing JSON is pretty much faster here than YAML.\n\n### Add unmatched entries to the testcases file\n\n````\nnode device.js -u myuseragents.txt -t --tcout mytests.json -o\n````\n\n### Generate a new testcases file\n\nTo generate a complete new set of testcases\n\n````\nnode ua.js -u myuseragents.txt -t --tcin no --tcout mytests.json\n````\n\n### Run tests against you testcases file\n\n````\nnode ua.js -u myuseragents.txt -t --tcin mytests.json --tcout mytestsout.json\n````\n\n## Special Tools\n\nTo generate a uniq set of test-vectors from text files containing user-agent strings the tool `uniq.js` together with a previous `sort.js` can be used.\n\nCheck `sort.js -h` and `uniq.js -h` for usage.\n\nTypical use:\n\n    # sort the useragents - Mozilla ... gets first\n    ./sort.js -u useragents.txt \u003e u.tmp\n    # the debuginfo needs to be present for `uniq.js` to work\n    ./debuginfo.js\n    ./uniq.js -t device -u u.tmp\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommenthol%2Fua-parser-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcommenthol%2Fua-parser-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommenthol%2Fua-parser-tools/lists"}