{"id":24540668,"url":"https://github.com/mistralys/jtokenizer","last_synced_at":"2025-06-13T20:07:38.475Z","repository":{"id":57017604,"uuid":"346838631","full_name":"Mistralys/jtokenizer","owner":"Mistralys","description":"Javascript parser and tokenizer for PHP","archived":false,"fork":false,"pushed_at":"2021-04-23T06:56:18.000Z","size":34,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-17T05:51:12.309Z","etag":null,"topics":["javascript","library","parser-library","php","tokenizer"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mistralys.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-11T21:09:28.000Z","updated_at":"2024-01-01T06:52:44.000Z","dependencies_parsed_at":"2022-08-22T11:31:37.416Z","dependency_job_id":null,"html_url":"https://github.com/Mistralys/jtokenizer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/Mistralys/jtokenizer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mistralys%2Fjtokenizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mistralys%2Fjtokenizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mistralys%2Fjtokenizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mistralys%2Fjtokenizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mistralys","download_url":"https://codeload.github.com/Mistralys/jtokenizer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mistralys%2Fjtokenizer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259712410,"owners_count":22900038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["javascript","library","parser-library","php","tokenizer"],"created_at":"2025-01-22T18:14:38.716Z","updated_at":"2025-06-13T20:07:38.466Z","avatar_url":"https://github.com/Mistralys.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JTokenizer\n\n**Archived project**: Please use [Peast][] instead, as this library is no longer actively maintained.\n\n----\n\nModernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.\n\n  \u003e NOTE: This package is a good choice for very basic access to JavaScript tokens.\n    A more modern and complete solution is [Peast][].\n\n## Overview\n\nThe JavaScript tokenizer is designed to mimic the PHP tokenizer. Possible uses for the \ntokenizer include code highlighting and simple manipulation of JavaScript source code.\n\n## Quickstart\n\nTokenize a JavaScript file:\n\n```php\n$tokens = \\JTokenizer\\JTokenizer::getFileTokens('javascript.js');\n```\n\nTokenize a JavaScript string:\n\n```php\n$jsCode = '(JavaScript source code)';\n$tokens = \\JTokenizer\\JTokenizer::getTokens($jsCode);\n```\n\nBoth methods behave like the PHP `token_get_all` function, with the addition of \na column number as well as a line number. \n\n## Token information\n\nThe `getTokens()` method returns an indexed array with a list of token arrays. Each token \narray is an indexed array with the following information:\n\n1) Token identifier\n2) Matched literal\n3) Line number\n4) Column number\n\nThe token identifier can be either an Integer, or a string. To get the according token\ntype name, use the `getTokenName()` method.\n\n## Source\n\nDerived from Tim Whitlock's JTokenizer package, released back in 2009. As far as \nmy research online showed, no one else published a modernized version of it that\ncan run without deprecated messages on PHP7.4+.\n\nThe original sources are here:\n\nhttps://timwhitlock.info/blog/2009/11/jparser-and-jtokenizer-released/\n\n[Peast]: https://github.com/mck89/peast\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmistralys%2Fjtokenizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmistralys%2Fjtokenizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmistralys%2Fjtokenizer/lists"}