{"id":28356442,"url":"https://github.com/hexydec/tokenise","last_synced_at":"2026-03-09T08:31:41.408Z","repository":{"id":56984064,"uuid":"358743651","full_name":"hexydec/tokenise","owner":"hexydec","description":"A Regexp Based PHP Tokeniser","archived":false,"fork":false,"pushed_at":"2026-01-28T01:24:11.000Z","size":51,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-02-24T05:03:45.172Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hexydec.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-04-16T23:39:56.000Z","updated_at":"2025-07-15T16:08:40.000Z","dependencies_parsed_at":"2025-07-15T19:17:44.504Z","dependency_job_id":"3edaffe7-ed0f-4e86-a473-fc2bd0daad3b","html_url":"https://github.com/hexydec/tokenise","commit_stats":{"total_commits":27,"total_committers":2,"mean_commits":13.5,"dds":0.03703703703703709,"last_synced_commit":"bf54715cc968ad54719e1f527aec2fb492ce5210"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/hexydec/tokenise","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hexydec%2Ftokenise","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hexydec%2Ftokenise/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hexydec%2Ftokenise/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hexydec%2Ftokenise/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hexydec","download_url":"https://codeload.github.com/hexydec/tokenise/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hexydec%2Ftokenise/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30287821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T02:57:19.223Z","status":"ssl_error","status_checked_at":"2026-03-09T02:56:26.373Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-28T06:30:25.687Z","updated_at":"2026-03-09T08:31:41.398Z","avatar_url":"https://github.com/hexydec.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tokenise: A Regexp Based PHP Tokeniser\n\nA program for performing lexical analysis, written in PHP. Designed to supply tokens to a parser in order to analyse the syntax of programming languages.\n\n![Licence](https://img.shields.io/badge/Licence-MIT-lightgrey.svg)\n![Status: Stable](https://img.shields.io/badge/Status-Stable-Green.svg)\n[![Tests Status](https://github.com/hexydec/tokenise/actions/workflows/tests.yml/badge.svg)](https://github.com/hexydec/tokenise/actions/workflows/tests.yml)\n[![Code Coverage](https://codecov.io/gh/hexydec/tokenise/branch/master/graph/badge.svg)](https://app.codecov.io/gh/hexydec/tokenise)\n\n## Description\n\nUsing an array of regular expressions, tokens are extracted from the input code in sequence and upon request. Previous tokens can also be re-requested, and custom regular expressions can also be used to retrieve the next token.\n\nCurrently used in my other projects:\n\n- [HTMLdoc: PHP HTML Document Parser and Minifier](http://github.com/hexydec/htmldoc)\n- [CSSdoc: PHP CSS Document Parser and Minifier](http://github.com/hexydec/cssdoc)\n- [JSlite: PHP Javascript Minifier](http://github.com/hexydec/jslite)\n\n## Usage\n\nBasic usage within a program:\n\n```php\n$obj = new \\hexydec\\tokens\\tokenise($tokens, $value);\n\nwhile (($token = $obj-\u003enext()) !== null) {\n\t// parse each token\n}\n```\n\n## API\n\nFor more detailed usage documentation, [read the API document](docs/api.md).\n\n## Support\n\nTokenise supports PHP version 8.1+.\n\n## Contributing\n\nIf you find an issue with tokenise, please create an issue in the tracker.\n\nIf you wish to fix an issue yourself, please fork the code, fix the issue, then create a pull request, and I will evaluate your submission.\n\n## Licence\n\nThe MIT License (MIT). Please see [License File](LICENCE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhexydec%2Ftokenise","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhexydec%2Ftokenise","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhexydec%2Ftokenise/lists"}