{"id":15490112,"url":"https://github.com/tmilos/lexer","last_synced_at":"2025-04-15T11:52:42.439Z","repository":{"id":57070668,"uuid":"76837910","full_name":"tmilos/lexer","owner":"tmilos","description":"PHP lexical analyzer","archived":false,"fork":false,"pushed_at":"2018-11-17T00:00:42.000Z","size":11,"stargazers_count":21,"open_issues_count":1,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-19T09:04:25.615Z","etag":null,"topics":["lexer","php"],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tmilos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-19T07:17:27.000Z","updated_at":"2024-07-25T17:20:35.000Z","dependencies_parsed_at":"2022-08-24T14:54:24.169Z","dependency_job_id":null,"html_url":"https://github.com/tmilos/lexer","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmilos%2Flexer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmilos%2Flexer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmilos%2Flexer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tmilos%2Flexer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tmilos","download_url":"https://codeload.github.com/tmilos/lexer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249067755,"owners_count":21207395,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lexer","php"],"created_at":"2024-10-02T07:09:23.080Z","updated_at":"2025-04-15T11:52:42.410Z","avatar_url":"https://github.com/tmilos.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PHP lexical analyzer\n\nPHP implementation of Lexical Analyzer.\n\n[![Author](http://img.shields.io/badge/author-@tmilos-blue.svg?style=flat-square)](https://twitter.com/tmilos77)\n[![Build Status](https://travis-ci.org/tmilos/lexer.svg?branch=master)](https://travis-ci.org/tmilos/lexer)\n[![Coverage Status](https://coveralls.io/repos/github/tmilos/lexer/badge.svg?branch=master)](https://coveralls.io/github/tmilos/lexer?branch=master)\n[![License](https://img.shields.io/packagist/l/tmilos/lexer.svg)](https://packagist.org/packages/tmilos/lexer)\n[![SensioLabsInsight](https://insight.sensiolabs.com/projects/2027f29b-b950-45ca-a185-54a1e3379bf7/small.png)](https://insight.sensiolabs.com/projects/2027f29b-b950-45ca-a185-54a1e3379bf7)\n\n\u003e **Warning**\n\u003e This is not a GENERATOR like classical lex is. It does not produce any php code. It's a simple plain scanner\n\u003e of the given input string and tokenizer into given set of tokens by matching regular expressions.\n\u003e Thus, at runtime you can change the token definition and use one same code for any token set.\n\n# Token Definition\n\nTokens are defined with ``TokenDefinition`` class that holds token name and regular expression. Token name can be\nempty, and in that case, lexer will ignore/skip such tokens.\n\n# Lexer Configuration\n\nThe lexer configuration holds a list of all token definitions. With ``LexerArrayConfig`` it can be easily created from an array\nwhere keys are regular expressions and values are names of tokens.\n\n# Full scan\n\nLexer's static method ``scan($config, $input)`` can be used to scan given input string and return an array of tokens.\n\n# Lexer with state\n\nInstance of the ``Lexer`` class be used to walk trough scanned tokens with single look-ahead token.\n\nIt's similar in API to [doctrine/lexer](https://github.com/doctrine/lexer), just tokens are defined and scanned differently, w/out\nthe need for recognizing the token type/name from the tokenized value - rather the token type/name is given by the same ``TokenDefn``\nthat gave the regex to recognize the token.\n\n\n# Examples\n\n``` php\n\u003c?php\n$config = new LexerArrayConfig([\n            '\\\\s' =\u003e '',\n            '\\\\d+' =\u003e 'number',\n            '\\\\+' =\u003e 'plus',\n            '-' =\u003e 'minus',\n            '\\\\*' =\u003e 'mul',\n            '/' =\u003e 'div',\n        ]);\n\n// static scan method that returns an array of\n$tokens = Lexer::scan($config, '2 + 3');\narray_map(function ($t) { return $t-\u003egetName(); }, $tokens); // ['number', 'plus', 'number']\n\n// lexer instance\n$lexer = new Lexer($config);\n$lexer-\u003esetInput('2 + 3');\n$lexer-\u003emoveNext();\nwhile ($lexer-\u003egetLookahead()) {\n    print $lexer-\u003egetLookahead()-\u003egetName();\n    $lexer-\u003emoveNext();\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmilos%2Flexer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftmilos%2Flexer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftmilos%2Flexer/lists"}