{"id":42669045,"url":"https://github.com/mdm-code/scanner","last_synced_at":"2026-01-29T10:22:35.283Z","repository":{"id":196385973,"uuid":"695997118","full_name":"mdm-code/scanner","owner":"mdm-code","description":"Custom Go text token scanner implementation.","archived":false,"fork":false,"pushed_at":"2024-01-25T20:32:52.000Z","size":33,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-01-26T21:42:29.018Z","etag":null,"topics":["go","golang","scan","scanner","token","tokenizer"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/mdm-code/scanner","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mdm-code.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-09-24T20:48:00.000Z","updated_at":"2023-09-26T11:59:53.000Z","dependencies_parsed_at":"2023-11-24T21:28:01.218Z","dependency_job_id":"c3c3e0a8-03f1-4899-a8a4-77fee8f921e9","html_url":"https://github.com/mdm-code/scanner","commit_stats":null,"previous_names":["mdm-code/scanner"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/mdm-code/scanner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdm-code%2Fscanner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdm-code%2Fscanner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdm-code%2Fscanner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdm-code%2Fscanner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mdm-code","download_url":"https://codeload.github.com/mdm-code/scanner/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdm-code%2Fscanner/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28875450,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-29T09:47:23.353Z","status":"ssl_error","status_checked_at":"2026-01-29T09:47:19.357Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","scan","scanner","token","tokenizer"],"created_at":"2026-01-29T10:22:34.423Z","updated_at":"2026-01-29T10:22:35.266Z","avatar_url":"https://github.com/mdm-code.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cdiv\u003e\n    \u003cimg\n      src=\"https://raw.githubusercontent.com/mdm-code/mdm-code.github.io/main/scanner_logo.png\"\n      alt=\"logo\"\n      style=\"object-fit: contain\"\n      width=\"40%\"\n    /\u003e\n  \u003c/div\u003e\n\u003c/h1\u003e\n\n\u003ch4 align=\"center\"\u003eCustom Go text token scanner implementation\u003c/h4\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cp\u003e\n    \u003ca href=\"https://github.com/mdm-code/scanner/actions?query=workflow%3ACI\"\u003e\n        \u003cimg alt=\"Build status\" src=\"https://github.com/mdm-code/scanner/workflows/CI/badge.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://app.codecov.io/gh/mdm-code/scanner\"\u003e\n        \u003cimg alt=\"Code coverage\" src=\"https://codecov.io/gh/mdm-code/scanner/branch/main/graphs/badge.svg?branch=main\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://opensource.org/licenses/MIT\" rel=\"nofollow\"\u003e\n        \u003cimg alt=\"MIT license\" src=\"https://img.shields.io/github/license/mdm-code/scanner\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://goreportcard.com/report/github.com/mdm-code/scanner\"\u003e\n        \u003cimg alt=\"Go report card\" src=\"https://goreportcard.com/badge/github.com/mdm-code/scanner\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://pkg.go.dev/github.com/mdm-code/scanner\"\u003e\n        \u003cimg alt=\"Go package docs\" src=\"https://img.shields.io/badge/go.dev-reference-007d9c?logo=go\u0026logoColor=white\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\u003c/div\u003e\n\nPackage `scanner` is a custom text scanner implementation. It has the same\nidiomatic Go scanner programming interface, and it lets the client to freely\nnavigate the buffer. The scanner is also capable of peeking ahead of the\ncursor. Read runes are rendered as tokens with additional information on their\nposition in the buffer. Consult the [package documentation](https://pkg.go.dev/github.com/mdm-code/scanner) or see\n[Usage](#usage) to see how to use it.\n\n\n## Installation\n\nUse the following command to add the package to an existing project.\n\n```sh\ngo get github.com/mdm-code/scanner\n```\n\n\n## Usage\n\nHere is a snippet showing the basic usage of the scanner to read text as a stream\nof tokens using the public API of the `scanner` package.\n\n```go\npackage main\n\nimport (\n    \"bufio\"\n    \"fmt\"\n    \"log\"\n    \"os\"\n\n    \"github.com/mdm-code/scanner\"\n)\n\nfunc main() {\n    r := bufio.NewReader(os.Stdin)\n    s, err := scanner.New(r)\n    if err != nil {\n        log.Fatalln(err)\n    }\n    var ts []scanner.Token\n    for s.Scan() {\n        t := s.Token()\n        ts = append(ts, t)\n    }\n    fmt.Println(ts)\n}\n```\n\n\n## Development\n\nConsult [Makefile](Makefile) to see how to format, examine code with `go vet`,\nrun unit test, run code linter with `golint` in order to get test coverage and\ncheck if the package builds all right.\n\nRemember to install `golint` before you try to run tests and test the build:\n\n```sh\ngo install golang.org/x/lint/golint@latest\n```\n\n\n## License\n\nCopyright (c) 2023 Michał Adamczyk.\n\nThis project is licensed under the [MIT license](https://opensource.org/licenses/MIT).\nSee [LICENSE](LICENSE) for more details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdm-code%2Fscanner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmdm-code%2Fscanner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdm-code%2Fscanner/lists"}