{"id":35725725,"url":"https://github.com/ferchd/tm2hsl","last_synced_at":"2026-01-24T15:36:24.126Z","repository":{"id":332411762,"uuid":"1128752552","full_name":"ferchd/tm2hsl","owner":"ferchd","description":"A compiler that transforms TextMate syntax grammars into optimized HSL bytecode for efficient language support in code editors","archived":false,"fork":false,"pushed_at":"2026-01-06T16:45:36.000Z","size":91,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-13T19:55:23.906Z","etag":null,"topics":["bytecode","compiler","go","hsl","syntax-highlighting","textmate"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ferchd.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-06T05:22:32.000Z","updated_at":"2026-01-06T16:38:21.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ferchd/tm2hsl","commit_stats":null,"previous_names":["ferchd/tm2hsl"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/ferchd/tm2hsl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferchd%2Ftm2hsl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferchd%2Ftm2hsl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferchd%2Ftm2hsl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferchd%2Ftm2hsl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ferchd","download_url":"https://codeload.github.com/ferchd/tm2hsl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ferchd%2Ftm2hsl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28730323,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T10:24:43.181Z","status":"ssl_error","status_checked_at":"2026-01-24T10:24:36.112Z","response_time":89,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bytecode","compiler","go","hsl","syntax-highlighting","textmate"],"created_at":"2026-01-06T08:18:41.720Z","updated_at":"2026-01-24T15:36:24.121Z","avatar_url":"https://github.com/ferchd.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tm2hsl\n\n[![CI](https://github.com/ferchd/tm2hsl/workflows/CI/badge.svg)](https://github.com/ferchd/tm2hsl/actions)\n[![Go Report Card](https://goreportcard.com/badge/github.com/ferchd/tm2hsl)](https://goreportcard.com/report/github.com/ferchd/tm2hsl)\n[![GoDoc](https://godoc.org/github.com/ferchd/tm2hsl?status.svg)](https://godoc.org/github.com/ferchd/tm2hsl)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n**tm2hsl** is a compiler that transforms TextMate syntax grammars into optimized HSL bytecode, revolutionizing language support in code editors.\n\n## Vision\n\nModern editors (VSCode, Sublime Text, etc.) interpret complex TextMate grammars at runtime, causing latency, high memory usage, and limited scalability. **tm2hsl** changes this paradigm by compiling grammars once into deterministic bytecode that editors execute directly.\n\n## Features\n\n- **Ahead-of-time compilation**: Transforms TextMate grammars into optimized HSL bytecode\n- **Deterministic execution**: Same input produces same bytecode\n- **Massive scalability**: Efficient support for hundreds of languages\n- **Layer separation**: Compiled languages vs. execution engine\n- **Binary format**: Memory-mappable, versioned, and compact\n- **Complete CLI**: Tools for development and testing\n\n## Installation\n\n### Pre-built Binaries\n\nDownload binaries from the [releases page](https://github.com/ferchd/tm2hsl/releases).\n\n### From Source\n\n```bash\n# Clone the repository\ngit clone https://github.com/ferchd/tm2hsl.git\ncd tm2hsl\n\n# Setup development environment\nmake dev-setup\n\n# Build the project\nmake build\n\n# Install globally\nmake install\n```\n\n### Requirements\n\n- Go 1.21 or higher\n- Git\n\n## Usage\n\n### Basic Compilation\n\n```bash\n# Compile a TextMate grammar\ntm2hsl compile --config language.toml --output output.hsl\n\n# Validate without generating bytecode\ntm2hsl compile --config language.toml --validate-only\n```\n\n### Configuration File\n\nCreate a `language.toml`:\n\n```toml\nname = \"MyLanguage\"\nscope = \"source.mylanguage\"\ngrammar = \"grammars/mylanguage.json\"\n\n[metadata]\nversion = \"1.0.0\"\ndescription = \"Support for MyLanguage\"\n```\n\n## Architecture\n\n```\ntm2hsl\n├── cmd/tm2hsl/          # Main CLI\n├── internal/             # Private code\n│   ├── cli/             # Command interface\n│   ├── compiler/        # Compilation logic\n│   ├── parser/          # TextMate parsing\n│   ├── ir/              # Intermediate representation\n│   ├── normalizer/      # Grammar normalization\n│   ├── optimizer/       # Optimizations\n│   ├── codegen/         # Bytecode generation\n│   ├── serializer/      # HSL serialization\n│   └── config/          # Configuration handling\n├── pkg/                 # Public packages\n│   ├── hsl/            # HSL bytecode format\n│   └── textmate/       # TextMate types\n└── docs/               # Documentation\n```\n\n### Compilation Flow\n\n1. **Parsing**: Load and validate TextMate grammar (JSON/plist)\n2. **Normalization**: Convert to deterministic state machine\n3. **IR**: Generate optimized intermediate representation\n4. **Optimization**: Apply structural transformations\n5. **Bytecode**: Generate HSL binary bytecode\n6. **Serialization**: Write final `.hsl` file\n\n## HSL Format\n\nHSL bytecode is a binary format designed for:\n\n- **Sequential execution**: Efficient disk reading\n- **Memory-mapping**: Zero-copy loading\n- **Versioning**: Forward compatibility\n- **Compression**: Optimized and deduplicated tables\n\n### Bytecode Structure\n\n```\nHSL Header (64 bytes)\n├── Magic: \"HSL1\"\n├── Version: uint16\n├── Checksum: uint32\n└── Offset table...\n\nString Table\nRegex Table\nState Table\nRule Table\nScope Table\n```\n\n\n\n## Contributing\n\nContributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guides.\n\n### Quick Development\n\n```bash\n# Setup environment\n./scripts/setup-dev.sh\n\n# Iterative development\nmake          # build + test + lint\nmake build    # build only\nmake test     # tests only\n```\n\n### Conventional Commits\n\nWe use [Conventional Commits](https://conventionalcommits.org/) for messages:\n\n```bash\nfeat: add support for recursive includes\nfix: fix regex lookbehind parsing\ndocs: update HSL specification\n```\n\n## Documentation\n\n- [HSL Specification](docs/HSL_SPEC.md) - Detailed bytecode format and execution model\n- [Migration Guide](docs/MIGRATION.md) - From TextMate to HSL\n- [Internal API](docs/API.md) - Developer reference\n- [Examples](examples/) - Sample language compilations\n\n## Project Status\n\n**Current version**: 0.x (active development)\n\n### Supported (v0)\n- `match` rules with basic regex\n- `begin`/`end` rules with content\n- `contentName` for internal scopes\n- `captures` with simple names\n- Includes: `$self`, `$base`\n- Line and block comments\n\n### Not Supported (future)\n- Repository with `#name` references\n- Captures in `begin`/`end`\n- `while` rules\n- Complex back-references\n- Advanced lookahead/lookbehind\n\n## License\n\nThis project is licensed under Apache License 2.0 - see [LICENSE](LICENSE) for details.\n\n## Acknowledgments\n\n- [TextMate](https://macromates.com/) for the grammar format\n- [VSCode](https://code.visualstudio.com/) for popularizing TextMate\n- Open source community for inspiration and tools\n\n## Contact\n\n- **Issues**: [GitHub Issues](https://github.com/ferchd/tm2hsl/issues)\n- **Discussions**: [GitHub Discussions](https://github.com/ferchd/tm2hsl/discussions)\n- **Email**: [fernando@example.com](mailto:fernando@example.com)\n\n---\n\n**tm2hsl**: Compiling languages, accelerating editors.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fferchd%2Ftm2hsl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fferchd%2Ftm2hsl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fferchd%2Ftm2hsl/lists"}