https://github.com/jamesgober/source-lang

Source file and buffer management with multi-file source maps.
https://github.com/jamesgober/source-lang

compiler language parser rust source-map

Last synced: about 4 hours ago
JSON representation

Source file and buffer management with multi-file source maps.

• Host: GitHub
• URL: https://github.com/jamesgober/source-lang
• Owner: jamesgober
• License: apache-2.0
• Created: 2026-06-19T03:36:53.000Z (19 days ago)
• Default Branch: main
• Last Pushed: 2026-06-20T18:05:54.000Z (18 days ago)
• Last Synced: 2026-06-20T20:03:30.353Z (18 days ago)
• Topics: compiler, language, parser, rust, source-map
• Language: Rust
• Size: 57.6 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE-APACHE

Awesome Lists containing this project

README

          


    

    


    source-lang

    


    SOURCE FILES & MAPS





    

    

    

    

    








    


        source-lang manages the text a compiler front-end reads: it loads files and buffers, assigns each a stable id, and maps a global position back to the specific file and local offset it came from. It is the multi-file coordinate layer that sits above raw spans, so a diagnostic can say not just where in a buffer an error is, but which file.

    

    


    


    


        MSRV is 1.85+ (Rust 2024 edition).

    

    


        Status: stable (1.0). The public API is frozen and follows Semantic Versioning — no breaking changes before 2.0. See the SemVer promise and CHANGELOG.md.

    








## Installation

```toml

[dependencies]

source-lang = "1.0"

```

Or from the terminal:

```bash

cargo add source-lang

```




## Usage

Add sources to a map and resolve a global position back to the file and local

offset it came from.

```rust

use source_lang::{BytePos, SourceMap};

let mut map = SourceMap::new();

let main = map.add("main.rs", "fn main() {}")?;   // global 0..12

let util = map.add("util.rs", "fn helper() {}")?; // global 12..26

// Which file does global position 13 belong to, and where inside it?

let (id, local) = map.locate(BytePos::new(13)).expect("inside util.rs");

assert_eq!(id, util);

assert_eq!(local, BytePos::new(1)); // 13 - 12

// The id is a stable handle back to the source for the life of the map.

assert_eq!(map.source(main).unwrap().name(), "main.rs");

# Ok::<(), source_lang::SourceMapError>(())

```

Read the located text back out of the resolved source:

```rust

use source_lang::{BytePos, SourceMap};

let mut map = SourceMap::new();

map.add("a", "let x = 1;")?;

let two = map.add("b", "let y = 2;")?;

let (id, local) = map.locate(BytePos::new(14)).expect("in range");

assert_eq!(id, two);

let file = map.source(id).unwrap();

assert_eq!(&file.text()[local.to_usize()..], "y = 2;");

# Ok::<(), source_lang::SourceMapError>(())

```

Resolve a global position to its file and 1-based line/column in one step — what a

diagnostic renderer needs to print `file:line:col`:

```rust

use source_lang::{BytePos, LineCol, SourceMap};

let mut map = SourceMap::new();

map.add("a.rs", "fn a() {}")?;                  // global 0..9

let b = map.add("b.rs", "let x = 1;\nlet y = 2;")?; // global 9..30

let (id, lc) = map.line_col(BytePos::new(20)).expect("in range");

assert_eq!(id, b);

assert_eq!(lc, LineCol::new(2, 1)); // second line of b.rs

# Ok::<(), source_lang::SourceMapError>(())

```

Load untrusted input — a file from disk or raw bytes from a buffer — through the

same checks, so bad input is a defined error rather than a panic:

```rust

use source_lang::{SourceMap, SourceMapError};

let mut map = SourceMap::new();

map.set_max_source_len(1 << 20); // cap any single source at 1 MiB

// Raw bytes are validated as UTF-8 before they are stored.

let id = map.add_bytes("config.toml", b"name = \"demo\"")?;

assert_eq!(map.source(id).unwrap().text(), "name = \"demo\"");

// Non-UTF-8 input is rejected, naming the source.

let err = map.add_bytes("blob.bin", &[0xff, 0xfe]).unwrap_err();

assert!(matches!(err, SourceMapError::NotUtf8 { .. }));

# Ok::<(), source_lang::SourceMapError>(())

```

With the default `std` feature, `map.add_file("src/main.rs")` reads a path from

disk through those same checks, rejecting an oversize file from its metadata before

a byte is read.

Walk every loaded source in order — id order is also global-offset order:

```rust

use source_lang::SourceMap;

let mut map = SourceMap::new();

map.add("a.txt", "one")?;

map.add("b.txt", "two")?;

let names: Vec<_> = map.iter().map(|(_, f)| f.name()).collect();

assert_eq!(names, ["a.txt", "b.txt"]);

# Ok::<(), source_lang::SourceMapError>(())

```

With the `serde` feature, a whole `SourceMap` round-trips through any serde format —

its spans and ids are regenerated on load, so the layout is validated rather than

trusted:

```rust,ignore

let mut map = SourceMap::new();

map.add("main.rs", "fn main() {}")?;

let json = serde_json::to_string(&map)?;

let restored: SourceMap = serde_json::from_str(&json)?;

assert_eq!(map, restored);

```

See docs/API.md for the full reference.




## How it works

Sources are placed end to end in the order they are added: the first occupies

global offsets `0..len₀`, the next `len₀..len₀ + len₁`, and so on. The ranges never

overlap, and because each base is the running total of all earlier sources, the

internal list stays sorted by offset — so locate is a binary search,

O(log files), that borrows the resolved source rather than copying it.

The shared space is 32 bits wide (the same envelope a single BytePos

addresses), so the combined length of every source is capped at 4 GiB;

overrunning it is a defined error, never a silent wrap into a neighbour's range.




## Status

v1.0.0 is the stable release: the public API is frozen and follows

Semantic Versioning, with no breaking changes before 2.0. The surface is

the full source map — the SourceMap, stable SourceIds,

and the non-overlapping global position space with its O(log files)

resolver, disk and buffer loading, line_col resolution, and optional

serde — each invariant property-tested against a naive linear

scan, verified on Linux, macOS, and Windows. See the

SemVer promise.






## Contributing

See dev/DIRECTIVES.md for engineering standards and the definition of done. Before a PR: `cargo fmt --all`, `cargo clippy --all-targets --all-features -- -D warnings`, and `cargo test --all-features` must be clean.






    
License

    
Licensed under either of

    


        


Apache License, Version 2.0 — LICENSE-APACHE



        


MIT License — LICENSE-MIT



    

    
at your option.





  


  COPYRIGHT © 2026 James Gober .