{"id":31274135,"url":"https://github.com/thanos/fastset-nif","last_synced_at":"2026-05-14T20:32:09.020Z","repository":{"id":313825335,"uuid":"1048055549","full_name":"thanos/fastset-nif","owner":"thanos","description":"The is a NIF wrapping Rust's HashSet -  POC: NOT FOR PRODUCTION","archived":false,"fork":false,"pushed_at":"2025-09-01T13:44:42.000Z","size":14,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-09T02:35:07.903Z","etag":null,"topics":["data-structures-and-algorithms","elixir","nif","rust","rust-lang"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thanos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-31T20:18:08.000Z","updated_at":"2025-09-01T13:46:55.000Z","dependencies_parsed_at":"2025-09-09T02:06:28.605Z","dependency_job_id":"790024fa-1319-41da-843e-9d8bc63ace23","html_url":"https://github.com/thanos/fastset-nif","commit_stats":null,"previous_names":["thanos/fastset-nif"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thanos/fastset-nif","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanos%2Ffastset-nif","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanos%2Ffastset-nif/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanos%2Ffastset-nif/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanos%2Ffastset-nif/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thanos","download_url":"https://codeload.github.com/thanos/fastset-nif/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanos%2Ffastset-nif/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33042133,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-13T13:14:54.681Z","status":"online","status_checked_at":"2026-05-14T02:00:06.663Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-structures-and-algorithms","elixir","nif","rust","rust-lang"],"created_at":"2025-09-23T22:37:27.672Z","updated_at":"2026-05-14T20:32:09.015Z","avatar_url":"https://github.com/thanos.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FastsetNif\n\nFastsetNif is an Elixir module that provides fast set operations through Rust NIFs using Rustler. This module leverages Rust's standard library HashSet for high-performance set operations, providing a familiar API similar to Elixir's MapSet but with native performance.\n\n## Features\n\n- **High Performance**: Uses Rust's optimized HashSet implementation\n- **Memory Safe**: Leverages Rust's memory safety guarantees\n- **Rich API**: Comprehensive set operations including union, intersection, and difference\n- **Type Safe**: Proper struct-based API with Rustler\n- **Cross-platform**: Works on all platforms supported by Rust\n\n## Installation\n\nAdd `fastset_nif` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:fastset_nif, \"~\u003e 0.1.0\"}\n  ]\nend\n```\n\n## Prerequisites\n\n- Rust toolchain (install via [rustup](https://rustup.rs/))\n- Elixir 1.14 or later\n\n## Usage\n\n```elixir\n# Create a new set\nset = FastsetNif.new()\n\n# Add elements\nset = FastsetNif.add(set, \"hello\")\nset = FastsetNif.add(set, \"world\")\n\n# Check membership\nFastsetNif.member?(set, \"hello\")  # Returns true\nFastsetNif.member?(set, \"missing\")  # Returns false\n\n# Get set size\nFastsetNif.size(set)  # Returns 2\n\n# Convert to list\nFastsetNif.to_list(set)  # Returns [\"hello\", \"world\"]\n\n# Set operations\nset2 = FastsetNif.new()\nset2 = FastsetNif.add(set2, \"world\")\nset2 = FastsetNif.add(set2, \"elixir\")\n\n# Union\nunion_set = FastsetNif.union(set, set2)\n\n# Intersection\nintersection_set = FastsetNif.intersection(set, set2)\n\n# Difference\ndifference_set = FastsetNif.difference(set, set2)\n```\n\n## API Reference\n\n### Core Operations\n\n- `new()` - Creates a new empty set\n- `add(set, element)` - Adds an element to the set\n- `remove(set, element)` - Removes an element from the set\n- `member?(set, element)` - Checks if an element exists in the set\n- `size(set)` - Gets the number of elements in the set\n- `to_list(set)` - Converts the set to a list\n\n### Set Operations\n\n- `union(set1, set2)` - Returns a new set with all elements from both sets\n- `intersection(set1, set2)` - Returns a new set with elements present in both sets\n- `difference(set1, set2)` - Returns a new set with elements from set1 not in set2\n\n## Performance Benchmarks\n\n### Intersection Operation Performance\n\nRecent benchmark results comparing FastSetNif with Elixir's built-in MapSet for intersection operations:\n\n#### System Information\n- **Operating System**: macOS\n- **CPU**: Apple M1 Max (10 cores)\n- **Memory**: 64 GB\n- **Elixir Version**: 1.18.4\n- **Erlang Version**: 28.0.1\n- **JIT**: Enabled\n\n#### Benchmark Configuration\n- **Warmup**: 1 second\n- **Execution Time**: 5 seconds\n- **Memory Measurement Time**: 2 seconds\n- **Reduction Time**: 2 seconds\n- **Parallel**: 1 process\n\n#### Performance Results\n\n| Metric | FastSetNif | MapSet | Comparison |\n|--------|------------|---------|------------|\n| **Speed** | 0.25 K ips | 1.19 K ips | **4.75x slower** |\n| **Average Time** | 3.99 ms | 0.84 ms | +3.15 ms |\n| **Median Time** | 3.86 ms | 0.83 ms | +3.03 ms |\n| **99th Percentile** | 4.70 ms | 0.93 ms | +3.77 ms |\n\n#### Memory Usage\n\n| Metric | FastSetNif | MapSet | Comparison |\n|--------|------------|---------|------------|\n| **Memory Usage** | 167.91 KB | 476.88 KB | **0.35x memory usage** (-308.98 KB) |\n\n#### Reduction Count\n\n| Metric | FastSetNif | MapSet | Comparison |\n|--------|------------|---------|------------|\n| **Average** | 0.87 K | 74.82 K | **0.01x reduction count** |\n| **Median** | 0.80 K | 74.81 K | -74.01 K |\n| **99th Percentile** | 3.07 K | 75.07 K | -72.00 K |\n\n#### Summary\n\n- **Performance**: FastSetNif is currently **4.75x slower** than MapSet for intersection operations\n- **Memory**: FastSetNif uses **significantly less memory** (0.35x) compared to MapSet\n- **Reduction Count**: FastSetNif shows much lower reduction counts, indicating more efficient internal processing\n\n\u003e **Note**: These benchmarks represent the current implementation state. Performance optimizations are ongoing, and the memory efficiency advantage may be beneficial for memory-constrained environments.\n\n## Building from Source\n\n### Prerequisites\n\n- Rust toolchain (install via [rustup](https://rustup.rs/))\n- Elixir 1.14 or later\n- Erlang/OTP 24 or later\n\n### Build Steps\n\n1. Clone the repository:\n   ```bash\n   git clone \u003crepository-url\u003e\n   cd fastset-nif\n   ```\n\n2. Install dependencies:\n   ```bash\n   mix deps.get\n   ```\n\n3. Build the project (this will compile the Rust code):\n   ```bash\n   mix compile\n   ```\n\n4. Run tests:\n   ```bash\n   mix test\n   ```\n\n## Development\n\n### Project Structure\n\n```\nfastset-nif/\n├── lib/\n│   └── fastset_nif.ex      # Main Elixir module\n├── native/\n│   └── fastset_nif/        # Rust NIF implementation\n│       ├── Cargo.toml      # Rust dependencies\n│       └── src/\n│           └── lib.rs      # Rust NIF code\n├── mix.exs                 # Mix project configuration\n└── README.md               # This file\n```\n\n### Adding New Operations\n\nTo add new set operations:\n\n1. Add the function declaration to `lib/fastset_nif.ex`\n2. Implement the Rust function in `native/fastset_nif/src/lib.rs`\n3. Add the function to the `rustler::init!` macro\n4. Rebuild the project with `mix compile`\n\n### Testing\n\nThe project includes basic tests to verify functionality:\n\n```bash\nmix test\n```\n\n## Performance Considerations\n\n- NIFs run in the same thread as the calling Erlang process\n- Long-running NIFs can block the Erlang scheduler\n- Use NIFs for CPU-intensive operations, not I/O operations\n- Consider using dirty NIFs for long-running operations\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests for new functionality\n5. Ensure all tests pass\n6. Submit a pull request\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## Acknowledgments\n\n- Built with [Rustler](https://github.com/rusterlium/rustler)\n- Uses Rust's standard library HashSet for optimal performance\n- Leverages Rust's memory safety and performance guarantees\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthanos%2Ffastset-nif","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthanos%2Ffastset-nif","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthanos%2Ffastset-nif/lists"}