{"id":26555253,"url":"https://github.com/ave-sergeev/tictonix","last_synced_at":"2025-06-11T11:33:13.249Z","repository":{"id":282715129,"uuid":"949404716","full_name":"Ave-Sergeev/Tictonix","owner":"Ave-Sergeev","description":"Crate for `Embedings` and `Positional Encoding` (Rust) 2025","archived":false,"fork":false,"pushed_at":"2025-03-16T13:22:07.000Z","size":13,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-16T14:26:29.880Z","etag":null,"topics":["embeddings","positional-encoding","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ave-Sergeev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-MIT","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-16T11:42:35.000Z","updated_at":"2025-03-16T13:22:11.000Z","dependencies_parsed_at":"2025-03-16T14:36:50.337Z","dependency_job_id":null,"html_url":"https://github.com/Ave-Sergeev/Tictonix","commit_stats":null,"previous_names":["ave-sergeev/tictonix"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ave-Sergeev%2FTictonix","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ave-Sergeev%2FTictonix/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ave-Sergeev%2FTictonix/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ave-Sergeev%2FTictonix/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ave-Sergeev","download_url":"https://codeload.github.com/Ave-Sergeev/Tictonix/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244940161,"owners_count":20535565,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embeddings","positional-encoding","rust"],"created_at":"2025-03-22T10:25:39.186Z","updated_at":"2025-06-11T11:33:13.231Z","avatar_url":"https://github.com/Ave-Sergeev.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/logo.png\" alt=\"Project Preview\" style=\"width: 55%; height: auto; border-radius: 15px;\"\u003e\n\u003c/p\u003e\n\u003ch1 align=\"center\"\u003e \n  Tictonix \n\u003c/h1\u003e\n\n[Russian version](https://github.com/Ave-Sergeev/Tictonix/blob/main/README.ru.md)\n\n### Description\n\nThis crate provides functionality for working with vector representations of words (embeddings) and positional encoding.\nIt is intended for use in NLP tasks, deep learning, and your custom projects.\n\nAlso, this project is the second step (step 1 [tokenizer](https://github.com/Ave-Sergeev/Tokenomicon)) towards\nown implementation of `LLM` on `Transformer` architecture.\n\n### Provided functionality:\n\n#### Structure of Embeddings\n\n- Creating a new embedding matrix by various methods such as: `Gaussian`, `Xavier`, `Uniform`.\n- Constructing the resulting embedding matrix for an array of tokens (indices), and obtaining a specific embedding for a token (index).\n- Updating (replacing) the embedding for a particular token (index).\n\n#### Structure of PositionalEncoding\n\n- Creating a new positional encoding matrix by various methods such as: `Sinusoidal PE`, `Relative PE`, `Rotary PE`.\n- Applying positional encodings to the embedding matrix.\n- Returning a part of the positional encoding matrix for a sequence, and a particular positional encoding by its position.\n\n#### Structure of MatrixIO\n\n- Saving to a file, and retrieving the embedding matrix from the file. Available formats are .safetensors and .npy.\n\n**UPD: Important clarifications:**\n- In this implementation, the embedding matrix has columns corresponding to tokens (each column is an embedding for one token).\n- All calculations are performed exclusively on CPU (without GPU).\n\n### Installing\n\nAdd to your `Cargo.toml`:\n```toml\n[dependencies]\ntictonix = \"1.0.1\"\n```\n\n### Logging\n\nThe project uses [crate log](https://github.com/rust-lang/log) which provides a logging facade.\n\nAt the moment logging is implemented only for IO operations (saving and loading files), with a success message.\n\nYou can use any compatible implementation (e.g. `env_logger`, `fern`, `simple_logger`, `tracing`, ...)\nTo do this, just initialize the selected logger in your application before starting.\n\nExample of initialization with `env_logger`:\n```Rust\nfn main() {\n    Builder::new()\n        .filter_level(LevelFilter::Info)\n        .filter_module(\"tictonix\", LevelFilter::Info)\n        .init();\n    // Your code\n}\n```\n\n### Usage\n\nSee [examples for usage](https://github.com/Ave-Sergeev/Tictonix/blob/main/example/src/main.rs).\n\n### Documentation\n\nSee [documentation](https://docs.rs/tictonix/1.0.1/tictonix/) for the project.\n\n### Glossary\n\n- Tokenization is the process of breaking text into separate elements called tokens.\n  Tokens can be words, characters, sub-words, or other units, depending on the chosen tokenization method.\n  This process is an important step in text preprocessing for Natural Language Processing (NLP) tasks.\n- LLMs (large language models) are large language models based on deep learning architectures (e.g.,\n  Transformer) that are trained on huge amounts of textual data. They are designed to perform a wide\n  range of tasks related to natural language processing, such as text generation, translation, question answering,\n  classification, and others. LLMs are capable of generalizing knowledge and performing tasks on which they have not\n  been explicitly trained (zero-shot or few-shot learning).\n- Transformer is a neural network architecture proposed in 2017 that uses the attention mechanism to process sequences\n  of data such as text.\n  The main advantage of Transformer is its ability to process long sequences and take context into account regardless of\n  the distance between elements of the sequence.\n  This architecture is the basis for most modern LLMs (such as GPT, BERT and others).\n- Embedding is a numerical (vector) representation of text data (tokens, words, phrases or sentences).\n- Positional Encoding is a technique used in the Transformer architecture to convey information about the order of\n  elements in a sequence. Since Transformer has no built-in order information (unlike recurrent networks),\n  positional encoding adds special signals to token embeddings that depend on their position in the sequence.\n  sequence. This allows the model to take into account the order of words or other elements in the input data.\n\n### P.S.\n\nDon't forget to leave a ⭐ if you found [this project](https://github.com/Ave-Sergeev/Tictonix) useful.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fave-sergeev%2Ftictonix","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fave-sergeev%2Ftictonix","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fave-sergeev%2Ftictonix/lists"}