{"id":16277511,"url":"https://github.com/rbitr/ferrite","last_synced_at":"2026-01-22T16:35:06.710Z","repository":{"id":200152699,"uuid":"704659327","full_name":"rbitr/ferrite","owner":"rbitr","description":"Simple, lightweight transformers in Fortran","archived":false,"fork":false,"pushed_at":"2023-11-17T19:51:06.000Z","size":29,"stargazers_count":16,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-14T13:45:57.512Z","etag":null,"topics":["embedding-models","embeddings","sentence-embeddings","sentence-transformers","transformer-models","transformers"],"latest_commit_sha":null,"homepage":"","language":"Fortran","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rbitr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-13T18:51:15.000Z","updated_at":"2024-09-26T14:33:34.000Z","dependencies_parsed_at":"2023-11-17T20:36:56.524Z","dependency_job_id":null,"html_url":"https://github.com/rbitr/ferrite","commit_stats":null,"previous_names":["rbitr/ferrite"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbitr%2Fferrite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbitr%2Fferrite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbitr%2Fferrite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbitr%2Fferrite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rbitr","download_url":"https://codeload.github.com/rbitr/ferrite/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247889613,"owners_count":21013212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embedding-models","embeddings","sentence-embeddings","sentence-transformers","transformer-models","transformers"],"created_at":"2024-10-10T18:55:14.458Z","updated_at":"2026-01-22T16:35:01.686Z","avatar_url":"https://github.com/rbitr.png","language":"Fortran","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Ferrite - Simple, lightweight transformers in Fortran\n\nModern ML frameworks like HF transformers are easy to use but extremely abstract. There are times the abstraction makes sense, particularly model training. For inference using transformers, the \"real\" inference code is less complex then the abstraction, and it can be faster and more transparent to just write the code. That way you can plainly see what your model is doing under the hood and adapt it to your use case, rather than picking through layer after layer of aabstraction over what is effectively a for-loop with some matrix multiplications inside. \n\nTo that end, as a complement to the [llama.f90](rbitr/llama.f90) Fortran LLM, this project demonstrates a [Sentence Transformer](https://www.sbert.net/index.html) in \"pure\" Fortran with no dependencies (you still need python to convert pytorch models if you want to use them).\n\nI plan to evolve this to make sure it can work with general transformer models, and add performance optimization as required. That said, I don't want to add any abstraction so I only want to add generalizations that don't obscure what is going on. The code can easily be adapted for architectural variations.\n\n## Setup and running\n\n```bash\n# clone the repo\ngit clone https://github.com/rbitr/ferrite\ncd ferrite\n# download a model\nwget https://huggingface.co/SDFASDGA/llm/resolve/main/msmarco-distilbert-base-dot-prod-v3-f32.gguf\n# compile\nmake\n#run\n./tx -m msmarco-distilbert-base-dot-prod-v3-f32.gguf -v -p \"I alwas feel like somebody's watching me\" # (sic)\n\nGGUF Header Info\n Magic number:   1179993927\n Version:            3\n Tensor Count:                   101\n Key-Value Pairs:                    15\n general.architecture                                            \n distilbert                                                      \n general.name                                                    \n DistilBert                                                      \n distilbert.context_length                                       \n         512\n distilbert.embedding_length                                     \n         768\n distilbert.feed_forward_length                                  \n        3072\n distilbert.block_count                                          \n           6\n distilbert.attention.head_count                                 \n          12\n distilbert.attention.head_count_kv                              \n           1\n general.file_type                                               \n           0\n tokenizer.ggml.model                                            \n gpt2                                                            \n tokenizer.ggml.tokens                                           \n       30522\n tokenizer.ggml.token_type                                       \n       30522\n tokenizer.ggml.unknown_token_id                                 \n         100\n tokenizer.ggml.seperator_token_id                               \n         102\n tokenizer.ggml.padding_token_id                                 \n           0\n Position      573471\n Deficit          30\n data offset      573473\n Embedding dimension:          768\n Hidden dimension:         3072\n Layers:            6\n Heads:           12\n kv Heads:            1\n Vocabulary Size:        30522\n Sequence Length:          512\n head size           64\n kv head Size           64\n loaded word embedding weights:    23440896\n loaded position embedding weights:      393216\n loaded embedding layernorm weights:         768\n loaded embedding layernorm bias:         768\n loaded wq weights:     3538944\n loaded wq bias:        4608\n loaded wk weights:     3538944\n loaded wk bias:        4608\n loaded wv weights:     3538944\n loaded wv bias:        4608\n loaded wo weights:     3538944\n loaded wo bias:        4608\n loaded sa layernorm weights:        4608\n loaded sa layernorm bias:        4608\n loaded w1 weights:    14155776\n loaded w1 bias:       18432\n loaded w2 (down) weights:    14155776\n loaded w2 (down) bias:        4608\n loaded output norm weights:        4608\n loaded output norm bias:        4608\n loaded classifier weights:      589824\n loading tokens\nfound 30522 tokens\n maximum token length           18\nToken 4081 is andrew                                                          \n simple token: i                 \n wordpiece tokens: i                 \n simple token: alwas             \n wordpiece tokens: al                ##was             \n simple token: feel              \n wordpiece tokens: feel              \n simple token: like              \n wordpiece tokens: like              \n simple token: somebody          \n wordpiece tokens: somebody          \n simple token: '                 \n wordpiece tokens: '                 \n simple token: s                 \n wordpiece tokens: s                 \n simple token: watching          \n wordpiece tokens: watching          \n simple token: me                \n wordpiece tokens: me                \n         102        1046        2633       17312        2515        2067        8308        1006        1056        3667        2034         103\n  0.117702775      0.268108070     -0.412374288     -0.684159577     -0.272519588     -0.633238137 ...\n```\n\nRight now I've only tested it with the `msmarco-distilbert-base-dot-prod-v3` model from sbert.net. This is a DistilBbert transformer with a pooling and linear layer used for generating embeddings for semantic search. See https://www.sbert.net/docs/pretrained-models/msmarco-v3.html for more information. \n\nCommand line arguments are as follows:\n\n```bash\ncase ('-m', '--model')\n! path to model file\n--\ncase ('-p', '--prompt')\n! prompt string\n--\ncase ('-s', '--tokenizer')\n! path to custom tokenizer\n--\ncase ('-t', '--temperature')\n! temperature scaling (not used)\n--\ncase ('-n', '--num_tokens')\n! number of tokens to generate, including prompt (not used)\n--\ncase ('-v', '--verbose')\n! print additional information\n--\ncase ('-1', '--single_line')\n! print each element on single line\n--\ncase ('-q', '--quiet')\n! don't print embedding\n\n```\n\n## Getting models\n\nModels are in gguf format, see https://github.com/ggerganov/ggml/blob/master/docs/gguf.md\n\nYou can use the `convert-hf-to-gguf.py` file from https://github.com/rbitr/llama.cpp to convery HF model files, ie \n\n```bash\ngit clone https://github.com/rbitr/llama.cpp\n# get the model\ngit clone https://huggingface.co/sentence-transformers/msmarco-distilbert-base-dot-prod-v3\n# convert\npython ./llama.cpp/convert-hf-to-gguf.py msmarco-distilbert-base-dot-prod-v3 --outtype f32\n```\n\nNote that only distilbert models are supported and it has not been extensively tested. Support is currently limited to the fork referenced above, it's not part of the original repo.\n\n\n## Examples (currently using the old model file format, adjust accordingly)\n\nIncluded in the repo is a file `sentence_ex` made up of some sentences from wikipedia about Europe and the saxoaphone. We save temporary embeddings for each sentence with a bash one-liner:\n\n```bash\nx=1; while read s; do echo $x $s; ./tx -m msmarco-distilbert-base-dot-prod-v3_converted_full.bin -1 -p \"$s\" \u003e tmp/emb${x}.txt; x=$((x+1)); done \u003c sentence_ex.txt\n1 Europe is a continent located entirely in the Northern Hemisphere and mostly in the Eastern Hemisphere.\n2 It comprises the westernmost part of Eurasia and is bordered by the Arctic Ocean to the north, the Atlantic Ocean to the west, the Mediterranean Sea to the south, and Asia to the east.\n3 Europe is commonly considered to be separated from Asia by the watershed of the Ural Mountains, the Ural River, the Caspian Sea, the Greater Caucasus, the Black Sea, and the waterways of the Turkish Straits.\n4 Although some of this border is over land, Europe is generally accorded the status of a full continent because of its great physical size and the weight of history and tradition.\n5 Europe covers about 10,180,000 square kilometres (3,930,000 sq mi), or 2% of the Earth's surface (6.8% of land area), making it the second smallest continent.\n6 Politically, Europe is divided into about fifty sovereign states, of which Russia is the largest and most populous, spanning 39% of the continent and comprising 15% of its population.\n7 Europe had a total population of about 741 million (about 11% of the world population) as of 2018.\n8 The European climate is largely affected by warm Atlantic currents that temper winters and summers on much of the continent, even at latitudes along which the climate in Asia and North America is severe.\n9 Further from the sea, seasonal differences are more noticeable than close to the coast.\n10 European culture is the root of Western civilization, which traces its lineage back to ancient Greece and ancient Rome.\n11 The fall of the Western Roman Empire in 476 AD and the subsequent Migration Period marked the end of Europe's ancient history and the beginning of the Middle Ages.\n12 A saxophone is a type of musical instrument in the woodwind family.\n13 The saxophone uses a piece of wood, called a reed, to make sound.\n14 The player blows air into the mouthpiece, which vibrates the reed.\n15 The saxophone also uses keys to change pitch, and the player closes or opens holes to choose the note.\n16 Commonly, saxophones have about 22 keys.\n17 The saxophone is most commonly found in four voices: soprano, alto, tenor, and baritone saxophones.\n18 However, uncommon saxophones include the bass and contrabass saxophones (lower than a baritone saxophone), the C-melody saxophone (between the tenor and alto saxophones), and the sopranino saxophone (higher than a soprano saxophone).\n19 It was invented in 1840 by Adolphe Sax and is used in classical, jazz, and occasionally in rock, pop, and other styles.\n20 The saxophone was originally created for military bands, but was commonly used in jazz big bands in the 1940s and 1950s.\n21 Famous saxophone players include Marcel Mule (classical music), John Coltrane (jazz music), and Charlie Parker (jazz music).\n```\n\nThen we can lookup queries by making an embedding and finding the entry with the largest dot-product (computed here in awk)\n\n```bash\n./tx -m msmarco-distilbert-base-dot-prod-v3_converted_full.bin -1 -p \"What bodies of water are in europe?\" \u003e tmp/embq.txt\nfor x in {1..21}; do echo $x; paste tmp/emb${x}.txt tmp/embq.txt | awk '{dp+=$1*$2} END {print dp}'; done\n1\n35.505\n2\n33.9245\n3\n37.5551\n4\n29.1835\n5\n36.0957\n6\n31.6795\n7\n29.0034\n8\n31.7701\n9\n17.0193\n10\n26.859\n11\n20.4201\n12\n10.0551\n13\n9.95383\n14\n14.5428\n15\n8.84668\n16\n10.5251\n17\n10.2478\n18\n8.90325\n19\n12.1863\n20\n6.15891\n21\n7.62652\n```\n\nThe question was about Europe so the scores are higher on the first 11 entries, and the maximum is #3 which talks about waterways.\n\nBelow we ask who invented the saxophone and get the highest score at sentence 19 which contains the answer. (Note I misspelled saxophone in the query and it still works).\n\n```bash\n./tx -m msmarco-distilbert-base-dot-prod-v3_converted_full.bin -1 -p \"Who invented the saxaphone?\" \u003e tmp/embq.txt\nfor x in {1..21}; do echo $x; paste tmp/emb${x}.txt tmp/embq.txt | awk '{dp+=$1*$2} END {print dp}'; done\n1\n8.78761\n2\n11.0401\n3\n11.4972\n4\n5.93544\n5\n3.17357\n6\n6.38081\n7\n9.9643\n8\n16.3048\n9\n12.8389\n10\n22.387\n11\n22.8647\n12\n31.1579\n13\n32.949\n14\n24.6756\n15\n28.0059\n16\n24.3043\n17\n25.446\n18\n29.0274\n19\n42.3414\n20\n30.9246\n21\n33.6924\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbitr%2Fferrite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frbitr%2Fferrite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbitr%2Fferrite/lists"}