{"id":19785396,"url":"https://github.com/michaelfeil/hf-hub-ctranslate2","last_synced_at":"2025-04-22T16:05:15.307Z","repository":{"id":160079136,"uuid":"635021370","full_name":"michaelfeil/hf-hub-ctranslate2","owner":"michaelfeil","description":"Connecting Transformers on HuggingFace Hub with CTranslate2","archived":false,"fork":false,"pushed_at":"2024-08-27T02:31:37.000Z","size":3226,"stargazers_count":36,"open_issues_count":8,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-18T17:22:42.850Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://michaelfeil.github.io/hf-hub-ctranslate2/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/michaelfeil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-01T19:39:09.000Z","updated_at":"2024-11-25T05:58:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"84e67214-80d4-43db-8494-7e8fc5d41200","html_url":"https://github.com/michaelfeil/hf-hub-ctranslate2","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fhf-hub-ctranslate2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fhf-hub-ctranslate2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fhf-hub-ctranslate2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fhf-hub-ctranslate2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/michaelfeil","download_url":"https://codeload.github.com/michaelfeil/hf-hub-ctranslate2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250275471,"owners_count":21403675,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T06:14:33.410Z","updated_at":"2025-04-22T16:05:15.249Z","avatar_url":"https://github.com/michaelfeil.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"hf_hub_ctranslate2\n==============================\n\nConnecting Transformers on HuggingfaceHub with Ctranslate2 - a small utility for keeping tokenizer and model around Huggingface Hub.\n\n[![codecov](https://codecov.io/gh/michaelfeil/hf-hub-ctranslate2/branch/main/graph/badge.svg?token=U9VIEFEELS)](https://codecov.io/gh/michaelfeil/hf-hub-ctranslate2)![CI pytest](https://github.com/michaelfeil/hf-hub-ctranslate2/actions/workflows/test_release.yml/badge.svg)\n\n[Read the docs](https://michaelfeil.github.io/hf-hub-ctranslate2/)\n\n\u003c!-- PROJECT SHIELDS --\u003e\n[![Contributors][contributors-shield]][contributors-url]\n[![Forks][forks-shield]][forks-url]\n[![Stargazers][stars-shield]][stars-url]\n[![Issues][issues-shield]][issues-url]\n[![MIT License][license-shield]][license-url]\n[![LinkedIn][linkedin-shield]][linkedin-url]\n\n--------\n## Usage:\n\n### PYPI Install\n```bash\npip install hf-hub-ctranslate2\n```\n--------\n\n## Decoder-only Transformer:\n```python\n# download ctranslate.Generator repos from Huggingface Hub (GPT-J, ..)\nfrom hf_hub_ctranslate2 import TranslatorCT2fromHfHub, GeneratorCT2fromHfHub\n\nmodel_name_1=\"michaelfeil/ct2fast-pythia-160m\"\nmodel = GeneratorCT2fromHfHub(\n    # load in int8 on CPU\n    model_name_or_path=model_name_1, device=\"cpu\", compute_type=\"int8\"\n)\noutputs = model.generate(\n    text=[\"How do you call a fast Flan-ingo?\", \"User: How are you doing?\"]\n    # add arguments specifically to ctranslate2.Generator here\n)\n```\n## Encoder-Decoder:\n```python\nfrom hf_hub_ctranslate2 import TranslatorCT2fromHfHub\n# download ctranslate.Translator repos from Huggingface Hub (T5, ..)\nmodel_name_2 = \"michaelfeil/ct2fast-flan-alpaca-base\"\nmodel = TranslatorCT2fromHfHub(\n        # load in int8 on CUDA\n        model_name_or_path=model_name_2, device=\"cuda\", compute_type=\"int8_float16\"\n)\noutputs = model.generate(\n    text=[\"How do you call a fast Flan-ingo?\", \"Translate to german: How are you doing?\"],\n    # use arguments specifically to ctranslate2.Translator below:\n    min_decoding_length=8,\n    max_decoding_length=16,\n    max_input_length=512,\n    beam_size=3\n)\nprint(outputs)\n```\n## Encoder-Decoder for multilingual translations (m2m-100):\n```python\nfrom hf_hub_ctranslate2 import MultiLingualTranslatorCT2fromHfHub\nmodel = MultiLingualTranslatorCT2fromHfHub(\n    model_name_or_path=\"michaelfeil/ct2fast-m2m100_418M\", device=\"cpu\", compute_type=\"int8\",\n    tokenizer=AutoTokenizer.from_pretrained(f\"facebook/m2m100_418M\")\n)\n\noutputs = model.generate(\n    [\"How do you call a fast Flamingo?\", \"Wie geht es dir?\"],\n    src_lang=[\"en\", \"de\"],\n    tgt_lang=[\"de\", \"fr\"]\n)\n```\n## Encoder-only Sentence Transformers\nFeel free to try out a new repo, using CTranslate2 for vector-embeddings: \nhttps://github.com/michaelfeil/infinity\n\n```python\nfrom hf_hub_ctranslate2 import CT2SentenceTransformer\nmodel_name_pytorch = \"intfloat/e5-small\"\nmodel = CT2SentenceTransformer(\n    model_name_pytorch, compute_type=\"int8\", device=\"cuda\", \n)\nembeddings = model.encode(\n    [\"I like soccer\", \"I like tennis\", \"The eiffel tower is in Paris\"],\n    batch_size=32,\n    convert_to_numpy=True,\n    normalize_embeddings=True,\n)\nprint(embeddings.shape, embeddings)\nscores = (embeddings @ embeddings.T) * 100\n```\n\n## Encoder-only -\u003e no longer recommended\n```python\nfrom hf_hub_ctranslate2 import EncoderCT2fromHfHub\nmodel_name = \"michaelfeil/ct2fast-e5-small\"\nmodel = EncoderCT2fromHfHub(\n        # load in int8 on CUDA\n        model_name_or_path=model_name,\n        device=\"cuda\",\n        compute_type=\"int8_float16\",\n)\noutputs = model.generate(\n    text=[\"I like soccer\", \"I like tennis\", \"The eiffel tower is in Paris\"],\n    max_length=64,\n)\n```\n\n\n\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n\u003c!-- https://www.markdownguide.org/basic-syntax/#reference-style-links --\u003e\n[contributors-shield]: https://img.shields.io/github/contributors/michaelfeil/hf-hub-ctranslate2.svg?style=for-the-badge\n[contributors-url]: https://github.com/michaelfeil/hf-hub-ctranslate2/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/michaelfeil/hf-hub-ctranslate2.svg?style=for-the-badge\n[forks-url]: https://github.com/michaelfeil/hf-hub-ctranslate2/network/members\n[stars-shield]: https://img.shields.io/github/stars/michaelfeil/hf-hub-ctranslate2.svg?style=for-the-badge\n[stars-url]: https://github.com/michaelfeil/hf-hub-ctranslate2/stargazers\n[issues-shield]: https://img.shields.io/github/issues/michaelfeil/hf-hub-ctranslate2.svg?style=for-the-badge\n[issues-url]: https://github.com/michaelfeil/hf-hub-ctranslate2/issues\n[license-shield]: https://img.shields.io/github/license/michaelfeil/hf-hub-ctranslate2.svg?style=for-the-badge\n[license-url]: https://github.com/michaelfeil/hf-hub-ctranslate2/blob/master/LICENSE.txt\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge\u0026logo=linkedin\u0026colorB=555\n[linkedin-url]: https://linkedin.com/in/michael-feil\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelfeil%2Fhf-hub-ctranslate2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmichaelfeil%2Fhf-hub-ctranslate2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelfeil%2Fhf-hub-ctranslate2/lists"}