Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/huggingface/swift-coreml-transformers
Swift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT for Question answering. Other Transformers coming soon!
https://github.com/huggingface/swift-coreml-transformers
Last synced: 20 days ago
JSON representation
Swift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT for Question answering. Other Transformers coming soon!
- Host: GitHub
- URL: https://github.com/huggingface/swift-coreml-transformers
- Owner: huggingface
- License: apache-2.0
- Created: 2019-06-27T13:50:53.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-11-24T19:48:07.000Z (12 months ago)
- Last Synced: 2024-10-01T21:21:36.732Z (about 1 month ago)
- Language: Swift
- Homepage:
- Size: 9.21 MB
- Stars: 1,606
- Watchers: 105
- Forks: 174
- Open Issues: 23
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-ChatGPT-repositories - swift-coreml-transformers - Swift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT for Question answering. Other Transformers coming soon! (Reimplementations)
- fucking-open-source-ios-apps - Question answering and text generation - 2, DistilGPT-2, BERT, and DistilBERT (Misc / Notes)
README
# This repo is not actively maintained and has been archived. For an in-development replacement, please head over to [swift-transformers](https://github.com/huggingface/swift-transformers)!
# Swift Core ML implementations of Transformers: GPT-2, DistilGPT-2, BERT, DistilBERT, more coming soon!
This repository contains:
- For **BERT** and **DistilBERT**:
- pretrained [Google BERT](https://github.com/google-research/bert) and [Hugging Face DistilBERT](https://arxiv.org/abs/1910.01108) models fine-tuned for Question answering on the SQuAD dataset.
- Swift implementations of the [BERT tokenizer](https://github.com/huggingface/swift-coreml-transformers/blob/master/Sources/BertTokenizer.swift) (`BasicTokenizer` and `WordpieceTokenizer`) and SQuAD dataset parsing utilities.
- A neat demo question answering app.
- For **GPT-2** and **DistilGPT-2**:
- a [conversion script](https://github.com/huggingface/swift-coreml-transformers/blob/master/model_generation/gpt2.py) from PyTorch trained GPT-2 models (see our [`transformers`](https://github.com/huggingface/transformers) repo) to CoreML models.
- The [GPT-2 generation model](https://github.com/huggingface/swift-coreml-transformers/blob/master/Sources/GPT2.swift) itself, including decoding strategies (greedy and TopK are currently implemented) and GPT-2 Byte-pair encoder and decoder.
- A neat demo app showcasing on-device text generation.
# 🦄 GPT-2 and DistilGPT-2
Unleash the full power of text generation with GPT-2 on device!!
# 🐸 BERT and DistilBERT
The `BERTSQUADFP16` Core ML model was packaged by Apple and is linked from the [main ML models page](https://developer.apple.com/machine-learning/models/#text). It was demoed at WWDC 2019 as part of the Core ML 3 launch.
The `DistilBERT` Core ML models were converted from [`🤗/transformers`](https://github.com/huggingface/transformers) exports using the scripts in this repo.
## 🦄 Demo Time 🔥
Apple demo at WWDC 2019
full video [here](https://developer.apple.com/videos/play/wwdc2019/704)
## BERT Architecture (wwdc slide)
## Notes
We use `git-lfs` to store large model files and it is required to obtain some of the files the app needs to run.
See how to install `git-lfs`on the [installation page](https://git-lfs.github.com/)