Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kohulan/decimer-image_transformer
DECIMER Image Transformer is a deep-learning-based tool designed for automated recognition of chemical structure images. Leveraging transformer architectures, the model converts chemical images into SMILES strings, enabling the digitization of chemical data from scanned documents, literature, and patents.
https://github.com/kohulan/decimer-image_transformer
chemical-image-recognition decimer deep-learning image-data-mining python tensorflow tpu transformers
Last synced: 4 days ago
JSON representation
DECIMER Image Transformer is a deep-learning-based tool designed for automated recognition of chemical structure images. Leveraging transformer architectures, the model converts chemical images into SMILES strings, enabling the digitization of chemical data from scanned documents, literature, and patents.
- Host: GitHub
- URL: https://github.com/kohulan/decimer-image_transformer
- Owner: Kohulan
- License: mit
- Created: 2020-09-07T16:00:46.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-11-04T08:30:04.000Z (2 months ago)
- Last Synced: 2025-01-11T11:04:21.820Z (4 days ago)
- Topics: chemical-image-recognition, decimer, deep-learning, image-data-mining, python, tensorflow, tpu, transformers
- Language: Python
- Homepage:
- Size: 24.5 MB
- Stars: 227
- Watchers: 8
- Forks: 56
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
Awesome Lists containing this project
README
# 🧪 DECIMER Image Transformer 🖼️
### Deep Learning for Chemical Image Recognition using Efficient-Net V2 + Transformer
[](https://opensource.org/licenses/MIT)
[](https://GitHub.com/Kohulan/DECIMER-Image_Transformer/graphs/commit-activity)
[](https://GitHub.com/Kohulan/DECIMER-Image_Transformer/issues/)
[](https://GitHub.com/Kohulan/DECIMER-Image_Transformer/graphs/contributors/)
[](https://www.tensorflow.org)
[](https://zenodo.org/badge/latestdoi/293572361)
[](https://decimer-image-transformer.readthedocs.io/en/latest/?badge=latest)
[](https://GitHub.com/Kohulan/DECIMER-Image_Transformer/releases/)
[](https://pypi.python.org/pypi/decimer/)
---
## 📚 Table of Contents
- [Abstract](#-abstract)
- [Method and Model Changes](#-method-and-model-changes)
- [Installation](#-installation)
- [Usage](#-usage)
- [Hand-drawn Model](#-decimer---hand-drawn-model)
- [Citation](#-citation)
- [Acknowledgements](#-acknowledgements)
- [Author](#-author-kohulan)
- [Project Website](#-project-website)
- [Research Group](#-research-group)
---
## 🔬 Abstract
The DECIMER 2.2 project tackles the OCSR (Optical Chemical Structure Recognition) challenge using cutting-edge computational intelligence methods. Our goal? To provide an automated, open-source software solution for chemical image recognition.
We've supercharged DECIMER with Google's TPU (Tensor Processing Unit) to handle datasets of over 1 million images with lightning speed!
---
## 🧠 Method and Model Changes
🖼️ Image Feature Extraction
Now utilizing EfficientNet-V2 for superior image analysis
🔮 SMILES Prediction
Employing a state-of-the-art transformer model
### 🚀 Training Enhancements
1. **TFRecord Files**: Lightning-fast data reading
2. **Google Cloud Buckets**: Efficient cloud storage solution
3. **TensorFlow Data Pipeline**: Optimized data loading
4. **TPU Strategy**: Harnessing the power of Google's TPUs
---
## 💻 Installation
```bash
# Create a conda wonderland
conda create --name DECIMER python=3.10.0 -y
conda activate DECIMER
# Equip yourself with DECIMER
pip install decimer
```
---
## 🎮 Usage
```python
from DECIMER import predict_SMILES
# Unleash the power of DECIMER
image_path = "path/to/your/chemical/masterpiece.jpg"
SMILES = predict_SMILES(image_path)
print(f"🎉 Decoded SMILES: {SMILES}")
```
---
## ✍️ DECIMER - Hand-drawn Model
🌟 **New Feature Alert!** 🌟
Our latest model brings the magic of AI to hand-drawn chemical structures!
[](https://doi.org/10.5281/zenodo.10781330)
---
## 📜 Citation
If DECIMER helps your research, please cite:
1. Rajan K, et al. "DECIMER.ai - An open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications." *Nat. Commun.* 14, 5045 (2023).
2. Rajan, K., et al. "DECIMER 1.0: deep learning for chemical image recognition using transformers." *J Cheminform* 13, 61 (2021).
3. Rajan, K., et al. "Advancements in hand-drawn chemical structure recognition through an enhanced DECIMER architecture," *J Cheminform* 16, 78 (2024).
---
## 🙏 Acknowledgements
- A big thank you to [Charles Tapley Hoyt](https://github.com/cthoyt) for his invaluable contributions!
- Powered by Google's TPU Research Cloud (TRC)
---
## 👨🔬 Author: [Kohulan](https://kohulanr.com)
---
## 🌐 Project Website
Experience DECIMER in action at [decimer.ai](https://decimer.ai), brilliantly implemented by [Otto Brinkhaus](https://github.com/OBrink)!
---
## 🏫 Research Group
---
### 📊 Project Analytics
