{"id":15700439,"url":"https://github.com/dexterp37/fontina","last_synced_at":"2025-05-12T13:14:22.221Z","repository":{"id":197433587,"uuid":"675567313","full_name":"Dexterp37/fontina","owner":"Dexterp37","description":"Data generation, model training and inference for Visual Font Recognition using PyTorch","archived":false,"fork":false,"pushed_at":"2023-12-05T09:26:03.000Z","size":58,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-20T03:03:21.251Z","etag":null,"topics":["deep-learning","machine-learning","pytorch","pytorch-lightning","visual-font-recognition"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Dexterp37.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-07T08:12:55.000Z","updated_at":"2025-04-19T16:25:48.000Z","dependencies_parsed_at":"2023-12-05T10:45:26.492Z","dependency_job_id":null,"html_url":"https://github.com/Dexterp37/fontina","commit_stats":null,"previous_names":["dexterp37/fontina"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dexterp37%2Ffontina","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dexterp37%2Ffontina/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dexterp37%2Ffontina/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dexterp37%2Ffontina/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Dexterp37","download_url":"https://codeload.github.com/Dexterp37/fontina/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253745196,"owners_count":21957319,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","machine-learning","pytorch","pytorch-lightning","visual-font-recognition"],"created_at":"2024-10-03T19:48:43.599Z","updated_at":"2025-05-12T13:14:22.177Z","avatar_url":"https://github.com/Dexterp37.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fontina\n`fontina` is a PyTorch library that helps with training models for the task of Visual Font\nRecognition and doing inference with them.\n\n## Feature highlights\n\n* DeepFont-like network architecture. See [​​Z. Wang, J. Yang, H. Jin, E. Shechtman, A. Agarwala, J. Brandt and T. Huang, “DeepFont: Identify Your Font from An Image”, In Proceedings of ACM International Conference on Multimedia (ACM MM) , 2015](https://arxiv.org/abs/1507.03196)\n* Configuration-based synthetic dataset generation\n* Configuration-based model training via [PyTorch Lightning](https://lightning.ai/pytorch-lightning)\n* Supports training and inference on Linux, MacOS and Windows.\n\n## Using `fontina`\n\n### Installing the dependencies\nStarting from a cloned repository directory:\n\n```bash\n# Create a virtual environment: this uses venv but any system\n# would work!\npython -m venv .venv\n\n# Activate the virtual environment: this depends on the OS. See\n# the two options below.\n# .venv/Scripts/activate # Windows\nsource .venv/bin/activate # Linux\n\n# Install the dependencies needed to use .\npip install .\n```\n\n\u003e **Note**\nWindows users **must** manually install the CUDA-based version of PyTorch, as pip will\nonly install the CPU version on this platform. See [PyTorch Get Started](https://pytorch.org/get-started/locally/)\nfor the specific command to ru, which should be something along the lines of `pip install torch torchvision --index-url https://download.pytorch.org/whl/cu117`.\n\n### **(Optional)** - Installing development dependencies\nThe following dependencies are only needed to develop `fontina`.\n\n```bash\n# Install the developer dependencies.\npip install .[linting]\n\n# Run linting and tests!\nmake lint\nmake test\n```\n\n### Generating a synthetic dataset\nIf needed, the model can be trained on synthetic data. `fontina` provides a synthetic\ndataset generator that follows part of the recommendations from the [DeepFont paper](https://arxiv.org/abs/1507.03196)\nto make the synthetic data look closer to the real data. To use the generator:\n\n1. Make a copy of `configs/sample.yaml`, e.g. `configs/mymodel.yaml`\n2. Open `configs/mymodel.yaml` and tweak the `fonts` section:\n\n```yaml\nfonts:\n\n  # ...\n\n  # Force an uniform white background for the generated images.\n  # backgrounds_path: \"assets/backgrounds\"\n\n  samples_per_font: 1000\n\n  # Fill in the paths of the fonts that need to be used to generate\n  # the data.\n  classes:\n    - name: Test Font\n      path: \"assets/fonts/test/Test.ttf\"\n    - name: Other Test Font\n      path: \"assets/fonts/test2/Test2.ttf\"\n```\n\n3. Run the generation:\n\n```bash\nfontina-generate -c configs/mymodel.yaml -o outputs/font-images/mymodel\n```\n\nAfter this completes, there should be one directory per configured font in `outputs/font-images/mymodel`.\n\n### Training\n`fontina` currently only supports training a DeepFont-like architecture. The training process\nhas two major steps: unsupervised training of the stacked autoencoders and supervised training\nof the full network.\n\nBefore starting, make a copy of `configs/sample.yaml`, e.g. `configs/mymodel.yaml` (or use the\nexisting one that was created for the dataset generation step).\n\n#### Part 1 - Unsupervised training\n1. Open `configs/mymodel.yaml` and tweak the `training` section:\n\n```yaml\ntraining:\n  # Set this to True to train the stacked autoencoders.\n  only_autoencoder: True\n\n  # Don't use an existing checkpoint for the unsupervised training.\n  # scae_checkpoint_file: \"outputs/models/autoenc/good-checkpoint.ckpt\"\n\n  data_root: \"outputs/font-images/mymodel\"\n\n  # The directory that will contain the model checkpoints.\n  output_dir: \"outputs/models/mymodel-scae\"\n\n  # The size of the batch to use for training.\n  batch_size: 128\n\n  # The initial learning rate to use for training.\n  learning_rate: 0.01\n\n  epochs: 20\n\n  # Whether or not to use a fraction of the data to run a\n  # test cycle on the trained model.\n  run_test_cycle: True\n```\n\n2. Then run the training with:\n\n```bash\npython src/fontina/train.py -c configs/mymodel.yaml\n```\n\n#### Part 2 - Supervised training\n1. Open `configs/mymodel.yaml` (or create a new one!) and tweak the `training` section:\n\n```yaml\ntraining:\n  # This will freeze the stacked autoencoders.\n  only_autoencoder: False\n\n  # Pick the best checkpoint from the unsupervised training.\n  scae_checkpoint_file: \"outputs/models/mymodel-scae/good-checkpoint.ckpt\"\n\n  data_root: \"outputs/font-images/mymodel\"\n\n  # The directory that will contain the model checkpoints.\n  output_dir: \"outputs/models/mymodel-full\"\n\n  # The size of the batch to use for training.\n  batch_size: 128\n\n  # The initial learning rate to use for training.\n  learning_rate: 0.01\n\n  epochs: 20\n\n  # Whether or not to use a fraction of the data to run a\n  # test cycle on the trained model.\n  run_test_cycle: True\n```\n\n2. Then run the training with:\n\n```bash\nfontina-train -c configs/mymodel.yaml\n```\n\n### **(Optional)** - Monitor performance using TensorBoard\n`fontina` automatically captures the performances of the training runs in a [TensorBoard](https://www.tensorflow.org/tensorboard)-compatible\nway. It should be possible to visualize the recorded data by pointing TensorBoard to the logs\ndirectory as follows:\n\n```bash\ntensorboard --logdir=lightning_logs\n```\n\n### Inference\nOnce training is complete, the resulting model can be used to run inference.\n\n```bash\nfontina-predict -n 6 -w \"outputs/models/mymodel-full/best_checkpoint.ckpt\" -i \"assets/images/test.png\"\n```\n\n## AdobeVFR Pre-trained model\nThe AdobeVFR dataset is currently available for download [at Dropbox, here](https://www.dropbox.com/sh/o320sowg790cxpe/AADDmdwQ08GbciWnaC20oAmna?dl=0). The license for using and distributing the dataset is available [here](https://www.dropbox.com/sh/o320sowg790cxpe/AADDmdwQ08GbciWnaC20oAmna?dl=0\u0026preview=license.txt), which cites:\n\n\u003e This dataset ('Licensed Material') is made available to the scientific community for non-commercial research purposes such as academic research, teaching, scientific publications or personal experimentation.\n\nThe model, being trained on that dataset, retains the same spirit and the same license applies: the release model can only be used for non-commercial purposes.\n\n### How to train\n\n1. Download the dataset to `assets/AdobeVFR`\n2. Unpack `assets/AdobeVFR/Raw Image/VFR_real_u/scrape-wtf-new.zip` in that directory so that the `assets/AdobeVFR/Raw Image/VFR_real_u/scrape-wtf-new/` path exists\n3. Run `fontina-train -c configs/adobe-vfr-autoencoder.yaml`. This will take a long while but progress can be checked with Tensorboard (see the previous sections) during training\n4. Change `configs/adobe-vfr.yaml` so that `scae_checkpoint_file` points to the best checkpoint from step (3).\n5. Run `fontina-train -c configs/adobe-vfr.yaml`. This will take a long while (but less than the unsupervised training round)\n\n### Downloading the models\nWhile only the full model is needed, the stand-alone autonencoder model is being released as well.\n\n* Stand-alone autoencoder model: [Google Drive](https://drive.google.com/file/d/107Ontyg2FGxOKvhE7KM7HSaJ1Wn2Merr/view?usp=sharing)\n* Full model: [Google Drive](https://drive.google.com/file/d/1Fw-bjmapCXe0aCiYvOyGLmYocZDvmptK/view?usp=drive_link)\n\n\u003e **Note**\nThe pre-trained model achieves a validation loss of 0.3523, with an accuracy of 0.8855 after 14 epochs.\nUnfortunately the test performance on `VFR_real_test` is much worse, with a top-1 accuracy of 0.05.\nI'm releasing the model in the hope that somebody could help me fixing this 😊😅\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdexterp37%2Ffontina","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdexterp37%2Ffontina","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdexterp37%2Ffontina/lists"}