{"id":19902712,"url":"https://github.com/argilla-io/get_started_with_deep_learning_for_text_with_allennlp","last_synced_at":"2025-05-02T23:32:28.775Z","repository":{"id":96511417,"uuid":"110685289","full_name":"argilla-io/get_started_with_deep_learning_for_text_with_allennlp","owner":"argilla-io","description":"Getting started with AllenNLP and PyTorch by training a tweet classifier","archived":false,"fork":false,"pushed_at":"2017-11-15T06:16:21.000Z","size":198,"stargazers_count":66,"open_issues_count":0,"forks_count":17,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-07T08:37:50.479Z","etag":null,"topics":["artificial-intelligence","natural-language-processing","neural-networks","pytorch","pytorch-tutorial"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/argilla-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-14T12:06:18.000Z","updated_at":"2025-01-21T03:50:01.000Z","dependencies_parsed_at":"2023-07-06T17:01:29.040Z","dependency_job_id":null,"html_url":"https://github.com/argilla-io/get_started_with_deep_learning_for_text_with_allennlp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/argilla-io%2Fget_started_with_deep_learning_for_text_with_allennlp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/argilla-io%2Fget_started_with_deep_learning_for_text_with_allennlp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/argilla-io%2Fget_started_with_deep_learning_for_text_with_allennlp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/argilla-io%2Fget_started_with_deep_learning_for_text_with_allennlp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/argilla-io","download_url":"https://codeload.github.com/argilla-io/get_started_with_deep_learning_for_text_with_allennlp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252122421,"owners_count":21698307,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","natural-language-processing","neural-networks","pytorch","pytorch-tutorial"],"created_at":"2024-11-12T20:19:07.621Z","updated_at":"2025-05-02T23:32:28.769Z","avatar_url":"https://github.com/argilla-io.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Introduction\n\nThis repository contains code and experiments using PyTorch, AllenNLP and spaCy and is intended as a learning resource for getting started with this libraries and with deep learning for NLP technologies.\n\nIn particular, it contains:\n\n1. Custom modules for defining a SequenceClassifier and its Predictor.\n2. A basic custom DataReader for reading CSV files.\n3. An experiments folder containing several experiment JSON files to show how to define a baseline and refine it with more sophisticated approaches.\n\nThe overall goal is to classify tweets in Spanish corresponding to the COSET challenge dataset: a collection of tweets for a recent Spanish Election. The winning approach of the challenge is described in the following paper: http://ceur-ws.org/Vol-1881/COSET_paper_7.pdf.\n\n# Setup\n\nUse a virtual environment, Conda for example:\n\n```shell\nconda create -n allennlp_spacy\n```\n\n```shell\nsource activate allennlp_spacy\n```\n\nInstall PyTorch for your platform:\n```shell\npip install http://download.pytorch.org/whl/torch-0.2.0.post3-cp36-cp36m-macosx_10_7_x86_64.whl\n```\n\nInstall spaCy Spanish model:\n```shell\npython -m spacy download es\n```\n\nInstall AllenNLP and other dependencies:\n```shell\npip install -r requirements.txt\n```\n\nInstall custom module for running AllenNLP commands with custom models:\n```shell\npython setup.py develop\n```\n\nInstall Tensorboard:\n```shell\npip install tensorboard\n```\n\n\nDownload pre-trained  and prepare word vectors from fastText project:\n```shell\ndownload_prepare_fasttext.sh\n```\n\n# Goals\n\n1. Understand the basic components of AllenNLP and PyTorch.\n\n2. Understand how to configure AllenNLP to use spaCy models in different languages, in this case Spanish model.\n\n3. Understand how to create and connect custom models using AllenNLP and extending its command-line.\n\n4. Design and compare several experiments on a simple Tweet classification tasks in Spanish. Start by defining a simple baseline and progressively use more complex models.\n\n5. Use Tensorboard for monitoring the experiments.\n\n6. Compare your results with existing literature (i.e., results of the COSET Tweet classification challenge)\n\n7. Learn how to prepare and use external pre-trained word embeddings, in this case fastText's wikipedia-based word vectors.\n\n# Exercises\n\n## Inspecting Seq2VecEncoders and understanding the basic building blocks of AllenNLP:\n\nCheck the basic structure of these modules in AllenNLP.\n\n## Defining and running our baseline:\n\nIn the folder ``experiments/definitions/`` you can find the definition of our baseline, using a BagOfEmbeddingsEncoder.\n\nRun the experiment using:\n```shell\npython -m recognai.run train experiments/definitions/baseline_boe_classifier.json -s experiments/output/baseline\n```\n\n## Monitor your experiments using Tensorboard:\n\nYou can monitor your experiments by running TensorBoard and pointing it to the experiments output folder:\n\n```shell\ntensorboard --logdir=experiments/output\n```\n\n## Defining and running a CNN classifier:\n\nIn the folder ``experiments/definitions/`` you can find the definition of a CNN classifier. As you see, we only need to configure a new encoder using a CNN.\n\nRun the experiment using:\n\n```shell\npython -m recognai.run train experiments/definitions/cnn_classifier.json -s experiments/output/cnn\n```\n\n## Using pre-trained word embeddings:\n\nFacebook fastText's team has made available pre-trained word embeddings for 294 languages (see https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md). Using the ``download_prepare_fasttext.sh`` script, you can download the Spanish vectors and use them as pre-trained weights in either of the models.\n\nTo use pre-trained embeddings, you can run the experiment using:\n```shell\npython -m recognai.run train experiments/definitions/cnn_classifier_fasttext_embeddings_fixed.json -s experiments/output/cnn_embeddings_fixed\n```\n\nOr use pre-trained embeddings and let the network tune their weights, using:\n```shell\npython -m recognai.run train experiments/definitions/cnn_classifier_fasttext_embeddings_tunable.json -s experiments/output/cnn_embeddings_tuned\n```\n\n## Extra:\n\n- Check https://github.com/recognai/custom_models_allennlp/tree/master/experiments/tweet-classification-spanish and run an RNN classifier. How are the results? Tip: Initialization is key when training LSTMs.\n\n- The network quickly overfits, what strategies would you follow?\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fargilla-io%2Fget_started_with_deep_learning_for_text_with_allennlp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fargilla-io%2Fget_started_with_deep_learning_for_text_with_allennlp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fargilla-io%2Fget_started_with_deep_learning_for_text_with_allennlp/lists"}