{"id":28290599,"url":"https://github.com/erechtheus/geolocation","last_synced_at":"2025-06-13T12:31:50.252Z","repository":{"id":19786979,"uuid":"87903207","full_name":"Erechtheus/geolocation","owner":"Erechtheus","description":"Geolocation prediction for a given Tweet","archived":false,"fork":false,"pushed_at":"2023-05-01T20:35:43.000Z","size":129,"stargazers_count":36,"open_issues_count":4,"forks_count":13,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-22T02:14:04.601Z","etag":null,"topics":["geolocation","geolocation-prediction","neural-network","pretrained-models","tweets"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Erechtheus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-11T07:25:08.000Z","updated_at":"2024-04-05T21:00:55.000Z","dependencies_parsed_at":"2023-01-11T20:36:11.168Z","dependency_job_id":null,"html_url":"https://github.com/Erechtheus/geolocation","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/Erechtheus/geolocation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Erechtheus%2Fgeolocation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Erechtheus%2Fgeolocation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Erechtheus%2Fgeolocation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Erechtheus%2Fgeolocation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Erechtheus","download_url":"https://codeload.github.com/Erechtheus/geolocation/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Erechtheus%2Fgeolocation/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259644914,"owners_count":22889501,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["geolocation","geolocation-prediction","neural-network","pretrained-models","tweets"],"created_at":"2025-05-22T02:14:04.456Z","updated_at":"2025-06-13T12:31:50.227Z","avatar_url":"https://github.com/Erechtheus.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# geolocation\nGeolocation prediction for a given Tweet, or a short text. The system trains a neural net, as described in\n\n\tPhilippe Thomas and Leonhard Hennig (2017), \"Twitter Geolocation Prediction using Neural Networks.\" In Proceedings of GSCL\n\n# Performance\nThis section briefly provides some information about the performance of our method. We removed the original model and only provide information about the new model.\n\n|Model   \t|   Acc\t| Median  \t| Mean  \t|  Acc \t| Median \t| Mean |\n|---\t|---\t|---\t|---\t|---\t|---\t|---\t|\n|   Location\t| 0.366| 203.9  | 4514.1 | 0.448 | 41.7   | 3821.0 |\n|  Text \t    | 0.201| 1834.8 | 4320.1 | 0.330 | 213.9  | 2441.7 |\n|  Description \t| 0.096| 3335.7 | 5837.4 | 0.121 | 2800.3 | 5491.0 |\n|  User-name \t| 0.060| 3852.3 | 5909.3 | 0.058 | 4154.9 | 6131.7 |\n|  Timezone     | 0.057| 5280.1 | 5554.8 | 0.061 | 5489.9 | 5481.4 |\n|  User-lang \t| 0.061| 6465.1 | 7306.9 | 0.047 | 8903.7 | 8523.1 |\n|  Links \t    | 0.033| 7606.3 | 6984.0 | 0.045 | 6734.4 | 6571.7 |\n|  UTC \t        | 0.045| 5387.4 | 5570.0 | 0.048 | 5365.8 | 5412.7 |\n|  Source \t    | 0.045| 8026.4 | 7539.6 | 0.045 | 6903.5 | 6901.1 |\n|  Tweet-time \t| 0.028| 8442.5 | 7694.4 | 0.024 | 11720.6| 10275.5 |\n|  Full-fixed \t| **0.442**| **43.6**| **1151.0**  |**0.530**| **14.1**| **771.2** |\n|  Baseline \t| 0.028  \t| 11,723.0  | 10,264.3  | 0.024  \t| 11,771.5  | 10,584.4  |\n\n\n# Usage\n\n## Download \nSource code from this repository has been published here (https://github.com/Erechtheus/geolocation/releases). \n- [Version 2.3](https://github.com/Erechtheus/geolocation/releases/tag/V2.3)   uses keras functional API (instead of Keras sequential API). This code runs with Keras Version 2, whereas the original [release](https://github.com/Erechtheus/geolocation/releases/tag/V1.0) worked with Keras Version 1. Also has some minor improvement regarding preprocessing and has a REST-API.\n\n\n##  Local installation (python)\nThis section briefly explains the steps to download the source code, installs python dependencies in Anaconda, downloads the models and processors and performs text classification for one text example.\nDownload model and preprocessor [relevantData.tar.lzma](https://drive.google.com/open?id=12L8BNQ9zTi6MehoQEILGKFy4pw0mVZGa).\n\n```bash\ngit clone https://github.com/Erechtheus/geolocation.git\ncd geolocation/\nconda create --name geoloc  --file requirements.txt\nconda activate geoloc\n\ntar xfva relevantData.tar.lzma\n\npython predictText.py\n```\n\n\n## Local installation (docker image)\n\nWe provide a [docker image](https://drive.google.com/file/d/19AA3M6dZHK8gogC8qxmt2vVDXrvs98MR/view?usp=sharing) of our code using functional API and a REST Service\n```bash\nunlzma geolocationV2.3.tar.lzma\ndocker load --input geolocationV2.3.tar\ndocker run -d -p   5000:5000 --name geolocation --network host  geoloc:latest\n```\n\nAlternatively, you can download the model from [docker hub](https://hub.docker.com/r/erechtheus79/geolocation).\n\n```bash\ndocker pull erechtheus79/geolocation\ndocker run -d -p   5000:8080 --name geolocation --network host  erechtheus79/geolocation\n```\n\nAccess the simple text model using the [URL](http://127.0.0.1:5000/predictText?text=Montmartre%20is%20truly%20beautiful) and it returns\n\n```json\n{\n    \"query\": \"Montmartre is truly beautiful\",\n    \"results\": [\n        {\n            \"city\": \"paris-a875-fr\",\n            \"lat\": 48.857779087136095,\n            \"lon\": 2.3539118329464914,\n            \"score\": 0.18563927710056305\n        },\n        {\n            \"city\": \"city of london-enggla-gb\",\n            \"lat\": 51.50090096289424,\n            \"lon\": -0.09162320754762229,\n            \"score\": 0.04953022673726082\n        },\n        {\n            \"city\": \"boulogne billancourt-a892-fr\",\n            \"lat\": 48.82956285864007,\n            \"lon\": 2.2603947479966044,\n            \"score\": 0.04159574210643768\n        },\n        {\n            \"city\": \"saint denis-a893-fr\",\n            \"lat\": 48.947253923722585,\n            \"lon\": 2.4314893304822607,\n            \"score\": 0.02842172235250473\n        },\n        {\n            \"city\": \"argenteuil-a895-fr\",\n            \"lat\": 48.97509961545753,\n            \"lon\": 2.1906891017164387,\n            \"score\": 0.021229125559329987\n        }\n    ]\n}\n```\n\n\n## Example usage to predict location of a text snippet:\nThe code below briefly describes how to use our neural network, trained on text only. For other examples (e.g., using Twitter text and metadata), we refer to the examples in the two evaluation scripts\n\n```python\nfrom keras.models import load_model\nimport pickle\nfrom keras.preprocessing.sequence import pad_sequences\nimport numpy as np\n\n\n#Load Model\ntextBranch = load_model('data/models/textBranchNorm.h5')\n\n#Load tokenizers, and mapping\nfile = open(\"data/binaries/processors.obj\",'rb')\ndescriptionTokenizer, domainEncoder, tldEncoder, locationTokenizer, sourceEncoder, textTokenizer, nameTokenizer, timeZoneTokenizer, utcEncoder, langEncoder, placeMedian, colnames, classEncoder  = pickle.load(file)\n\n#Predict text (e.g., 'Montmartre is truly beautiful')\ntestTexts=[];\ntestTexts.append(\"Montmartre is truly beautiful\")\n\ntextSequences = textTokenizer.texts_to_sequences(testTexts)\ntextSequences = np.asarray(textSequences)\ntextSequences = pad_sequences(textSequences)\n\npredict = textBranch.predict(textSequences)\n\n#Print the top 5\nfor index in reversed(predict.argsort()[0][-5:]):\n    print(\"%s with score=%.3f\" % (colnames[index], float(predict[0][index])) )\n\n```\n\n#### The output is (scores might vary between different model versions):\n    paris-a875-fr with score=0.413\n    boulogne billancourt-a892-fr with score=0.070\n    saint denis-a893-fr with score=0.058\n    creteil-a894-fr with score=0.029\n    argenteuil-a895-fr with score=0.026\n\n## Train and apply models\nTo train models, training data (tweets and gold labels) needs to be retrieved. As Tweets can not be shared directly, we refer to the [WNUT'16 workshop page](http://noisy-text.github.io/2016/geo-shared-task.html) for further information.\n\nAfter retrieving the training files, the [preprocess](https://github.com/Erechtheus/geolocation/blob/master/Preprocess.py) script converts tweets into the desired representation to train a neural network. Models can be trained from scratch using the [trainindividual](https://github.com/Erechtheus/geolocation/blob/master/TrainIndividualModels.py) script.\nPretrained models and preprocessors (e.g., used tokenizer)  are available [here](https://drive.google.com/open?id=1BA_Rj5FJ30nTzvfJvnhgx3k-bzC6Sn9D).\nThe evaluation of models is implemented [here](https://github.com/Erechtheus/geolocation/blob/master/EvaluateTweet.py).\n\n# Licence\nErechtheus/geolocation is licensed under the \"GNU General Public License v3.0\". See also the licence file [here](https://github.com/Erechtheus/geolocation/blob/master/LICENSE).\n\n# Possible improvements\n - Transformer models\n - LSTM representation via Keras generators to save memory\n - REST API with twitter JSON object as input\n - How's the performance for the full network when we only feed partial info? (E.g. only text, timezone, ...)\n - Incorporate user-graph for prediction (e.g. using neural structure learning)\n - Character CNN (memory consumption pretty high in my implementation, needs generators)\n - Use image data\n - Train a worldwide country-model? Clustering?\n \n# Tested improvements\n - FastText as embedding method -\u003e Performance for text-model is below our current methods. But, we did not use a fast-text model explicitly learned on social-media data\n - LSTM using recurrent dropout -\u003e no improvement can be oberved (TrainIndividualModelsCNN.py)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferechtheus%2Fgeolocation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferechtheus%2Fgeolocation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferechtheus%2Fgeolocation/lists"}