{"id":23236995,"url":"https://github.com/nicolay-r/arelight","last_synced_at":"2025-08-19T23:31:11.067Z","repository":{"id":37711265,"uuid":"467513232","full_name":"nicolay-r/ARElight","owner":"nicolay-r","description":"Granular Viewer of Sentiments Between Entities in Massively Large Documents and Collections of Texts, powered by AREkit","archived":false,"fork":false,"pushed_at":"2024-12-10T10:41:00.000Z","size":26153,"stargazers_count":37,"open_issues_count":9,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-12-10T11:35:29.409Z","etag":null,"topics":["arekit","attitudes","bert","bert-model","brat","deep-learning","deeppavlov","deeppavlov-framework","machine-learning","named-entity-recognition","natural-language-processing","nlp","relation-extraction","russian-language","sentiment-analysis","tensorflow"],"latest_commit_sha":null,"homepage":"https://link.springer.com/chapter/10.1007/978-3-031-56069-9_23","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nicolay-r.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-08T13:00:05.000Z","updated_at":"2024-09-22T15:48:13.000Z","dependencies_parsed_at":"2024-01-13T12:15:18.224Z","dependency_job_id":"95806578-cfdb-465b-bea6-349d6ce4dd0c","html_url":"https://github.com/nicolay-r/ARElight","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicolay-r%2FARElight","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicolay-r%2FARElight/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicolay-r%2FARElight/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicolay-r%2FARElight/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nicolay-r","download_url":"https://codeload.github.com/nicolay-r/ARElight/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230374271,"owners_count":18216044,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arekit","attitudes","bert","bert-model","brat","deep-learning","deeppavlov","deeppavlov-framework","machine-learning","named-entity-recognition","natural-language-processing","nlp","relation-extraction","russian-language","sentiment-analysis","tensorflow"],"created_at":"2024-12-19T04:13:24.009Z","updated_at":"2025-08-19T23:31:11.058Z","avatar_url":"https://github.com/nicolay-r.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ARElight 0.25.1\n\n![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)\n![](https://img.shields.io/badge/AREkit-0.25.2-orange.svg)\n[![](https://img.shields.io/badge/demo-0.24.0-purple.svg)](https://guardeec.github.io/arelight_demo/template.html)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/ARElight/blob/v0.24.0/ARElight.ipynb)\n[![PyPI downloads](https://img.shields.io/pypi/dm/arelight.svg)](https://pypistats.org/packages/arelight)\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"logo.png\"/\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://guardeec.github.io/arelight_demo/template.html\"\u003e👉\u003cb\u003eDEMO\u003c/b\u003e👈\u003c/a\u003e\n  \u003cbr\u003e\n  \u003ca href=\"https://github.com/nicolay-r/ARElight-server\"\u003e👉GUI server setup👈\u003c/a\u003e\n\u003c/p\u003e\n\nARElight is an application for a granular view onto sentiments between mentioned named entities \nin texts. \nThis repository is a part of the **ECIR-2024** demo paper: \n[ARElight: Context Sampling of Large Texts for Deep Learning Relation Extraction](https://link.springer.com/chapter/10.1007/978-3-031-56069-9_23).\n\n\n# Installation\n\n```bash\npip install git+https://github.com/nicolay-r/arelight@v0.25.1\n```\n\n# GUI Interface \n\nSince the version `0.25.0` ARElight has an updated GUI server\n![image](https://github.com/user-attachments/assets/b7a1189a-b5b0-479f-8413-f8a16801e06b)\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/nicolay-r/ARElight-server\"\u003e👉GUI server setup👈\u003c/a\u003e\n\u003c/p\u003e\n\n## Usage: Inference\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/ARElight/blob/v0.24.0/ARElight.ipynb)\n\nInfer sentiment attitudes from text file **in English**:\n```bash\npython3 -m arelight.run.infer  \\\n    --sampling-framework \"arekit\" \\\n    --ner-framework \"deeppavlov\" \\\n    --ner-model-name \"ner_ontonotes_bert\" \\\n    --ner-types \"ORG|PERSON|LOC|GPE\" \\\n    --terms-per-context 50 \\\n    --sentence-parser \"nltk:english\" \\\n    --tokens-per-context 128 \\\n    --bert-framework \"opennre\" \\\n    --batch-size 10 \\\n    --pretrained-bert \"bert-base-cased\" \\\n    --bert-torch-checkpoint \"ra4-rsr1_bert-base-cased_cls.pth.tar\" \\\n    --backend \"d3js_graphs\" \\\n    --docs-limit 500 \\\n    -o \"output\" \\\n    --from-files \"\u003cPATH-TO-TEXT-FILE\u003e\"\n```\n\n\u003e **NOTE:** [Applying ARElight for **non-english texts**](https://github.com/nicolay-r/ARElight/wiki/Language-Specific-Application)\n\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\n\n### Parameters\n\u003c/summary\u003e\n\nThe complete documentation is available via `-h` flag:\n```bash\npython3 -m arelight.run.infer -h\n```\n\nParameters:\n* `sampling-framework` we consider only `arekit` framework by default.\n    * `from-files` -- list of filepaths to the related documents.\n      * for the `.csv` files we consider that each line of the particular `column` as a separated document.\n        * `csv-sep` -- separator between columns.\n        * `csv-column` -- name of the column in CSV file.\n    * `collection-name` -- name of the result files based on sampled documents.\n    * `terms-per-context` -- total amount of words for a single sample.\n    * `sentence-parser` -- parser utilized for document split into sentences; list of the [[supported parsers]](https://github.com/nicolay-r/ARElight/blob/a17088a98729e3092de1666bef9ba8327ef30b80/arelight/run/utils.py#L15).\n    * `synonyms-filepath` -- text file with listed synonymous entries, grouped by lines. [[example]](https://github.com/nicolay-r/RuSentRel/blob/master/synonyms.txt).\n    * `stemmer` -- for words lemmatization (optional); we support [[PyMystem]](https://pypi.org/project/pymystem3/).\n    * NER parameters:  \n      * `ner-framework` -- type of the framework:\n        * `deeppavlov` -- [[DeepPavlov]](https://docs.deeppavlov.ai/en/master/features/models/NER.html#6.-Models-list) list of models.\n        * `transformers` -- [[Transformers]](https://huggingface.co/models?library=transformers\u0026other=named-entity-recognition\u0026sort=downloads) list of models.\n      * `ner-model-name` -- model name within utilized NER framework.\n      * `ner-types` -- list of types to be considered for annotation, separated by `|`.\n    * `docs-limit` -- the total limit of documents for sampling.\n  * [Translation specific parameters](https://github.com/nicolay-r/ARElight/wiki/Language-Specific-Application#any-other-languages)\n      * `translate-framework` -- text translation backend (optional); we support [[googletrans]](https://github.com/nicolay-r/ARElight/blob/a17088a98729e3092de1666bef9ba8327ef30b80/arelight/run/utils.py#L31)\n      * `translate-entity` -- (optional) source and target language supported by backend, separated by `:`.\n      * `translate-text` -- (optional) source and target language supported by backend, separated by `:`.\n* `bert-framework` -- samples classification framework; we support [[OpenNRE]](https://github.com/thunlp/OpenNRE).\n    * `text-b-type` -- (optional) `NLI` or None [[supported]](https://github.com/nicolay-r/ARElight/blob/a17088a98729e3092de1666bef9ba8327ef30b80/arelight/samplers/bert.py#L14).\n    * `pretrained-bert` -- pretrained state name.\n    * `batch-size` -- amount of samples per single inference iteration.\n    * `tokens-per-context` -- size of input.\n    * `bert-torch-checkpoint` -- fine-tuned state.\n    * `device-type` -- `cpu` or `gpu`.\n    * `labels-fmt` -- list of the mappings from `label` to integer value; is a `p:1,n:2,u:0` by default, where:\n        * `p` -- positive label, which is mapped to `1`.\n        * `n` -- negative label, which is mapped to `2`.\n        * `u` -- undefined label (optional), which is mapped to `0`.\n* `backend` -- type of the backend (`d3js_graphs` by default).\n    * `host` -- port on which we expect to launch localhost server.\n    * `label-names` -- default mapping is `p:pos,n:neg,u:neu`.\n* `-o` -- output folder for result collections and demo.\n\nFramework parameters mentioned above as well as their related setups might be ommited.\n\n\u003c/details\u003e\n \n## Usage: Graph Operations\n\nFor graph analysis you can perform several graph operations by this script:\n\n1. Arguments mode:\n\n```bash\npython3 -m arelight.run.operations \\\n\t--operation \"\u003cOPERATION-NAME\u003e\" \\\n\t--graph_a_file output/force/boris.json \\\n  \t--graph_b_file output/force/rishi.json \\\n  \t--weights y \\\n  \t-o output \\\n  \t--description \"[OPERATION] between Boris Johnson and Rishi Sunak on X/Twitter\"\n```\n\n2. Interactive mode:\n\n```bash\npython3 -m arelight.run.operations\n```\n\n`arelight.run.operations` allows you to operate ARElight's outputs using graphs: you can merge graphs, find their similarities or differences.\n\n\n\u003cdetails\u003e\n\u003csummary\u003e\n\n### Parameters\n\n\u003c/summary\u003e\n\n* `--graph_a_file` and `--graph_b_file` are used to specify the paths to the `.json` files for graphs A and B, which are used in the operations.\n  These files should be located in the `\u003cyour_output/force\u003e` folder.\n* `--name` -- name of the new graph.\n* `--description` -- description of the new graph.\n* `--host` -- determines the server port to host after the calculations.\n* `-o` -- option allows you to specify the path to the folder where you want to store the output.\n  You can either create a new output folder or use an existing one that has been created by ARElight.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\n\n### Parameter `operation`\n\u003c/summary\u003e\n\n#### Preparation\n\nConsider that you used ARElight script for X/Twitter \nto [infer relations](#usage-inference) from\nmessages of UK politicians `Boris Johnson` and `Rishi Sunak`:\n\n```bash\npython3 -m arelight.run.infer ...other arguments... \\\n\t-o output --collection-name \"boris\" --from-files \"twitter_boris.txt\"\n\t\npython3 -m arelight.run.infer  ...other arguments... \\\n\t-o output --collection-name \"rishi\" --from-files \"twitter_rishi.txt\"\n```\nAccording to the [results section](#layout-of-the-files-in-output), you will have `output` directory with 2 files `force` layout graphs:\n```lua\noutput/\n├── force/\n    ├──  rishi.json\n    └──  boris.json\n```\n\n#### List of Operations\n\nYou can do the following operations to combine several outputs, ot better understand similarities, and differences between them:\n\n**UNION** $(G_1 \\cup G_2)$ - combine multiple graphs together.\n* The result graph contains all the vertices and edges that are in $G_1$ and $G_2$. \nThe edge weight is given by $W_e = W_{e1} + W_{e2}$, and the vertex weight is its weighted degree centrality: $W_v = \\sum_{e \\in E_v} W_e(e)$.\n  ```bash\n  python3 -m arelight.run.operations --operation UNION \\\n      --graph_a_file output/force/boris.json \\\n      --graph_b_file output/force/rishi.json \\\n      --weights y -o output --name boris_UNION_rishi \\\n      --description \"UNION of Boris Johnson and Rishi Sunak Twits\"\n  ```\n  ![union](https://github.com/nicolay-r/ARElight/assets/14871187/eaac6758-69f7-4cc1-a631-7ce132757b29)\n\n**INTERSECTION** $(G_1 \\cap G_2)$ - what is similar between 2 graphs?\n* The result graph contains only the vertices and edges common to $G_1$ and $G_2$. \nThe edge weight is given by $W_e = \\min(W_{e1},W_{e2})$, and the vertex weight is its weighted degree centrality: $W_v = \\sum_{e \\in E_v} W_e(e)$.\n  ```bash\n  python3 -m arelight.run.operations --operation INTERSECTION \\\n      --graph_a_file output/force/boris.json \\\n      --graph_b_file output/force/rishi.json \\\n      --weights y -o output --name boris_INTERSECTION_rishi \\\n      --description \"INTERSECTION between Twits of Boris Johnson and Rishi Sunak\"\n  ```\n  ![intersection](https://github.com/nicolay-r/ARElight/assets/14871187/286bd1ce-dbb0-4370-bfbe-245330ae6204)\n\n\n**DIFFERENCE** $(G_1 - G_2)$ - what is unique in one graph, that another graph doesn't have? \n\n* **NOTE:** this operation is not commutative $(G_1 - G_2) ≠ G_2 - G_1)$)_\n* The results graph contains all the vertices from $G_1$ but only includes edges from $E_1$ that either don't appear in $E_2$ or have larger weights in $G_1$ compared to $G_2$. \nThe edge weight is given by $W_e = W_{e1} - W_{e2}$ if $e \\in E_1$, $e \\in E_1 \\cap E_2$ and $W_{e1}(e) \u003e W_{e2}(e)$.\n  ```bash\n  python3 -m arelight.run.operations --operation DIFFERENCE \\\n      --graph_a_file output/force/boris.json \\\n      --graph_b_file output/force/rishi.json \\\n      --weights y -o output --name boris_DIFFERENCE_rishi \\\n      --description \"Difference between Twits of Boris Johnson and Rishi Sunak\"\n  ```\n  ![difference](https://github.com/nicolay-r/ARElight/assets/14871187/8b036ce6-6607-4588-b0cf-4704647f55ff)\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\n\n### Parameter `weights`\n\u003c/summary\u003e\n\nYou have the option to specify whether to include edge weights in calculations or not. \nThese weights represent the frequencies of discovered edges, indicating how often a relation between two instances was found in the text analyzed by ARElight.\n* `--weights`\n  * `y`: the result will be based on the union, intersection, or difference of these frequencies.\n  * `n`: all weights of input graphs will be set to 1. In this case, the result will reflect the union, intersection, or difference of the graph topologies, regardless of the frequencies. This can be useful when the existence of relations is more important to you, and the number of times they appear in the text is not a significant factor.\n  \u003e Note that using or not using the `weights` option may yield different topologies:\n  \u003e \n  ![weights](https://github.com/nicolay-r/ARElight/assets/14871187/43ad2054-d413-47ee-ac8b-d06af6921214)\n\n\u003c/details\u003e\n\n## Powered by\n\n* AREkit [[github]](https://github.com/nicolay-r/AREkit)\n\n\u003cp float=\"left\"\u003e\n\u003ca href=\"https://github.com/nicolay-r/AREkit\"\u003e\u003cimg src=\"https://github.com/nicolay-r/ARElight/assets/14871187/01232f7a-970f-416c-b7a4-1cda48506afe\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## How to cite\nOur one and my personal interest is to help you better explore and analyze attitude and relation extraction related tasks with ARElight. \nA great research is also accompanied with the faithful reference. \nif you use or extend our work, please cite as follows:\n\n```bibtex\n@inproceedings{rusnachenko2024arelight,\n  title={ARElight: Context Sampling of Large Texts for Deep Learning Relation Extraction},\n  author={Rusnachenko, Nicolay and Liang, Huizhi and Kolomeets, Maxim and Shi, Lei},\n  booktitle={European Conference on Information Retrieval},\n  year={2024},\n  organization={Springer}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicolay-r%2Farelight","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnicolay-r%2Farelight","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicolay-r%2Farelight/lists"}