https://github.com/sodalabsio/textstellar

Mapping research capabilities using contextual text embeddings
https://github.com/sodalabsio/textstellar

nlproc semantic-similarity topic-modeling visualization

Last synced: about 1 year ago
JSON representation

Mapping research capabilities using contextual text embeddings

• Host: GitHub
• URL: https://github.com/sodalabsio/textstellar
• Owner: sodalabsio
• License: mit
• Created: 2022-11-29T07:29:00.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2022-12-13T02:12:59.000Z (over 3 years ago)
• Last Synced: 2025-01-28T15:30:56.273Z (over 1 year ago)
• Topics: nlproc, semantic-similarity, topic-modeling, visualization
• Language: Jupyter Notebook
• Homepage: https://textstellar.com
• Size: 2.08 MB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Textstellar 🌌

![preview](/assets/preview.png)

Capability and skill mapping using transformer-based/contextual text embeddings.

## Overview

On a high level, the `Textstellar` pipeline broadly consists of three modules:

1. Semantic Ranking:

    - Given a definition for "X" (a list of *reference* sentences capturing X), which could be a theme, challenge, or a concept, we find the top-K related items based on their semantic similarity. The reference sentences could be either handcrafted, or GPT-3 prompted

    - We apply this to finding relevant research outcomes (and researchers) that are most salient for a given excercise

2. Topic Clustering:

    - Perform unsupervised clustering for topic discovery

    - Using Topic Coherence to automatically select the optimal cluster size etc.

3. Visualization:

    - Plot highest matching outputs and their corresponding authors

    - Generate a 2D "night sky" visualization of topics

## Setup 

1. Clone this repo and get started with `textstellar.ipynb` notebook to use on your own dataset

2. Preferably run on a GPU (recommended to use Google Colab)

3. Replace all system path(s) as needed

Most importantly, explore the low-dimensional semantic space—at your leisure.

Click [here](https://textstellar.com) for a live demo.