https://github.com/capjamesg/pysurprisal

Calculate surprisal for words in text.
https://github.com/capjamesg/pysurprisal

nlp surprisal

Last synced: over 1 year ago
JSON representation

Calculate surprisal for words in text.

• Host: GitHub
• URL: https://github.com/capjamesg/pysurprisal
• Owner: capjamesg
• License: mit
• Created: 2023-06-07T10:10:19.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2023-12-15T12:25:27.000Z (over 2 years ago)
• Last Synced: 2025-05-09T01:15:15.696Z (over 1 year ago)
• Topics: nlp, surprisal
• Language: Python
• Homepage: https://capjamesg.github.com/pysurprial/
• Size: 567 KB
• Stars: 7
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

          
# pysurprisal

[![version](https://badge.fury.io/py/pysurprisal.svg)](https://badge.fury.io/py/pysurprisal)

[![downloads](https://img.shields.io/pypi/dm/pysurprisal)](https://pypistats.org/packages/pysurprisal)

[![license](https://img.shields.io/pypi/l/pysurprisal)](https://github.com/capjamesg/pysurprisal/blob/main/LICENSE)

[![python-version](https://img.shields.io/pypi/pyversions/pysurprisal)](https://badge.fury.io/py/pysurprisal)

An implementation of surprisal in Python.

## Installation

```bash

pip install pysurprisal

```

## Usage

```python

from pysurprisal import Surprisal

text = "..."

# calculate surprisal for each word in the text

data = Surprisal(text)

surprisals = data.calculate_surprisals()

# get the top 10 most surprising words

top_k = data.get_top_k(10)

# print dictionary of all surprisals

# key = word, value = surprisal

print(surprisals.surprisals)

```

## Contributors

- capjamesg

## License

This project is licensed under an [MIT license](LICENSE.md).