https://github.com/andrefs/we-srds-eval
Calculate the correlation between cosine similarity of word embeddings and human judgements of semantic relations.
https://github.com/andrefs/we-srds-eval
Last synced: 11 months ago
JSON representation
Calculate the correlation between cosine similarity of word embeddings and human judgements of semantic relations.
- Host: GitHub
- URL: https://github.com/andrefs/we-srds-eval
- Owner: andrefs
- Created: 2025-02-05T23:31:37.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-17T23:44:25.000Z (over 1 year ago)
- Last Synced: 2025-04-06T01:15:49.510Z (about 1 year ago)
- Language: TypeScript
- Size: 62.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# we-srds-eval
## Overview
`we-srds-eval` is a tool designed to calculate the correlation between the cosine similarity of word embeddings and human judgments of semantic relations. It processes datasets from `punuy-datasets` to evaluate how well word embeddings align with human semantic understanding.
## Features
- Computes cosine similarity for word pairs.
- Calculates the correlation between cosine similarity scores and human-judged scores.
- Supports multiple datasets and languages.
## Installation
To install the necessary dependencies, run:
```bash
npm install
```
## Usage
To process all datasets and calculate correlations, execute the following command:
```bash
tsx src/bin/proc-datasets.ts
```
This will output the correlation results for each dataset.
## Project Structure
- **src/bin/proc-datasets.ts**: Entry point for processing datasets.
- **src/lib/we-srd-corr.ts**: Core library functions for calculating cosine similarity and correlation.
- **src/lib/utils.ts**: Utility functions used across the project.
- **src/lib/ds-adapters.ts**: Adapters for handling different dataset formats.
## Development
This project uses TypeScript for type safety and Vitest for testing. To run tests, use:
```bash
npm test
```
## License
This project is licensed under the ISC License.
## Author
André Santos