https://github.com/ktibow/surprisingly-simple-embeddings

Old text search techniques + filtering = vector search on the client
https://github.com/ktibow/surprisingly-simple-embeddings

Last synced: 6 months ago
JSON representation

Old text search techniques + filtering = vector search on the client

• Host: GitHub
• URL: https://github.com/ktibow/surprisingly-simple-embeddings
• Owner: KTibow
• Created: 2024-04-21T01:55:14.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-05-23T02:07:26.000Z (over 1 year ago)
• Last Synced: 2025-07-11T12:48:44.436Z (6 months ago)
• Language: JavaScript
• Homepage:
• Size: 3.36 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

          
# surprisingly simple embeddings

these days, if you research embeddings, all you hear about is language models. these embeddings are great but usually require 5mb or even 100mb of a download to run locally.

turns out we had this figured out a while ago. just

1. tokenize your text

2. assign each token a vector

3. average these vectors based on how common the token is

4. use the average vector as your embedding

here, we use glove's 50d twitter-based token embeddings, but we remove uncommon tokens. that way, when we compress the embeddings, it's relatively small.