https://github.com/kefirski/pytorch_NEG_loss

NEG loss implemented in pytorch
https://github.com/kefirski/pytorch_NEG_loss

python pytorch word2vec

Last synced: 3 months ago
JSON representation

NEG loss implemented in pytorch

• Host: GitHub
• URL: https://github.com/kefirski/pytorch_NEG_loss
• Owner: kefirski
• License: mit
• Created: 2017-03-04T07:32:11.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2017-07-07T08:49:15.000Z (over 7 years ago)
• Last Synced: 2024-05-22T07:52:38.979Z (6 months ago)
• Topics: python, pytorch, word2vec
• Language: Python
• Size: 33.2 KB
• Stars: 125
• Watchers: 7
• Forks: 26
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Pytorch Negative Sampling Loss

[Negative Sampling Loss](https://arxiv.org/abs/1310.4546) implemented in [PyTorch](http://www.pytorch.org).

![NEG Loss Equation](images/neg.png)

## Usage

```python

neg_loss = NEG_loss(num_classes, embedding_size)

    

optimizer = SGD(neg_loss.parameters(), 0.1)

    

for i in range(num_iterations):

    ''' 

    input is [batch_size] shaped tensors of Long type

    while target has shape of [batch_size, window_size]

    '''

    input, target = next_batch(batch_size)

        

    loss = neg_loss(input, target, num_sample)

    

    optimizer.zero_grad()

    loss.backward()

    optimizer.step()

    

word_embeddings = neg_loss.input_embeddings()        

```