https://github.com/josuerhea/tfidf

TFIDF implementation in Go
https://github.com/josuerhea/tfidf

go golang

Last synced: about 2 months ago
JSON representation

TFIDF implementation in Go

• Host: GitHub
• URL: https://github.com/josuerhea/tfidf
• Owner: JosueRhea
• Created: 2023-02-15T21:09:06.000Z (almost 2 years ago)
• Default Branch: master
• Last Pushed: 2023-02-23T18:13:31.000Z (almost 2 years ago)
• Last Synced: 2024-06-22T08:57:56.316Z (7 months ago)
• Topics: go, golang
• Language: Go
• Homepage:
• Size: 1.97 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# tfidf

An package for my search engine using tfidf

## Example

```go

package main

import (

	"fmt"

	"log"

	"tfidf/tfidf"

	"tfidf/util"

	"github.com/goccy/go-json"

)

func main() {

	fileContent, err := util.ReadFileToString("test-data.json")

	if err != nil {

		log.Fatal("Error reading the file", err)

	}

	var parsedData []map[string]interface{}

	json.Unmarshal([]byte(fileContent), &parsedData)

	t := tfidf.New()

	t.AddDocs(parsedData)

	{

		stopWordsFilePath := "stopwords-en.txt"

		error := t.AddStopWordsFile(stopWordsFilePath)

		if error != nil {

			log.Printf("Cannot add %s because %s", stopWordsFilePath, error.Error())

		}

	}

	{

		stopWordsFilePath := "stopwords-es.txt"

		error := t.AddStopWordsFile(stopWordsFilePath)

		if error != nil {

			log.Printf("Cannot add %s because %s", stopWordsFilePath, error.Error())

		}

	}

	t.AddStopWordsFile("stopwords-es.txt")

	search := t.CalculateTFIDF("marvel")

	fmt.Println(len(search))

	for _, value := range search {

		fmt.Printf("id: %s -> %f\n", value.ID, value.Rank)

		fmt.Printf("value.Data: %v\n", value.Data["description"])

	}

}

```