https://github.com/jloveric/neuralphrasex
Mixed neural network / fuzzy similarity based phrase database matching. Uses universal sentence embeddings from Tensorflow combined with KNN and fuzzy similarity to perform slot filling based on a list of potential phrases. The approach allows for one shot learning and the efficiency will be determined by KNN (or other approximate nearest neighbor). The name "PhraseX" comes from "Regex", as it is a more flexible way of matching phrases than using regex and better suited for natural language.
https://github.com/jloveric/neuralphrasex
Last synced: 6 months ago
JSON representation
Mixed neural network / fuzzy similarity based phrase database matching. Uses universal sentence embeddings from Tensorflow combined with KNN and fuzzy similarity to perform slot filling based on a list of potential phrases. The approach allows for one shot learning and the efficiency will be determined by KNN (or other approximate nearest neighbor). The name "PhraseX" comes from "Regex", as it is a more flexible way of matching phrases than using regex and better suited for natural language.
- Host: GitHub
- URL: https://github.com/jloveric/neuralphrasex
- Owner: jloveric
- License: mit
- Created: 2019-12-07T18:39:36.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-01-05T02:46:15.000Z (over 2 years ago)
- Last Synced: 2024-12-16T03:50:24.200Z (6 months ago)
- Language: JavaScript
- Homepage:
- Size: 381 KB
- Stars: 7
- Watchers: 3
- Forks: 3
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://travis-ci.org/jloveric/NeuralPhraseX)
# NeuralPhraseX
Takes a javascript object that contains a set of possible pattern matches, phrases and wildcards. These phrases are converted to sentence embeddings in tensorflow. A search phrase is passed to Phrasex and an object is returned containing a list of close phrases as well is the slots that are filled in. The algorithm relies on npm modules, neural-sentence-search, slot-filler and sentence-similarity. Each "match" is scored based on how well it matched. This can be used directly in processing text in a chatbot or a one shot information extraction pipeline.
## Install
```bash
npm install neural-phrasex
```
## How to use
```javascript
let {Phrasex, UserData, BasicPhrasexDatabase} = require("neural-phrasex");
let simpleDatabase = {
data: [
{
//The simplest entry has just a phrase and a class name - phraseType
phrase: ["It is what it is."],
phraseType: "isWhatIs"
},
{
//A phrase and a response to a phrase can be included.
phrase: ["what is your name?"],
response: ["My name is Bot"],
phraseType: "whatIsName",
implies: ["whatIsName"]
}, {
//You can include just simple phrases with no wildcards
exampleWildcards: { value: ["pig"], ans: ["animal"] },
phrase: ["What is a (value)?"],
response: ["A (value) is an (ans)"],
phraseType: "whatIsThing",
}, {
//The wildcards are just examples of what could be put in the slots
//These are necessary so that they can be passed to the neural network
//to construct a sentence vector.
exampleWildcards: { value: ["Seattle"], ans: ["Washington"] },
phrase: ["where is (value)"],
response: ["(value) is in (ans)"],
phraseType: "whereIsThing"
}
]
}
let compute = async () => {
let ans = BasicPhrasexDatabase.generatePhraseDatabase(simpleDatabase)
let phrasex = new Phrasex(ans)
let res = await phrasex.initialize()
let userData = new UserData();
userData.initialize();
let res1 = await phrasex.getWildcardsAndMatch("Where is Boston", "", userData)
console.log(res1[0])
let res2 = await phrasex.getWildcardsAndMatch("What is a coconut", "", userData)
console.log(res2[0])
})
compute()
```
with result
```javascript
{
source: {
exampleWildcards: { value: [Array], ans: [Array] },
phrase: 'where is (value)',
response: [ '(value) is in (ans)' ],
phraseType: 'whereIsThing',
implies: [ 'whereIsThing', 'whereIsThing' ],
meta: { groupInex: 6 },
example: 'where is Seattle',
storage: null,
words: 'where is'
},
wildcards: { matched: true, value: 'Boston' },
confidence: 1,
wcScore: { score: 1, count: 1 },
score: {
queryIndex: [ [Object], [Object], [Object] ],
score: 2,
order: 1,
size: 0.5,
semantic: 0.580953918415687
}
}
{
source: {
exampleWildcards: { value: [Array], ans: [Array] },
phrase: 'What is a (value)?',
response: [ 'A (value) is an (ans).' ],
phraseType: 'whatIsThing',
implies: [ 'whatIsThing', 'whatIsThing' ],
meta: { groupInex: 4 },
example: 'What is a pig',
storage: null,
words: 'What is a ?'
},
wildcards: { matched: true, value: 'coconut' },
confidence: 1,
wcScore: { score: 1, count: 1 },
score: {
queryIndex: [ [Object], [Object], [Object], [Object] ],
score: 3,
order: 1,
size: 0.3333333333333333,
semantic: 0.5399621217119523
}
}
```
The result is a variation of the original database source plus the wildcards for filling
in the data.