https://github.com/eiso/go-engine
This library is a learning project that implements the source{d} engine in Go using Gleam
https://github.com/eiso/go-engine
Last synced: about 1 year ago
JSON representation
This library is a learning project that implements the source{d} engine in Go using Gleam
- Host: GitHub
- URL: https://github.com/eiso/go-engine
- Owner: eiso
- Created: 2018-02-12T18:35:06.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-03-08T08:40:15.000Z (over 8 years ago)
- Last Synced: 2025-04-01T14:21:04.761Z (about 1 year ago)
- Language: Go
- Homepage:
- Size: 35 MB
- Stars: 5
- Watchers: 2
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
go-engine
-------------------
**WARNING: This library is a WIP and is not production ready.**
This library is currently a learning project that implements the [source{d} engine](https://github.com/src-d/engine) in Go using [Gleam](https://github.com/chrislusf/gleam/) (Go implementation of MapReduce).
The heavy lifting of working with Git repositories is done by [go-git](https://github.com/src-d/go-git).
### To-do
- [ ] Split remotes properly in the repositories reader; for siva into seperate repos
- [ ] Implement the [queries from QuerySetApp](https://github.com/mcarmonaa/QuerySetApp/blob/master/src/main/scala/tech/sourced/queryset/SourcedQueries.scala#L26)
- [ ] Generalize the filter function
- [ ] Improve the siva reading to turn rooted repositories into individual ones
- [ ] Add a Babelfish deployment to k8s
- [ ] UDF's:
- [x] `readBlob` read the content of a blob based on its hash
- [x] `classifyLanguage` implements [enry](https://github.com/src-d/enry) to classify the programming language of the blobs content
- [ ] `extractUAST` parses a blob using [Babelfish](https://doc.bblf.sh/)
### Future ideas:
- [ ] Research how to add named columns to Gleam
- [ ] Update gleam to use both inner & outer IP's so that binaries can be sent to agents from any IP
- [ ] Research on adding bitmap reader to go-git:
- https://kscherer.github.io/git/2015/05/15/git-and-bitmaps
- https://githubengineering.com/counting-objects/