https://github.com/nlpie/nlp-adapt-kube

Kubernetes cluster that supports NLP workflow to generate and interrogate artifacts from various annotator systems.
https://github.com/nlpie/nlp-adapt-kube

Last synced: 3 days ago
JSON representation

Kubernetes cluster that supports NLP workflow to generate and interrogate artifacts from various annotator systems.

• Host: GitHub
• URL: https://github.com/nlpie/nlp-adapt-kube
• Owner: nlpie
• License: apache-2.0
• Created: 2018-08-01T16:38:59.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2022-01-27T16:16:56.000Z (almost 3 years ago)
• Last Synced: 2024-04-16T07:14:37.036Z (7 months ago)
• Language: XSLT
• Size: 61.2 MB
• Stars: 0
• Watchers: 7
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# University of Minnesota NLP-ADAPT (Artifact Discovery and Preparation Toolkit) 

The Artifact Discovery and Preparation Toolkit Kubernetes Cluster (NLP-ADAPT-Kube) is designed to allow researchers to increase the scale of the tools found in the [NLP-ADAPT VM](https://github.com/nlpie/nlp-adapt) for processing larger volumes of data at scale for use in ensembling of multiple annotator engines. 

### Project Goals

Scalability and performance: NLP-ADAPT-Kube will allow researchers to distribute annotator engine processing in parallel across multiple systems. 

Usability: We simplify the process of deploying a kubernetes cluster by utilizing standard utilities, including

- `ansible-playbook` for building local dependencies

- `docker` for building Docker images

- `kubectl` for deployment and monitoring of kubernetes cluster

- `argo` for workflow management of kubernetes cluster

- `kubeadm` for simplification of creating a multi-node kubernetes cluster

- `minikube` for localized testing of kubernetes cluster

### Status of Project

We utilize [`minikube`](https://kubernetes.io/docs/setup/minikube/) for testing as a single node Kubernetes cluster. We also successfully deployed a Kubernetes cluster across multiple nodes with a single master node using [`kubeadm`](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/).

All local devlopment can be done using `minikube` and then pushed to a production cluster. Our production cluster uses barebones native Linux OS access using `calico` as a virtualized overlay network to manage internal requests within cluster.

If you are interested in using this, please submit an issue. We can provide assistance in installation and configuration, as well as providing requisite `Docker` images needed to run this workflow.

This project, including the [Wiki](https://github.com/nlpie/nlp-adapt-kube/wiki) is a work in progress...

### Acknowledgements

Funding for this work was provided by:

* 5U01TR002062-02