https://github.com/instill-ai/instill-core
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
https://github.com/instill-ai/instill-core
ai api cli developer-tools etl generative-ai golang gpt hacktoberfest llm low-code no-code open-source pipeline python stable-diffusion typescript unstructured-data
Last synced: 15 days ago
JSON representation
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
- Host: GitHub
- URL: https://github.com/instill-ai/instill-core
- Owner: instill-ai
- License: other
- Created: 2022-01-13T13:27:40.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-12-03T09:50:05.000Z (5 months ago)
- Last Synced: 2024-12-06T21:28:34.776Z (5 months ago)
- Topics: ai, api, cli, developer-tools, etl, generative-ai, golang, gpt, hacktoberfest, llm, low-code, no-code, open-source, pipeline, python, stable-diffusion, typescript, unstructured-data
- Language: Makefile
- Homepage: https://www.instill.tech
- Size: 9.52 MB
- Stars: 2,174
- Watchers: 30
- Forks: 109
- Open Issues: 31
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
- StarryDivineSky - instill-ai/instill-core - core 存储库进行自托管。 (其他_机器学习与深度学习)
- awesome-production-machine-learning - Instill VDP - ai/instill-core.svg?style=social) - Instill VDP (Versatile Data Pipeline) aims to streamline the data processing pipelines from inception to completion. (Data Pipeline)
README
# 🔮 **Instill Core**
[](https://github.com/instill-ai/instill-core/actions/workflows/integration-test-latest.yml?branch=main&event=push)
[](https://github.com/instill-ai/instill-core/releases)
[](https://artifacthub.io/packages/helm/instill-ai/core)
[](https://discord.gg/sevxWsqpGh)
[](#contributors-)A complete unstructured data solution: ETL processing, AI-readiness, open-source LLM hosting, and RAG capabilities in one powerful platform.
## Quick start
Follow the [installation](#installation) steps below or [documentation](https://www.instill-ai.dev/docs/latest/core/deployment/?utm_source=github&utm_medium=readme&utm_campaign=org_readme) for more details to build versatile AI applications locally.
## What is Instill Core?
[**Instill Core**](https://github.com/instill-ai/instill-core) is an end-to-end AI platform for data, pipeline and model orchestration.
🔮 **Instill Core** simplifies infrastructure hassle and encompasses these core features:
- 💧 [**Pipeline**](https://www.instill-ai.dev/docs/pipeline/introduction/?utm_source=github&utm_medium=readme&utm_campaign=org_readme): Quickly build versatile AI-first APIs or automated workflows.
- ⚗️ [**Model**](https://www.instill-ai.dev/docs/model/introduction/?utm_source=github&utm_medium=readme&utm_campaign=org_readme): Deploy and monitor AI models without GPU infrastructure hassles.
- 💾 [**Artifact**](https://www.instill-ai.dev/docs/artifact/introduction/?utm_source=github&utm_medium=readme&utm_campaign=org_readme): Transform unstructured data (e.g., documents, images, audio, video) into AI-ready formats.
- ⚙️ [**Component**](https://www.instill-ai.dev/docs/component/introduction/?utm_source=github&utm_medium=readme&utm_campaign=org_readme): Connect essential building blocks to construct powerful pipelines.## What can you build?
- 📖 Parsing PDF Files to Markdown: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Parsing_pdf_files.ipynb)
- 🧱 Generating Structured Outputs from LLMs: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Generating_structured_outputs.ipynb) & [Tutorial](https://www.instill-ai.com/blog/llm-structured-outputs)
- 🕸️ Web scraping & Google Search with Structured Insights
- 🌱 Instance segmentation on microscopic plant stomata images: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Instance_segmentation_stomavision.ipynb)See [Examples](https://www.instill-ai.dev/docs/examples) for more!
## Installation
### Prerequisites
| Operating System | Requirements and Instructions |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| macOS or Linux | Instill Core works natively |
| Windows | • Use Windows Subsystem for Linux (WSL2)
• Install latest `yq` from [GitHub Repository](https://github.com/mikefarah/yq)
• Install latest Docker Desktop and enable WSL2 integration ([tutorial](https://docs.docker.com/desktop/wsl))
• (Optional) Install `cuda-toolkit` on WSL2 ([NVIDIA tutorial](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl-2)) |
| All Systems | • Docker Engine v25 or later
• Docker Compose v2 or later
• Install latest stable [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) |### Steps
#### Use stable release version
Execute the following commands to pull pre-built images with all the dependencies to launch:
```bash
$ git clone -b v0.50.2-beta https://github.com/instill-ai/instill-core.git && cd instill-core# Launch all services
$ make all
```> [!NOTE]
> We have restructured our project repositories. If you need to access **🔮 Instill Core** projects up to version `v0.13.0-beta`, please refer to the [instill-ai/deprecated-core](https://github.com/instill-ai/deprecated-core) repository.#### Use the latest version for local development
Execute the following commands to build images with all the dependencies to launch:
```bash
$ git clone https://github.com/instill-ai/instill-core.git && cd instill-core# Launch all services
$ make latest PROFILE=all
```> [!IMPORTANT]
> Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use [latest release](https://github.com/instill-ai/instill-core/releases).🚀 That's it! Once all the services are up with health status, the UI is ready to go at . Please find the default login credentials in the [documentation](https://www.instill-ai.dev/docs/latest/quickstart#self-hosted-instill-core).
> [!NOTE]
> If you change to a different version of **Instill Core**, you will need to re-build the docker images by running `make build-latest` instead of `make all`.To shut down all running services:
```shell
make down
```## Deployment
Visit the [Deployment Overview](https://www.instill-ai.dev/docs/latest/core/deployment/?utm_source=github&utm_medium=readme&utm_campaign=org_readme) for more details.
## Client Access
- 📺 Console
- ⌨️ CLI
- 📦 SDK:
- [Python SDK](https://github.com/instill-ai/python-sdk)
- [TypeScript SDK](https://github.com/instill-ai/typescript-sdk)
- Stay tuned, as more SDKs are on the way!## Documentation
Please visit our [official documentation](https://www.instill-ai.dev/docs?utm_source=github&utm_medium=link&utm_campaign=instill-core) for more.
Additional resources:
- [API Reference](https://openapi.instill.tech)
- [Cookbooks](https://github.com/instill-ai/cookbook)
- [Tutorials](https://www.instill-ai.dev/blog)
- [Examples](https://www.instill-ai.dev/docs/examples)## Contributing
We welcome contributions from our community! Checkout the methods below:
1. **Cookbooks**: Help us create helpful pipelines and guides for the community. Visit our [Cookbook repository](https://github.com/instill-ai/cookbook) to get started.
2. **Issues**: Contribute to improvements by raising tickets using [templates here](https://github.com/instill-ai/instill-core/issues/new/choose) or discuss in existing ones you think you can help with.
### Community Standards
We are committed to maintaining a respectful and welcoming atmosphere for all contributors. Before contributing, please read:
- [Contributing Guidelines](./.github/CONTRIBUTING.md)
- [Code of Conduct](https://github.com/instill-ai/.github/blob/main/.github/CODE_OF_CONDUCT.md)### Support
Get help by joining our [Discord community](https://discord.gg/sevxWsqpGh) where you can post any questions on our `#ask-for-help` channel.
## Contributors ✨
Thank you to all these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
Vibhor Bhatt
Miguel Ortiz
Sajda Kabir
Henry Chen
Hari Bhandari
Shiva Gaire
Zubeen
ShihChun-H
Ikko Eltociear Ashimine
Farookh Zaheer Siddiqui
Brian Gallagher
hairyputtar
David Marx
Deniz Parlak
Po-Yu Chen
Po Chun Chiu
Sarthak
HR Wu
phelan
Chang, Hui-Tang
Xiaofei Du
Ping-Lin Chang
Tony Wang
Pratik date
Juan Vallés
Naman Anand
totuslink
Praharsh Jain
Utsav Paul
CaCaBlocker
Rafael Melo
Jeremy Shih
Romit Mohane
ChunHao
Amelia C
楊竣凱
andre.liang
Zoodane
George Strong
Anni
Mubeen Kodvavi
RCKT
Wojciech Bandzerewicz
Gary
Leo
felixcorleone
Zoe
Daniel
Manul Thanura
Akash Jana
Anish0203
Prathamesh Tugaonkar
Shubham
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
## License
See the [LICENSE](./LICENSE) file for licensing information.
