Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/instill-ai/instill-core
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
https://github.com/instill-ai/instill-core
ai api cli developer-tools etl generative-ai golang gpt hacktoberfest llm low-code no-code open-source pipeline python stable-diffusion typescript unstructured-data
Last synced: 2 days ago
JSON representation
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
- Host: GitHub
- URL: https://github.com/instill-ai/instill-core
- Owner: instill-ai
- License: other
- Created: 2022-01-13T13:27:40.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-12-03T09:50:05.000Z (about 1 month ago)
- Last Synced: 2024-12-06T21:28:34.776Z (about 1 month ago)
- Topics: ai, api, cli, developer-tools, etl, generative-ai, golang, gpt, hacktoberfest, llm, low-code, no-code, open-source, pipeline, python, stable-diffusion, typescript, unstructured-data
- Language: Makefile
- Homepage: https://www.instill.tech
- Size: 9.52 MB
- Stars: 2,174
- Watchers: 30
- Forks: 109
- Open Issues: 31
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
- StarryDivineSky - instill-ai/instill-core - core 存储库进行自托管。 (其他_机器学习与深度学习)
- awesome-production-machine-learning - Instill VDP - ai/instill-core.svg?style=social) - Instill VDP (Versatile Data Pipeline) aims to streamline the data processing pipelines from inception to completion. (Data Pipeline)
README
# 🔮 **Instill Core**
[![Integration Test](https://img.shields.io/github/actions/workflow/status/instill-ai/instill-core/integration-test-latest.yml?branch=main&label=Integration%20Test&logoColor=fff&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMTYiIGhlaWdodD0iMTYiIHZpZXdCb3g9IjAgMCAxNiAxNiIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZmlsbC1ydWxlPSJldmVub2RkIiBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik0wIDEuNzVDMCAwLjc4NCAwLjc4NCAwIDEuNzUgMEg1LjI1QzYuMjE2IDAgNyAwLjc4NCA3IDEuNzVWNS4yNUM3IDUuNzE0MTMgNi44MTU2MyA2LjE1OTI1IDYuNDg3NDQgNi40ODc0NEM2LjE1OTI1IDYuODE1NjMgNS43MTQxMyA3IDUuMjUgN0g0VjExQzQgMTEuMjY1MiA0LjEwNTM2IDExLjUxOTYgNC4yOTI4OSAxMS43MDcxQzQuNDgwNDMgMTEuODk0NiA0LjczNDc4IDEyIDUgMTJIOVYxMC43NUM5IDkuNzg0IDkuNzg0IDkgMTAuNzUgOUgxNC4yNUMxNS4yMTYgOSAxNiA5Ljc4NCAxNiAxMC43NVYxNC4yNUMxNiAxNC43MTQxIDE1LjgxNTYgMTUuMTU5MiAxNS40ODc0IDE1LjQ4NzRDMTUuMTU5MiAxNS44MTU2IDE0LjcxNDEgMTYgMTQuMjUgMTZIMTAuNzVDMTAuMjg1OSAxNiA5Ljg0MDc1IDE1LjgxNTYgOS41MTI1NiAxNS40ODc0QzkuMTg0MzcgMTUuMTU5MiA5IDE0LjcxNDEgOSAxNC4yNVYxMy41SDVDNC4zMzY5NiAxMy41IDMuNzAxMDcgMTMuMjM2NiAzLjIzMjIzIDEyLjc2NzhDMi43NjMzOSAxMi4yOTg5IDIuNSAxMS42NjMgMi41IDExVjdIMS43NUMxLjI4NTg3IDcgMC44NDA3NTIgNi44MTU2MyAwLjUxMjU2MyA2LjQ4NzQ0QzAuMTg0Mzc0IDYuMTU5MjUgMCA1LjcxNDEzIDAgNS4yNUwwIDEuNzVaTTEuNzUgMS41QzEuNjgzNyAxLjUgMS42MjAxMSAxLjUyNjM0IDEuNTczMjIgMS41NzMyMkMxLjUyNjM0IDEuNjIwMTEgMS41IDEuNjgzNyAxLjUgMS43NVY1LjI1QzEuNSA1LjM4OCAxLjYxMiA1LjUgMS43NSA1LjVINS4yNUM1LjMxNjMgNS41IDUuMzc5ODkgNS40NzM2NiA1LjQyNjc4IDUuNDI2NzhDNS40NzM2NiA1LjM3OTg5IDUuNSA1LjMxNjMgNS41IDUuMjVWMS43NUM1LjUgMS42ODM3IDUuNDczNjYgMS42MjAxMSA1LjQyNjc4IDEuNTczMjJDNS4zNzk4OSAxLjUyNjM0IDUuMzE2MyAxLjUgNS4yNSAxLjVIMS43NVpNMTAuNzUgMTAuNUMxMC42ODM3IDEwLjUgMTAuNjIwMSAxMC41MjYzIDEwLjU3MzIgMTAuNTczMkMxMC41MjYzIDEwLjYyMDEgMTAuNSAxMC42ODM3IDEwLjUgMTAuNzVWMTQuMjVDMTAuNSAxNC4zODggMTAuNjEyIDE0LjUgMTAuNzUgMTQuNUgxNC4yNUMxNC4zMTYzIDE0LjUgMTQuMzc5OSAxNC40NzM3IDE0LjQyNjggMTQuNDI2OEMxNC40NzM3IDE0LjM3OTkgMTQuNSAxNC4zMTYzIDE0LjUgMTQuMjVWMTAuNzVDMTQuNSAxMC42ODM3IDE0LjQ3MzcgMTAuNjIwMSAxNC40MjY4IDEwLjU3MzJDMTQuMzc5OSAxMC41MjYzIDE0LjMxNjMgMTAuNSAxNC4yNSAxMC41SDEwLjc1WiIgZmlsbD0id2hpdGUiLz4KPC9zdmc+Cg==)](https://github.com/instill-ai/instill-core/actions/workflows/integration-test-latest.yml?branch=main&event=push)
[![GitHub release (latest SemVer including pre-releases)](https://img.shields.io/github/v/release/instill-ai/instill-core?&label=Release&color=blue&include_prereleases&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjQiIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTQgOEg3VjRIMjNWMTdIMjFDMjEgMTguNjYgMTkuNjYgMjAgMTggMjBDMTYuMzQgMjAgMTUgMTguNjYgMTUgMTdIOUM5IDE4LjY2IDcuNjYgMjAgNiAyMEM0LjM0IDIwIDMgMTguNjYgMyAxN0gxVjEyTDQgOFpNMTggMThDMTguNTUgMTggMTkgMTcuNTUgMTkgMTdDMTkgMTYuNDUgMTguNTUgMTYgMTggMTZDMTcuNDUgMTYgMTcgMTYuNDUgMTcgMTdDMTcgMTcuNTUgMTcuNDUgMTggMTggMThaTTQuNSA5LjVMMi41NCAxMkg3VjkuNUg0LjVaTTYgMThDNi41NSAxOCA3IDE3LjU1IDcgMTdDNyAxNi40NSA2LjU1IDE2IDYgMTZDNS40NSAxNiA1IDE2LjQ1IDUgMTdDNSAxNy41NSA1LjQ1IDE4IDYgMThaIiBmaWxsPSJ3aGl0ZSIvPgo8L3N2Zz4K)](https://github.com/instill-ai/instill-core/releases)
[![Artifact Hub](https://img.shields.io/endpoint?labelColor=gray&color=blue&url=https://artifacthub.io/badge/repository/instill-ai)](https://artifacthub.io/packages/helm/instill-ai/core)
[![Discord](https://img.shields.io/discord/928991293856681984?color=blue&label=Discord&logo=discord&logoColor=fff)](https://discord.gg/sevxWsqpGh)
[![All Contributors](https://img.shields.io/badge/all_contributors-31-blue.svg?label=All%20Contributors)](#contributors-)✨ Make your unstructured data **AI-ready** FAST & build unique versatile **AI-first** applications EASILY. ✨
## Quick start
There are a few ways to start exploring:
1. **[Instill Cloud](https://instill.tech/featured/pipelines/?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: Explore our public cloud powered by [Instill Core](https://github.com/instill-ai/instill-core) with ready-to-use APIs, automations, AI Assistants and more. See the [quickstart](https://www.instill.tech/docs/quickstart#%EF%B8%8F-instill-cloud) on how to get started.
2. **Self-hosting Instill Core**: Follow the [installation](#installation) steps below or [documentation](https://www.instill.tech/docs/latest/core/deployment/?utm_source=github&utm_medium=readme&utm_campaign=org_readme) for more details to build versatile AI applications locally.## What is Instill Core?
**[Instill Core](https://github.com/instill-ai/instill-core)** is your end-to-end AI platform for data, pipeline and model orchestration.
**Instill Core** simplifies infrastructure hassle and encompasses these core features:
- **DATA - [Instill Artifact](https://www.instill.tech/docs/artifact/introduction//?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: Transforms unstructured data (documents, images, audio, video) into AI-ready formats.
- **PIPELINE - [Instill VDP](https://www.instill.tech/docs/vdp/introduction//?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: Quickly build any versatile AI-first APIs or automated workflows.
- **MODEL - [Instill Model](https://www.instill.tech/docs/model/introduction//?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: MLOps/LLMOps platform for model serving and monitoring.## What can you build?
- 📖 Parsing PDF Files to Markdown: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Parsing_pdf_files.ipynb)
- 🧱 Generating Structured Outputs from LLMs: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Generating_structured_outputs.ipynb) & [Tutorial](https://www.instill.tech/blog/llm-structured-outputs)
- 🕸️ Web scraping & Google Search with Structured Insights: [Tutorial](https://www.instill.tech/blog/mastering-openai-structured-outputs) & [YouTube](https://www.youtube.com/watch?v=x0XsmuCrit8)
- 🌱 Instance segmentation on microscopic plant stomata images: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Instance_segmentation_stomavision.ipynb)See [Examples](https://www.instill.tech/docs/examples) for more!
## Installation
### Prerequisites
| Operating System | Requirements and Instructions |
|------------------|--------------------------------|
| macOS or Linux | Instill Core works natively |
| Windows | • Use Windows Subsystem for Linux (WSL2)
• Install latest `yq` from [GitHub Repository](https://github.com/mikefarah/yq)
• Install latest Docker Desktop and enable WSL2 integration ([tutorial](https://docs.docker.com/desktop/wsl))
• (Optional) Install `cuda-toolkit` on WSL2 ([NVIDIA tutorial](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl-2)) |
| All Systems | • Docker Engine v25 or later
• Docker Compose v2 or later
• Install latest stable [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) |### Steps
**Use stable release version**
Execute the following commands to pull pre-built images with all the dependencies to launch:
```bash
$ git clone -b v0.48.0-beta https://github.com/instill-ai/instill-core.git && cd instill-core# Launch all services
$ make all
```> [!NOTE]
> We have restructured our project repositories. If you need to access **🔮 Instill Core** projects up to version `v0.13.0-beta`, please refer to the [instill-ai/deprecated-core](https://github.com/instill-ai/deprecated-core) repository.**Use the latest version for local development**
Execute the following commands to build images with all the dependencies to launch:
```bash
$ git clone https://github.com/instill-ai/instill-core.git && cd instill-core# Launch all services
$ make latest PROFILE=all
```> [!IMPORTANT]
> Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use [latest release](https://github.com/instill-ai/instill-core/releases).🚀 That's it! Once all the services are up with health status, the UI is ready to go at http://localhost:3000. Please find the default login credentials in the [documentation](https://www.instill.tech/docs/latest/quickstart#self-hosted-instill-core).
To shut down all running services:
```
$ make down
```Explore the [documentation](https://www.instill.tech/docs/latest/core/deployment) to discover all available deployment options.
## Deployment
You can either self-host **🔮 Instill Core** or choose our fully managed **☁️ Instill Cloud** to get access to all the latest features. Visit the [Deployment Overview](https://www.instill.tech/docs/core/deployment) for more details.
## Client Access
- 📺 Instill Console
- ⌨️ Instill CLI
- 📦 Instill SDK:
- [Python SDK](https://github.com/instill-ai/python-sdk)
- [TypeScript SDK](https://github.com/instill-ai/typescript-sdk)
- Stay tuned, as more SDKs are on the way!## Documentation
Please visit our [official documentation](https://www.instill.tech/docs?utm_source=github&utm_medium=link&utm_campaign=instill-core) for more.
Additional resources:
- [API Reference](https://openapi.instill.tech)
- [Cookbooks](https://github.com/instill-ai/cookbook)
- [Tutorials](https://www.instill.tech/blog)
- [Examples](https://www.instill.tech/docs/examples)## Contributing
We welcome contributions from our community! Checkout the methods below:
1. **Cookbooks**: Help us create helpful pipelines and guides for the community. Visit our [Cookbook repository](https://github.com/instill-ai/cookbook) to get started.
2. **Issues**: Contribute to improvements by raising tickets using [templates here](https://github.com/instill-ai/instill-core/issues/new/choose) or discuss in existing ones you think you can help with.
### Community Standards
We are committed to maintaining a respectful and welcoming atmosphere for all contributors. Before contributing, please read:
- [Contributing Guidelines](./.github/CONTRIBUTING.md)
- [Code of Conduct](https://github.com/instill-ai/.github/blob/main/.github/CODE_OF_CONDUCT.md)### Support
Get help by joining our [Discord community](https://discord.gg/sevxWsqpGh) where you can post any questions on our `#ask-for-help` channel.
## Contributors ✨
Thank you to all these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
Vibhor Bhatt
Miguel Ortiz
Sajda Kabir
Henry Chen
Hari Bhandari
Shiva Gaire
Zubeen
ShihChun-H
Ikko Eltociear Ashimine
Farookh Zaheer Siddiqui
Brian Gallagher
hairyputtar
David Marx
Deniz Parlak
Po-Yu Chen
Po Chun Chiu
Sarthak
HR Wu
phelan
Chang, Hui-Tang
Xiaofei Du
Ping-Lin Chang
Tony Wang
Pratik date
Juan Vallés
Naman Anand
totuslink
Praharsh Jain
Utsav Paul
CaCaBlocker
Rafael Melo
Jeremy Shih
Romit Mohane
ChunHao
Amelia C
楊竣凱
andre.liang
Zoodane
George Strong
Anni
Mubeen Kodvavi
RCKT
Wojciech Bandzerewicz
Gary
Leo
felixcorleone
Zoe
Daniel
Manul Thanura
Akash Jana
Anish0203
Prathamesh Tugaonkar
Shubham
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
## License
See the [LICENSE](./LICENSE) file for licensing information.