Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/instill-ai/instill-core

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
https://github.com/instill-ai/instill-core

ai api cli developer-tools etl generative-ai golang gpt hacktoberfest llm low-code no-code open-source pipeline python stable-diffusion typescript unstructured-data

Last synced: 2 days ago
JSON representation

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

Awesome Lists containing this project

README

        

# 🔮 **Instill Core**

[![Integration Test](https://img.shields.io/github/actions/workflow/status/instill-ai/instill-core/integration-test-latest.yml?branch=main&label=Integration%20Test&logoColor=fff&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMTYiIGhlaWdodD0iMTYiIHZpZXdCb3g9IjAgMCAxNiAxNiIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZmlsbC1ydWxlPSJldmVub2RkIiBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik0wIDEuNzVDMCAwLjc4NCAwLjc4NCAwIDEuNzUgMEg1LjI1QzYuMjE2IDAgNyAwLjc4NCA3IDEuNzVWNS4yNUM3IDUuNzE0MTMgNi44MTU2MyA2LjE1OTI1IDYuNDg3NDQgNi40ODc0NEM2LjE1OTI1IDYuODE1NjMgNS43MTQxMyA3IDUuMjUgN0g0VjExQzQgMTEuMjY1MiA0LjEwNTM2IDExLjUxOTYgNC4yOTI4OSAxMS43MDcxQzQuNDgwNDMgMTEuODk0NiA0LjczNDc4IDEyIDUgMTJIOVYxMC43NUM5IDkuNzg0IDkuNzg0IDkgMTAuNzUgOUgxNC4yNUMxNS4yMTYgOSAxNiA5Ljc4NCAxNiAxMC43NVYxNC4yNUMxNiAxNC43MTQxIDE1LjgxNTYgMTUuMTU5MiAxNS40ODc0IDE1LjQ4NzRDMTUuMTU5MiAxNS44MTU2IDE0LjcxNDEgMTYgMTQuMjUgMTZIMTAuNzVDMTAuMjg1OSAxNiA5Ljg0MDc1IDE1LjgxNTYgOS41MTI1NiAxNS40ODc0QzkuMTg0MzcgMTUuMTU5MiA5IDE0LjcxNDEgOSAxNC4yNVYxMy41SDVDNC4zMzY5NiAxMy41IDMuNzAxMDcgMTMuMjM2NiAzLjIzMjIzIDEyLjc2NzhDMi43NjMzOSAxMi4yOTg5IDIuNSAxMS42NjMgMi41IDExVjdIMS43NUMxLjI4NTg3IDcgMC44NDA3NTIgNi44MTU2MyAwLjUxMjU2MyA2LjQ4NzQ0QzAuMTg0Mzc0IDYuMTU5MjUgMCA1LjcxNDEzIDAgNS4yNUwwIDEuNzVaTTEuNzUgMS41QzEuNjgzNyAxLjUgMS42MjAxMSAxLjUyNjM0IDEuNTczMjIgMS41NzMyMkMxLjUyNjM0IDEuNjIwMTEgMS41IDEuNjgzNyAxLjUgMS43NVY1LjI1QzEuNSA1LjM4OCAxLjYxMiA1LjUgMS43NSA1LjVINS4yNUM1LjMxNjMgNS41IDUuMzc5ODkgNS40NzM2NiA1LjQyNjc4IDUuNDI2NzhDNS40NzM2NiA1LjM3OTg5IDUuNSA1LjMxNjMgNS41IDUuMjVWMS43NUM1LjUgMS42ODM3IDUuNDczNjYgMS42MjAxMSA1LjQyNjc4IDEuNTczMjJDNS4zNzk4OSAxLjUyNjM0IDUuMzE2MyAxLjUgNS4yNSAxLjVIMS43NVpNMTAuNzUgMTAuNUMxMC42ODM3IDEwLjUgMTAuNjIwMSAxMC41MjYzIDEwLjU3MzIgMTAuNTczMkMxMC41MjYzIDEwLjYyMDEgMTAuNSAxMC42ODM3IDEwLjUgMTAuNzVWMTQuMjVDMTAuNSAxNC4zODggMTAuNjEyIDE0LjUgMTAuNzUgMTQuNUgxNC4yNUMxNC4zMTYzIDE0LjUgMTQuMzc5OSAxNC40NzM3IDE0LjQyNjggMTQuNDI2OEMxNC40NzM3IDE0LjM3OTkgMTQuNSAxNC4zMTYzIDE0LjUgMTQuMjVWMTAuNzVDMTQuNSAxMC42ODM3IDE0LjQ3MzcgMTAuNjIwMSAxNC40MjY4IDEwLjU3MzJDMTQuMzc5OSAxMC41MjYzIDE0LjMxNjMgMTAuNSAxNC4yNSAxMC41SDEwLjc1WiIgZmlsbD0id2hpdGUiLz4KPC9zdmc+Cg==)](https://github.com/instill-ai/instill-core/actions/workflows/integration-test-latest.yml?branch=main&event=push)
[![GitHub release (latest SemVer including pre-releases)](https://img.shields.io/github/v/release/instill-ai/instill-core?&label=Release&color=blue&include_prereleases&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjQiIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTQgOEg3VjRIMjNWMTdIMjFDMjEgMTguNjYgMTkuNjYgMjAgMTggMjBDMTYuMzQgMjAgMTUgMTguNjYgMTUgMTdIOUM5IDE4LjY2IDcuNjYgMjAgNiAyMEM0LjM0IDIwIDMgMTguNjYgMyAxN0gxVjEyTDQgOFpNMTggMThDMTguNTUgMTggMTkgMTcuNTUgMTkgMTdDMTkgMTYuNDUgMTguNTUgMTYgMTggMTZDMTcuNDUgMTYgMTcgMTYuNDUgMTcgMTdDMTcgMTcuNTUgMTcuNDUgMTggMTggMThaTTQuNSA5LjVMMi41NCAxMkg3VjkuNUg0LjVaTTYgMThDNi41NSAxOCA3IDE3LjU1IDcgMTdDNyAxNi40NSA2LjU1IDE2IDYgMTZDNS40NSAxNiA1IDE2LjQ1IDUgMTdDNSAxNy41NSA1LjQ1IDE4IDYgMThaIiBmaWxsPSJ3aGl0ZSIvPgo8L3N2Zz4K)](https://github.com/instill-ai/instill-core/releases)
[![Artifact Hub](https://img.shields.io/endpoint?labelColor=gray&color=blue&url=https://artifacthub.io/badge/repository/instill-ai)](https://artifacthub.io/packages/helm/instill-ai/core)
[![Discord](https://img.shields.io/discord/928991293856681984?color=blue&label=Discord&logo=discord&logoColor=fff)](https://discord.gg/sevxWsqpGh)
[![All Contributors](https://img.shields.io/badge/all_contributors-31-blue.svg?label=All%20Contributors)](#contributors-)

✨ Make your unstructured data **AI-ready** FAST & build unique versatile **AI-first** applications EASILY. ✨

## Quick start

There are a few ways to start exploring:

1. **[Instill Cloud](https://instill.tech/featured/pipelines/?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: Explore our public cloud powered by [Instill Core](https://github.com/instill-ai/instill-core) with ready-to-use APIs, automations, AI Assistants and more. See the [quickstart](https://www.instill.tech/docs/quickstart#%EF%B8%8F-instill-cloud) on how to get started.
2. **Self-hosting Instill Core**: Follow the [installation](#installation) steps below or [documentation](https://www.instill.tech/docs/latest/core/deployment/?utm_source=github&utm_medium=readme&utm_campaign=org_readme) for more details to build versatile AI applications locally.

## What is Instill Core?

**[Instill Core](https://github.com/instill-ai/instill-core)** is your end-to-end AI platform for data, pipeline and model orchestration.





🔮 Instill Core - The full-stack AI infrastructure tool

**Instill Core** simplifies infrastructure hassle and encompasses these core features:

- **DATA - [Instill Artifact](https://www.instill.tech/docs/artifact/introduction//?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: Transforms unstructured data (documents, images, audio, video) into AI-ready formats.
- **PIPELINE - [Instill VDP](https://www.instill.tech/docs/vdp/introduction//?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: Quickly build any versatile AI-first APIs or automated workflows.
- **MODEL - [Instill Model](https://www.instill.tech/docs/model/introduction//?utm_source=github&utm_medium=readme&utm_campaign=org_readme)**: MLOps/LLMOps platform for model serving and monitoring.

## What can you build?

- 📖 Parsing PDF Files to Markdown: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Parsing_pdf_files.ipynb)
- 🧱 Generating Structured Outputs from LLMs: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Generating_structured_outputs.ipynb) & [Tutorial](https://www.instill.tech/blog/llm-structured-outputs)
- 🕸️ Web scraping & Google Search with Structured Insights: [Tutorial](https://www.instill.tech/blog/mastering-openai-structured-outputs) & [YouTube](https://www.youtube.com/watch?v=x0XsmuCrit8)
- 🌱 Instance segmentation on microscopic plant stomata images: [Cookbook](https://github.com/instill-ai/cookbook/blob/main/examples/Instance_segmentation_stomavision.ipynb)

See [Examples](https://www.instill.tech/docs/examples) for more!

## Installation

### Prerequisites

| Operating System | Requirements and Instructions |
|------------------|--------------------------------|
| macOS or Linux | Instill Core works natively |
| Windows | • Use Windows Subsystem for Linux (WSL2)
• Install latest `yq` from [GitHub Repository](https://github.com/mikefarah/yq)
• Install latest Docker Desktop and enable WSL2 integration ([tutorial](https://docs.docker.com/desktop/wsl))
• (Optional) Install `cuda-toolkit` on WSL2 ([NVIDIA tutorial](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl-2)) |
| All Systems | • Docker Engine v25 or later
• Docker Compose v2 or later
• Install latest stable [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) |

### Steps

**Use stable release version**

Execute the following commands to pull pre-built images with all the dependencies to launch:

```bash
$ git clone -b v0.48.0-beta https://github.com/instill-ai/instill-core.git && cd instill-core

# Launch all services
$ make all
```

> [!NOTE]
> We have restructured our project repositories. If you need to access **🔮 Instill Core** projects up to version `v0.13.0-beta`, please refer to the [instill-ai/deprecated-core](https://github.com/instill-ai/deprecated-core) repository.

**Use the latest version for local development**

Execute the following commands to build images with all the dependencies to launch:

```bash
$ git clone https://github.com/instill-ai/instill-core.git && cd instill-core

# Launch all services
$ make latest PROFILE=all
```

> [!IMPORTANT]
> Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use [latest release](https://github.com/instill-ai/instill-core/releases).

🚀 That's it! Once all the services are up with health status, the UI is ready to go at http://localhost:3000. Please find the default login credentials in the [documentation](https://www.instill.tech/docs/latest/quickstart#self-hosted-instill-core).

To shut down all running services:
```
$ make down
```

Explore the [documentation](https://www.instill.tech/docs/latest/core/deployment) to discover all available deployment options.

## Deployment

You can either self-host **🔮 Instill Core** or choose our fully managed **☁️ Instill Cloud** to get access to all the latest features. Visit the [Deployment Overview](https://www.instill.tech/docs/core/deployment) for more details.





🔮 Instill Core - Deployment

## Client Access

- 📺 Instill Console
- ⌨️ Instill CLI
- 📦 Instill SDK:
- [Python SDK](https://github.com/instill-ai/python-sdk)
- [TypeScript SDK](https://github.com/instill-ai/typescript-sdk)
- Stay tuned, as more SDKs are on the way!

## Documentation

Please visit our [official documentation](https://www.instill.tech/docs?utm_source=github&utm_medium=link&utm_campaign=instill-core) for more.

Additional resources:

- [API Reference](https://openapi.instill.tech)
- [Cookbooks](https://github.com/instill-ai/cookbook)
- [Tutorials](https://www.instill.tech/blog)
- [Examples](https://www.instill.tech/docs/examples)

## Contributing

We welcome contributions from our community! Checkout the methods below:

1. **Cookbooks**: Help us create helpful pipelines and guides for the community. Visit our [Cookbook repository](https://github.com/instill-ai/cookbook) to get started.

2. **Issues**: Contribute to improvements by raising tickets using [templates here](https://github.com/instill-ai/instill-core/issues/new/choose) or discuss in existing ones you think you can help with.

### Community Standards

We are committed to maintaining a respectful and welcoming atmosphere for all contributors. Before contributing, please read:

- [Contributing Guidelines](./.github/CONTRIBUTING.md)
- [Code of Conduct](https://github.com/instill-ai/.github/blob/main/.github/CODE_OF_CONDUCT.md)

### Support

Get help by joining our [Discord community](https://discord.gg/sevxWsqpGh) where you can post any questions on our `#ask-for-help` channel.

## Contributors ✨

Thank you to all these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):




Vibhor Bhatt


Miguel Ortiz


Sajda Kabir


Henry Chen


Hari Bhandari


Shiva Gaire




Zubeen


ShihChun-H


Ikko Eltociear Ashimine


Farookh Zaheer Siddiqui


Brian Gallagher


hairyputtar




David Marx


Deniz Parlak


Po-Yu Chen


Po Chun Chiu


Sarthak


HR Wu




phelan


Chang, Hui-Tang


Xiaofei Du


Ping-Lin Chang


Tony Wang


Pratik date




Juan Vallés


Naman Anand


totuslink


Praharsh Jain


Utsav Paul


CaCaBlocker




Rafael Melo


Jeremy Shih


Romit Mohane


ChunHao


Amelia C


楊竣凱




andre.liang


Zoodane


George Strong


Anni


Mubeen Kodvavi


RCKT




Wojciech Bandzerewicz

Gary

Leo

felixcorleone

Zoe

Daniel




Manul Thanura


Akash Jana


Anish0203


Prathamesh Tugaonkar


Shubham


This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!

## License

See the [LICENSE](./LICENSE) file for licensing information.