{"id":13640359,"url":"https://github.com/ForestFlow/ForestFlow","last_synced_at":"2025-04-20T02:33:21.291Z","repository":{"id":43852075,"uuid":"214001313","full_name":"ForestFlow/ForestFlow","owner":"ForestFlow","description":"ForestFlow is a policy-driven Machine Learning Model Server. It is an LF AI Foundation incubation project.","archived":false,"fork":false,"pushed_at":"2024-02-21T00:11:21.000Z","size":838,"stargazers_count":72,"open_issues_count":6,"forks_count":18,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-09T10:39:05.013Z","etag":null,"topics":["h2o","machine-learning","model-deployment"],"latest_commit_sha":null,"homepage":"https://forestflow.ai/","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ForestFlow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-09T19:03:58.000Z","updated_at":"2024-09-29T17:03:43.000Z","dependencies_parsed_at":"2024-03-05T09:00:23.107Z","dependency_job_id":null,"html_url":"https://github.com/ForestFlow/ForestFlow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForestFlow%2FForestFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForestFlow%2FForestFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForestFlow%2FForestFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForestFlow%2FForestFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ForestFlow","download_url":"https://codeload.github.com/ForestFlow/ForestFlow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249840693,"owners_count":21332925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["h2o","machine-learning","model-deployment"],"created_at":"2024-08-02T01:01:10.395Z","updated_at":"2025-04-20T02:33:20.840Z","avatar_url":"https://github.com/ForestFlow.png","language":"Scala","funding_links":[],"categories":["Model Serving and Monitoring","Frameworks/Servers for Serving","人工智能"],"sub_categories":["模型训练"],"readme":"\u003c!--\n    Copyright 2020 DreamWorks Animation L.L.C.\n    Licensed under the Apache License, Version 2.0 (the \"License\");\n    you may not use this file except in compliance with the License.\n    You may obtain a copy of the License at\n    http://www.apache.org/licenses/LICENSE-2.0\n    Unless required by applicable law or agreed to in writing, software\n    distributed under the License is distributed on an \"AS IS\" BASIS,\n    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n    See the License for the specific language governing permissions and\n    limitations under the License.\n--\u003e\n[![ForestFlow](docs/static/forestflow_logo_text.png)](https://dreamworksanimation.github.io/ForestFlow/)\n\n\nForestFlow is a scalable policy-based cloud-native machine learning model server. ForestFlow strives to strike a balance between the flexibility it offers data scientists and the adoption of standards while reducing friction between Data Science, Engineering and Operations teams.\n\nForestFlow is policy-based because we believe automation for Machine Learning/Deep Learning operations is critical to scaling human resources. ForestFlow lends itself well to workflows based on automatic retraining, version control, A/B testing, Canary Model deployments, Shadow testing, automatic time or performance-based model deprecation and time or performance-based model routing in real-time.\n\nOur aim with ForestFlow is to provide data scientists a simple means to deploy models to a production system with minimal friction accelerating the development to production value proposition.\n\nTo achieve these goals, ForestFlow looks to address the proliferation of model serving formats and standards for inference API specifications by adopting, what we believe, are currently, or are becoming widely adopted open source frameworks, formats, and API specifications. We do this in a pluggable format such that we can continue to evolve ForestFlow as the industry and space matures and we see a need for additional support.\n\n\n\n# Contents\n\n  - [Overview](./docs/overview)\n      - [Why ForestFlow?](./docs/overview.md#why-forestflow)\n      - [Model Deployment](./docs/overview.md#model-deployment)\n      - [Inference](./docs/overview.md#inference)\n      - [Currently Supported model formats](./docs/overview.md#currently-supported-model-formats)\n  - [Quick Start Guide](./docs/quickstart.md)\n  - [Building and Configuration](./docs/buildconfig.md)\n      - [Building ForestFlow](./docs/buildconfig.md#building-forestflow)\n         - [Create JAR without Kubernetes dependencies](./docs/buildconfig.md#create-jar-without-kubernetes-dependencies)\n         - [Create a JAR with Kubernetes dependencies](./docs/buildconfig.md#create-a-jar-with-kubernetes-dependencies)\n      - [ForestFlow Configuration](./docs/buildconfig.md#forestflow-configuration)\n      - [Creating an OCI-compliant Image](./docs/buildconfig.md#creating-an-oci-compliant-image)\n  - [Inference](./docs/inference.md)\n      - [Servable implementation interface support matrix](./docs/inference.md#servable-implementation-interface-support-matrix)\n      - [Inference - Using the BASIC REST API](./docs/inference.md#using-the-basic-rest-api)\n      - [Inference - Using the GraphPipe API](./docs/inference.md#using-the-graphpipe-api)\n  - [Concepts](./docs/concepts.md)\n      - [A ForestFlow Cluster](./docs/concepts.md#a-forestflow-cluster)\n          - [Servable](./docs/concepts.md#servable)\n          - [Fully Qualified Release Version (FQRV)](./docs/concepts.md#fully-qualified-release-version-fqrv)\n      - [Servable (Model) Deployment](./docs/concepts.md#servable-model-deployment)\n          - [Creating a Contract](./docs/concepts.md#creating-a-contract)\n          - [Creating a Servable](./docs/concepts.md#creating-a-servable)\n  - [Contributing to ForestFlow](#contributing)\n\n# Overview\n## Why ForestFlow?\nContinuous deployment and lifecycle management of Machine Learning/Deep Learning models is currently widely accepted as a primary bottleneck for gaining value out of ML projects.\n\nWe first set out to find a solution to deploy our own models. The model server implementations we found were either proprietary, closed-source solutions or had too many limitations in what we wanted to achieve.\nThe main concerns for creating ForestFlow can be summarized as:\n   - We wanted to reduce friction between our data science, engineering and operations teams\n   - We wanted to give data scientists the flexibility to use the tools they wanted (H2O, TensorFlow, Spark export to PFA etc..)\n   - We wanted to automate certain lifecycle management aspects of model deployments like automatic performance or time-based routing and retirement of stale models\n   - We wanted a model server that allows easy A/B testing, Shadow (listen only) deployments and and Canary deployments. This allows our Data Scientists to experiment with real production data without impacting production and using the same tooling they would when deployment to production. \n   - We wanted something that was easy to deploy and scale for different deployment scenarios (on-prem local data center single instance, cluster of instances, Kubernetes managed, Cloud native etc..)\n   - We wanted the ability to treat inference requests as a stream and log predictions as a stream. This allows us to test new models against a stream of older infer requests.\n   - We wanted to avoid the \"super-hero\" data scientist that knows how to dockerize an application, apply the science, build an API and deploy to production. This does not scale well and is difficult to support and maintain.\n   - Most of all, we wanted repeatability. We didn't want to re-invent the wheel once we had support for a specific framework. \n\n## Model Deployment\nFor model deployment, ForestFlow supports models described via [MLfLow Model](https://mlflow.org/docs/latest/models.html) format which allows for different flavors i..e, frameworks \u0026 storage formats.\n\nForestFlow also supports a BASIC REST API for model deployment as well that mimics the MLflow Model format but does not require it.\n\n## Inference\nFor inference, we’ve adopted a similar approach. ForestFlow provides 2 interfaces for maximum flexibility; \na [BASIC REST API](./docs/inference.md#using-the-basic-rest-api) in addition to \nstandardizing on the [GraphPipe](https://oracle.github.io/graphpipe) \n[API specification](https://oracle.github.io/graphpipe/#/guide/user-guide/spec).\n\nRelying on standards, for example using GraphPipe’s specification means immediate availability of client libraries in a variety of languages that already support working with ForestFlow; see [GraphPipe clients](https://oracle.github.io/graphpipe/#/guide/clients/overview).\n\nPlease visit the [quickstart guide](./docs/quickstart.md) to get a quick overview of setting up ForestFlow and an example on inference.\nAlso please visit the [Inference](./docs/inference.md) documentation for a deeper dive. \n\n## Currently Supported model formats\n - H2O - Mojo Model\n - TensorFlow \u0026 Keras - Planned\n - PFA - Planned\n - Spark ML Models and Pipelines via [Aardpfark](https://github.com/CODAIT/aardpfark) and PFA - Planned\n\nGo to the [Quick Start Guide](./docs/quickstart.md) to get started then dive a little deeper and \nlearn about [ForestFlow Concepts](./docs/concepts.md) and how you can \n[tailor](./docs/concepts.md#servable-model-deployment) it to fit your own use-cases.\n\n\n# Contributing\nWhile ForestFlow has already delivered tremendous value for us in production, it's still in early phases of development\nas there are plenty of features we have planned and this continues to evolve at a rapid pace. \nWe appreciate and consistently, make use of and, contribute open source projects back to the community. \nWe realize the problems we're facing aren't unique to us so we welcome feedback, ideas and contributions from the \ncommunity to help develop our roadmap and implementation of ForestFlow.\n\nCheck out [Contribution Guide](./docs/contributing.md) for more details on contributing to ForestFlow.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FForestFlow%2FForestFlow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FForestFlow%2FForestFlow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FForestFlow%2FForestFlow/lists"}