{"id":22848403,"url":"https://github.com/feedzai/fos-core","last_synced_at":"2025-04-30T04:48:59.805Z","repository":{"id":13388950,"uuid":"16077092","full_name":"feedzai/fos-core","owner":"feedzai","description":"Feedzai Open Scoring Server","archived":false,"fork":false,"pushed_at":"2016-03-09T19:22:35.000Z","size":134,"stargazers_count":7,"open_issues_count":1,"forks_count":4,"subscribers_count":32,"default_branch":"master","last_synced_at":"2025-04-30T04:48:54.583Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/feedzai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-01-20T17:05:17.000Z","updated_at":"2023-07-27T11:10:25.000Z","dependencies_parsed_at":"2022-09-26T19:11:12.643Z","dependency_job_id":null,"html_url":"https://github.com/feedzai/fos-core","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedzai%2Ffos-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedzai%2Ffos-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedzai%2Ffos-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedzai%2Ffos-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/feedzai","download_url":"https://codeload.github.com/feedzai/fos-core/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251644827,"owners_count":21620630,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-13T04:12:07.186Z","updated_at":"2025-04-30T04:48:59.771Z","avatar_url":"https://github.com/feedzai.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# feedzai Open Scoring Server (FOS)\n\nFOS is machine learning model training and scoring server in Java.\n\n[![Build Status](https://feedzaios.ci.cloudbees.com/buildStatus/icon?job=fos-core)](https://feedzaios.ci.cloudbees.com/job/fos-core/)\n\n[![CloudbeesDevCloud](http://www.cloudbees.com/sites/default/files/Button-Built-on-CB-1.png)](http://www.cloudbees.com/dev)\n\n\n## Why FOS\n\nThere are pretty good machine learning training and scoring frameworks/libraries out there, but they don't provide the\nfollowing benefits:\n\n1. Common API: fos provides a common abstraction for model attributes, model training and model scoring. Using a [Weka]\nbased classifier will use have exactly the same API as using a R based classifier.\n1. Scoring \u0026 Training as a remote service: Training and scoring can be farmed to dedicated servers in the network\nenabling both vertical and horizontal scaling.\n1. Import and Export models: A model could be trained in a development box and imported seamlessly into a remote server\n1. Scalable and low latency scoring: Marshalling and Unmarshalling scoring requests/responses can be responsible\nfor a significant amount of overhead. Along with the slow RMI based interface, fos also supports scoring using [Kryo].\n\n## Compiling FOS\n\nYou need:\n\n1. [Java SDK]: Java 7\n1. [Maven]: Tested with Maven 3\n1. Access to maven central repo (or a local proxy)\n\nAfter both the [Java SDK] and [Maven] are installed run the following command\n\n`mvn clean install`\n\nThis should compile fos-core, ran all the tests and install all modules into your local maven repo.\n\n# FOS Quickstart\n\n## Running FOS\n\nIn order to start a FOS sever you need to create a bundle that contains both the core components and one or more fos backend implementations.\n\n## Creating a FOS server bundle\n\nTo create a bundle type \n`make package` on the `fos-core` project root. This will:\n\n1. Build fos core\n2. Copy all dependencies listed in `non-core-dependencies.xml` into fos-server lib directory (this includes the weka and R API implementations)\n3. Create a tar.gz bundle with all the necessary code plus shell scripts to bootstrap the process. The file will be available as `fos-server/target/fos-server-bin.tar.gz`. \n\n## Running FOS\n\n1. Untar the server bundle on your install location. Let's assume FOS will be installed in the home dir\n\n```\n$ cd ~\n$ tar xf \u003cgit clone dir\u003e/fos-server/target/fos-server-bin.tar.gz\n```\n\nBy now you should have a `fos-server` on your home.\n\n```\n$ cd fos-server\n$ ~/fos-server $ ls\nbin  conf  lib  LICENSE.txt  log  models  README.txt\n```\n\nInside there are a couple directories:\n\n1. `bin` with startup scripts\n1. `conf` with FOS configuration scripts\n1. `models` where trained models are going to be found\n\nTo start fos, run the bundled startup script `bin/startup.sh`\n\n```\n$ bin/startup.sh\n03-Feb 04:28:31  INFO   com.feedzai.fos.server.Runner                  Starting fos server using configuration from conf/fos.properties\n03-Feb 04:28:31  INFO   com.feedzai.fos.server.Runner                  FOS Server started in 272ms\n```\n\nWe've just started a FOS server with [Weka] support built into the [fos-weka] fos module.\nCurrently FOS can only support an active module per runtime instance. The active server is\nset in the `fos.factoryName` configuration option inside `conf/fos.properties` file:\n\n```\n# the fos implementation to launch\nfos.factoryName=com.feedzai.fos.impl.weka.WekaManagerFactory\n```\n## Training and scoring my first model\n\nWe've prepared a couple [FOS samples]. Check them out\n\n## Implementing a FOS Module\n\n\n### Creating your own manager\n\nfos-core does not provide any concrete implementation. However, a bundled fos server includes both a [fos-weka] (active by default) and a [fos-r] implementation. It is pretty easy to create a new implementation if you want to leverage existing code.\n\n\nYour first step will be to understand [ManagerFactory] interface. A [ManagerFactory] should perform all the necessary boostrapping for a given [Manager] implementation, which must provide implementations for:\n\n1. Model training\n2. Model management (add, removal, import and export)\n3. A [Scorer] implementation \n\nLets dissect a real example:\n\n```java\npublic class WekaManagerFactory implements ManagerFactory {\n\n    @Override\n    public Manager createManager(FosConfig configuration) {\n        WekaManagerConfig wekaManagerConfig = new WekaManagerConfig(configuration);\n        return new WekaManager(wekaManagerConfig);\n    }\n}\n```\n\nThe first step is parse [Manager] specific configuration parameters\n\n```java\n    WekaManagerConfig wekaManagerConfig = new WekaManagerConfig(configuration);\n```\n\nNow that we have specific config, we can create a new [WekaManager]:\n\n```java\n    return new WekaManager(wekaManagerConfig);\n```\n\nMost of the heavy lifting is done by [WekaManager] implementation. Since most of them are Weka specifc, it is not worthwhile to go into implementation details here. The following operations are performed:\n\n1. Search for previously saved models and load their configuration. \n1. Create a `fos-weka` [Scorer] implementation.\n1. Start listening to requests via RMI and [Kryo].\n1. Start a thread pool to allow parallel scoring.\n\n\n### Implementing model training\n\nIn order to implement [model training], you need to supply the [model configuration] and training instances. You can see a practical example in [fos training sample]. \nA model configuration is composed by:\n\n1. A set of key-value properties relevant from each implementation. We recomend to define all configuration options in a dedicated class (see [weka model configuration] for example).\n\n2. An [attribute] list. The number of atributes in the training data must match the configuration attribute list. An attribute can be:\n    1. [Numerical attribute]: Numeric attributes can be real or integer types.\n    2. [Categorical attribute]: For attributes with a limited set of valid values.\n\n\n\nYou'll have to convert these FOS abstractions to a format your implementation understands. \n\n\nThere are multiple training entry points:\n\n```java\n UUID trainAndAdd(ModelConfig config,List\u003cObject[]\u003e instances) throws FOSException;\n```\n\n`traindAndAdd`  must train a new classifier with the given configuration and using the given `instances`. It should return the serialized classifier and automatically make it avaiable for scoring\n\n```java\n UUID trainAndAddFile(ModelConfig config,String path) throws FOSException;\n```\n`trainAndAddFile`  Same as above, but instances are read from a CSV file. \n\n\n```java\nModel train(ModelConfig config,List\u003cObject[]\u003e instances) throws FOSException;\n```\n`train` Trains a model and returns a Model. The Model implementation can be either a ModelBinary, which contains its serialized representation, or a ModelPMML, which contains a String with its representation in PMML. The model is not made available for scoring.\n\n\n```java\nModel trainFile(ModelConfig config, String path) throws FOSException;\n```\n`trainFile` same as above, but instances are read from a CSV file.\n\n\n### Implementing model Scoring\n\nAlong with training models, the manager is also responsible for providing a [Scorer] implementation. There is a weka [weka scoring] example for reference.\n\n\n\n\n\n### Managing models\n\n\n\n\n\n\n[Kryo]: https://github.com/EsotericSoftware/kryo\n[fos-r]: https://github.com/feedzai/fos-r\n[fos-weka]: https://github.com/feedzai/fos-weka\n[Weka]: http://www.cs.waikato.ac.nz/ml/weka/\n[R]: http://www.r-project.org/\n[Maven]: http://maven.apache.org/\n[Java SDK]: http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html\n[FOS samples]: https://github.com/feedzai/FosSample\n[ManagerFactory]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/ManagerFactory.java?source=cc\n[Manager]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/Manager.java?source=cc\n[Scorer]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/Scorer.java?source=cc\n[WekaManager]: https://github.com/feedzai/fos-weka/blob/master/src/main/java/com/feedzai/fos/impl/weka/WekaManager.java?source=cc\n[model training]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/Manager.java?source=c#L120\n[fos training sample]: https://github.com/feedzai/fos-sample/blob/master/src/main/java/com/feedzai/fos/samples/weka/WekaTraining.java#L65\n[attribute]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/Attribute.java?source=cc#L37\n[Numerical attribute]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/NumericAttribute.java?source=c#L32\n[Categorical attribute]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/CategoricalAttribute.java#L46\n[model configuration]: https://github.com/feedzai/fos-core/blob/master/fos-api/src/main/java/com/feedzai/fos/api/ModelConfig.java?source=cc#L45\n[weka model configuration]: https://github.com/feedzai/fos-weka/blob/master/src/main/java/com/feedzai/fos/impl/weka/config/WekaModelConfig.java?source=c#L45\n[weka scoring]: https://github.com/feedzai/fos-sample/blob/master/src/main/java/com/feedzai/fos/samples/weka/WekaScoring.java#L45\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeedzai%2Ffos-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffeedzai%2Ffos-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeedzai%2Ffos-core/lists"}