{"id":13482134,"url":"https://github.com/paypal/yurita","last_synced_at":"2025-03-27T12:32:15.793Z","repository":{"id":66229164,"uuid":"178290225","full_name":"paypal/yurita","owner":"paypal","description":"Anomaly detection framework @ PayPal","archived":true,"fork":false,"pushed_at":"2019-09-02T20:54:00.000Z","size":4218,"stargazers_count":107,"open_issues_count":0,"forks_count":32,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-10-30T15:51:16.775Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paypal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-03-28T22:16:17.000Z","updated_at":"2024-10-26T02:36:00.000Z","dependencies_parsed_at":"2024-01-07T11:14:24.856Z","dependency_job_id":null,"html_url":"https://github.com/paypal/yurita","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fyurita","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fyurita/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fyurita/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paypal%2Fyurita/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paypal","download_url":"https://codeload.github.com/paypal/yurita/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245845246,"owners_count":20681872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T17:00:59.348Z","updated_at":"2025-03-27T12:32:15.273Z","avatar_url":"https://github.com/paypal.png","language":"Scala","funding_links":[],"categories":["Table of Contents","人工智能"],"sub_categories":["Online Machine Learning"],"readme":"\u003c!--\n\n  Licensed under the Apache License, Version 2.0 (the \"License\");\n  you may not use this file except in compliance with the License.\n  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n--\u003e\n\n[![logo](docs/YuritaLogo.png)](https://yurita.readthedocs.io)\n# Yurita\n\n[![Join the chat at https://gitter.im/pp-yurita](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/pp-yurita?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n[![Build Status](https://travis-ci.org/paypal/yurita.svg?branch=master)](https://travis-ci.org/paypal/yurita)\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/4536adca78704f699198a03f9b92a133)](https://app.codacy.com/app/r39132/yurita?utm_source=github.com\u0026utm_medium=referral\u0026utm_content=paypal/yurita\u0026utm_campaign=Badge_Grade_Dashboard)\n[![License](https://img.shields.io/badge/License-Apache%202.0-red.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Documentation Status](https://readthedocs.org/projects/yurita/badge/?version=latest)](https://yurita.readthedocs.io)\n\n\nYurita is an open source project for developing large scale anomaly detection models\n[Site](https://github.com/paypal/yurita/)\n\n## Getting Started\n\n### Documentation\nDocumentation on Yurita's architecture, statistical models available, anomaly detection pipeline/data flow, etc can be found here: \u003chttps://yurita.readthedocs.io/en/latest/\u003e\n\n### Build from source\n```console\nfoo@bar:~/yurita$ ./gradlew clean build\nfoo@bar:~/yurita$ ./gradlew publishToMavenLocal\n```\n### Install from Maven Central\n\n*Please build the project from source at this time or try our dockerized Yurita demo application to build automatically as we make the project jar available on Maven Central in upcoming few days.*\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.github.paypal\u003c/groupId\u003e\n    \u003cartifactId\u003eyurita\u003c/artifactId\u003e\n    \u003cversion\u003e1.0.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\nOther Required Dependencies:\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eorg.apache.spark\u003c/groupId\u003e\n    \u003cartifactId\u003espark-core_2.11\u003c/artifactId\u003e\n    \u003cversion\u003e2.4.1\u003c/version\u003e\n\u003c/dependency\u003e\n\u003cdependency\u003e\n    \u003cgroupId\u003eorg.apache.spark\u003c/groupId\u003e\n    \u003cartifactId\u003espark-sql_2.11\u003c/artifactId\u003e\n    \u003cversion\u003e2.4.1\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Running Dockerized Demo Application\n\n`YuritaSampleApp` directory in the Yurita project root path contains a standalone scala project for you to play around with. Run the demo through Docker inside `YuritaSampleApp` directory as shown below.\n\n### Build Docker Image\n```console\nfoo@bar:~/YuritaSampleApp$ docker build -f Dockerfile -t yuritademo .\n```\n\n### Run Docker Container\n```console\nfoo@bar:~/YuritaSampleApp$ docker run -p 8080:8080 -t yuritademo\n```\n\n## Writing Your First App\nCreate SparkSession with your own configurations\n```scala\nval appName = \"AnomalyDetectionAPI\"\nval sparkConf = new SparkConf().setAppName(appName).setMaster(\"local[*]\")\nval spark = SparkSession\n    .builder()\n    .config(sparkConf)\n    .getOrCreate()\n```\n\n\u003cbr/\u003e\n\nCreate dataframe of your data points/attributes with what time interval they occur on\n```scala\n//sample window timestamp\nval window1 = (dateFormat.parse(\"2011-01-18 01:00:00.0\"), dateFormat.parse(\"2011-01-18 01:00:10.0\"))\n```\n```scala\nval inputDF: DataFrame = Seq(\n    Person(\"Ned\", \"Stark\", 40, 40.6, \"M\", Array(5.5), getTimestamp(window1)),\n    Person(\"Arya\", \"Stark\", 9, 40.1, \"F\", Array(5.6), getTimestamp(window2)),\n    Person(\"Sansa\", \"Stark\", 13, 46.3, \"F\", Array(5.6), getTimestamp(window3)),\n    Person(\"Jon Snow\", \"Stark\", 17, 11.4, \"M\", Array(12.4), getTimestamp(window1),\n    ...\n).toDF()\n```\n\u003cbr/\u003e\n\nCreate a data pipe that will perform specified stastical methods on set columns of dataframe within the window size.\n```scala\nval categoricalPipe = PipelineBuilder()\n    .onColumns(Seq(\"surname\", \"gender\"))\n    .setWindowing(Window.fixed(\"1 hour\"))\n    .setWindowReferencing(windowRef)\n    .buildCategoricalModel(\n    Functions.Categorical.avgRef,\n    Functions.Categorical.entropy,\n    Functions.statResultThreshold(3.0))\n```\n\nCombine multiple pipelines\n```scala\nval workload = AnomalyWorkload.builder()\n    .addAllPipelines(categoricalPipe)\n    .addPartitioner(\"surname\")\n    .buildWithWatermark(\"timestamp\", \"2 hours\")\n```\n\nDataset extended api\n```scala\ndf.detectAnomalies(workload).map(_.toString).foreach(println(_))\n```\n\n\u003cbr/\u003e\nFull demo application code can be viewed in our YuritaSampleApp project.\n\n## Contributing to Yurita\n\nThank you very much for contributing to Yurita. Please read the [contribution guidelines](CONTRIBUTING.md) for the process.\n\n## License\n\nYurita is licensed under the [Apache License, v2.0](LICENSE.txt)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaypal%2Fyurita","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaypal%2Fyurita","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaypal%2Fyurita/lists"}