{"id":17025816,"url":"https://github.com/runwayml/processing-library","last_synced_at":"2025-05-03T16:31:17.260Z","repository":{"id":80902578,"uuid":"197783148","full_name":"runwayml/processing-library","owner":"runwayml","description":"RunwayML-for-Processing","archived":true,"fork":false,"pushed_at":"2020-04-11T22:18:21.000Z","size":31709,"stargazers_count":137,"open_issues_count":2,"forks_count":24,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-07T18:11:46.427Z","etag":null,"topics":["java","machine-learning","processing","runwayml"],"latest_commit_sha":null,"homepage":"https://runwayml.github.io/processing-library/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/runwayml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-07-19T14:05:44.000Z","updated_at":"2025-03-06T12:42:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"d3b059f9-c616-4390-811e-bd7a5391d03c","html_url":"https://github.com/runwayml/processing-library","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":"processing/processing-library-template-ant","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2Fprocessing-library","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2Fprocessing-library/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2Fprocessing-library/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2Fprocessing-library/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/runwayml","download_url":"https://codeload.github.com/runwayml/processing-library/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252216089,"owners_count":21713097,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["java","machine-learning","processing","runwayml"],"created_at":"2024-10-14T07:30:02.490Z","updated_at":"2025-05-03T16:31:14.740Z","avatar_url":"https://github.com/runwayml.png","language":"Java","readme":"# RunwayML library for Processing\n\nA library to easily use [RunwayML](https://runwayml.com) with [Processing](https://processing.org/).\n\n## Prerequisites\n\n* [RunwayML](https://runwayml.com/): Download the latest release of [RunwayML](https://runwayml.com/download) and sign up for an account. Visit our [installation guide](https://learn.runwayml.com/#/getting-started/installation) for more details. If you encounter any issues installing RunwayML, feel free to contact us through the [support page](https://support.runwayml.com).\n\n* [Processing](https://processing.org/): Processing version 3.0 or greater is required.\n\n## Installation\n\n1. Download **[RunwayML.zip](https://github.com/runwayml/processing-library/releases/download/latest/RunwayML.zip)**\n2. Unzip into **Documents \u003e Processing \u003e libraries**\n3. Restart Processing (if it was already running)\n\n[📽 Watch How to Install and Use the RunwayML Processing Library](https://www.youtube.com/watch?v=zGdOKaLOjck\u0026list=PLj598ZXODDO_oWYAiO5c0Ac05IyrPUG8t\u0026index=6\u0026t=0s)\n\n## Testing installation\n\nBefore using the library please make sure RunwayML is started and is running one model.\n\nYou can test by opening your browser and navigating to [http://localhost:8000/info](http://localhost:8000/info)\nto view info about your model.\n\nIf you see a JSON string with inputs and outputs information use the Basic Example below !\n\nFeel free to test the Basic Example below.\n\n## Basic Example\n\nFor this example run the im2txt model.\n\nThis example will print information from RunwayML (e.g. running im2txt)\n\n```processing\n// import Runway library\nimport com.runwayml.*;\n// reference to runway instance\nRunwayHTTP runway;\n\nvoid setup() {\n  // setup Runway HTTP connection using default host (localhost) and port (8000)\n  runway = new RunwayHTTP(this);\n}\n\nvoid draw() {\n  \n}\n\n// this is called when new Runway data is available\nvoid runwayDataEvent(JSONObject runwayData){\n  println(runwayData);\n}\n\n// this is called each time Processing connects to Runway\n// Runway sends information about the current model\npublic void runwayInfoEvent(JSONObject info){\n  println(info);\n}\n// if anything goes wrong\npublic void runwayErrorEvent(String message){\n  println(message);\n}\n```\n\nNotice the text running displayed in console. \n\nWhat if we want to display an image from RunwayML ?\n\nLet's stop im2txt and use CygleGAN.\nNotice on info event details about the model: the output in particular is an image with the key \"image\".\n(You can see the information within RunwayML on the top right Network area of the Model).\n\nImages to/from RunwayML are Base64 encoded and the library's `ModelUtils` helps convert between this format and the familiar `PImage` format.\n\nWith the CycleGAN model running we can tweak the above example to display an image received from RunwayML:\n\n```processing\n// import Runway library\nimport com.runwayml.*;\n// reference to runway instance\nRunwayHTTP runway;\n// reference to image received\nPImage resultImage;\n\nvoid setup() {\n  // setup Runway HTTP connection using default host (localhost) and port (8000)\n  runway = new RunwayHTTP(this);\n}\n\nvoid draw() {\n\t// if there's an image received:\n\tif(resultImage != null){\n\t\t// display the image\n\t\timage(resultImage,0,0);\n\t}  \n}\n\n// this is called when new Runway data is available\nvoid runwayDataEvent(JSONObject runwayData){\n  // try to access \"image\" key and convert the received that to a PImage\n  try{\n  \tString base64Image = runwayData.getString(\"image\");\n  \tresultImage = ModelUtils.fromBase64(base64Image);\n  }catch(Exception e){\n  \t// print an error message if the above fails\n  \tprintln(\"error parsing received data: \" + runwayData);\n  \te.printStackTrace();\n  }\n}\n\n// this is called each time Processing connects to Runway\n// Runway sends information about the current model\npublic void runwayInfoEvent(JSONObject info){\n  println(info);\n}\n// if anything goes wrong\npublic void runwayErrorEvent(String message){\n  println(message);\n}\n```\n\n## Existing Examples\n\nYou can access the following examples via **Processing \u003e Examples \u003e Contributed Libraries \u003e RunwayML**\n\n- AdaIN-Style-Transfer ![preview](assets/examples/ada-in-style-transfer.jpg)\n(content image: \"Still Life with fruit dish\" by Paul Cézanne,\nstyle image: \"Fruit Dish\" by Georges Braques)\n  \n- Adaptive-Style-Transfer ![preview](assets/examples/adaptive-style-transfer.jpg)\n(content image: \"Fruit Dish\" by Georges Braques, inference using Cézanne (landscapes) style)  \n- Arbitrary-Image-Stylization ![preview](assets/examples/arbitrary-image-stylization.jpg)\n(content image: \"Girl with a Mandolin\" by Pablo Picasso,\nstyle image: \"Man with a guitar\" by Georges Braques)\n  \n- AttnGAN:![preview](assets/examples/attn-gan.jpg)\n  \n- BigGAN: ![preview](assets/examples/big-gan.jpg)\n\n- cnOCR: ![preview](assets/examples/cn-ocr.jpg)\n\n- COCO-SSD: ![preview](assets/examples/coco-ssd.jpg)\n(content image: \"The Card Players\" by Paul Cézanne)\n\n- CycleGAN: ![preview](assets/examples/cycle-gan.jpg)\n(content image: \"Fruit Dish\" by Georges Braques)\n- DeepLab: ![preview](assets/examples/deep-lab.jpg)\n(content image: \"The Card Players\" by Paul Cézanne)\n- DeepLabV3: ![preview](assets/examples/deep-lab-v3.jpg)\n(content image: \"The Card Players\" by Paul Cézanne)\n- DeepPrivacy: ![preview](assets/examples/deep-privacy.jpg)\n(image credits: \"Portrait of Ada Lovelace\" by Margaret Sarah Carpenter)\n- DenseCap: ![preview](assets/examples/dense-cap.jpg)\n(content image: \"The Card Players\" by Paul Cézanne)\n- DenseDepth: ![preview](assets/examples/dense-depth.jpg)\n(content image: \"The Card Players\" by Paul Cézanne)\n- DensePose: ![preview](assets/examples/dense-pose.jpg)\n(image credits: Margaret Hamilton in 1969 with the source code her team developed for the Apollo missions. Photograph: Science History Images/Alamy Stock Photo via theguardian.com)\n- DeOldify: ![preview](assets/examples/de-oldify.jpg)\n(content image: Grace Hopper, image credits: Courtesy of the Computer History Museum, source: https://news.yale.edu/2017/02/10/grace-murray-hopper-1906-1992-legacy-innovation-and-service)\n- DeRaindrop: ![preview](assets/examples/de-raindrop.jpg)\n(content image: \"Sudden shower over Shin Ōhashi bridge and Atake\" by Hiroshige)\n- EdgeConnect: ![preview](assets/examples/edge-connect.jpg)\n(content image: \"Self-Portrait with Striped Shirt\" by Egon Schiele)\n- ESRGAN: ![preview](assets/examples/esrgan.jpg)\n(image credits: SuSuMa original source: https://en.wikipedia.org/wiki/White_tiger#/media/File:White-tiger-2407799_1280.jpg)\n- Face-Landmarks: ![preview](assets/examples/face-landmarks.jpg)\n(content image: \"Celestina\" by Pablo Picasso)\n- Face-Parser: ![preview](assets/examples/face-parser.jpg)\n(content image: \"Celestina\" by Pablo Picasso)\n- Face-Recognition: ![preview](assets/examples/face-recognition.jpg)\n(image from 12 Angry Men (1957) directed by Sidney Lumet)\n- Fast-Photo-Style ![preview](assets/examples/fast-photo-style.jpg)\n(content image: \"Still Life with fruit dish\" by Paul Cézanne,\nstyle image: \"Fruit Dish\" by Georges Braques)\n- Fast-Style-Transfer: ![preview](assets/examples/fast-style-transfer.jpg)\n(content image: \"Nude Descending a Staircase, No. 2\" by Marcel Duchamp)\n- GLOW: ![preview](assets/examples/glow.jpg)\n(content image: \"Self-Portrait with Striped Shirt\" by Egon Schiele)\n- GPT-2: ![preview](assets/examples/gpt-2.jpg)\n- im2txt: ![preview](assets/examples/im2txt.jpg)\n(content image: \"The Card Players\" by Paul Cézanne)\n- Image-Inpainting-GMCNN: ![preview](assets/examples/inpainting-gmcnn.jpg)\n(content image: \"Study after Velázquez's Portrait of Pope Innocent X\" by Francis Bacon)\n- Image-Super-Resolution: ![preview](assets/examples/image-super-resolution.jpg)\n(image credits: SuSuMa original source: https://en.wikipedia.org/wiki/White_tiger#/media/File:White-tiger-2407799_1280.jpg)\n- MaskRCNN: ![preview](assets/examples/mask-rcnn.jpg)\n(image credits: Margaret Hamilton in 1969 with the source code her team developed for the Apollo missions. Photograph: Science History Images/Alamy Stock Photo via theguardian.com)\n- MobileNet: ![preview](assets/examples/mobilenet.jpg)\n(content image: \"Still Life with fruit dish\" by Paul Cézanne)\n- OpenGPT-2: ![preview](assets/examples/open-gpt-2.jpg)\n- OpenPifPaf-Pose: ![preview](assets/examples/open-pif-paf.jpg)\n(image credits: Margaret Hamilton in 1969 with the source code her team developed for the Apollo missions. Photograph: Science History Images/Alamy Stock Photo via theguardian.com)\n- Photo-Sketch: ![preview](assets/examples/photo-sketch.jpg)\n- Pix2Pix: ![preview](assets/examples/pix2pix.jpg)\n(content image: \"Broadway Boogie Woogie\" by Piet Mondrian)\n- Places365: ![preview](assets/examples/places-365.jpg)\n(image credits: \"London: The Thames from Somerset House Terrace towards the City\" by Canaletto)\n- PoseNet: ![preview](assets/examples/posenet.jpg)\n(image credits: Margaret Hamilton in 1969 with the source code her team developed for the Apollo missions. Photograph: Science History Images/Alamy Stock Photo via theguardian.com)\n- SPADE-COCO: ![preview](assets/examples/spade-coco.jpg)\n(uses segmentation based on \"The Card Players\" by Paul Cézanne)\n- SPADE-Landscapes: ![preview](assets/examples/spade-landscapes.jpg)\n(uses segmentation based on \"The Card Players\" by Paul Cézanne)\n- Style2Paints: ![preview](assets/examples/style2paints.jpg)\n(content image: \"Self-Portrait with Striped Shirt\" by Egon Schiele)\n- StyleGAN: ![preview](assets/examples/style-gan.jpg)\n(image credits: \"Portrait of Ada Lovelace\" by Margaret Sarah Carpenter)\n- YOLACT: ![preview](assets/examples/yolact.jpg)\n(content image: \"The Card Players\" by Paul Cézanne)  \n- UGATIT: ![preview](assets/examples/ugatit.jpg)\n(content image: \"Still Life with fruit dish\" by Paul Cézanne)\n\n## Extra platforms\n\nAdditionally you can use RunwayML on a machine on the local network and connect from another device capable of running Processing.\n\n### Raspberry PI\n\nAs long as you have a Raspberry PI running Processing, all you need to do is specify the IP (and optionally port) of the computer running the RunwayML app: that's it!\n(e.g. if your computer's LAN IP is 192.168.0.12 you would initialize Runway like so: `runway = new RunwayHTTP(this,\"192.168.0.12\",8000);`)\n\nTo setup Processing on a Raspberry Pi please follow the official [Get Started tutorial](https://pi.processing.org/get-started/)\n\n![Raspberry Pi library example preview](assets/examples/attn-gan-raspberry-pi.jpg)\n\n\n### Android\n\n1. please [install Android Mode for Processing](https://android.processing.org/install.html) (if it's not already installed on your system).\n2. allow `INTERNET` permissions in **Processing \u003e Android \u003e Sketch Permissions** ![Android library example preview](assets/examples//android-sketch-permissions.jpg)\n3. run the code (e.g. attnGANAndroid)\n\nThe one major caveat at the moment is the Base64 \u003c-\u003e PImage conversion is handled by a separate class: `ModelUtilsAndroid`.\n\nThis means on Android\n\n```processing\nPImage result = ModelUtils.fromBase64(base64ImageString);\n```\n\nbecomes:\n\n```processing\nPImage result = ModelUtilsAndroid.fromBase64(base64ImageString);\n```\n\n\n\n![Android library example preview](assets/examples/attn-gan-android.png)\n\n(Additionally, it's even possible to write RunwayML Processing sketchs on the Android device using [APDE](https://github.com/Calsign/APDE/releases))\n\n\n## Dependencies\n\nThis Processing Library manages the OSC connection to Runway relying on the [oscP5](http://www.sojamo.de/libraries/oscP5/) library.\n\nPlease install oscP5 before using the OSC connection with this library via **Sketch \u003e Import Library... \u003e Add Library... \u003e Contribution Manager Filter \u003e oscP5**\n\n## Contributing\n\nThis is still a work in progress. Contributions are welcomed!\n\n## Credits\n\nSpecial thanks for mentoring and support from [Cris Valenzuela, Anastasis Germanidis](https://runwayml.com) and [Daniel Shiffman](https://github.com/shiffman)\n\nAdditional thanks to [Jen Sykes](http://www.gsa.ac.uk/about-gsa/our-people/our-staff/s/sykes,-jen/) for updating the DenseCap and GPT2 examples\n\nMain library development by [George Profenza](https://github.com/orgicus)\n\nLibrary examples are based partially on [Runway Processing Examples](https://github.com/runwayml/processing) by:\n\n- [Cris Valenzuela](https://github.com/cvalenzuela)\n- [Anastasis Germanidis](https://github.com/agermanidis)\n- [Gene Kogan](https://github.com/genekogan)\n- [maybay21](https://github.com/maybay21)\n- [JP Yepez](https://github.com/jpyepez)\n- [Joel Matthys](https://github.com/jwmatthys)\n- [Alejandro Matamala Ortiz](https://github.com/matamalaortiz)\n","funding_links":[],"categories":["Java","Libraries"],"sub_categories":["Contributions"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunwayml%2Fprocessing-library","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frunwayml%2Fprocessing-library","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunwayml%2Fprocessing-library/lists"}