{"id":13599563,"url":"https://github.com/runwayml/ofxRunway","last_synced_at":"2025-04-10T13:30:46.152Z","repository":{"id":51139449,"uuid":"129554549","full_name":"runwayml/ofxRunway","owner":"runwayml","description":"RunwayML + openFrameworks ☄️","archived":false,"fork":false,"pushed_at":"2021-08-27T21:20:20.000Z","size":623,"stargazers_count":102,"open_issues_count":1,"forks_count":11,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-03-01T13:42:10.328Z","etag":null,"topics":["openframeworks","runway","runway-openframeworks"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/runwayml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-14T21:07:35.000Z","updated_at":"2024-11-29T14:54:02.000Z","dependencies_parsed_at":"2022-08-28T22:12:04.674Z","dependency_job_id":null,"html_url":"https://github.com/runwayml/ofxRunway","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2FofxRunway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2FofxRunway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2FofxRunway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/runwayml%2FofxRunway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/runwayml","download_url":"https://codeload.github.com/runwayml/ofxRunway/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248225705,"owners_count":21068078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["openframeworks","runway","runway-openframeworks"],"created_at":"2024-08-01T17:01:06.878Z","updated_at":"2025-04-10T13:30:43.126Z","avatar_url":"https://github.com/runwayml.png","language":"Makefile","readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/cover_runway_openframeworks_github.jpg\"\u003e\n\u003c/p\u003e\n\n#  ofxRunway\n### RunwayML + openFrameworks\n\n[openFrameworks](http://openframeworks.cc/) addon that allows you to easily integrate [RunwayML](https://runwayml.com/) into your openFrameworks App.\nIt contains examples on most of the currently available models.\n\n## Install\nDownload and copy or `git clone` this addon into your openFrameworks addons folder.\n\nGenerate or update the example projects by dragging their folders onto the projectGenerator and hitting \"update\" or \"generate\".\n\n## Dependencies\n\nYou'll need to install the following addons for this addon to work:\n\n* [ofxHTTP](https://github.com/bakercp/ofxHTTP)\n* [ofxIO](https://github.com/bakercp/ofxIO)\n* [ofxMediaType](https://github.com/bakercp/ofxMediaType)\n* [ofxNetworkUtils](https://github.com/bakercp/ofxNetworkUtils)\n* [ofxSSLManager](https://github.com/bakercp/ofxSSLManager)\n\n\n## Instructions\n\nOpen an example and compile. When running it you must have your RunwayML app running, with the same model as the example running in a Workspace. The examples are named after each model.\n\nFor models that accept images as input, there are usually two examples: one which uses your computer's webcam and continously streams frames to RunwayML, and the other one opens an image file and processes it once.\n\n## Warning \n### Visual Studio \nSome users have [reported](https://github.com/runwayml/ofxRunway/issues/11) a compilation error on Windows, particularly but probably not limited to Visual Studio 2019, producing the following error:\n\n```\n1\u003e------ Build started: Project: example-GPT-2, Configuration: Debug x64 ------\n1\u003eofxNetworkUtils.cpp\n1\u003eC:\\of\\addons\\ofxNetworkUtils\\src\\ofxNetworkUtils.cpp(16,1): fatal error C1021: invalid preprocessor command 'warning'\n1\u003eDone building project \"example-GPT-2.vcxproj\" -- FAILED.\n========== Build: 0 succeeded, 1 failed, 1 up-to-date, 0 skipped ==========\n```\n The fix is to simply comment out the lines that start with `#warning` in file `openframeworks_folder\\addons\\ofxNetworkUtils\\src\\ofxNetworkUtils.cpp`\n\n\n\n## Explanation\n\nRunwayML's method of communication with other applications is using some network protocol. So far, HTTP is the one implemented.\n\nofxRunway takes charge of making the needed requests via HTTP on its own thread, as well as receiving the response and processing it when needed; if the response is an image it will decode it, and put into an ofImage.\n\nRunwayML expects to receive and return data in the JSON format.\n\nThe particular format for what is sent and received can be queried by calling\n`http://localhost:8000/info` which will return a JSON file with the structure for the input and output packages.\n\nYou can find the host name as well as the input and output specification of the model you want to run directly inside the RunwayML app. Select your model in the Workspace, and then select Network at the top of the right side bar to find the specifications.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/appScreenshot.jpg\"\u003e\n\u003c/p\u003e\n\n### Classes\n\nofxRunway has the following classes:\n\n#### **ofxRunwayListener**\n  * This is an abstract class.\n  * The idea is that you extend your `ofApp` class by delcaring it like this\n    ```cpp\n    class ofApp : public ofBaseApp, public ofxRunwayListener`\n    ```\n  as well as adding and implementing the following callback functions:\n\t```cpp\n\tvoid runwayInfoEvent(ofJson\u0026 info){}\n\tvoid runwayErrorEvent(string\u0026 message){}\n\t```\n   which will get called whenever needed by `ofxRunway`. You should use this as a means for communication with `ofxRunway`.\n  * In order for these callbacks to work you need to setup your `ofxRunway` instance by calling\n```cpp\nrunway.setup(this, \"http://localhost:8000\");\n```\nassuming that you are calling it from within `ofApp` and the second parameter is the correct one.\n\n\n\n\n####  **ofxRunway**\n\n  * Is the main class, which is in charge of managing the sending and receiving of data.\n  * All the HTTP calls are executed on a different thread, so it won't block your app while waiting for a response.\n  *  It includes several handy functions for sending and receiving different data types.\n  *  It is designed to be thread safe. You don't need to take care of mutexes.\n\n##### **Public Methods**\n######  setup\n* Run any of these during setup or if for some reason you want to reset the connection during runtime.\n* Only use the version with two arguments when you have implemented your class extending `ofxRunwayListener` as described previously.\n* You can get the host string from RunwayML app, which is usually defaulted to `\"http://localhost:8000\"`;\n\n    ```cpp\n    bool setup(const string\u0026 host);\n\t bool setup(ofxRunwayListener\u0026 listenerClass, const string\u0026 host);\n\t bool setup(ofxRunwayListener* listenerClass, const string\u0026 host);   \n\t ```\n\n######  send\n* Send an `ofxRunwayData` instance with the necessary data by calling ` void send(ofxRunwayData \u0026 data);`\n* This will create the call on its own thread, so it won't block your main app.\n* There are also the following handy functions to be used when you only need to send a single item:\n    ```cpp\n    bool send(const string\u0026 name, const ofBaseHasPixels\u0026 img, ofxRunwayImageType type =OFX_RUNWAY_JPG, int resize_width = 0, int resize_height = 0);\n    bool send(const string\u0026 name, const ofPixels\u0026 pix, ofxRunwayImageType type = OFX_RUNWAY_JPG, int resize_width = 0, int resize_height = 0);\n    bool send(const string\u0026 name, string\u0026 data);\n    ```\n\n######  tryReceive\n```cpp\nbool tryReceive(ofxRunwayData \u0026 data);\n```\n\nThis function will check if there is any new data, and if there is, it will move or copy this new data to the `ofxRunwayData` instance you pass to it and will return true. If there is no new data it will return false. It is intended to be called from within your `ofApp::update` function. There is no need to make any other checks or conditions; if there is nothing it will return immediately with very little overhead.\nUse it when you want to access directly the data received from the RunwayML app, otherwise use the handy get functions further down.\n\n```cpp\n\tofxRunwayData dataToReceive;\n\twhile (runway.tryReceive(dataToReceive)) {\n//\t\t dataToReceive.data contains the raw JSON data sent by RunwayML. And it is here where you should perform what ever you need with it.\n\t\tbreak;\n\t}\n```\n\n######  get\n\nJust like the send methods there are a few handy functions that you can use when expecting a single element to be returned:\n```cpp\nbool get(const string\u0026 name, ofImage\u0026 img);\nbool get(const string\u0026 name, ofPixels\u0026 pix);\nbool get(const string\u0026 name, string\u0026 data);\nbool get(vector\u003cofxRunwayCaption\u003e\u0026 captions, float imgWidth, float imgHeight);\nbool get(vector\u003cofxRunwayPose\u003e\u0026 poses, float imgWidth, float imgHeight, ofxRunwayPoseType poseType);\nbool get(vector\u003cofxRunwayFaceLandmarks\u003e\u0026 landmarks, float imgWidth, float imgHeight);\n```\n\n###### Other methods\n\n* `bool isBusy();`\n\n  Will return true if `ofxRunway` has sent a request and is waiting for a response, otherwise will return false;\n\n\n* `ofxRunwayState getState();`\n\n  Will return the current state of `ofxRunway`. This is a thread-safe function.\n `ofxRunwayState` is an enum which has the following possible values\n\n  * `OFX_RUNWAY_DISCONNECTED` initial default state\n  * `OFX_RUNWAY_SETUP` it has been set up but no attempt of getting any data from the RunwayML server has been done\n  * `OFX_RUNWAY_CONNECTED`  set up and connected to RunwayML server but no data has been received back\n  * `OFX_RUNWAY_RUNNING`  connected and getting data from server\n  * `OFX_RUNWAY_CONNECTION_REFUSED` There was an error while trying to connect to server. Probably RunwayML is not running or the model in the RunwayML Workspace is not running.\n\n* `string getStateAsString(bool bVerbose = false);`\n\n  * Will return the status as as string. If you pass `true` as an argument it will return extra info.\n  * Used by `drawStatus()`\n\n\n* `const string\u0026 getHost();`\n  * Returns the host address.\n\n* `ofRectangle drawStatus(int x = 20, int y = 20,  bool bVerbose = false);`\n  * Draws the current status using ofBitmapFont\n  * `x` X-axis coordinate where to draw the text\n  * `y` y-axis coordinate where to draw the text\n  * `bVerbose` if true draw the verbose (full text) for the current state.\n  * returns the bounding box of the text drawn.\n* `const ofJson\u0026 getInputType(const string\u0026 name);`\n  * Returns the `ofJson` object with info for the input types with the name passed as an argument.\n* `const ofJson\u0026 getOutputType(const string\u0026 name);`\n  * Returns the `ofJson` object with info for the output types with the name passed as an argument.\n* `const ofJson\u0026 getInputTypes();`\n  * Returns the full `ofJson` object with info about the sent data.\n* `const ofJson\u0026 getOutputTypes();`\n  * Returns the full `ofJson` object with info about the received data.\n\n* `bool isServerAvailable();`\n  * Returns true if connected to the server.\n\n##### **Public variables**\n\n* `ofEvent\u003cofJson\u003e infoEvent;`\n  * Event triggered when a new info event has arrived. This will happen as soon as `ofxRunway` successfully connects to the RunwayML app. It will usually only happen once.\n\n* `ofEvent\u003cstring\u003e errorEvent;`\n  * Event triggered when a there has been an error, either in sending or receiving. The event will contain a string with information about the error.\n\n\n####  **ofxRunwayData**\n\n  * This class provides useful functions for setting and getting data into the JSON data required by RunwayML.\n  * Essentially it stores the data in an `ofJson` object, although it has a lot of very useful functions for setting and getting the data.\n\n##### **Public methods**\n###### Getters\n\n  * The following `get` functions require a string as the first argument which is the name of the element you want to get or set. These names are the ones given by RunwayML in its infoEvent (you'll see these printed in the console).\n  * The second argument of these is a reference to an object of the type you want to get.\n  * It will return true if the element was found and its info was copied into the reference passed in the second argument.  \n  * All these behave in the same way and only differ on the data type asked for.\n  * These are named according to the data type that these will get.\n\n\t* `bool getImage(const string\u0026 name,ofBaseHasPixels\u0026 pixels);`\n\t* `bool getImage(const string\u0026 name, ofPixels\u0026 pixels);`\n\t* `bool getBoolean(const string\u0026 name, bool \u0026b);`\n\t* `bool getInt(const string\u0026 name, int\u0026 i);`\n\t* `bool getFloat(const string\u0026 name, float\u0026 f);`\n\t* `bool getString(const string\u0026 name, string\u0026 s);`\n\t* `bool getFloats(const string\u0026 name, vector\u003cfloat\u003e\u0026 f);`\n\t* `bool getStrings(const string\u0026 name, vector\u003cstring\u003e\u0026 s);`\n\t* `bool getFloatVectors(const string\u0026 name, vector\u003cvector\u003cfloat\u003e \u003e\u0026 v);`\n\n* example\n```cpp\n    ofxRunwayData dataToReceive;// this might be declared elsewhere\n    ofImage img;// this might be declared elsewhere\n\t\tif(dataToReceive.getImage(\"image\", img)){\n\t\t    //if it returned true, then it means that an image was extracted form the `dataToReceive` object, decoded and copied into the `img` object.\n\t\t}\n```\n###### Get captions\nThere is the`ofxRunwayCaption` struct, whose declaration is the following.\n```cpp\n  struct ofxRunwayCaption{\n\tofRectangle rect;\n\tstring label;\n\tvoid draw();\n};\n```\n\n  * The following two functions allow you to get a collection of `ofxRunwayCaption` objects. The difference between the two is that one is `static` while the other is not.\n\n  * You pass a reference to a `vector\u003cofxRunwayCaption\u003e` along with the width and height of the image over which the captions go, so these can get scaled accordingly.\n\n\t* `bool getCaptions(vector\u003cofxRunwayCaption\u003e\u0026 captions, float imgWidth, float imgHeight);`\n\t* `static bool getCaptions(vector\u003cofxRunwayCaption\u003e\u0026 captions, const ofJson\u0026 data, float imgWidth, float imgHeight);`\n\n\n###### Get segmentation map\n  * The segmentation map is simply a bunch of unique colors which have been associated with a label (string).\n\n\ttypedef std::map\u003cofColor, string, colorComp\u003e SegmentationMap;\n\n\t* static bool getSegmentationMap(SegmentationMap \u0026 segMap, const ofJson\u0026 info);\n\t* static string findSegmentationLabel(const SegmentationMap \u0026 segMap, const ofBaseHasPixels\u0026 pixels, size_t x, size_t y);\n\t* static string findSegmentationLabel(const SegmentationMap \u0026 segMap, const ofPixels\u0026 pixels, size_t x, size_t y);\n\n\n\n##### **Public variables**\n\n* `ofJson data;`\n  This is where the actual data is stored.\n\n## Contributing\n\n🎉Community Contribution\n\n\nThis is still a work in progress. Contributions are welcomed!\n\n## History\n* v1. [Original addon](https://github.com/genekogan/ofxRunway) by [Gene Kogan](https://github.com/genekogan/) 🎉\n* v2. Updated, with many examples by [Roy Macdonald](https://github.com/roymacdonald/) 🎉\n\nNote: v1 and v2 are not compatible even though they share a lot of code.\n\nNote: This project is heavily inspired by the [RunwayML Processing Library](https://github.com/runwayml/processing-library) made by [George Profenza](https://github.com/orgicus)\n","funding_links":[],"categories":["Makefile"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunwayml%2FofxRunway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frunwayml%2FofxRunway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frunwayml%2FofxRunway/lists"}