{"id":15398090,"url":"https://github.com/farzonl/cs6250","last_synced_at":"2026-02-12T20:32:13.676Z","repository":{"id":48373623,"uuid":"136876775","full_name":"farzonl/cs6250","owner":"farzonl","description":"An android client app that runs AR, CV, \u0026 ML algorthims  both locally or offloaded.","archived":false,"fork":false,"pushed_at":"2021-07-29T10:15:19.000Z","size":138393,"stargazers_count":2,"open_issues_count":7,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-09-12T23:15:01.807Z","etag":null,"topics":["android","augmented-reality","computer-photography","computer-vision","distributed-systems","iperf3","networking","opencv","opencv-java"],"latest_commit_sha":null,"homepage":"https://youtu.be/2qbBWnnJzZo","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/farzonl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-11T05:02:48.000Z","updated_at":"2020-06-11T07:52:36.000Z","dependencies_parsed_at":"2022-09-21T17:00:50.914Z","dependency_job_id":null,"html_url":"https://github.com/farzonl/cs6250","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/farzonl/cs6250","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farzonl%2Fcs6250","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farzonl%2Fcs6250/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farzonl%2Fcs6250/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farzonl%2Fcs6250/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/farzonl","download_url":"https://codeload.github.com/farzonl/cs6250/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farzonl%2Fcs6250/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29380604,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-12T19:05:20.189Z","status":"ssl_error","status_checked_at":"2026-02-12T19:01:44.216Z","response_time":55,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","augmented-reality","computer-photography","computer-vision","distributed-systems","iperf3","networking","opencv","opencv-java"],"created_at":"2024-10-01T15:40:47.201Z","updated_at":"2026-02-12T20:32:13.663Z","avatar_url":"https://github.com/farzonl.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Profiling and Benchmarking Computational Offloading\n\n**THIS IS A SUBMODULE PROJECT ONLY, DO NOT CLONE DIRECTLY**\nGo to [Parent Project](https://github.com/farzonl/cs6250-server) for installation a dependency instrutions.\n\n## Code Status\n[![Build Status](https://travis-ci.com/farzonl/cs6250.svg?branch=master)](https://travis-ci.com/farzonl/cs6250)\n[![Total alerts](https://img.shields.io/lgtm/alerts/g/farzonl/cs6250.svg?logo=lgtm\u0026logoWidth=18)](https://lgtm.com/projects/g/farzonl/nexuswars/alerts/)\n[![Language grade: Java](https://img.shields.io/lgtm/grade/java/g/farzonl/cs6250.svg?logo=lgtm\u0026logoWidth=18)](https://lgtm.com/projects/g/farzonl/nexuswars/context:java)\n\n## Demo\n\u003cobject width=\"560\" height=\"315\" data=\"https://www.youtube.com/embed/2qbBWnnJzZo\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen\u003e\n\u003cp\u003eGithub does not support embedded youtube. \u003ca href=\"https://youtu.be/2qbBWnnJzZo\"\u003e Click here to view\u003c/a\u003e.\u003c/p\u003e\n\u003c/object\u003e\n\nAbstract\n========\n\nMobile devices are ubiquitous and resource constrained. However, users\nneed an exceptional experience despite these constraints. To provide\nsuch an experience, we are looking to exploit the plentiful resources\navailable in the cloud.This project hopes to solve this problem with a\nremote processing strategy known as cloud offloading. Many benchmarks\nwere made that take pictures of a scene but allow an offsite server to\ndo the image processing and then send the resulting image back to the\ndevice on which the images were captured. The essence of this project is\nto strategize offloading for different applications to enhance user\nexperience.\n\nIntroduction\n============\n\nToday everyone has a mobile device but these devices remain\ncomputationally constrained compared to laptops, desktops, servers, and\nsuper computers. Other limitations include battery, heat, memory, and\nthreading limiting the potential for code optimization on device with\ncurrent general purpose CPUs. Despite all of this users still demand and\nexceptional experience when using a mobile application. Increasing the\napplications that consume most of users time are photo and video\nmanipulation apps like Snapchat and Instagram. The future likely holds\nmore applications in the computer vision, graphics, and augmented\nreality space. Our solution to these pressing constraints on what can be\ndone with on device resources and what is expected to be done is to look\ntowards the cloud as computational accelerator. In this paper we present\na benchmarking application we built to analyze which problem sets are\nbest to be offloaded and what optimization techniques we can use to\nimprove speed and bandwidth.\n\nMotivation\n==========\n\nComputational offloading is a broad problem where depending on the goal\nyou want to optimize for will result in different outcomes. Much of the\nfocus in the literature today seeks to find answers for performance\nimprovements, extending battery life, optimizing for Heat and power\nconstraints (typically classified under the dark silicon problem),\ncomputational partitioning where you divide computation between the\ndevice and the cloud, and much more. We itemize the interesting areas of\nresearch below.\n\n-   *Performance*: This is where much of the research we found\n    interesting for our project lies. Essentially here we identify Where\n    we can improve efficiency by offloading suitable algorithms to the\n    cloud. We will do a literature review on this area in the background\n    section of this paper.\n\n-   *Battery Life*: Offloading might help us save battery on the mobile\n    system. For an application like ours we would expect the screen and\n    camera to be constantly running which we could treat as constant\n    power draw. experimentation could be cpu utilization vs wifi or\n    modem power draw. Experimentation could further be done around\n    distance from a cell tower or wifi access point as a way of varying\n    any heuristic we derive for determining whether or not to offload.\n    [@wifiBattery]\n\n-   *Computation Partitioning*: It is possible to partition an\n    application to use both the cloud and local resources\n    simultaneously.[@partitioning]\n\n-   *Dark Silicon*: Most of the research around Dark silicon in mobile\n    has been focused around coming up with new architectures like ARM\n    big.LITTLE and the experimental GreenDroid. These are all solutions\n    to try and overcome *Dennard scaling* which states that as as\n    transistors get smaller their energy density stays constant since\n    voltage and current in the transistor scale downward as well.\n    [@darkSilicon] GreenDroid is particularly interesting because they\n    are creating an on-chip network router as a way of distributing\n    processing across the die to minimize heat by utilizing as much\n    surface area as possible. [@greenDroid] Given they are already using\n    distributive computing concepts to solve this problem it would not\n    be too far fetch to think that if current chip design improvements\n    fail to fully address this area, off device computation could be a\n    path forward.\n\n-   *Server Availability*: This topic attempts to address the question\n    can you offload if your server is unreliable? If the cloud might not\n    always be available for offloading can we gather a metric for the\n    quality of the network? How much do we consider qualitative metrics\n    like diminishing user experience?\n\n-   *Decision Overhead*: There is an an overhead that needs to be\n    consider when we periodically tests the network quality and makes\n    dynamic decisions for offloading. The iPerf network profiling\n    processing periods would routinely be one of our most high profile\n    threading events in our Traceview dumps. These were done\n    asynchronously so it did not show up in performance metrics but\n    could have implications for some of the other goals to consider for\n    this research topic like partitioning, and battery life.\n\nLack of a budget for this project made battery life and power\nexperimentation impossible as all of our testing were done on emulated\nandroid devices. We chose to focus instead on performance optimization\nand simulate 3G, 4G, and Wifi networks. The Server Availability and\nDecision overhead head questions required us to first build out a\nplatform before we could make any progress on addressing these\nquestions, so for scoping we do not address them in this paper. We\nshould note that the literature around using computational offloading to\nsolve the dark silicon problem is slim to none and could be an\ninteresting focus of research. Since we will be focusing on performance\nwe will try to constrain the problem sets we will try to improve to:\n\n-   Image/Video processing applications\n\n-   Augmented Reality (AR) applications\n\nBackground\n==========\n\nThe basic idea of Computational offloading as laid out by *Shi and Ammar\nCOSMOS (Computation Offloading as a Service for Mobile Devices)* is to\nhave a client component and a server component that can run the same\ncode.[@cosmos] The client component needs to perform 3 basics functions.\nFirst, monitor and predict network performance. Second, track execution\nrequirements Third, choose some portion of code to execute remotely that\ncan reduce computation time. Our implementation and paper will deviate\nfrom COSMOS in a few significant ways, for one we have no hardware or\nbudget for on demand VM allocation. Or performance improvements will be\nlimited to that of a single laptop server. Second, our platform is a\nbenchmark suite similar to something like spec2000, So we want to answer\nthe question of what types of problems are good to offload and what are\nnot. COSMOS on the other hand is better suited for problems where you\nknow what you want to offload. COSMOS also had a metric of cost it\nwanted to optimize for. This is a requirement we are ignoring for this\npaper, and is not as relevant for us as we do not lease computation time\nfor our experiments.\n\nAnother research paper, CloneCloud [@clonecloud] by Chun and Ihm focuses\non dynamically offloading only a part of the execution from the mobile\ndevices to the cloud. CloneCloud was far more focused on taking any\napplication and running it through a static analyzer to determine what\nimprovements could be done for offloading. Our paper will try to pick a\nmore constrained problem set to work with. Our paper will focuses on the\noffloading experimentation around image processing applications. We\nchose to focus only on the image processing applications.\n\nDecision Engine\n---------------\n\nThe COSMOS system achieves high performance offloading at a low cost by\nsharing cloud resources among the mobile devices. An optimization\nalgorithm was proposed by the authors to determine the cloud resource\nmanagement, the decision to offload and the task allocation, based on\nthe assumption that the cloud can simultaneously run N virtual machine\ninstances. [@cosmos]\n\nThe maximum speedup of using the COSMOS system against the local\nexecution can be computed by solving the optimization algorithm.\n\n  K                   Total computation tasks\n  ------------------- ------------------------------------------------------------\n  M$_{i}$             The type of the i$^{th}$ VM instance\n  T$_{i}$             The leasing periods of the i$^{th}$ VM instance\n  $\\psi$(M,T)         The pricing function for leasing a VM instance\n  O$_{k}$             The k$^{th}$ computation task\n  I(i,k)              The indicator function for offloading task O$_{k}$ to VM i\n  I$_{l}$(k)          The indicator function for executing task O$_{k}$ locally\n  L(O$_{k}$)          The local execution time of the task O$_{k}$\n  R$_{i}$ (O$_{k}$)   The response time of offloading the task O$_{k}$ to VM i\n\n$$Max  \\space \\sum_{k = 1}^{K} \\frac{L({O_{k}})}{\\min_{ \\frac{L({O_{k}})}{I_l(k)}, \\frac{R_i({O_{k}})}{I(i,k)} |\\forall_{i}}}$$\n$$s.t. \\space I_{t}(k) +\\sum_{t = 1}^{N} I(i,k) = 1$$\n\nIn the COSMOS decision engine, to decide whether to offload or not\ndepends on setting the Il(k) to 0 or 1. This is a challenge because of\nvarious network connectivity, program execution and resource contention\nissues. These uncertainties have to be dealt with for proper utilization\nof cloud resources and optimized speedup. [@cosmos]\n\nFor our uses we can drop variables T$_{i}$ and $\\psi$(M,T) because cost\nis not a concern of this paper. Further we will be redefining L(O$_{k}$)\nas T$_{computation\\_device}$ and R$_{i}$ (O$_{k}$) as\nT$_{total\\_server}$. We will further simplify our algorithm from being\nthe sum of all compute tasks to each individual task as our heuristic\nwill be highly specialized for the effect we want to optimize. For\nexample the indicator for grayscale would likely be I$_{l}$(k) \\\u003e .95\nwhile one for something like Face swap would take more network health\ninto consideration and thus I(i,k) would be larger. More on this in\n*Simplified Dynamic Offloading Decision Engine* section.\n\nDesign\n======\n\nObjective\n---------\n\nCreate a video editor that allows different effects to be applied to the\nvideo, such as gray scale, blurring, negative, Face Swap, etc. The video\nshould be captured directly from the camera. The editor should also have\na well usable UI to apply any effects or editing features. The UI should\nconsider best practices for an application that targets phone or tablet\nform-factors. Finally, the application is expected to have good\nprogrammatic design and well-documented function or method calls to\nallow for reproducibility of results by an impartial red team.\n\nCore Android (Anchored Code)\n----------------------------\n\n![overview of the Android specific code.](pics/coreUML.png)\n\nDesign patterns make up a big part of enabling whether or not an\napplication can be offloaded. We took best practices from *Refactoring\nAndroid Java Code for On-Demand Computation Offloading*. The most\nimportant way that we enable this is through pipelineing our effects. We\nneed to be able to categorize our codebase into *Anchored* and\n*Moveable* class. Effects have no dependency on Android code and the\ncamera image fetches are abstracted to the beginning of the pipeline and\nhave no barring on later section of the pipeline. [@AndroidRefactor]\n\n![Application overview.](pics/ApplicationOverview.png)\n\n![Displaying a Frame](pics/FramePipline.png)\n\nEffects (Moveable Code)\n-----------------------\n\n![overview of the Effects.](pics/Effects.png)\n\n### Definitions\n\nBelow you will find a list of effects we implemented for this project\n\n-   *Blur*\n\n-   *Circle Detection*\n\n-   *Color Saturation*\n\n-   *Drawing*\n\n-   *Edge Detection*\n\n-   *Gradient Magnitude*\n\n-   *Gray Scale*\n\n-   *Horizontal Flip*\n\n-   *Line Detection*\n\n-   *Motion History*\n\n-   *Negative*\n\n-   *Sepia*\n\n-   *Vertical Flip*\n\n-   *Object Detection*\n\n-   *Cartoon*\n\n-   *Face Detection*\n\n-   *Face Landmark Detection*\n\n-   *Mask*\n\n-   *Face Swap*\n\n![Adding an Effect to the pipeline](pics/EffectsPipline.png)\n\n### Example of Effects\n\nThe effects we chose covered a broad set of Image processing\nrequirements.\n\n-   Image Processing\n\n      ------------------------- ------------------------- -----------------------------\n      ![image](pics/OrigImage.png)    ![image](pics/grayScale.png)    ![image](pics/CartoonEffect.png)\n      Original                          Grayscale                               Cartoon\n      ------------------------- ------------------------- -----------------------------\n\n-   Artificial Intelligence\n\n      ------------------------------- ------------------------------------\n      ![image](pics/ObjectDetection.png)     ![image](pics/FaceFeatureDetection.png)\n      Object Detection                              Face Feature Detection\n      ------------------------------- ------------------------------------\n\n-   Augmented Reality\n\n      ----------------------------- -------------------------- ------------------------\n      ![image](pics/FaceLandMarks.png)    ![image](pics/MaskEffect.png)    ![image](pics/FaceSwap.png)\n      Face Landmark Detection                  Mask                           Face Swap\n      ----------------------------- -------------------------- ------------------------\n\nImplementation\n==============\n\nOur project is a testing framework to measure the practicality of\noffloading these image processing jobs to a remote server. We do this\nwith an Android application and Java server such that the Java code can\nbe written once and run anywhere. Both of these programs are capable of\napplying these effects whether or not the choice to offload has been\nenabled. Our experimentation will be to use a suite of image effects to\nmeasure the efficiency of offloading.\n\nClient Side\n-----------\n\nWe built and image manipulation application that acts as a benchmark\nsuite for common computation problems performed by those in the Computer\nVision, Augmented Reality, Artificial Intelligence, and Computer\ngraphics space. We apply multiple filters including some that use neural\nnets using OpenCV and Dlib. The Effects themselves were written entirely\nin Java and thus could be shared with the server side. The Effects\nclasses are inspired by functional programming, they have no global\ndependencies and perform their effects without causing any side effects.\nThey take in one input matrix perform and action and return an output\nmatrix. This makes these effects good candidates to take advantage of\npipeline parallelism.\n\nFor this project we tried to distinguish ourselves by working with real\ntime data that we get straight from the camera. The effects we have\nchose make up a range of high, to low computationally complex problem\nsets. We also implemented a set of compression optimization's on the\nclient to reduce the number of bits we send.\n\nServer Side\n-----------\n\nTo facilitate the ease of offloading, we use the Apache Avro Interface\nDescription Language, which runs on top of the non-blocking Netty HTTP\nWeb Server. This IDL is written in a language independent form which is\nthen serialized into JSON and compiled into dependency-free Java.\n\n![Overview of the full system pipeline](pics/ServerOverview.png)\n\nSome benefits of this interface are that we can ship\nplatform-independent code to the client and server which requires no\nstate or previous interaction to complete a task. The compiled Java code\nis a loose interface which abstracts device dependant implementations.\nBecause each task is modeled as a function call, the profiling tools we\nuse can simply watch the call stack to measure elapsed times, and the\nJava runtime can take care of any native code discrepancies.\n\n### Details of Offloading\n\nWe use Apache Avro to generate the Interface which will be used to\nimplement the offloading framework. An example of an offloadable method,\nits corresponding AIDL signature and the generated Interface method are\ngiven in figure below.\n\n  ---------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------\n  ![AIDL file and the Generate Proxy function's JSON representation](pics/AIDLFile.png)     ![AIDL file and the Generate Proxy function's JSON representation](pics/AvroGenerated.png)\n  ---------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------\n\nTo write RPC signatures in AIDL like in the above figure we manually\nidentify which methods can be offloaded. We covered a broad range of\nalgorithms that both should and should not be good at offloading. With\nprofiling we can come up with a minimization subset that we may find\nsuitable for offloading. We need to define some qualitative and\nquantitative heuristics for offloading. The qualities of methods the\nliterature suggest are suitable for offloading are:\n\n-   *clean signature* Don't pass around global memory all data passed in\n    or out should be serializable\n\n-   *large execution times* Significant chunk of computation required\n\n-   *System resource independence* Should not require OS or system\n    specific hardware solved with wrappers and interfaces) Examples\n    include camera, storage, and OS specific APIs.\n\n-   *Parallelizable* If a method performs a computation which is much\n    more optimally implemented in the cloud, for example, GPGPU\n    computations, we can choose to offload such a method. For our uses\n    linear algebra and matrices are a very parallelizable problem space.\n\n-   *No native code dependence* The literature encourages the use of VM\n    only code. This is not something we had the luxury of doing, as both\n    Dlib and OpenCV are native libraries. We get close though by using\n    existing and writing our own JNI wrappers.\n\nProfiling\n=========\n\nEverything we have presented so far has been for the sake of answering\nthe one important question of when should we offload? Using Android's\nTraceview and iPerf we hope to answer this question.\n\nTraceview\n---------\n\nWe use Android Traceview to help us determine which Effects are good\ncandidates to offload. Traceview is an android tool used for profiling\nan applications performance and produces execution trace logs with per\nmethod execution times that we can comb over with out own scripts. It\ngives us detailed view of the execution times of a method as well as\ntheir call stacks. Using a Traceview profile, we can build up a\nheuristic to determine offload candidates.\n\niPerf3\n------\n\nTraceview above let us know either the time to run on device or the time\nto run on the server plus communication time. However, this alone is not\nenough information to determine if we should offload For this we need\niPerf3, a bandwidth measurement tool. We ported iPerf3 to android and\nthen began measuring the available bandwidth for our app and the health\nof our network. With the information provided by iPerf and Traceview we\ncan finally do something smart, like dynamic offloading.\n\nExperimentation\n===============\n\nMethod\n------\n\nWe evaluated our project across three different simulated mobile\nnetworking scenarios:\n\n-   **WiFi** a connection over enterprise-grade 802.11 wireless access\n    point between our RPC server on one machine and an emulator on\n    another.\n\n-   **3G** Here we rate limit the emulators network speeds to that of a\n    3G modem.\n\n-   **4G** Here we rate limit the emulators network speed to that of a\n    4G modem.\n\nFor a baseline we measured the on-device computation of each image\neffect defined in the effects section of this paper.Then for each of the\nscenarios listed above we calculated the performance of each effect when\noffloaded. We also wanted to experiment with compression optimizations.\nSo we had a test looking for both the fastest and most reduced sized\npayload. The best algorithm was then run across the full set of effects\nto evaluate performance. Each run had both iPerf as well as Traceview\nruns. This generated server and client side iPerf files as well as one\ntrace dump file that we evaluated for performance metrics.\n\nTo reduce variability among results we all used an x86 QEMU emulated\nPixel android phone running Android 8 Oreo.\n\nNext, we decided to run the results on different network link speeds.\nThis is because mobile applications today run applications on both\ncellular data at 3G or 4G/LTE speeds as well as on WiFi. The experiments\nwere conducted by limiting the emulator network rates to match the real\nworld data rates as closely as possible.\n\nThe Traceview data was collected for all effects initially to determine\nthe CPU usage for each effect on device and on server. Next, the iPerf\nlogs were used to determine the network utilization during offloading.\nSo, the idea was to use the data from Traceview and iPerf to determine\nthe CPU and network utilization for different types of effects.\n\nThe compression bandwidths vs Time Figures show that the gzip and bzip2\noffer the best maintenance of bandwidth when the effect type was held\nconstant across multiple network types. The two anomalous data entries\nin the Face Landmark and Face Swap in the Time to Compute graph were\nlikely caused by invocation of the android dynamic loader for the dLib\nmodule.\n\nResults\n-------\n\n![CPU Utilization on device.](pics/Graph1.png)\n\n![CPU Utilization on Server.](pics/Graph2.png)\n\nFuture Work\n===========\n\nSimplified Dynamic Offloading Decision Engine\n---------------------------------------------\n\nThe decision engine we have experimented with but need more data to\ncomplete is based on the following:\n\n-   bandwidth to the server\n\n-   computation time on the server\n\n-   computation time locally\n\nWe need to determine what our communication costs will be for this we\nwill define the following formula for time to communicate. (Note we\nmultiply by 2 to account for sends and receives.)\n$$T_{total\\_communication} = 2 * \\frac{Size_{image}}{bandwidth}$$\n\nWe can estimate the amount of time taken to process the image on the\nserver when offloaded as follows:\n\n$$T_{total\\_server} = T_{total\\_communication} + T_{computation\\_server}$$\n\nTo determine what are good candidates for offloading we need to\nstatically compute the average amount of time required to apply an\neffect to an image both on the server (*T$_{computation\\_server}$*) and\non the local device (*T$_{computation\\_device}$*). This is where our\nTraceview Runs come in handy.\n\nIf T$_{computation\\_server}$ is less than T$_{computation\\_device}$ then\nwe have a candidate for offloading. But as mentioned above in the iPerf\nsection that is not enough data. We start with a fuzzed value of 100 ms\nto represent communication costs at first until we can update with a\nmore accurate number representing the current state of the network. If\nT$_{total\\_server}$ is greater than T$_{comp\\_dev}$, it means that it is\nmore efficient to do the computation locally than offloading. This\ndecision is taken dynamically. Using the information surmised above we\ncan use the following heuristic (equation 3) to determine when to\noffload. $$T_{computation\\_device} \u003e T_{total\\_server}$$\n\n![The Decision Engine pipeline](pics/DecisionEngine.png)\n\nVideo Codec Optimizations\n-------------------------\n\nThe project can be extended to work with live video streams, to process\nthese video streams on device or on cloud. Video streams carry large\namounts of data. Consider a 480p video stream at 30 frames per second\nwith 24 bits per pixel (colored video). It generates data at the rate of\n221 Mbps. It is not plausible to support such large data rates on\ntoday's internet. Hence compression is crucial. [@codec]\n\nVideo streams can be broken down into frames where each frame is\nessentially an image. A video stream can be compressed spatially and\ntemporally. Spatial video compression is when each frame is compressed\nindividually and temporal compression is when frames are compressed in\ntime. Temporal compression exploits the fact that pixels do not change\ndrastically across each frame. [@codec]\n\nSpatial compression uses JPEG (Joint Photographic Experts Group) for\ncompressing each frame. JPEG uses DCT (Discrete Cosine Transform)\nencoding with quantization to achieve compression. Another aspect is\nusing YUV and chroma subsampling. Using chroma subsampling, we can send\ncolor data at half the resolution. This is because human eye cannot\ndetect changes in color as sharply as changes in brightness. [@codec]\n\nHowever, in high motion frames,it is more important to send more number\nof frames than maintaining the resolution of each frame. Hence, a new\nmacroblock encoding technique can be used, \\\"Flat\\\", to reduce the bits\nneeded for each frame. Flat format is:\n\n6 bits - for each color channel quantized by 4 [@codec]\n\nDCT and quantization is done to reduce the coefficient values to zero to\nsave bandwidth. However, what is more important is getting the\nconsecutive coefficients to zero so that we can use RLE (Run Length\nEncoding) to actually reduce the number of bits to be sent. [@codec]\n\nFor this, we need to focus on the entropy coding. JPEG uses Huffman\ncoder to reduce the number of bits sent for coefficients closer to zero.\nAn artithmetic coder like RANS can aslo be used to save bandwidth.\n[@codec]\n\nConclusion\n==========\n\nIn the paper, we have implemented a testing framework to measure the\npracticality of offloading image processing jobs to a remote server. The\nexperimental results show that our system enables computation offloading\nat low cost and high efficiency. We have noticed the best offloading\nperformance in our ML based algorithms like face swap, Even when\naccounting for the additional performance cost at start up of shared\nlibrary loading we still see reliable performance improvements.\n\nThere are some future directions to extend our testing framework. We\nneed more data to complete bandwidth to the server, computation time on\nthe server and computation time locally for simplified dynamic\noffloading decision engine. The project can be extended to work with\nlive video streams, to process these video streams on device or on\ncloud.\n\n2\n\nCong Shi, Karim Habak, Pranesh Pandurangan,Mostafa Ammar, Mayur Naik,\nEllen Zegura *COSMOS: Computation Offloading as a Service for\nMobileDevices*.\n\nByung-Gon Chun, Sunghwan Ihm, Petros Maniatis, Mayur Naik, Ashwin Patti\n*CloneCloud: Elastic Execution between Mobile Device and Cloud*.\n\nNing Ding Daniel Wagner Xiaomeng Chen Abhinav Pathak Y. Charlie Hu\nAndrew Rice *Characterizing and Modeling the Impact of Wireless Signal\nStrength on Smartphone Battery Drain*.\n\nMax Menges *Dark Silicon and its Implications for Future Processor\nDesign*\n\nJianqiang Li, Luxiang Huang, Yaoming Zhou, Suiqiang He, and Zhong Ming\n*Computation Partitioning for Mobile Cloud Computing in a Big Data\nEnvironment*\n\nNathan Goulding, Jack Sampson, Ganesh Venkatesh, Saturnino Garcia, Joe\nAuricchio, Jonathan Babb, Michael B. Taylor, and Steven Swanson\n*GreenDroid: A Mobile Application Processor for a Future of Dark\nSilicon*\n\n\u003chttp://docs.opencv.org/doc/tutorials/imgproc/imgtrans/hough_lines/hough_lines.html\u003e\n\n\u003chttp://www.aishack.in/2010/04/hough-transform-in-opencv/\u003e\n\n\u003chttp://docs.opencv.org/doc/tutorials/imgproc/imgtrans/sobel_derivatives/sobel_derivatives.html\u003e\n\n\u003chttp://docs.opencv.org/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html\u003e\n\nYing Zhang, Gang Huang , Xuanzhe Liu , Wei Zhang, Hong Mei, Shunxiang\nYang *Refactoring Android Java Code for On-Demand Computation\nOffloading*\n\nBen Garney\n*https://bengarney.com/2016/06/25/video-conference-part-2-joint-photographic-hell-for-beginners/\nVideo Conference: Joint Photographic Hell (For Beginners)*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarzonl%2Fcs6250","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffarzonl%2Fcs6250","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarzonl%2Fcs6250/lists"}