{"id":23591212,"url":"https://github.com/socaity/fasttaskapi","last_synced_at":"2025-07-17T13:34:34.067Z","repository":{"id":234749260,"uuid":"789455988","full_name":"SocAIty/FastTaskAPI","owner":"SocAIty","description":"Create web-APIs for long-running tasks. Job based task handling. Get the result with the job id later. FastTaskAPI creates threaded jobs and job queues on the fly. Router functionality for Runpod. Run services anywhere, be it local, hosted or serverless.","archived":false,"fork":false,"pushed_at":"2025-06-26T08:56:27.000Z","size":1168,"stargazers_count":17,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-26T09:43:18.797Z","etag":null,"topics":["azure-blob-storage","celery","deployment","fastapi","job-queue","json","machine-learning","openapi","rest","runpod","s3-storage","starlette","threading","uvicorn"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SocAIty.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"SocAIty","patreon":"w4hns1nn","buy_me_a_coffee":"socaity","custom":["paypal.me/SocaityRieger"]}},"created_at":"2024-04-20T15:46:38.000Z","updated_at":"2025-06-26T08:56:30.000Z","dependencies_parsed_at":"2024-04-20T17:11:11.722Z","dependency_job_id":"f6f96531-0910-4d7a-8647-46a027b8e3d2","html_url":"https://github.com/SocAIty/FastTaskAPI","commit_stats":null,"previous_names":["socaity/socaity_router","socaity/socaity-router","socaity/fasttaskapi"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SocAIty/FastTaskAPI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SocAIty%2FFastTaskAPI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SocAIty%2FFastTaskAPI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SocAIty%2FFastTaskAPI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SocAIty%2FFastTaskAPI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SocAIty","download_url":"https://codeload.github.com/SocAIty/FastTaskAPI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SocAIty%2FFastTaskAPI/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265611382,"owners_count":23797877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-blob-storage","celery","deployment","fastapi","job-queue","json","machine-learning","openapi","rest","runpod","s3-storage","starlette","threading","uvicorn"],"created_at":"2024-12-27T07:34:54.786Z","updated_at":"2025-07-17T13:34:34.037Z","avatar_url":"https://github.com/SocAIty.png","language":"Python","funding_links":["https://github.com/sponsors/SocAIty","https://patreon.com/w4hns1nn","https://buymeacoffee.com/socaity","paypal.me/SocaityRieger"],"categories":[],"sub_categories":[],"readme":"\n  \u003ch1 align=\"center\" style=\"margin-top:-25px\"\u003eFastTaskAPI\u003c/h1\u003e\n  \u003ch3 align=\"center\" style=\"margin-top:-10px\"\u003eCreate web-APIs for long-running tasks\u003c/h3\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg align=\"center\" src=\"docs/example_images/socaity_router_icon.png\" height=\"200\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\nWe at SocAIty developed FastTaskAPI to create and deploy our AI services as easy and standardized as possible.\nBuilt on-top of FastAPI and runpod you can built high quality endpoints with proven stability. \nRun services anywhere, be it local, hosted or serverless.\u003c/br\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  Package machine learning models standardized and production ready.\n  Deploy them to your own infrastructure, runpod or to \u003ca href=\"https://www.socaity.com\"\u003esocaity.ai\u003c/a\u003e.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\nFastTaskAPI creates threaded jobs and job queues on the fly.\nCall the api and return a job id. Get the result with the job id later.\u003c/br\u003e\n\u003c/p\u003e\n\n## Table of contents\n\nIntroduction\n- [Why is this useful?](#why-is-this-useful): A section explaining why you should use this package.\n- [What does this do?](#what-does-this-do): A section explaining the features of this package.\n\nGet started:\n- [Installation](#installation): A section explaining how to install the package.\n- [First-steps](#how-to-use): Create your first service with the socaity router.\n- [Jobs and job queues](#jobs-and-job-queues): A section explaining how to use the job queue functionality.\n- [File uploads and files](#file-uploads-and-files): A section explaining how to use file uploads and files.\n- [Backends and deploying a service](#backends-and-deploying-a-service): Deploy serverless for example with runpod.\n\n\n## Why is this useful?\n\nCreating services for long-running tasks is hard.\n- In AI services inference time makes realtime results difficult. Parallel jobs, and a Job queue is often required. For example as a client you would not like to wait for a server response instead do some work until the server produced the result.\n- Serverless deployments like runpod often DO NOT provide routing functionality. This router works in this conditions.\n- Scaling AI services is hard.\n- Everybody handles file uploads differently. Different formats, request-toolkits, base64, byte-streams. This leads to messy code. We standardized file handling with [MediaToolkit](https://github.com/SocAIty/media-toolkit).\n- Streaming services (for example for generative models) is complicated to setup.\n- Same Syntax for all kind of hosting providers and services.\n\nThis package solves these problems, by providing a simple well-known interface to deploy AI services anywhere.\u003c/br\u003e\nThe syntax is oriented by the simplicity of fastapi. Other hazards are taken care of by our router.\n\n\n## What does this do?\n\u003cimg align=\"right\" src=\"docs/example_images/socaity_services.png\" height=\"400\" style=\"margin-top: 50px\" /\u003e\n\n\n- Jobs, job queues for your service (no code required).\n- Routing functionality: for serverless providers like [Runpod](Runpod.io)\n- Async, sync and streaming functionality.\n  - Including progress bars.\n- File support, also for serverless providers like [Runpod](https://docs.runpod.io/serverless/workers/handlers/overview) \n  - Simplified sending files to the services with [fastSDK](https://github.com/SocAIty/fastSDK) \n  - One line file response with [MediaToolkit](https://github.com/SocAIty/media-toolkit) including images, audio, video and more.\n- Integration: integrates neatly into the SOCAITY ecosystem for running AI services like python functions with our [Client](https://github.com/SocAIty/socaity-client)/[fastSDK](https://github.com/SocAIty/socaity).\n- Monitoring server state.\n\nThe code is fast, lightweight, flexible and pure python.\n\n## Installation \nYou can install the package with PIP, or clone the repository.\n\n```python\n# install from pypi\npip install fast-task-api\n# install from github for the newest version\npip install git+git://github.com/SocAIty/FastTaskAPI\n```\n\n# How to use\n## Create your first service\nUse the decorator syntax @app.task_endpoint to add an endpoint. This syntax is similar to [fastapi](https://fastapi.tiangolo.com/tutorial/first-steps/)'s @app.get syntax.\n\n```python\nfrom fast_task_api import FastTaskAPI, ImageFile\n\n# define the app including your provider (fastapi, runpod..)\napp = FastTaskAPI()\n\n# add endpoints to your service\n@app.task_endpoint(\"/predict\")\ndef predict(my_param1: str, my_param2: int = 0):\n  return f\"my_awesome_prediction {my_param1} {my_param2}\"\n\n@app.task_endpoint(\"/img2img\", queue_size=10)\ndef my_image_manipulator(upload_img: ImageFile):\n  img_as_numpy = upload_img.to_np_array() \n  # Do some hard work here...\n  # img_as_numpy = img2img(img_as_numpy)\n  return ImageFile().from_np_array(img_as_numpy)\n\n# start and run the server\napp.start()\n```\nIf you execute this code you should see the following page under http://localhost:8000/docs.\n\u003cimg align=\"center\" src=\"docs/example_images/demo_service.png\" /\u003e\n\n## Jobs and job queues\n\nIf you have a long running task, you can use the job queue functionality. \n\n```python\n@app.task_endpoint(path=\"/make_fries\", queue_size=100)\ndef make_fries(job_progress: JobProgress, fries_name: str, amount: int = 1):\n    job_progress.set_status(0.1, f\"started new fries creation {fries_name}\")\n    time.sleep(1)\n    job_progress.set_status(0.5, f\"I am working on it. Lots of work to do {amount}\")\n    time.sleep(2)\n    job_progress.set_status(0.8, \"Still working on it. Almost done\")\n    time.sleep(2)\n    return f\"Your fries {fries_name} are ready\"\n```\nWhat will happen now is: \n- The method will return a \"Job\" object instead of the result, including a job id. This json is send back to the client.\n- By calling the status endpoint with status?job_id=... one gets the result / status of the job.\n\nNote: in case of \"runpod\", \"serverless\" this is not necessary, as the job mechanism is handled by runpod deployment.\n\nYou can provide status updates by changing the values of the job_progress object. \nIf you add a parameter named job_progress to the function we will pass that object to the function.\nIf then a client asks for the status of the task, he will get the messages and the process bar. \nThis is for example in [fastSDK](https://github.com/SocAIty/fastSDK) used to provide a progress bar.\n\n\n### Calling the endpoints -\u003e Getting the job result\n\nYou can call the endpoints with a simple http request.\nYou can try them out in the browser, with curl or Postman. \nFor more convenience with the socaity package, you can use the endpoints like functions.\n\n### Use the endpoints like functions with [fastSDK](https://github.com/SocAIty/fastSDK).\nWith fastSDK, you can use the endpoints like a function. FastSDK will deal with the job id and the status requests in the background.\nThis makes it insanely useful for complex scenarios where you use multiple models and endpoints.\nCheckout the [fastSDK](https://github.com/SocAIty/fastSDK) documentation for more information.\n\n\n### Normal openapi (no-task) endpoints\n\nIf you don't want to use the job queue functionality, you can use the ```@app.endpoint``` syntax\n```python\n@app.endpoint(\"/my_normal_endpoint\", methods=[\"GET\", \"POST\"]):\ndef my_normal_endpoint(image: str, my_param2: int = 0):\n  return f\"my_awesome_prediction {my_param1} {my_param2}\"\n```\nThis will return a regular endpoint -\u003e No job_result object with job-id is returned.\nThe method also supports file uploads.\n\n## File uploads and files.\n\nThe library supports file uploads out of the box. \nUse the parameter type hints in your method definition to get the file.\n\n```python\nfrom fast_task_api import MediaFile, ImageFile, AudioFile, VideoFile\n\n@app.task_endpoint(\"/my_upload\")\ndef my_upload(anyfile: MediaFile):\n    return anyfile\n```\nFastTaskAPI supports all file-types of [media-toolkit](https://github.com/SocAIty/media-toolkit). This includes common file types like: ImageFile, AudioFile and VideoFile.\n```python\nfrom fast_task_api import ImageFile, AudioFile, VideoFile\n\n@app.task_endpoint(\"/my_file_upload\")\ndef my_file_upload(image: ImageFile, audio: AudioFile, video: VideoFile):\n    image_as_np_array = np.array(image)\n    return [ImageFile().from_np_array(image_as_np_array) for i in range(10)]\n```\nYou can call the endpoints, either with bytes or b64 encoded strings. \n\n\n### Sending requests (and files) to the service with FastSDK\n\nFastSDK also supports MediaFiles and for this reason it natively supports file up/downloads.\nOnce you have added the service in FastSDK you can call it like a python function\n```\nmysdk = mySDK() # follow the fastSDK tutorial to set up correctly.\ntask = upload_image(my_imageFile, myAudioFile, myVideoFile) # uploads the data to the service. Retrieves a job back.\nresult = task.get_result()  # constantly trigger the get_job endpoint in the background until the server finished.\n```\n\n### Sending files to the service with httpx / requests\n\n```python\nimport httpx\nwith open('my_image_file.png', 'rb') as f:\n    image_file_content = f.read()\n\nmy_files = {\n  \"image\": (\"file_name\", image_file_content, 'image/png')\n  ...\n}\nresponse = httpx.Client().post(url, files=my_files)\n```\nNote: In case of runpod you need to convert the file to a b64 encoded string.\n\n### Sending file with URLs\nOne property of media-toolkit is, that it support files from URLs. \nThus instead of sending a file directly (as bytes) to the endpoints, you can also send a URL to the file location.\n```python\nmy_files = {\n  \"image\": \"https:/my_cloud_storage/my_image_file.png\"\n  ...\n}\nresponse = httpx.Client().post(url, files=my_files)\n```\n\n### Upload file size limits\n\nYou can define a default upload size limit in the settings with the environment variable:\n``DEFAULT_MAX_UPLOAD_FILE_SIZE_MB``. If it is None, there is no limit.\n\nAlso you can define individual limit for each task_endpoint like.\n```python\n@app.task_endpoint(\"/my_upload\", max_upload_file_size_mb=10)\n```\nFinally you could also define the limit fo a specific file by using\n```python\n@app.task_endpoint(\"/my_upload\")\ndef my_upload(file: LimitedUploadFile = LimitedUploadFile(max_size_mb=10)):\n    return file.content\n```\n\nNote: MaxUploadSize currently is only respected for fastapi backend.\n\n\n# Deployment of the service with different backends (hosting providers)\n\n\n## Locally\nJust run the server by running your script ```app.start()``` this will spin-up uvicorn on localhost:port.\n\n## Docker\nPrerequisite: You have created a python module \"yourmodule.server\" with the code that starts the server.\nThen to start the fast-task-api server in docker, add the following command at the end of your Dockerfile.\n\n```dockerfile \n# Start the fast-task-api server which is instantiated in the module -m yourmodule.server\nCMD [ \"python\", \"-m\", \"yourmodule.server\"]\n```\n\n### Additional configuration: Backend, deployment type, host, port\nYou can change the backend (hosting provider) either by setting it in the constructor or by setting the environment variable.\n```dockerfile \n# Options: \"fastapi\", \"runpod\"\nENV FTAPI_BACKEND=\"runpod\"\n# Options: \"localhost\", \"serverless\"\nENV FTAPI_DEPLOYMENT=\"serverless\"\n```\n\nDepending on the environment it is also necessary to specify the host and port.\n```dockerfile\n# allows any IP from the computer to connect to the host\nENV FTAPI_HOST=\"0.0.0.0\"\n# allows the docker container to use the port\nARG port=8080\nENV FTAPI_PORT=$port\nEXPOSE $port \n```\n\n## Runpod\nIt is not required to write a [handler](https://docs.runpod.io/serverless/workers/handlers/overview) function anymore. The fast-task-api magic handles it :D\nJust change the ENV variable and described above.\nThis brings you additional benefits:\n- [x] Same syntax as with fastapi\n- [x] Router functionalities.\n- [x] Better file handling\n- [x] ultra easy deploy. \n\nWhen deployed with runpod, fast-task-api will use the builtin runpod job handling instead \nof using the own job queue implementation.\n\n# FastSDK :two_hearts: FastTaskAPI\n\n\u003cimg src=\"docs/example_images/fastsdk_to_fasttaskapi.png\" width=\"50%\" /\u003e\n\n[FastSDK](https://github.com/SocAIty/fastSDK) allows you to easily invoke your FastTaskAPI services with python in an efficient parallel manner.\nTherefore you can natively work with them as if they are python functions.\nRead the [fastSDK](https://github.com/SocAIty/fastSDK). documentation to get started.\n\n# Related projects and its differences\n\n## Cog\n\nCog comes with several cons:\n - it is mainly focused on packaging. Additional features for ML services are missing.\n - it does NOT support multiple endpoints and router functionality. \n - it is focused on coping with DOCKER and CUDA problems. \n - it forces you to learn a new syntax.\n - It doesn't simplify deployment on RUNPOD\n\n## Starlette Background Tasks\n\nThe fastapi documentation recommends using starlette background tasks for long-running tasks like sending an e-mail.\nHowever starlette background tasks have several drawbacks.\n- Without a job queue, the server can be overloaded with tasks pretty fast.\n- It's not clear when and how to return the result to the client. So you need to write your own \"webhook\" code.\n- No job progress and monitoring functionality for the client. \n\nStarlette-Background-Tasks are a good solution for simple tasks, but you'll hit a wall pretty soon.\n\n\n## Celery\n\nCelery is a great tool for running jobs in the background on distributed systems.\nHowever it comes with several drawbacks: \n- Hard to setup\n- Doesn't run everywhere\n- Overkill for most projects.\n\nSocaity router is lightweight and provides background task functionality abstracted from the developer.\nThis doesn't mean that we don't recommend celery. Indeed, it is planned to integrate celery as possible backend.\n\n\n# Roadmap\n\n- [x] streaming support\n- [x] support other job-queuing systems like celery\n\n\n#### LEAVE A STAR TO SUPPORT US. ANY BUG REPORT OR CONTRIBUTION IS HIGHLY APPRECIATED.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsocaity%2Ffasttaskapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsocaity%2Ffasttaskapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsocaity%2Ffasttaskapi/lists"}