https://github.com/gelbpunkt/aioscheduler

Scalable, high-performance AsyncIO task scheduler
https://github.com/gelbpunkt/aioscheduler

asyncio coroutines high-performance multiple-schedulers scheduler

Last synced: 5 days ago
JSON representation

Scalable, high-performance AsyncIO task scheduler

• Host: GitHub
• URL: https://github.com/gelbpunkt/aioscheduler
• Owner: Gelbpunkt
• License: mit
• Created: 2020-05-16T11:46:20.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2023-05-28T01:53:04.000Z (over 1 year ago)
• Last Synced: 2024-11-09T02:18:22.259Z (2 months ago)
• Topics: asyncio, coroutines, high-performance, multiple-schedulers, scheduler
• Language: Python
• Homepage: https://pypi.org/project/aioscheduler/
• Size: 64.5 KB
• Stars: 26
• Watchers: 2
• Forks: 7
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        
# aioscheduler

aioscheduler is a scalable and high-performance task scheduler for asyncio.

It schedules execution of coroutines at a specific time in a single task, making it lightweight and extremely scalable by adding a manager for multiple schedulers.

Tests have shown that aioscheduler can run up to 10 million timed tasks with up to 20 finishing per second when using 20 schedulers. Single tasks can easily schedule up to 10.000 tasks. This is based on tests on a Xeon E5 1650v3.

## Installation

`pip install aioscheduler`

## Usage

aioscheduler provides several Scheduler classes that runs a main task to consume coroutines.

There are `QueuedScheduler/LifoQueuedScheduler` and `TimedScheduler`, whereas TimedScheduler is the default for Managers.

The TimedScheduler compares datetime objects to UTC by default, to disable it, pass `prefer_utc=False` to the constructor.

```py

import asyncio

from datetime import datetime, timedelta

from aioscheduler import TimedScheduler

async def work(n: int) -> None:

    print(f"I am doing heavy work: {n}")

async def main() -> None:

    starting_time = datetime.utcnow()

    scheduler = TimedScheduler()

    scheduler.start()

    for i in range(60):

        scheduler.schedule(work(i), starting_time + timedelta(seconds=5 + i))

    await asyncio.sleep(65)

asyncio.run(main())

```

In this example, 60 tasks are scheduled to run in 5 seconds from now, 1 of them per second over a time of 1 minute.

The QueuedScheduler works identical, but consumes tasks in scheduling order immediately and only takes a single coroutine as argument to `schedule()`.

To scale even further, aioscheduler offers the Manager (example with QueuedScheduler backend):

```py

import asyncio

from datetime import datetime, timedelta

from aioscheduler import Manager, QueuedScheduler

async def work(n: int) -> None:

    print(f"I am doing heavy work: {n}")

async def main() -> None:

    starting_time = datetime.utcnow()

    manager = Manager(5, cls=QueuedScheduler)  # The number of Schedulers to use

                                               # Leaving out cls defaults to TimedScheduler

    manager.start()

    for i in range(30000):

        manager.schedule(work(i))

    await asyncio.sleep(5)

asyncio.run(main())

```

The manager distributes the tasks across multiple schedulers internally and acts as a load-balancer.

`schedule()` returns a Task object, you may cancel a task after scheduling by running `scheduler.cancel(task)` (or `manager.cancel(task)`). The manager is less efficient for cancelling.

To limit the amount of tasks scheduled, there is a `max_tasks` argument that takes a positive integer. It is advised to use this in production enviroments of known task queue sizes and available on both Scheduler and Manager.

## License

MIT