https://github.com/pgiri/pycos

Concurrent, Asynchronous, Distributed, Communicating Tasks with Python
https://github.com/pgiri/pycos

asynchronous-programming cluster-computing concurrent-programming distributed-computing message-passing python

Last synced: 2 months ago
JSON representation

Concurrent, Asynchronous, Distributed, Communicating Tasks with Python

• Host: GitHub
• URL: https://github.com/pgiri/pycos
• Owner: pgiri
• License: other
• Created: 2017-05-21T17:58:23.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2024-06-28T04:54:11.000Z (over 1 year ago)
• Last Synced: 2025-11-27T17:15:13.151Z (4 months ago)
• Topics: asynchronous-programming, cluster-computing, concurrent-programming, distributed-computing, message-passing, python
• Language: Python
• Homepage: https://pycos.org
• Size: 1.6 MB
• Stars: 49
• Watchers: 5
• Forks: 9
• Open Issues: 14
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

          
pycos

######

    .. note:: Full documentation for pycos is now available at `pycos.org

              `_.

`pycos `_ is a Python framework for asynchronous, concurrent, network /

distributed programming, distributed computing with *very* light weight tasks and message passing.

Unlike with other asynchronous frameworks, programs developed with pycos have

**same logic and structure** as programs with threads, except for a few

syntactic changes - mostly using *yield* with asynchronous completions that give

control to pycos's scheduler, which interleaves executions of tasks, similar to

the way an operating system executes multiple processes. In addition, pycos has

many additional features, including message passing for communication,

distributed computing/programming etc.

Unlike threads, creating tasks with pycos is very efficient. Moreover, with

pycos context switch occurs only when tasks use *yield* (typically with an

asychronous call), so there is no need for locking and there is no overhead of

unnecessary context switches.

pycos works with Python versions 2.7+ and 3.1+ on Linux, Mac OS X and Windows; it may

work on other platforms (e.g., FreeBSD and other BSD variants) too.

Features

--------

* No callbacks or event loops! No need to lock critical sections either,

* Efficient polling mechanisms epoll, kqueue, /dev/poll, Windows

  I/O Completion Ports (IOCP) for high performance and

  scalability,

* Asynchronous (non-blocking) sockets and pipes, for concurrent

  processing of I/O,

* SSL for security,

* Asynchronous locking primitives similar to Python threading module,

* Asynchronous timers and timeouts,

* `Message passing `_

  for (local and remote) tasks to exchange messages one-to-one

  with `Message Queue Pattern

  `_ or through

  broadcasting channels with `Publish-Subscribe Pattern

  `_,

* `Location transparency

  `_ with naming

  and locating (local and remote) resources,

* Remote Pico Service (RPS) for defining services that remote clients can run as tasks

  (with possibly message passing to communicate).

* Distributing computation components (code and data) for execution of

  distributed communicating processes, for wide range of use cases, covering

  `SIMD, MISD, MIMD `_ system

  architectures at the process level, `web interface

  `_ to

  monitor cluster/application status/performance; `in-memory processing

  `_, data streaming,

  real-time (live) analytics and cloud computing are supported as well,

* Monitoring and restarting of (local or remote) tasks, for fault detection

  and fault-tolerance,

* Hot-swapping of task functions, for dynamic system reconfiguration.

* Thread pools with asynchronous task completions, for executing (external)

  synchronous tasks, such as reading standard input.

Dependencies

------------

pycos is implemented with standard modules in Python.

If `psutil `_ is available on nodes, node

availability status (CPU, memory and disk) is sent in status messages, and shown

in web browser so node/application performance can be monitored.

Under Windows efficient polling notifier I/O Completion Ports (IOCP) is

supported only if `pywin32 `_ is available;

otherwise, inefficient *select* notifier is used.

Installation

------------

To install pycos, run::

   python -m pip install pycos

Authors

-------

* Giridhar Pemmasani

Links

-----

* Documentation is at `pycos.org`_.

* `GitHub (Code Repository) `_.