https://github.com/alexcepoi/pyscale

General purpose Python framework for writing highly scalable applications
https://github.com/alexcepoi/pyscale

Last synced: 24 days ago
JSON representation

General purpose Python framework for writing highly scalable applications

• Host: GitHub
• URL: https://github.com/alexcepoi/pyscale
• Owner: alexcepoi
• License: mit
• Created: 2012-02-07T12:33:42.000Z (almost 13 years ago)
• Default Branch: master
• Last Pushed: 2021-12-25T14:52:11.000Z (almost 3 years ago)
• Last Synced: 2024-04-25T20:21:37.675Z (8 months ago)
• Language: Python
• Homepage:
• Size: 24.4 KB
• Stars: 58
• Watchers: 9
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

• starred-awesome - pyscale - General purpose Python framework for writing highly scalable applications (Python)

README

        
General purpose Python framework for writing highly scalable applications.

About

---------------------------------------------------

A typical application consists of several modules. Each module has its own

process, stores a pidfile in 'tmp/pids', and has a logfile in 'logs'.

A RPC protocol is implemented on top of zeromq in order to allow for

inter-module communication. Modules have an auto-adjustable number of workers

in order to cope with a high number of requests. These rpc requests will block

until that module becomes available.

Read more about zeromq at http://zguide.zeromq.org/

Each module consists of several gevent greenlets. A basic module will already

contain a few greenlets that handle incoming rpc requests. You can spawn

additional greenlets for your own needs.

Read more about gevent at http://www.gevent.org/

Tasks

---------------------------------------------------

You can manage and debug your modules using built-in tasks. Type 'cake' at a

bash prompt when inside your project to see available tasks and what they do.

You can also define your own task.

Commands

---------------------------------------------------

To create a new project:

::

  $ pyscale new 

To generate a new module:

::

  $ pyscale generate 

To start, stop, debug, view logs and more check out available cake tasks:

::

  $ cake

  $ cake start

  $ cake stop

  $ cake status

  $ cake log

  $ cake console

Usage

---------------------------------------------------

To execute an rpc request on another module:

::

  self.sock('modname').method(*args, **kwargs)

You can also use properties, and chain requests:

::

  self.sock('modname').prop.method()

You can also issue requests on all available modules:

::

  self.multisock('*').method()

To spawn another greenlet in a module either use the 'job' decorator or:

::

  self.jobs.spawn(func)

To debug your application use logs and the console.

Requirements

---------------------------------------------------

System Dependencies:

 * zeromq

 * atd

Python Dependencies:

 * pyzmq

 * gevent

 * gevent_zeromq

 * cake

 * argparse

 * jinja2

 * nose