https://github.com/kaltura/nginx-parallel-module

Nginx module for boosting upstream requests by issuing them in parallel
https://github.com/kaltura/nginx-parallel-module

Last synced: 3 months ago
JSON representation

Nginx module for boosting upstream requests by issuing them in parallel

• Host: GitHub
• URL: https://github.com/kaltura/nginx-parallel-module
• Owner: kaltura
• License: agpl-3.0
• Created: 2015-06-08T08:46:39.000Z (over 10 years ago)
• Default Branch: master
• Last Pushed: 2020-11-16T18:01:18.000Z (almost 5 years ago)
• Last Synced: 2025-06-08T15:08:02.074Z (4 months ago)
• Language: C
• Size: 37.1 KB
• Stars: 18
• Watchers: 21
• Forks: 6
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Parallel module for Nginx [![Build Status](https://travis-ci.org/kaltura/nginx-parallel-module.svg?branch=master)](https://travis-ci.org/kaltura/nginx-parallel-module)

Reads data from an upstream-backed nginx location by running several concurrent range requests.

## Installation

Add `--add-module` when configuring nginx:

    ./configure --add-module=$PATH_TO_PARALLEL_MODULE

## Configuration

### Directives

#### parallel

* **syntax**: `parallel uri_prefix`

* **default**: `none`

* **context**: `location`

Enables the parallel module on the enclosing location, the uri_prefix is added to the request URI 

to form the URI of the subrequests

#### parallel_fiber_count

* **syntax**: `parallel_fiber_count count`

* **default**: `8`

* **context**: `http`, `server`, `location`

Sets the maximum number of concurrent subrequests

#### parallel_min_chunk_size

* **syntax**: `parallel_min_chunk_size size`

* **default**: `128k`

* **context**: `http`, `server`, `location`

Sets the minimum size that is requested in a single subrequest, only the request for the last chunk

is allowed to return less than this value. This parameter also controls the size that is requested in 

the initial subrequests, that are issued before the response size is known.

#### parallel_max_chunk_size

* **syntax**: `parallel_max_chunk_size size`

* **default**: `512k`

* **context**: `http`, `server`, `location`

Sets the maximum size that can be fetched in a single subrequest

#### parallel_max_headers_size

* **syntax**: `parallel_max_headers_size size`

* **default**: `4k`

* **context**: `http`, `server`, `location`

Sets the size that should be allocated for holding the subrequests headers

#### parallel_max_buffer_count

* **syntax**: `parallel_max_buffer_count count`

* **default**: `16`

* **context**: `http`, `server`, `location`

Sets the maximum number of buffers that the module is allowed to allocate.

Once the limit is reached, the module will until data is flushed to the client before issuing

more subrequests.

#### parallel_consistency_check_etag

* **syntax**: `parallel_consistency_check_etag on/off`

* **default**: `on`

* **context**: `http`, `server`, `location`

Enables/disables ETag consistency validation, when enabled, the server will make sure all ETag headers

returned for subrequests are consistent and fail the request otherwise.

#### parallel_consistency_check_last_modified

* **syntax**: `parallel_consistency_check_last_modified on/off`

* **default**: `on`

* **context**: `http`, `server`, `location`

Enables/disables Last-Modified consistency validation, when enabled, the server will make sure all Last-Modified headers

returned for subrequests are consistent and fail the request otherwise.

#### parallel_content_length_cache

* **syntax**: `parallel_content_length_cache zone_name zone_size`

* **default**: `off`

* **context**: `http`, `server`, `location`

Configures the size and shared memory object name of the content length cache.

This cache holds the content lengths of requested URLs in order to optimize their retrieval in the future.

Cached lengths serve as a hint to the module when making the initial batch of subrequests.

If the content length is large, a larger chunk size can be used, if the content length is small, 

the module can avoid making subrequests that will fail with 416 error.

An incorrect length that is saved in the cache can not affect the response generated by this module,

but it can lead to a less efficient pull from the origin.

Each saved content length record consumes about 80 bytes, so a cache of few hundred megabytes can probably last

for a long time.

### Sample configuration

http {

	upstream backend {

		server backendhost:80;

		keepalive 32;

	}

	server {

		location /parallel {

			parallel /proxy;

		}

		

		location /proxy/parallel/ {

			proxy_pass http://backend/;

			proxy_http_version 1.1;

			proxy_set_header Connection "";

			proxy_set_header Host $http_host;

			

			internal;

		}

	}

}

## Copyright & License

All code in this project is released under the [AGPLv3 license](http://www.gnu.org/licenses/agpl-3.0.html) unless a different license for a particular library is specified in the applicable library path. 

Copyright © Kaltura Inc. All rights reserved.