https://github.com/ged/thingfish-datastore-filesystem

Git mirror
https://github.com/ged/thingfish-datastore-filesystem

Last synced: 8 months ago
JSON representation

Git mirror

• Host: GitHub
• URL: https://github.com/ged/thingfish-datastore-filesystem
• Owner: ged
• License: bsd-3-clause
• Created: 2016-11-04T01:03:41.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2020-04-08T01:14:44.000Z (about 6 years ago)
• Last Synced: 2025-04-05T10:14:16.531Z (about 1 year ago)
• Language: Ruby
• Homepage: https://bitbucket.org/ged/thingfish-datastore-filesystem
• Size: 56.6 KB
• Stars: 1
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.rdoc
  • Changelog: History.rdoc
  • License: LICENSE.rdoc

Awesome Lists containing this project

README

          
= Thingfish Filesystem Datastore

home:: https://hg.sr.ht/~ged/Thingfish-Datastore-Filesystem

code:: https://hg.sr.ht/~ged/Thingfish-Datastore-Filesystem/browse

github:: https://github.com/ged/Thingfish-Datastore-Filesystem

docs:: http://deveiate.org/code/Thingfish-Datastore-Filesystem

== Description

This is a data storage plugin for the Thingfish digital asset manager.

It provides persistent storage for uploaded data to a simple filesystem

path.

== Installation

    $ gem install thingfish-datastore-filesystem

== Basic Usage

By default, data will be stored in a 'thingfish' directory, underneath

your system's tmpdir.  (For most systems, this will be /tmp/thingfish.)

This directory will need to exist before starting the Thingfish handler.

As with Thingfish itself, this plugin uses

Configurability[https://rubygems.org/gems/configurability] to modify

default behaviors.

Here's an example configuration file that enables this plugin and

modifies the default storage path:

  ---

  thingfish:

    datastore: filesystem

    filesystem_datastore:

      root_path: /net/storage/thingfish-data

== Advanced Usage

This plugin can utilize various advanced features of the Mongrel2 daemon

that can offload large data transfers to the Mongrel2 daemon directly.

This frees the Thingfish handler(s) up for servicing new requests

quickly, and dramatically improves performance for concurrent data

transfers and large files.

=== Assumptions

Because of Mongrel2's flexible architecture, you could equally run

everything under a single machine, or scale Thingfish out as a farm,

spanning hundreds of handlers across individual nodes, all with the same

configuration.  In any case, Your Thingfish handlers need access to the

same physical storage device(s) that the Mongrel2 daemon does.

How you accomplish this is entirely up to you, though NFS is a likely

choice.  There are two directories that the Mongrel2 server and the

Thingfish handlers will need access to -- one for downloads (this

plugin's 'root_path' directory), and one for uploads (the Mongrel2 async

spool directory.)

=== Downloads

To enable this functionality for downloads, you'll need to add the

'sendfile' filter in your Mongrel2 server configuration.  The path to

this library will vary, depending on where Mongrel2 was installed on

your system -- but it should look like something that resembles:

	xrequest '/usr/local/lib/mongrel2/filters/sendfile.so'

That filter is automatically built alongside Mongrel2, it just needs to

be switched on.  Ensure the 'root_path' is accessible from the Thingfish

handler(s) and the Mongrel2 daemon.

=== Uploads

Uploads also require altering Mongrel2 server settings:

	setting 'limits.content_length', 65536

This switches on async uploads, for any upload larger than 64k.

Anything under 64k is kept in RAM, and handled internally by the

Thingfish handler.  Adjust to taste.

	setting 'upload.temp_store', '/network/storage/mongrel2.upload.XXXXXXX'

For files larger than 'limits.content_length' size, asynchronously

spool the upload to this location.  When the file upload has completed,

Mongrel2 notifies Thingfish, and the spooled upload is properly stored.

This path must be accessible from the Thingfish handler(s) and the

Mongrel2 daemon.

	setting 'upload.temp_store_mode', '0600'

While this is not required, it is recommended to spool files so they

are readable only to the Mongrel2 process.  It is your responsibility

to ensure the user account running the Thingfish handler also has

read/write access to these files.

== Authors

- Michael Granger 

- Mahlon E. Smith 

== License

Copyright (c) 2014-2020, Michael Granger and Mahlon E. Smith

All rights reserved.

Redistribution and use in source and binary forms, with or without

modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice,

  this list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,

  this list of conditions and the following disclaimer in the documentation

  and/or other materials provided with the distribution.

* Neither the name of the author/s, nor the names of the project's

  contributors may be used to endorse or promote products derived from this

  software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"

AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE

IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE

DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE

FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL

DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR

SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER

CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,

OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE

OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.