{"id":15914615,"url":"https://github.com/bbengfort/flashcube","last_synced_at":"2025-07-08T23:07:15.115Z","repository":{"id":69222730,"uuid":"44417854","full_name":"bbengfort/flashcube","owner":"bbengfort","description":"A simple, standalone CryptoService for storing encrypted key value pairs.","archived":false,"fork":false,"pushed_at":"2015-10-17T03:51:44.000Z","size":172,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-03T03:44:14.264Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bbengfort.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-17T00:44:35.000Z","updated_at":"2015-10-17T03:42:40.000Z","dependencies_parsed_at":"2023-03-06T12:56:18.533Z","dependency_job_id":null,"html_url":"https://github.com/bbengfort/flashcube","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bbengfort/flashcube","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fflashcube","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fflashcube/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fflashcube/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fflashcube/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bbengfort","download_url":"https://codeload.github.com/bbengfort/flashcube/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fflashcube/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264363794,"owners_count":23596507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-06T17:04:39.815Z","updated_at":"2025-07-08T23:07:15.100Z","avatar_url":"https://github.com/bbengfort.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Before Instagram, there was Flashcube for Kodak Instamatic Cameras.][flashcube_img]\n\n# Flashcube\n\n[![Build Status][build_status_img]][build_status_page]\n[![Coverage Status][coveralls_img]][coveralls_page]\n[![Stories in Ready][waffle_img]][waffle_page]\n\n[build_status_img]: https://travis-ci.org/bbengfort/flashcube.svg \"Build Status\"\n[build_status_page]: https://travis-ci.org/bbengfort/flashcube \"Travis\"\n[coveralls_img]: https://coveralls.io/repos/bbengfort/flashcube/badge.svg?branch=master\u0026service=github\n[coveralls_page]: https://coveralls.io/github/bbengfort/flashcube?branch=master\n[waffle_img]: https://badge.waffle.io/bbengfort/flashcube.png?label=ready\u0026title=Ready \"Stories in Ready\"\n[waffle_page]: https://waffle.io/bbengfort/flashcube\n\n_A simple, standalone CryptoService for member data integrity._\n\nFlashcube is a simple WSGI service that RESTfully allows for the fetching\nand storage of data that needs to be encrypted, namely member-secure data.\nAPI access is secured with [HMAC Authentication][hmac_auth] and the\nservice itself provides all key storage, encryption, and decryption. In\nthe first implementation, it also maintains access to the secure data\nstore.\n\n[Before Instagram, there was Flashcube for Kodak Instamatic Cameras.][flashcube_vimeo]\n\n## Table of Contents ##\n\n1. [HMAC Authentication](#hmac_authentication)\n2. [Running Flashcube Locally](#running_locally)\n3. [Adding Clients](#adding_clients)\n4. [Generating Keys](#generating_keys)\n5. [Flashcube API v1](#flashcube_api_v1)\n6. [Encryption Methodology](#encryption_methodology)\n7. [TODO](#todo)\n\n\u003ca id=\"hmac_authentication\"\u003e\u003c/a\u003e\n## HMAC Authentication ##\nFlashcube requires HMAC style authentication added to the request body of\nevery request to the Flashcube service. This section details how to create\nan HMAC signature for use with the service.\n\nEvery client will register with Flashcube (their name, IP address, and a\ndescription of the service they will be using Flashcube for). In return\nthey will receive (via a command-line script for now, perhaps a web form\nin the future) an _API KEY_ and an _API SECRET_.\n\nThe _API KEY_ and _API SECRET_ are used together in order to create an\nHMAC (Hashed Message Authentication Code) that will be sent to Flashcube,\nalong with the API Key and a timestamp of the request. THE _API SERET_\nshould be kept secret, and if compromised, should be regenerated with the\nAPI service.\n\nHere are the steps for generating HMAC for _each_ request:\n\n1. Determine the current UTC timestamp integer with milliseconds\n   (millisecond epoch time).\n2. Concatenate the _API KEY_ with the timestamp\n3. Create a SHA256 signature of the concatenated key, stamp with the\n   _API SECRET_ - this is the HMAC\n4. Send the _API KEY_, timestamp, and the HMAC (base64 encoded) with the\n   request.\n\nNote that Flashcube will maintain the last request timestamp for each\nclient, and will not permit out of time order requests. The server will\nalso not accept requests whose timestamps are outside of a particular\ntime interval. This is to prevent replay attacks.\n\n**Sending the HMAC with the request**:\n\nThe HMAC is included in the request header before the body, as part of the\nHTTP Authorization. The string format for the authorization is actually\nvery formal:\n\n    Authorization: FLASHCUBE [APIKey]:[HMACb64]\n    Content-MD5: [MD5(BODY)]\n    Time: [TIMESTAMP INTEGER WITH MILLISECONDS]\n\nLet's inspect the Authorization header: The header value begins with the\nauthorization type- in this case \"FLASHCUBE\". It is followed by a \":\"\nseparated string value where the front of the colon is the APIKey and the\nrest of the string after the colon is the base64 encoded HMAC. Note that\nthis value should be fairly precise in terms of regular expression\nstandards, e.g. no whitespace between the key and the colon or the colon\nand the HMAC.\n\nThe other component of the Authorization is the timestamp. This should be\nincluded in a non RFC-2616 compliant header called \"Time\". This header\nshould be the integer time in milliseconds from epoch, used to compute the\nHMAC. This should not be confused with the \"Date\" header as this header\ncan be manipulated by other software objects and is part of the RFC-2616\nstandard.\n\nThe last, optional component is the Content-MD5: this is a MD5 hash of the\nbody and is used as a finger print of the body if sent over plaintext. This\nis currently optional for use in our HMAC authorization methodology.\n\n### Python Implementation ###\n\n    import time\n    import hmac\n    import base64\n    import hashlib\n    import calendar\n\n    from datetime import datetime\n\n    def get_utc_timestamp():\n        utcnow    = datetime.utcnow()\n        millisec  = utcnow.microsecond/1e3\n        timestamp = calendar.timegm(utcnow.timetuple())*1e3 + millisec\n        return int(timestamp)\n\n    def create_hmac(api_key, secret):\n\n        timestamp = get_utc_timestamp() # TS in Milliseconds\n        message   = api_key + str(timestamp)\n        signature = hmac.new(secret, message, digestmod=hashlib.sha256)\n        signature = base64.b64encode(signature.digest())\n\n        return {\n            'ts': timestamp,\n            'apiKey': api_key,\n            'hmac': signature,\n        }\n\n### Node Implementation ###\n\n**TODO: Document**\n\n### Ruby Implementation ###\n\n**TODO: Document**\n\n\u003ca id=\"running_locally\"\u003e\u003c/a\u003e\n## Running Flashcube Locally ##\nTo run Flashcube in a local environment:\n\n1. Create a new virtual environment for Flashcube (assuming you are using virtualenvwrapper):\n\n    $ mkvirtualenv flashcube\n\n2. Git clone the flashcube repository into the desired location:\n\n    $ git clone git@github.com:bbengfort/flashcube.git\n\n3. cd into the flashcube projects and install required Python packages within the virtualenv:\n\n    $ cd flashcube\n    $ pip install -r requirements.txt\n\n4. Create an environment variable for FLASHCUBE_SETTINGS that loads the Development Configuration\n(you can also put this in your virtualenv's activate hook or bash_profile):\n\n    $ export FLASHCUBE_SETTINGS = 'flashcube.conf.DevelopmentConfig'\n\n5. Flashcube's Development configurations assume that you have a Postgres database on localhost with\nname flashcube for user flashcube and password d4t41slava!\n\n    $ sudo su postgres\n    $ createuser -P flashcube (enter password d4t41slava! twice)\n    $ psql template1\n    psql\u003e CREATE DATABASE flashcube OWNER flashcube ENCODING 'UTF8';\n    psql\u003e \\q\n    $ exit\n\n6. Follow directions to generate a secret key and add clients below.\n\n7. Start flashcube:\n\n    $ bin/flashcube\n\n\n\u003ca id=\"adding_clients\"\u003e\u003c/a\u003e\n## Adding Clients ##\nAdding clients to Flashcube is performed through a command line utility at\nthe moment. This utility can be found in `bin/flashcube-addclient` (at\nleast the execution script) and the codebase can be found in\n`flashcube.console.addclient` - it extends the `simpleconsole` module that\nwe have leveraged in other projects like Breccia.\n\nWhen you run the script you will be asked for input to populate the client\ntable, but only the name option is required. You can also add this data\ndirectly without `stdin` by passing named arguments to the script.\n\n    $ bin/flashcube-addclient\n    Please enter the details for the new API client:\n        Name of client: Test Client\n        Description:    This is a test server that will access the main database\n        IP Address:     10.1.10.2\n    Is this correct? (yes|no): yes\n\n    ✓ Generating API Key and Secret\n    ✗ Database save not implemented yet.\n\n    ✫  API Key:    pm3FiuIhlFkyW9DzoTGwxQ\n    ✫  API Secret: XfWyCaznW8TgtteK2BdCEzQn4fIdXzBKrR5a9julizs\n\nUsage:\n\n    bin/flashcube-addclient [options]\n\n    Generates an API Key and Secret for Clients and saves them.\n\n    Options:\n      -n NAME, --name=NAME           Add the name via the command line\n      -d DESC, --description=DESC    Add a description via the command line\n      -i INET, --ipaddr=INET         Add the ip address via command line\n      --version                      Show program's version number and exit\n      -h, --help                     Show this help message and exit\n\nFuture work will include data type validations, and handling of database\nexceptions.\n\nNote that the API Key and API Secret are generated using a similar\nmethdology found in [Simple API Key Generation in Python][key_gen], which\ngenerates keys very similar to AWS or to Goodreads.\n\n\u003ca id=\"generating_keys\"\u003e\u003c/a\u003e\n## Generating Keys ##\nSecret keys should be randomly generated rather than by using some easy to\ndeciper plaintext key. Additionally, our secret keys written to disk should\nbe encrypted with a strong password that will be prompted for at runtime.\n\nTo aid in the generation of the keys for use in this app, there is a\ncommand line utility called `flashcube-keygen` that behaves in a similar\nmanner to `ssh-keygen`, which is used for creating public and private keys\nfor RSA encryption. This utility can be found in `bin/flashcube-keygen`\nand the codebase can be found in `flashcube.console.secret` as it extends\nthe `simpleconsole` lib in a similar manner to `flashcube-addclient`.\n\nWhen you run the script, it will prompt you for the details to create the\nkey, including the location to write to disk, and a password for\nencryption.\n\n    $ bin/flashcube-keygen\n    Generating Flashcube cryptographic secret.\n    Enter file in which to save the key (/home/ubuntu/.private/flashcube.key):\n    Enter passphrase (empty for no passphrase):\n    Enter same passphrase again:\n    The Flashcube secret has been saved in /home/ubuntu/.private/flashcube.key\n    The key fingerprint is:\n    ef:32:ff:35:55:bd:ce:c2:f3:96:29:e3:de:fa:9a:b0\n    $\n\n**Usage**:\n\n    bin/flashcube-keygen [options]\n\n    Generates, manages, and converts keys for flashcube cryptography\n\n    Options:\n      -f                    Force the writing of the key to the filesystem.\n      --version             show program's version number and exit\n      -h, --help            show this help message and exit\n\nFuture work should include more securely randomly generated keys (using\n`Crpyto.Random` instead of the Mersenne Twister Pseudo Random Number\nGenerator). We could also make the console program prettier (like\naddaclient) or update with a randomart image.\n\n\u003ca id=\"flashcube_api_v1\"\u003e\u003c/a\u003e\n## Flashcube API v1 ##\n\n\u003e **HOST**: `https://flashcube.herokuapp.com`\n\n### Cube Resources ###\nThe following section describes how to access cryptography (cube) services.\n\n**Note**: any base64 encoded string in URL resources must be URLEncoded with\n% escaping. See the note on encoding below.\n\n#### `POST /cube/` ####\nAdd member data to the crypto store\n\n    POST /cube/\n    \u003e Content-Type: application/x-www-form-urlencoded\n    \u003e Authorization: FLASHCUBE [APIKey]:[HMACb64]\n    \u003e Time: [UTCTS]\n    email_hash=tXL3Sp1ewZJCoCzUzqV%2FRrJjWZCt4ovwqMYLghV719U%3D\n    \u0026password=plaintextsecret\n    \u003c 201\n    \u003c Content-Type: application/json\n    { \"success\": true, \"status\": \"created\" }\n\n#### `GET /cube/:email_hash/` ####\nFetch member data from the crypto store\n\n    GET /cube/:email_hash/\n    \u003e Authorization: FLASHCUBE [APIKey]:[HMACb64]\n    \u003e Time: [UTCTS]\n    \u003c 200\n    \u003c Content-Type: application/json\n    { \"success\": true,\n      \"email_hash\": \"tXL3Sp1ewZJCoCzUzqV/RrJjWZCt4ovwqMYLghV719U=\",\n      \"password\": \"plaintextsecret\" }\n\n#### `PUT /cube/:email_hash/` ####\nUpdate member data in the crypto store\n\n    PUT /cube/:email_hash/\n    \u003e Content-Type: application/x-www-form-urlencoded\n    \u003e Authorization: FLASHCUBE [APIKey]:[HMACb64]\n    \u003e Time: [UTCTS]\n    password=plaintextsecret\n    \u003c 200\n    \u003c Content-Type: application/json\n    { \"success\": true, \"status\": \"updated\" }\n\n#### `DELETE /cube/:email_hash/` ####\nDelete member data from the crypto store\n\n    DELETE /cube/:email_hash/\n    \u003e Authorization: FLASHCUBE [APIKey]:[HMACb64]\n    \u003e Time: [UTCTS]\n    \u003c 200\n    \u003c Content-Type: application/json\n    { \"success\": true, \"status\": \"deleted\" }\n\n### Encoding Strings to Flashcube ###\n\nThere are two main encoding issues to deal with in Flashcube: unicode\nand base64 encoding. This section describes how to deal with both in\nFlashcube.\n\n**Base64**\n\nBase64 encoding is a scheme that represents binary data in a textual form,\nmuch like hexadecimal representations. The primary benfit of this encoding\nis that it provides a 4/3 byte ratio to the original binary data, as\nopposed to a 2/1 byte ratio for hexadecimal encoding.\n\nBase64 encoding, however, is not URL safe as it uses not just the characters\n`a-zA-Z0-9` (62 characters) but also the characters `+`, `/` and to represent\nnecessary padding at the end, `=`. In URL encoding schemes both `+` and `/`\nare special characters not considered safe because they identify either a\nspace or a path seperator. The `=` is used in parameterization for query\nstrings and url-encoded form data. Something must be done with these\ncharacters before transmission over HTTP.\n\nSeveral methodologies are proposed, including a URL safe Base64 encoding that\nreplaces, `+/` with `-_`, but we prefer to simply %-quote the characters and\ntherefore maintain a URL safe Base64 string at the cost of a few extra bytes,\nwhile allowing us to use normal Base64 operations outside of the URL context.\nTo do this, replace the `+` with `%2B`, replace `/` with `%2F` and replace\n`=` with `%3D`.\n\nFor more information on this topic see: [Base64 URL Applications](http://en.wikipedia.org/wiki/Base64#URL_applications).\n\n**Unicode**\n\nFlashcube supports unicode entries and expects all requests to be made in\nUTF-8. All passwords are stored as binary encoded strings, and are returned\nas unicode strings decoded in UTF-8.\n\nThis means that clients should expect that the raw ASCII body of the\nresponse will contain byte encoded unicode, e.g. `'exon\\u00e9r\\u00e9e'`.\nHowever, if the body is encoded to UTF-8 (and most JSON libraries will do\nthis, including Javascripts' `JSON.parse` function) the unicode string will\nbe correctly displayed e.g. `'exonérée'`.\n\n\u003ca id=\"encryption_methodology\"\u003e\u003c/a\u003e\n## Encryption Methodology ##\n\nThe encryption methodology takes into account the specific encryption\nrequirements for a lightweight credentials app, and the process is described below.\n\n**Key Requirements**\n\n1. Storage of a plaintext value that is less than 256 bytes.\n2. Use of a strong cryptographic cipher algorithm.\n3. Use of a strong key applied to the cryptographic cipher.\n4. Assertion that decryption was successful.\n5. Salting the cipher with an IV to prevent pattern awareness.\n6. Storage of unicode byte strings.\n\nBecause of these requirements, the following topics become very important:\n\n* Selection of cryptographic cipher algorithm and mode.\n* Key size and length.\n* Padding of both the key and the plaintext.\n* Encoding and decoding of the string value of the cipher.\n* Checksums appended to the plaintext before encryption.\n* IV prepended to the ciphertext for storage.\n7. Encoding and decoding of UTF-8 data.\n\nWithout getting into too much into details, here is an outline of the\nmethodology already implemented for review.\n\n**Encryption**:\n\n1. If the input is unicode, ASCII encode as UTF-8.\n2. Pad key to a length of 32 bytes (or use SHA256)\n3. Generate a random initialization vector of 16 bytes\n4. Generate a CRC32 checksum of the plaintext, and append\n5. Pad the plaintext+checksum with an ord/chr mechanism (see more below)\n6. Encrypt the padded plaintext+checksum with AES in CBC mode with the IV\n7. prepend the IV to the ciphertext\n8. Encode the IV+ciphertext in base64\n\n**Decryption**:\n\n1. Decode the IV+ciphertext from base64\n2. Split the first 16 bytes off the ciphertext = IV\n3. Decrypt the ciphertext with AES in CBC mode with IV\n4. Unpad the plaintext with ord/chr mechanism (see more below)\n5. Break off the last 4 bytes of the plaintext (checksum) and test checksum\n6. Return the plaintext without the checksum, decoded into UTF-8.\n\nSome notes:\n\n* Should we use a cipher other than AES?\n* The random IV/CBC mechanism salts our cryptography to prevent pattern matching.\n* The checksum is not cryptographically secure, but is encrypted with the plaintext.\n* Should we switch to a cryptographic checksum like SHA1 or MD5?\n\n**Padding**:\n\nPadding is very simple, if the length of the plaintext is not divisible\nin 16 byte chunks (or the cipher blocksize), then we pad the end to the\nnext number that is divisible by the blocksize. To do that, we first take\nthe difference of the blocksize to length then modulo the blocksize. This\nis the number of bytes we need to append for padding.\n\nWe also use that value to determine the char to pad with. Using the number\nas the ordinal of the ASCII char to pad with, we pad with that many of\nthat char. Unpadding then is simple- we rstrip the number of chars of the\nordinal of the last char.\n\nCollisions are prevented because in a block size of 16- the first 15\ncharacters are all whitespace chars in both ASCII and Unicode, and\nwhitespace is not valid in the text we're encrypting. Moreover, we only\nremove the amount of whitespace that we appended to the end of the string,\nso in the unlikely event the last char of the string is the same whitespace\ncharacter, only the number of the ordinal is removed.\n\nFinally, in the case of an input that is exactly the blocksize, we increase\nthe size by 16 bytes, appending extra whitespace to the end. This is to\nensure that the unpadding functions correctly.\n\n\u003ca id=\"todo\"\u003e\u003c/a\u003e\n## TODO ##\n\nAttached are some ideas to make this service better:\n\n* Should we protect the api client secrets in the database?\n* Web form client registration.\n* Can we utilize a keychain instead of disk storage?\n* Create manage.py-like utility instead of seperate utilities.\n* Add Content-MD5 checking to HMAC authorization.\n* Add timestamp and serial request checking to authorization.\n* Add logging functionality\n* Use create_app method in __init__.py\n* Add fixture for testing based on John the Ripper's password list.\n\n\u003c!-- Link References --\u003e\n\n[hmac_auth]: http://www.wolfe.id.au/2012/10/20/what-is-hmac-and-why-is-it-useful/ \"Mark Wolfe's Blog: What is HMAC and Why is it Useful?\"\n[flashcube_img]: http://b.vimeocdn.com/ts/295/454/295454469_640.jpg \"Before Instagram, there was Flashcube for Kodak Instamatic Cameras.\"\n[flashcube_vimeo]: https://vimeo.com/42583181 \"Vimeo\"\n[key_gen]: http://jetfar.com/simple-api-key-generation-in-python/ \"Simple API Key Generation in Python\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbengfort%2Fflashcube","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbbengfort%2Fflashcube","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbengfort%2Fflashcube/lists"}