Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bboe/deterministic_zip
A tool to generate consistent zip files (useful for terraform updates of aws lambda functions).
https://github.com/bboe/deterministic_zip
aws-lambda terraform zip
Last synced: about 2 months ago
JSON representation
A tool to generate consistent zip files (useful for terraform updates of aws lambda functions).
- Host: GitHub
- URL: https://github.com/bboe/deterministic_zip
- Owner: bboe
- License: bsd-2-clause
- Created: 2019-02-25T19:55:22.000Z (almost 6 years ago)
- Default Branch: main
- Last Pushed: 2021-07-17T06:44:01.000Z (over 3 years ago)
- Last Synced: 2024-10-20T15:44:03.983Z (2 months ago)
- Topics: aws-lambda, terraform, zip
- Language: Python
- Size: 14.6 KB
- Stars: 60
- Watchers: 2
- Forks: 9
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# deterministic_zip
A tool to generate consistent zip files.
This tool was specifically built to prevent zip file changes from triggering
AWS Lambda function updates when running `terraform apply`. Before this change,
every re-build of the zipfile would result in a different zip, even if its
contents had not changed.
## Requirements
This tool currently only runs on python3.7+ in order to use deflate compression
level 9.
## Installation
```sh
pip install deterministic_zip
```
## Creating a deterministic_zip
Run the following, and verify that your zip produces the same sha256 hash:
```sh
echo "The first file." > first
echo "The second file." > second
deterministic_zip archive.zip first second
sha256sum archive.zip
```
If you have `\n` line endings the result should be:
3afbd7c9b42bd5539ffd5c40499d3d1825157ed83791dce8d7ff2694189d28d6
If you have `\r\n` line endings (Windows) the result should be:
40e16270d62f15e7a192e88b1b301fa6540c86e7e897036b56be513341d376ed
## How does it work?
Great question! There are three tricks to building a deterministic zip:
1) Files must be added to the zip in the same order. Directory iteration order
may vary across machines, resulting in different zips. `deterministic_zip`
sorts all files before adding them to the zip archive.
2) Files in the zip must have consistent timestamps. If I share a directory to
another machine, the timestamps of individual files may differ, despite
having identical content. To achieve timestamp consistency,
`deterministic_zip` sets the timestamp of all added files to `2019-01-01
00:00:00`.
3) Files in the zip must have consistent permissions. File permissions look
like `-rw-r--r--` for a file that is readable by all users, and only
writable by the user who owns the file. Similarly executable files might
have permissions that look like: `-rwxr-xr-x` or
`-rwx------`. `deterministic_zip` sets the permission of all files added to
the archive to either `-r--r--r--`, or `-r-xr-xr-x`. The latter is only used
when the user running `deterministic_zip` has execute access on the file.
__Note__: `deterministic_zip` does not modify nor update timestamps of any
files it adds to archives. The techniques used above apply only to the copies
of files within archives `deterministic_zip` creates.