{"id":16612831,"url":"https://github.com/samkit-jain/aws-lambda-imagemagick-ghostscript","last_synced_at":"2025-03-10T19:46:21.934Z","repository":{"id":176661433,"uuid":"335383210","full_name":"samkit-jain/aws-lambda-imagemagick-ghostscript","owner":"samkit-jain","description":"Use ImageMagick and Ghostscript in AWS Lambda","archived":false,"fork":false,"pushed_at":"2021-02-02T18:55:55.000Z","size":82317,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-17T15:39:25.229Z","etag":null,"topics":["aws-lambda","ghostscript","imagemagick","python"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/samkit-jain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-02T18:20:55.000Z","updated_at":"2024-03-03T12:23:17.000Z","dependencies_parsed_at":"2023-07-24T23:31:49.672Z","dependency_job_id":null,"html_url":"https://github.com/samkit-jain/aws-lambda-imagemagick-ghostscript","commit_stats":null,"previous_names":["samkit-jain/aws-lambda-imagemagick-ghostscript"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samkit-jain%2Faws-lambda-imagemagick-ghostscript","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samkit-jain%2Faws-lambda-imagemagick-ghostscript/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samkit-jain%2Faws-lambda-imagemagick-ghostscript/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samkit-jain%2Faws-lambda-imagemagick-ghostscript/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/samkit-jain","download_url":"https://codeload.github.com/samkit-jain/aws-lambda-imagemagick-ghostscript/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242917516,"owners_count":20206514,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-lambda","ghostscript","imagemagick","python"],"created_at":"2024-10-12T01:43:49.586Z","updated_at":"2025-03-10T19:46:21.908Z","avatar_url":"https://github.com/samkit-jain.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Use ImageMagick and Ghostscript in AWS Lambda\n\n**Runtime:** Python 3.8\u003cbr/\u003e\n**TL;DR:** Download the ZIP [here](lambda-layer.zip) and skip to step 14.\n\nMy use-case was to convert pages in a PDF to PNG via `wand`. I followed the instructions provided by @chaddjohnson at https://gist.github.com/bensie/56f51bc33d4a55e2fc9a#gistcomment-3133859 but was getting errors. Following are the instructions on how I was able to resolve those:\n1. Start an EC2 instance and SSH into it. I used the AMI `amzn2-ami-hvm-2.0.20210126.0-x86_64-gp2`.\n2. Download ImageMagick 6.9.11.\n    ```bash\n    wget https://download.imagemagick.org/ImageMagick/download/ImageMagick-6.9.11-60.tar.gz\n    ```\n3. Extract the folder.\n    ```bash\n    tar zxvf ImageMagick-6.9.11-60.tar.gz\n    ```\n4. `cd` into the extracted folder.\n    ```bash\n    cd ImageMagick-6.9.11-60\n    ```\n5. Edit the `policy.xml` file to allow PDF to PNG conversion.\n    ```bash\n    nano config/policy.xml\n    ```\n    I copy-pasted the following content but you can modify it as needed.\n    ```xml\n    \u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n    \u003c!DOCTYPE policymap [\n    \u003c!ELEMENT policymap (policy)+\u003e\n    \u003c!ELEMENT policy (#PCDATA)\u003e\n    \u003c!ATTLIST policy domain (delegate|coder|filter|path|resource) #IMPLIED\u003e\n    \u003c!ATTLIST policy name CDATA #IMPLIED\u003e\n    \u003c!ATTLIST policy rights CDATA #IMPLIED\u003e\n    \u003c!ATTLIST policy pattern CDATA #IMPLIED\u003e\n    \u003c!ATTLIST policy value CDATA #IMPLIED\u003e\n    ]\u003e\n    \u003c!--\n      Configure ImageMagick policies.\n\n      Domains include system, delegate, coder, filter, path, or resource.\n\n      Rights include none, read, write, and execute.  Use | to combine them,\n      for example: \"read | write\" to permit read from, or write to, a path.\n\n      Use a glob expression as a pattern.\n\n      Suppose we do not want users to process MPEG video images:\n\n        \u003cpolicy domain=\"delegate\" rights=\"none\" pattern=\"mpeg:decode\" /\u003e\n\n      Here we do not want users reading images from HTTP:\n\n        \u003cpolicy domain=\"coder\" rights=\"none\" pattern=\"HTTP\" /\u003e\n\n      Lets prevent users from executing any image filters:\n\n        \u003cpolicy domain=\"filter\" rights=\"none\" pattern=\"*\" /\u003e\n\n      The /repository file system is restricted to read only.  We use a glob\n      expression to match all paths that start with /repository:\n      \n        \u003cpolicy domain=\"path\" rights=\"read\" pattern=\"/repository/*\" /\u003e\n\n      Let's prevent possible exploits by removing the right to use indirect reads.\n\n        \u003cpolicy domain=\"path\" rights=\"none\" pattern=\"@*\" /\u003e\n\n      Any large image is cached to disk rather than memory:\n\n        \u003cpolicy domain=\"resource\" name=\"area\" value=\"1GB\"/\u003e\n\n      Define arguments for the memory, map, area, width, height, and disk resources\n      with SI prefixes (.e.g 100MB).  In addition, resource policies are maximums\n      for each instance of ImageMagick (e.g. policy memory limit 1GB, -limit 2GB\n      exceeds policy maximum so memory limit is 1GB).\n    --\u003e\n    \u003cpolicymap\u003e\n      \u003c!-- \u003cpolicy domain=\"resource\" name=\"temporary-path\" value=\"/tmp\"/\u003e --\u003e\n      \u003cpolicy domain=\"resource\" name=\"memory\" value=\"256MiB\"/\u003e\n      \u003cpolicy domain=\"resource\" name=\"map\" value=\"512MiB\"/\u003e\n      \u003cpolicy domain=\"resource\" name=\"width\" value=\"16KP\"/\u003e\n      \u003cpolicy domain=\"resource\" name=\"height\" value=\"16KP\"/\u003e\n      \u003cpolicy domain=\"resource\" name=\"area\" value=\"128MB\"/\u003e\n      \u003cpolicy domain=\"resource\" name=\"disk\" value=\"1GiB\"/\u003e\n      \u003c!-- \u003cpolicy domain=\"resource\" name=\"file\" value=\"768\"/\u003e --\u003e\n      \u003c!-- \u003cpolicy domain=\"resource\" name=\"thread\" value=\"4\"/\u003e --\u003e\n      \u003c!-- \u003cpolicy domain=\"resource\" name=\"throttle\" value=\"0\"/\u003e --\u003e\n      \u003c!-- \u003cpolicy domain=\"resource\" name=\"time\" value=\"3600\"/\u003e --\u003e\n      \u003c!-- \u003cpolicy domain=\"system\" name=\"precision\" value=\"6\"/\u003e --\u003e\n      \u003c!-- not needed due to the need to use explicitly by mvg: --\u003e\n      \u003c!-- \u003cpolicy domain=\"delegate\" rights=\"none\" pattern=\"MVG\" /\u003e --\u003e\n      \u003c!-- use curl --\u003e\n      \u003cpolicy domain=\"delegate\" rights=\"none\" pattern=\"URL\" /\u003e\n      \u003cpolicy domain=\"delegate\" rights=\"none\" pattern=\"HTTPS\" /\u003e\n      \u003cpolicy domain=\"delegate\" rights=\"none\" pattern=\"HTTP\" /\u003e\n      \u003c!-- in order to avoid to get image with password text --\u003e\n      \u003cpolicy domain=\"path\" rights=\"none\" pattern=\"@*\"/\u003e\n      \u003cpolicy domain=\"cache\" name=\"shared-secret\" value=\"passphrase\" stealth=\"true\"/\u003e\n      \u003c!-- disable ghostscript format types --\u003e\n      \u003cpolicy domain=\"coder\" rights=\"none\" pattern=\"PS\" /\u003e\n      \u003cpolicy domain=\"coder\" rights=\"none\" pattern=\"EPI\" /\u003e\n      \u003cpolicy domain=\"coder\" rights=\"read|write\" pattern=\"PDF\" /\u003e\n      \u003cpolicy domain=\"coder\" rights=\"none\" pattern=\"XPS\" /\u003e\n      \u003cpolicy domain=\"coder\" rights=\"read|write\" pattern=\"LABEL\" /\u003e\n    \u003c/policymap\u003e\n    ```\n6. Configure and install ImageMagick.\n    ```bash\n    ./configure --prefix=/var/task/imagemagick --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared=no --enable-static=yes --with-modules --with-perl=no --with-x=no --with-gslib=no --with-lcms --without-rsvg --with-xml --without-dps --disable-hdri --with-quantum-depth=8 --disable-openmp\n    make\n    sudo make install\n    ```\n7. Copy the required `.so` files.\n    ```bash\n    mkdir lib\n    cd /usr/lib64/\n    cp -L libbz2.so.1 libexpat.so.1 libfontconfig.so.1 libfreetype.so.6 libgs.so.9 libjbig.so.2.0 libjpeg.so.62 liblcms2.so.2 liblzma.so.5 libpng15.so.15 libtiff.so.5 libxml2.so.2 libMagickCore-6.Q16.so.6 libMagickWand-6.Q16.so.6 libXext.so.6 libXt.so.6 libltdl.so.7 libSM.so.6 libICE.so.6 libX11.so.6 libgomp.so.1 libuuid.so.1 libxcb.so.1 libXau.so.6 libMagickCore-6.Q8.so.6 libMagickWand-6.Q8.so.6 libm.so.6 libz.so.1 libjasper.so.1 /home/ec2-user/lib/\n    cp -r ImageMagick-6.9.10/ ImageMagick-6.9.11/ /home/ec2-user/lib/\n    cd /home/ec2-user\n    tar zcf lib.tar.gz lib/\n    ```\n    Copy the `lib.tar.gz` file from the server to your local machine.\n8. Copy the required binary files.\n    ```bash\n    cd /var/task/imagemagick\n    sudo tar zcf bin.tar.gz bin/\n    cp bin.tar.gz /home/ec2-user/bin.tar.gz \n    ```\n    Copy the `bin.tar.gz` file from the server to your local machine.\n9. Copy the XML files required by ImageMagick.\n    ```bash\n    cd /etc/\n    sudo tar zcf etc.tar.gz ImageMagick-6/\n    cp etc.tar.gz /home/ec2-user/etc.tar.gz\n    ```\n    Copy the `etc.tar.gz` file from the server to your local machine.\n10. Close the SSH session.\n11. On your local machine, extract the contents of the 3 `*.tar.gz` files.\n12. Download ghostscript from https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs9533/ghostscript-9.53.3-linux-x86_64.tgz and extract the ghostscript binary into the `bin/` folder and rename it to `gs`. Run `chmod +x bin/gs` to make it executable.\n13. Compress the 3 - `lib`, `bin` and `etc` - folders into a ZIP file. The tree structure of the ZIP file would look like\n    ```\n    file.zip/\n    |-- bin\n    |   |-- convert\n    |   |-- ...\n    |   `-- gs\n    |-- etc\n    |   `-- ImageMagick-6\n    |       |-- coder.xml\n    |       |-- ...\n    |       `-- type.xml\n    `-- lib\n        |-- ImageMagick-6.9.10\n        |   |-- config-Q16\n        |   |   `-- configure.xml\n        |   `-- modules-Q16\n        |       |-- coders\n        |       |   |-- aai.la\n        |       |   |-- ...\n        |       |   `-- yuv.so\n        |       `-- filters\n        |           |-- analyze.la\n        |           `-- analyze.so\n        |-- ImageMagick-6.9.11\n        |   |-- config-Q8\n        |   |   `-- configure.xml\n        |   `-- modules-Q8\n        |       |-- coders\n        |       |   |-- aai.la\n        |       |   |-- ...\n        |       |   `-- yuv.so\n        |       `-- filters\n        |           |-- analyze.la\n        |           `-- analyze.so\n        |-- libICE.so.6\n        |-- ...\n        `-- libz.so.1\n    ```\n    I have used `...` wherever the folder contained more than 2 files to denote that there are more files present.\n14. Create a Python 3.8 runtime compatible layer on AWS Lambda and use the ZIP created in step 13.\n15. Add the layer to your AWS Lambda function code.\n16. Update environment variables in your lambda function.\n    ```python\n    import os\n\n    os.environ[\"PATH\"] = f\"/opt/bin:{os.environ['PATH']}\"\n    os.environ[\"LD_LIBRARY_PATH\"] = f\"/opt/lib:{os.environ['LD_LIBRARY_PATH']}\"\n    os.environ[\"MAGICK_HOME\"] = \"/opt/\"\n    os.environ[\"WAND_MAGICK_LIBRARY_SUFFIX\"] = \"-6.Q8\"\n    os.environ[\"MAGICK_CONFIGURE_PATH\"] = \"/opt/etc/ImageMagick-6/\"\n    os.environ[\"MAGICK_CODER_MODULE_PATH\"] = \"/opt/lib/ImageMagick-6.9.11/modules-Q8/coders/\"\n    ```\n\nNote: If the size of the uncompressed ZIP file is too large and you reach AWS Lambda size limits, remove the binaries that you don't need from the `bin/` folder. In my case, I only kept `Magick-config`, `MagickCore-config`, `MagickWand-config`, `Wand-config`, `convert` and `gs` and removed others.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamkit-jain%2Faws-lambda-imagemagick-ghostscript","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamkit-jain%2Faws-lambda-imagemagick-ghostscript","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamkit-jain%2Faws-lambda-imagemagick-ghostscript/lists"}