
By Patryk Antkiewicz, backend engineer at Unifa.
In some of our products we have a photo upload feature. The stable solution implemented a few years ago works like this: images are uploaded to an S3 bucket using a presigned URL, which triggers an SQS event that executes a Lambda function - the function processes the image and moves the result to a destination bucket, and the image is later referenced in the app via a presigned download URL. This solution was working reliably, but the stack was getting a bit old, so this spring I gave it a small upgrade, and I describe the process here.
Starting point
First, let's take a look at our starting point: an AWS Lambda implemented in Ruby 3.2 was using the ImageMagick library for photo processing. It accepts the S3 bucket and key of the uploaded image (PNG and JPEG formats are supported), along with the expected target file sizes and locations. The processing includes the following operations:
auto-rotating the image
stripping unneeded EXIF data
generating thumbnails of specified sizes
Since an ImageMagick build takes quite a long time, it was provided as a pre-built Lambda Layer, and the Lambda function code was uploaded directly using a zip file.
Ruby 3.2 was reaching EOL this spring, so together with upgrading it to 3.4 I decided to also try improving the implementation a little. The old stack had the following problems:
difficulty of local setup & testing (ImageMagick's local build takes ~20 minutes, and there was no Docker container)
inconvenient deployment (separate layer and code, deployed using a ZIP file)
using ImageMagick was overkill for such simple processing
Setting the goal
I searched for some more lightweight alternatives and libvips appeared to be the most promising choice - compared to ImageMagick it offers a smaller range of features, but opinions suggested that it is much faster and consumes less memory. To be more precise, instead of loading a whole image into memory, decoding it, applying every operation, and holding the full result in memory before encoding (roughly ImageMagick's model), it processes images in small "regions" and only computes the pixels it actually needs - which matters a lot for a Lambda function where both memory and time are billed directly. Just as importantly, libvips is a plain, statically-linkable C library we can easily build ourselves from source in a Docker image, instead of depending on a precompiled Lambda Layer.
For these reasons I decided to give it a go and also get rid of the existing deploy setup (Lambda Layer + Lambda code shipped as a Zip file) by replacing it with a single Docker image.
The migration process
From a zip layer to a container image
The core infrastructure change was moving the whole function from a Lambda zip package (function code + a separately-deployed ImageMagick layer) to a Lambda container image. The new Dockerfile is a two-stage build:
Builder stage - starts from
public.ecr.aws/lambda/ruby:3.4, installs build tooling (gcc-c++,meson,ninja-build, etc.) and the image codec dev headers (libjpeg-turbo-devel,libpng-devel,libwebp-devel,libexif-devel,lcms2-devel), downloads libvips source and builds it with Meson/Ninja, and runsbundle install.Runtime stage - starts fresh from the same base image, installs only the runtime versions of those codec libraries (no
-develheaders), and copies over the compiledvipsbinaries/libraries and the installed gems from the builder stage.
# --- Stage 1: Build libvips and Ruby Gems --- FROM public.ecr.aws/lambda/ruby:3.4 AS builder # Install AL2023 build dependencies RUN dnf install -y \ gcc-c++ make meson ninja-build pkgconfig tar xz \ glib2-devel expat-devel libjpeg-turbo-devel libpng-devel \ libwebp-devel orc-devel ruby-devel libexif-devel lcms2-devel # Download and build libvips from official source ARG VIPS_VERSION=8.18.0 WORKDIR /tmp RUN curl -L -O https://github.com/libvips/libvips/releases/download/v${VIPS_VERSION}/vips-${VIPS_VERSION}.tar.xz && \ tar xf vips-${VIPS_VERSION}.tar.xz && \ cd vips-${VIPS_VERSION} && \ meson setup build --prefix=/opt/vips --buildtype=release && \ cd build && \ ninja && \ ninja install # Install Ruby Gems WORKDIR ${LAMBDA_TASK_ROOT} COPY Gemfile Gemfile.lock ./ RUN gem install bundler:4.0.3 && \ bundle config set --local deployment 'true' && \ bundle config set --local path 'vendor/bundle' && \ bundle install # --- Stage 2: Final Runtime Image --- FROM public.ecr.aws/lambda/ruby:3.4 # 1. Install the runtime libraries that vips needs # These are the "base" versions (no -devel headers needed here) RUN dnf install -y \ expat \ glib2 \ libjpeg-turbo \ libpng \ libwebp \ orc \ libexif \ lcms2 \ && dnf clean all # Copy vips binaries and libraries from builder COPY --from=builder /opt/vips /opt/vips # Copy compiled gems WORKDIR ${LAMBDA_TASK_ROOT} COPY --from=builder ${LAMBDA_TASK_ROOT}/vendor/bundle vendor/bundle COPY --from=builder ${LAMBDA_TASK_ROOT}/.bundle .bundle # Set Environment Variables so the system finds 'vips' ENV LD_LIBRARY_PATH=/opt/vips/lib64:$LD_LIBRARY_PATH ENV PATH=/opt/vips/bin:$PATH # Copy application code COPY photo_processor.rb $LAMBDA_TASK_ROOT RUN chmod 644 photo_processor.rb CMD ["photo_processor.PhotoProcessor.main"]
This keeps the final image reasonably lean (~215 MB) while giving us full control over the libvips version and build flags. Building the whole image above takes about ~3.5 minutes, which is a big gain compared to ~20 minutes of just building the ImageMagick layer.
Deployment also moved from replacing the Lambda code (zip)/rebuilding layer to a more automated flow:
"create a Git tag with a specific format → CodeBuild automatically picks it up and builds the image → push to ECR → point a Lambda alias at the new image", driven by CodeBuild's buildspec.yml.
version: 0.2 phases: pre_build: commands: # ECR login - aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com - IMAGE_TAG=$(git rev-parse HEAD) build: commands: # build - docker build -t $REPO_NAME:latest -f Dockerfile . # tag - docker tag $REPO_NAME:latest $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$REPO_NAME:$ENV-release - docker tag $REPO_NAME:latest $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$REPO_NAME:$IMAGE_TAG # push - docker push $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$REPO_NAME:$ENV-release - docker push $ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$REPO_NAME:$IMAGE_TAG
Rewriting the image-processing commands
The core of the rewrite was swapping the two ImageMagick convert invocations for their vips equivalents (command was being called from Ruby using Open3.popen3):
Auto-orient original image
# ImageMagick (before) convert -auto-orient "src.jpg" "dest.jpg" # libvips (after) vips autorot "src.jpg" "dest.jpg[Q=92,optimize_coding=true]"
Generate thumbnail(s)
# ImageMagick (before) convert -define jpeg:size=NxN -resize NxN -sampling-factor 4:2:0 \ -density 1x1 -interlace line -quality 85 -auto-orient -strip \ "src.jpg" "dest.jpg" # libvips (after) vips thumbnail "src.jpg" "dest.jpg[Q=85,optimize_coding=true,strip=true,interlace=true]" N
Note: libvips isn't only available as a CLI - it also ships a Ruby gem
ruby-vips, so instead of building a shell command string and executing it directly, the same two
operations can be called as plain Ruby:
def generate_original(src_path, dest_path) options = dest_path.downcase.end_with?('.png') ? {} : { Q: 92, optimize_coding: true } Vips::Image.new_from_file(src_path).autorot.write_to_file(dest_path, **options) end def generate_thumbnail(src_path, dest_path, max_pixels) options = { Q: 85, optimize_coding: true, strip: true, interlace: true } Vips::Image.thumbnail(src_path, max_pixels).write_to_file(dest_path, **options) end
A few things to note:
vips thumbnailhandles resizing and auto-orientation together - there's no need for a separate rotate step before shrinking.- All of the output-format tuning (quality, chroma handling, metadata stripping, progressive/interlaced
encoding) lives in libvips's
[option=value]suffix syntax on the output filename rather than as separate CLI flags. - Thumbnails are always re-encoded as JPEG regardless of source format, to keep thumbnail size down - that behavior carried over unchanged from the ImageMagick version.
Getting the JPEG/PNG encoder options to actually match ImageMagick's defaults took a few iterations
(Q=95 → Q=92 to match IM's actual default; experimenting with subsample_mode, optimize_coding,
PNG compression/filter levels) before settling on the simplest set that produced visually
equivalent output.
The bug that cost me a lot of time: silent EXIF orientation failures
The part of this migration that took the longest wasn't the happy path - JPEGs without any odd EXIF
data rotated and resized correctly. The problem showed up specifically
with portrait photos carrying an EXIF orientation tag (e.g. orientation 6: a photo shot in
portrait on a camera that only records landscape pixel data with a "please rotate me" flag) - they were not being rotated properly.
I tried swapping vips autorot for vips copy "src[autorot=true]" dest and played with the JPEG encoder options, but it didn't help.
After a lot of debugging, the actual root cause turned out to be the list of system libraries installed in the Docker image, not the
Ruby code: libvips reads JPEG EXIF orientation through a library called libexif, and our runtime image wasn't installing it.
Without libexif present, vips autorot had nothing to read the orientation tag from, so it 'silently failed' - the command "succeeded", produced a valid JPEG, but just never rotated it.
result = PhotoProcessor.new(context).run(event) assert_equal(result[:result], 'success') # before the fix, this failed - the orientation tag never got normalized assert_equal(6, EXIFR::JPEG.new(local_src_path).orientation.to_i) # source: still tagged "rotate me" assert_equal(1, EXIFR::JPEG.new(output_path).orientation.to_i) # output: normalized / upright
Result
I compared the performance of both Lambdas (old and new) using relatively large JPG and PNG files (currently we handle files of up to 5 MB) - this includes auto-rotating the original, generating two thumbnails: small (max 250 pixels) and large (max 800 pixels), and uploading them to target bucket.
JPEG source (4.1 MB)
| Metric | ImageMagick | libvips |
|---|---|---|
| Duration | 4075.36 ms | 1522.80 ms |
| Billed Duration | 4824 ms | 2495 ms |
| Init Duration (cold start) | 748.50 ms | 972.42 ms |
| Max Memory Used | 333 MB | 282 MB |
PNG source (4.5 MB)
| Metric | ImageMagick | libvips |
|---|---|---|
| Duration | 4761.82 ms | 2359.93 ms |
| Billed Duration | 5426 ms | 3347 ms |
| Init Duration (cold start) | 663.99 ms | 986.98 ms |
| Max Memory Used | 162 MB | 152 MB |
Conclusions:
- Processing is meaningfully faster on libvips - roughly 2.7x for this specific JPEG case and 2x for the PNG
case, both in raw
Durationand inBilled Duration, which is what actually drives Lambda cost. - Peak memory is lower on libvips in both cases, although the difference is a bit smaller than I expected.
- Cold starts (
Init Duration) are slightly worse on libvips in both cases, which is most likely a result of spinning up the Docker container. This is an acceptable cost considering the overall advantage of making the deployment process much simpler.
The image below shows the average Lambda duration in production after the release, with purple line representing ImageMagick and green showing libvips - the difference is evident, although a bit smaller than in the simple comparison above (about a 25-30% improvement).

The current load is ~400 calls/min at peak time, and when it comes to the total monthly cost of Lambda execution, about a 40-50% reduction was achieved when comparing the three months before and after the release, so the impact was undoubtedly positive.
Where this could go next
While this change was just a slight refactoring of an established, working solution, it still has its shortcomings - the biggest one is the need to store the thumbnails (however small) in S3, which introduces some redundant costs. As a next step, I plan to explore the possibility of introducing a CloudFront image proxy, which would generate thumbnails "on the fly" and cache them.
Overall, this was a fairly contained change - the external interface (event shape, inputs, outputs) stayed exactly the same - but it paid off in every aspect: faster processing, lower memory and compute cost, and a much simpler deployment flow. The on-demand thumbnail idea above is the logical next step, but not an urgent one - the current setup should serve us well for a while yet.
Unifa is actively recruiting, please check our website for details: