3/30/2023 0 Comments Aws lambda imagemagick![]() ![]() Horizontal Scaling with AWS Lambdaīecause image uploads are very stochastic and unpredictable events, the best way to handle them is with horizontal scaling instead of vertical scaling. This solution may result in unacceptable processing times for creating the images’ variations, further deferring the resized images’ availability.įinally, don’t forget the cost of cool-down periods until the extra resources get shut down after the spike has passed. Handing this over to background workers would allow them to tackle it at their own pace, while auto scaling could bring up more resources as needed.Īlthough this solves the problem of over-provisioning to some extent, the eventual consistency has real-life implications. One could argue this is solvable by queuing the image processing. Working on this solution opened an opportunity to take the AWS Lambda route once LEVELS committed to using the AWS Cloud for their infrastructure needs. This was not the best way to approach the problem or spend the money. LEVELS ended up heavily over-provisioning the Amazon EC2 instances size just to handle occasional spikes. ![]() When adding multiple concurrent user uploads into the equation, the problem gets worse. Such processing times and RAM usage called for the use of bigger Amazon EC2 instances than what the application itself needs. ImageMagick used 1.5 GB of RAM-the maximum on AWS Lambda.Processing time exceeded two seconds per uploaded image.Setting up PoC (proof-of-concept) image processing, running on AWS Lambda, yielded the following results for the largest images: The LEVELS team’s initial idea was to simply resize images into the desired size variations on the API server, or as a background job on Amazon Elastic Compute Cloud (Amazon EC2) instances. Let’s take a look at the three solutions we proposed to LEVELS for resizing the user uploaded images. The main goal was to reduce the bandwidth usage by the web and mobile apps, thereby speeding up image delivery within the app. Because of this, there was a clear need for LEVELS to optimize the delivery of such images by providing smaller sizes and thumbnails. LEVELS accepts users’ uploaded images, which can easily reach more than 8000 x 5000 pixels in size and a few megabytes in weight on modern smartphones. This type of serverless solution can reduce the strain on your application programming interface (API) servers, and eliminate the need for running separate servers to handle spikes without crashing. In this post, I will review a horizontally scalable solution applied to LEVELS for their image upload processing. They can share their experiences with other users on the LEVELS social network through verified testimonials, and get endorsed by one of the LEVELS partners. This allows users to focus more on enjoying unique experiences and less on managing them. Payments are processed through LEVELS’ built-in “Get Up and Go” payment method, available in more than 6,000 locations globally. With LEVELS, a guest can visit a luxury restaurant, be immediately recognized by the staff, taken to preferred seating, and pay for dinner without ever taking out a credit card or managing logistics. Toptal helped build and automate the cloud infrastructure for LEVELS, a social network with an integrated payment function that also finds and applies VIP benefits users can get for the credit card programs they are a member of, things they own, or places they live. We provide application development and DevOps solutions for Amazon Web Services (AWS) customers. Toptalis an AWS Partner Network (APN) Advanced Consulting Partner with the AWS Lambda Service Delivery designation. As you scale, this process simply doesn’t work. The more concurrent uploads you get, the more they wreak havoc on your servers. Each image needs to be processed into several variations for thumbnails, previews, and reasonably-sized images for web and mobile display.Īll of that processing eats up the computing power (CPU) and random access memory (RAM). Let’s say you’ve got thousands of users who love to upload big and heavy images from their 40+ megapixel smartphone cameras. By Tomislav Capan, Principal Solution Architect at Toptal ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |