155. How We Made AWS Containers Start Up to 10× Faster!
What if you could make your AWS containers start up to 10× faster without changing your application code? In this episode of AWS Bites, Eoin and Luciano explore practical ways to reduce container startup time, from smaller Docker images and faster compression to lazy loading with SOCI (Seekable OCI).
We start with the fundamentals: multi-stage Docker builds, image layers, caching, and the trade-offs between gzip and zstd compression. Then we explain how SOCI lets containers start without downloading and unpacking every layer upfront, covering containerd snapshotters, FUSE, zTOCs, compressed spans, and HTTP Range requests.
To put these techniques to the test, we share our open-source benchmark covering 44 configurations and 1,320 ECS task executions. We compare image sizes, compression methods, ARM64 and x86-64 architectures, and Fargate against ECS on EC2 with Amazon Linux 2023 and Bottlerocket. In some of our large-image Fargate benchmarks, SOCI delivered up to a 10× startup improvement.
We also discuss SOCI support across Fargate, SageMaker, EKS, and ECS on EC2, and walk through the tools and workflow for generating SOCI indexes and publishing images to ECR. We close with the practical trade-offs: when zstd might be enough, when SOCI is worth adopting, and why time to useful work matters more than container startup time alone.
Thanks to fourTheorem for powering AWS Bites. We help teams build cloud systems that are simple, scalable, and cost effective. Visit fourTheorem.com.
In this episode, we mentioned the following resources:
- Container startup benchmark and interactive results
- The hidden design compromises of Docker layers
- Building x86 Rust containers from Apple Silicon
- AWS deep dive: Lazy loading container images with SOCI
- SOCI Snapshotter on GitHub
- AWS SageMaker SOCI documentation
- ECS roadmap: SOCI snapshotter support
- ECS roadmap: containerd support
We are back. If you have been wondering where AWS Bites disappeared to, we took a shortish summer break. Well, it was supposed to be a short summer break and yet winter is coming now at this point. But you know how it goes, holidays, work, family, random projects. Sometimes life just gets in the way of your perfectly optimized AWS podcast release schedule, right? That happens to everyone, I'm sure. So yes, we might've been a little bit slow at launching a new podcast episode, but that doesn't mean that your containers have to be slow at launching as well, right?
So that's exactly what we are going to talk about today, how to make containers start fast on AWS. And this is important because if you work with large containers, they can take significant amount of time to download, unpack, and eventually start. So we'll explore some of the common techniques to speed things up from when you have to deal with smaller images, but also when you have bigger images. We'll talk about compressions.
And most importantly, we'll talk about Soci, which is an optimization that many teams still don't know about or even use. So we'll explain what Soci is, how it works, where you can use it in AWS, and a few things that we have discovered by running our very own benchmarks. My name is Luciano, and today I'm joined by Eoin, and this is AWS Bites. OK, so just before we get into the meat of the topic of today, I just want to recap quickly how important it is that your containers start fast.
And there are a few reasons, but the main one to me is that if you are dealing with solutions that can have traffic spikes, and you built it in such a way that new infrastructure can come up to deal with the spike, and this infrastructure is using containers, guess what? If your containers are very slow at starting, then you're not going to be so efficient at dealing with the spike because it's going to take some time before those new containers are actually going to start to handle requests.
So you want your containers to start fast to be able to deal with spikes. And similarly, if you're doing quick or you're doing frequent deployments, and maybe you have an outage and you figure out there is a bug and you have a fix and you want to ship it as quick as possible. If you are doing all of that through containers, of course you want your new containers, the one that fixes the issue, to come up as quick as possible because that means restoring your environment quickly.
And similarly, if you run short-lived jobs, of course, you don't want to spend most of the time just bootstrapping a container that then finishes very quickly. So it's important to balance it out. This is very common when you have, I don't know, MapReduce jobs, when you are distributing lots of computation to many containers, and they may be not running for a long time, just giving you that scale of parallelizing the workload.
You want, of course, each container to come up quickly and start to do the job as quick as possible. then when you have, in general, user-facing latency, of course, the startup container time is in the critical path. So we want to minimize the time between needing to compute and when the compute actually starts and does useful work. So with all of that being said, and I hope that gives us a good context, what are some of the more common advices that we can leverage to make our kind of dummy container. We didn't think too much about it and optimize it a little bit. So it starts faster.
Well, the simple stuff is don't put things in your images you don't need because smaller images means you have less data to download and decompress, less IO and generally faster deployments. So that means using .dockerignore files to exclude files from your build context, which would then be picked up by copy commands within your Docker file. Also removing package manager caches and temporary files. Deleting files in a later Docker layer with a separate command won't make an earlier layer smaller, so you need to do it all in the one run instruction.
And Luciano actually was inspired to write a deep dive into Docker layers, which I really recommend reading. It emerged from all the work you did on researching this episode. So a link to that will be in the show notes. Another piece of advice is using multi-stage builds. That's quite common now. You see it quite a lot. So you can compile your application with all the necessary tools, then have a slimmer base image and copy all of the runtime artifacts into something clean and lean.
Then using leaner base images with like Slim or Distroless or even Scratch where appropriate. Just remember you can make debugging harder if you need to shell into a container. and open up Vim or even run things like PS. Structure layers for caching as well. So you can put infrequently changing dependencies first before your application code. And you can also use things like compiled languages, you know, Go, C, C++, Rust, Zig.
They can help you produce very small runtime images. Sometimes it could just be a little single binary. We won't spend the whole episode talking about Dockerfile optimization. That's a little bit standard, I guess. There's loads of resources out there for that. But by the way, there are other things you could do depending on which technology you're using to run your services. Like if you're using ECS on EC2, you can use bridge networking mode, and that cuts out the time to provision the ENI for each task. And that's something that could really add a lot of time to your launch. The trade-off is that you would lose the ability to have dedicated security groups for your tasks. You have to use the EC2's security group. But that's an aside, that depends where you're deploying to. Let's get into, I suppose, some more of the deployment and startup time optimizations, starting with compression.
Yes, compression, it's an interesting topic because, again, it's one of those things that I don't feel like many people are actually exploring or leveraging enough. So let's talk about it and let's see what's possible. And first of all, I think it's important to understand that a container image is effectively It's made up of layers and layers are typically stored as compressed archives in a registry and most often it's using something like gzip for compression.
And when we talk about compression we always need to remember what are the trade-offs there. We are effectively trying to reduce the number of bytes that we need to download because compression effectively minimizes the amount of bytes that that's what's built for. But at the same time, to be able to use the information, so in this case, the files within a Docker image, you need to decompress it first. So effectively, we are saving time at download because we are downloading less, but then we might be spending more time to actually make that download useful because we'll need to decompress and that's all CPU time that it spent.
So it's important to figure out what is the right balance, and different compression algorithms can also be tuned. They have different compression levels, for example, where you can say, OK, compress more, but that generally means that then you will need more CPU work to decompress, or compress less, which means you save less bytes, but then it's much faster at decompressing. And again, depending on the algorithm, there are lots of nuances there.
So it's not an obvious choice, like which algorithm will be the best for your specific use case, or which compression level will be the best. But when it comes to containers, we have two options. There is gzip, which we already mentioned, but there is also zstd, also called zstandard. which is supported by the OCI standard. So when you compare ZSTD with gzip, generally ZSTD has very good compression rates, but also significantly faster decompression.
I think that was one of the design goals of ZSTD when it was created. How can we achieve competitive levels of compression without burdening too much on the CPU for decompression? And AWS has done their own research, and you can probably find this somewhere in the AWS documentation. I don't remember exactly where we found it, but we found that they claim that if you use ZSTD, you get up to 27% faster Fargate startup times just by switching from gzip to ZSTD.
And of course, as I was saying, there is a tuning there. So you might use different levels of compression. And AWS suggests that, based on the research, level 3 is probably a good average that works well enough in most cases. So if you don't know where to start, probably that's a good number to start with. Now, we're going to be talking more about Soci and what all of that is. And I'm just going to mention up front that there is a little bit of a caveat.
So Soci doesn't support ZSTD, but only gzip. So I suppose that that's one of the things we'll need to keep in mind when we decide exactly which kind of optimizations we can use to make our container start fast. So I'm just going to give you a little bit of a spoiler on Soci, and then I'm going to let Eoin introduce all of that. But the main idea is that we are used to thinking about containers as it's a package that gives you everything you need to run an application. So in order to start that application, you need to download everything, somehow create kind of a file system that has all these files available, and then you can start up an isolated process with all of this information. The idea of Soci is a little bit different in the sense that it is trying to effectively start the container as quick as possible. So the idea is like, can we start it even before we have made all the files available? Then the files can be loaded lazily as needed, but the startup of that process can be much faster. So that's kind of the general idea. What else can we share about Soci?
So Soci is all about becoming faster by having this seekable OCI image. AWS developed it and open sourced it, and it's built around Containerd. It's been around for quite a few years, but it seems like it's still relatively unknown and underutilized. That's why we wanted to talk about it. So normal container startup works by having your images with multiple layers. Each of them has set of file system changes in it.
And they're stored in compressed archives in a registry like ECR. That's normally how it's done. When you start a container, a runtime like Fargate or the ECS agent on EC2 will have to download the required image layers from the repository. That could be ECR, which is backed by S3. Then it would decompress and unpack them locally and assemble a file system, typically using an overlay FS. After that, it can go about the process of starting the container.
And once the container starts, the application can access the files in the image normally. The problem is that all the downloading and unpacking happens up front, even if the application only needs a small percentage of those files immediately or at all. So if you imagine a five gigabyte container image with a large application and hundreds of dependencies and maybe assets to go along with it, the application might only need 100 or 200 megabytes of those files during startup, but we're still downloading and unpacking everything upfront. And this is the opportunity for optimization. So what changes with Soci?
Yeah, Soci introduces a different kind of snapshotter, which is the component responsible for preparing the container file system. So as you said, instead of downloading everything upfront, the SOCI snapshot works a little bit differently. It just downloads a small index. So effectively, there is this new piece of information. It's an index that describes which files exist and where they are located within the set of layers that the container image has available.
Then it creates a file system. that is effectively like a virtual file system using FUSE, the file system in user space. And it doesn't need to download all the layers to be able to do that because it can use the index to understand which files are going to be available and what is the structure of these files. At that point, it can start the container with the expected file system available. And it fetches only the relevant portion of the image as the application starts to access the file.
So it makes this virtual file system available to the application. And the application will ask for the specific files. And those files can be downloaded lazily. And there is, of course, a cache. So every time a file has been fetched, of course, it remains available in the cache. So it doesn't need to be redownloaded from scratch or re-unpacked from the container image if we want to frame it like that. Now, let's make a little bit of a more concrete example.
So imagine you have an application, I don't know, web server written for Node.js. You might have like an app.js that is kind of your entry point, so what starts the process, what brings up the web server and starts to listen for requests. So Soci will use the index to figure out where is this app.js and just make it available so it can be launched and the process starts. Then, of course, you can imagine, I don't know, maybe this web application has some assets like CSS or images or whatever.
As the users will start to send requests, the first time that it's needing specific files, those files will be loaded lazily and then cached. And the next time, they are going to be readily available. So the idea is that you are starting much faster and then getting the files might be a little bit slower because they are not immediately available and they need to be fetched on demand. So we can call this idea effectively lazy loading for containers.
Now, why it is particularly clever? I think the design is really interesting because it tries not to change what a Docker container image looks like. It just adds additional metadata on top. So you can just take an existing Docker image that you have already And you don't need to change anything on it. You just need to follow a process that allows you to build the SOCI index. So that allows you to incrementally adopt this idea, this technology.
And all of that is transparent to the application. You don't need to do any application change in order to support this particular approach of loading and starting container images. Also, interesting enough, there is native Fargate integration. So if you're running containers on Fargate, Soci automatically works. Of course, you need to publish your images in a way that you are also building and publishing this SOCI index.
But otherwise, you don't have to enable anything particular or install anything in particular at the Fargate level or at the application level. And it's important to mention that there are trade-offs. It's not solving all the problems and making everything magically faster. So it's important to understand what is the trade-off. The trade-off is starting that process as quick as possible. So, for example, you can start to send requests, but then, of course, you might be a little bit slower in serving the initial first request because the lazy loading is going to take some time.
It's going to be decompressing on the fly, for example, the first time it's loading a new file. So I suppose the trade-off is that, yes, you're starting faster, but your initial request might be a little bit slower. It's difficult to tell exactly what is the impact of that because it's very application specific. If you have very few files, large files, lots of files, small files, different kinds of files that compress differently, you might have very different performance characteristics.
AWS gives us a rough idea that around 250 megabytes of compressed image, that's where you start to see more benefits than downsides. So that's probably a good threshold to start to explore when you might want to. experiment with such and see if it's beneficial to you or not. If you have smaller images, probably it's not worth it, because I think that the extra cost of all this machinery is not justified for a smaller image. You're probably not going to see benefits, and you're probably just going to see more of the downsides rather than the benefits. So again, the key takeaway is that it's not something you should just enable all the time by default. It's not always going to make things faster. So try to understand the trade-offs, do your own tests, and see where those trade-offs are beneficial to you or not. Now, I already mentioned we can use Soci on Fargate. What else can we do, Eoin, on AWS?
Yeah, we mentioned Fargate, and we're going to talk about how you it all up and running. But basically, once you have the metadata there, it'll work on Fargate. So we'll go through that. And we'll also go through our benchmarking results in a bit, which is interesting. In AWS, the other services where you could use it are SageMaker, so SageMaker Studio and Unified Studio. AWS say that you can get 30% to 70% startup time reductions.
EKS is an interesting one. So Saatchi is a container D snapshotter. So it can be deployed wherever you control the container runtime. AWS has also introduced something called Soci Parallel Pull, which works with EKS and optimizes concurrent layer downloading. So it's the same tooling, the same ecosystem, but it's a different optimization mechanism. So on EKS, you can then benefit from pulling parts of the image in separate parallel threads, if you like. Now, ECS on EC2 is a funny one. Unfortunately, there's no native Soci support. It's still not available there. There's been a longstanding AWS roadmap request for Soci integration. And it depends on supporting containerd in ECS and the ECS agent that drives ECS on EC2. It's quite old and based on Docker, and there doesn't seem to be a lot of movement around evolving it. We'll have to watch that space. So for today, Fargate is really the obvious place to start. So before we go into our experiences, everything we built, how does Soci work, Luciano? Do you want to break it down?
Yes, I think we already mentioned the high-level idea. So let's deep dive a little bit more. So the idea is that if we take a closer look at container layers, we basically need to think about a container layer as an immutable set of file system changes. If you have in mind, for example, a Docker file, is defined, you know, you're kind of running different, let's call them instructions. And then each step is effectively building a layer that will become part of the final container image.
So, and those layers are kind of stacked together. And if you basically read all of them in sequence, you are effectively building a file system. The idea of each layer is that they are reusable, cacheable, and stored as a compressed archive. So it's effectively like a tar of all the files in that layer that can be compressed, and then you're kind of stacking up all these layers to accumulate file system changes.
So again, rolling them back from the first layer to the last gives you the file system that you can use for your application. Now imagine that one of these layers is called layer.tar.gz. What is the problem here? We want to build an index that tells us, if you ever need that file app.js, for example, where can I find this file among all these layers? And of course, you want the last version of that file, because of course, you might have a layer that replaces a file at some point.
So you need to have an index that tells you exactly how do you load efficiently only app.js and not having to unpack all the layers and rebuild the entire file system from scratch. And it's not as simple as it might sound when you bring compression into the mix, because you have to use gzip with SOCI. And if you say, give me that app.js, it's somewhere in a compressed form. in the middle of a layer. So first of all, you need to know which layer, and then you need to know, okay, in this layer, there are just a bunch of bytes, which bytes are relevant to me in a way that I can decompress those bytes and get the full file, and potentially not much else that I don't care about.
And so again, the issue is, if I have to phrase it differently, it's like you cannot efficiently jump in a random place, and I suppose in a gzip stream, and be reassured, OK, this is exactly where I'm going to find the file. You probably need to kind of unwind the entire stream until you find the specific file you're looking for. And this is where Soci does something a little bit clever, in my opinion, that they call ZTOC, or Z Table of Contents.
contents, I suppose. The idea of this table of contents is that when you create the index, you can effectively inspect the stream, the gzip stream, and see exactly which file starts at which byte. And if you are taking a note of all of that effectively, like you are bookkeeping and tracking as, okay, I know that this file is going to start at this byte, then the index is effectively what's kind of solving the problem for you later on, because you can then look at the index and know exactly at which byte you can read to get that particular file.
So again, this is not something that is natively supported by gzip, and that's why Soci needs to add this additional structure. So if gzip had something like a table of contents, of course, they didn't have to come up with all of this. But because it doesn't exist, that's effectively how they solved the problem. So once you have something like that, you can imagine that solving the give me the app.js problem becomes much simpler, because Soci can just see, OK, the entry point is running Node.js with app.js.
So I need to unpack app.js to be able to start that process. So it's going to look at, first of all, which layer contains the last version of that file. Then it's going to look at the TOC and figure out what is the offset in the gzip stream that I need to start unpacking for. Of course there are other additional details because it's not just the offset but you might also need to know what is the length that you need to read.
So it's going to read all of that. It's going to fetch the information. And if you deep dive, you might come across the concept of spans because, of course, compressed blobs are not covering precise files. They are kind of blocks. So you might need to read multiple blocks. So of course, all of that information is in the index. So the index will tell you where to start, how many blocks you need to read. Then they can be decompressed.
You have your file available. So again, interesting idea of Soci, they didn't rewrite any technology, they just layered more stuff on top, which is a great idea because it makes things incrementally adoptable without having to change what's already out there. And yeah, we'll probably have a link in the show notes with a deep dive from AWS with more details and more in-depth explanations if you're curious, but I honestly found all this stuff fascinating and that's why I wanted to share this particular design. Now, it's probably a good time to talk about benchmarks if you really want to know how much can you save and whether it could be worth to you or not.
Yeah, we didn't want to just trust the AWS benchmark, so we built our own. And I think it's fairly comprehensive, probably more comprehensive than it needed to be. And it's a bit like what we've done before for other optimization stories. We built a CDK application, CDK application, that deploys the infrastructure, repeatedly launches ECS tasks, and measures performance in a few different dimensions. The workload was a Python application using the Python 3.12 slim actually base image.
The application imports and uses its dependency. So we're testing realistic files, we think, rather than artificially bloated images. And then we have two image variants. A small one was just NumPy and Pandas. and a large one, which has a lot of big dependencies, PyTorch and Scikit-learn, PyRO and Transformers. So the variables we tested were, we did x86-64 and ARM64. We had the two image sizes, and then we had three compression values, gzip, zstandard and uncompressed.
And then we had Soci, either enabled or disabled, where supported. We also tested in Fargate and ECS on EC2. And just for kicks, we also tried two different OSs for EC2, Amazon Linux 2023, and Bottlerocket. So not every combination is possible. So for example, you can't combine Zstandard with Soci, as Luciano mentioned, and you can't use Soci with ECS on EC2. But overall, that gives us 44 different permutations.
We ran each one 30 times each, and that gives us 1,320 task executions. All of them were using one vCPU and two gigabytes of memory. So how we measured it is we used ECS timestamps to calculate the pull time. So you can get all of that information from ECS, like how long it takes to do image retrieval, the container startup time from task creation to the actual container start. But we also measured how long the application is taking to load its dependencies.
So lazy loading can dramatically reduce the initial image pull time with Soci. But if the first file access need to fetch data over the network, some of that time may come back later. So it's not just about container starts, but when the application can actually do work and how fast that application work is. We also verified which snapshot Fargate is using. So we're checking Soci versus overlay FS. So we know Soci is really active.
When we talk about the different tool links, some of it is not quite clear how to get it to work in each case. And so we're also just verifying that the build process we're using is doing what we expect for the benchmark. So this repository, which you can access, the link will be in the show notes, includes the benchmark code, the workflow, and there's also a visualization dashboard with Plotly for exploring the results.
So what were the results? Well, some interesting stuff, I think, in here. We're not going to go through it in painstaking detail. But let's start with our image sizes. So we had around 70 megabytes compressed for the small image and 3 gigabytes compressed for the large image. So first of all, how does compression affect it? So looking at gzip versus zstandard versus uncompressed, The large image goes from about 6 gigabytes to 3 gigabytes with GZIP, and it's a little bit less with Z-Standard, like 200 megabytes less.
Small goes from 250 megabytes to about 74 with GZIP, and 67 with Z-Standard. So in terms of performance just based on compression, Zstandard is only about three to five times faster than gzip on large Fargate images in our experience. Now, your mileage may vary, because it depends what's actually in your image, how compressible it is, et cetera. So if you look at Soci for a second, with the large image, we saw up to about 10 times improvement in container startup on Fargate.
So an uncompressed image was about 160 seconds. With gzip, you can get 120 seconds. So already, that's a pretty good improvement. But Soci, it goes way down to 16 seconds. gzip with Soci, so combining the two, was actually a bit slower than Soci alone, 17.5 seconds. So that I think is interesting. For the small image, uncompressed, 19 seconds startup time. With Soci, 13 seconds. gzip actually just makes it slower for small image.
We get 20 seconds. And if you combine gzip with SOCI, it's kind of in the middle. It's better than just gzip alone, but not as good as Soci on its own with an uncompressed image. So you can see the varying effects of combining compression with Soci. Again, your mileage will vary. This is our image, our context. A lot of these dependencies might have already some compression involved in them. So you might not get so much benefit.
But it's easy enough, I think, to experiment with the different permutations. So those are the primary findings. And I think the 10x improvement will obviously be attractive to a lot of people. So it is possible, especially with larger images. Looking at Fargate versus ECS then, when I say ECS, I mean ECS on EC2, the old school ECS. EC2 had a lot more variability in startup timings, but I guess that's to be expected, given that you need to account for instance provisioning sometimes as well, provisioning the ENI.
But once you start a warm instance, startup times can be four times faster than Fargate. Obviously, you can have image caching locally in your Docker agent, so that makes a lot of sense. But when you add Soci, especially for large images, you get very consistent low startup times in Fargate. You can get low startup times in EC2 as well, but it relies on having a warm instance and caching. You might be curious about Amazon Linux 2023 versus Bottlerocket.
Bottlerocket is an OS that's typically used for a security perspective because it's got a very minimal surface area, very lightweight base image, or sorry, very lightweight OS. But we actually saw cold instance startup times on EC2 around two times faster. I guess the AMI is just a lot smaller. So that's something that can make a difference on EC2. Looking at x86, 64 versus ARM as well. We actually saw Arm on Fargate showed about 10 to 15% improvement in Fargate startup times for basically the same image, right? You've got obviously multi-architecture images, so they might vary. They will vary in terms of the bytes that you're shipping, but we saw pretty consistently that Graviton processors for Fargate. seem to be a bit faster for startup. So lots of opportunity for optimization, depending on your context. So Satya used a big win. How do we go about doing it, Luciano?
Yeah, that's honestly the part that I found a little bit frustrating, mostly because there is a bunch of different tools that you might come across. And it can be a little bit confusing to figure out exactly which ones should you use when and for what, right? So let's try to describe what's the high-level workflow. And then I can probably let you describe the tools that you use, because you are more familiar with some of them.
And yeah, the high level workflows that you can build and push your normal container image as you always do every time you need to use a container, right? So that means going to ECR and pushing your image there and having it available. Then you need to inspect the layers. And there are tools that are Soci-aware, so that they basically understand what the Soci shape is going to look like at the end. So they can do that process of, given an image, let me inspect the layers.
And that allows the tool to generate the ZTOC, so the table of contents from the compressed streams for each layer. And that is basically what gives you the Soci index at the end. Once you've done all of that, so you publish the image, you create the index, then, of course, you also need to publish the Soci metadata in ECR. So effectively, you are making that index also available in ECR. And that is what enables, for example, Fargate to be able to use then that index when you spin up images, when you spin up tasks that use those images with Soci. So again, I'll defer that to you, Eoin, which tools did you use and which one you think people should care more about?
Yeah, it's difficult to know. So AWS provides the Soci purpose-built tool for generating and inspecting the metadata. There's also Crane, which is a popular enough OCI tool for inspecting manifests, layers, and registry contents. There's another one called Skopeo. which is useful for copying images and artifacts between registries. There's a whole suite of tools out there, depending on which camp you're in and which world you're in.
For our benchmarks, we used the Soci Convert standalone to prepare the images and Skopeo to publish them. Soci actually only runs on Linux, but the way we got around that, because we're Mac users when in our development environments, was by running a Docker container in privileged mode to be able to run Soci Convert, and it worked pretty well. And when you're doing Skopeo copy, by the way, you need to do dash dash all.
We've discovered the hard way a lot of these limitations because without the image, the SOCI index might not be published and Fargate will silently fall back to a normal pull. That's why it's a good idea to verify the manifests that are pushed to make sure you've got what you want. So I guess I'd recommend starting with the Soci tool and Crane maybe for inspection. You don't need to learn the whole suite of tools available across the OCI image ecosystem. And if you want to automate all of this in production, AWS actually has a solution, the Soci Index Builder. I haven't tried it, but apparently it can generate indexes automatically when your images are pushed to ECR. That's quite a nice idea if you don't want to worry about introducing all of this tool into your CI pipelines. So complete implementation is available in our benchmark repository. It does run a lot of tasks, so be mindful of cost before you kick it off.
Yeah, that's a good one. I think at this point, we can wrap up. We shared most of the things we wanted to talk about today. So let's give it a quick recap. So we started with the basics. So we mentioned that smaller, well-optimized container images and sensible compression is probably the first thing you should try, right? Because it's probably going to give you lots of value for your packs and your time. Then, of course, you can consider ZSTD as a compression mechanism, because it should give a little bit of an extra edge in terms of less bytes to download, faster to decompress, so probably better startup times overall.
If you have tried all of that and you still are not particularly happy with the startup times, then that's probably a good time to start to explore Soci. So generally, that means you have larger images, which might take longer, and especially in Fargate, where one application don't need or when application, I mean, don't need every file immediately. So yeah, we have seen, for example, lots of cases for ML workflows where you might have lots of files or big files.
I think those are cases where Soci could work really well. And the other thing is try to do your own benchmarks and try to be comprehensive. So of course, you should care about the startup time. But remember that this is not your only metric because a faster startup you will have slower time to access the files the first time that they are accessed. So just be sure what's the impact of that for your particular application.
It might be good overall, in which case Soci is a good win. It might be bad, so you probably want to revert and try something else instead. I think in general, we particularly like the idea of Soci just because it's a clever idea. The incremental adoption makes it for a very compelling story. It doesn't require you to change anything, but just maybe add a few extra steps in your pipeline. And if you're using Fargate, it's even enabled by default, so you don't even need to do anything in particular from kind of the infrastructure point of view.
So I think all of these are great wins and it's an optimization worth knowing and figuring out when it might be worth using it or not. Now, with that, I think we have a few questions for you, because we're curious to hear your stories. So of course, make sure to reach out to us and share your answers. We are curious to know, in general, what kind of workloads are they running? What kind of production containers, images do you have?
Have you ever measured how much time do you spend downloading and unpacking them? And if you have used Soci, was it beneficial to you? Or did you prefer to just stick with something else, like Zstandard compression? So that's a topic that we have been starting to explore only more recently when we are doing lots of optimization for certain customers. So definitely sharing your knowledge with us would be useful to us, and hopefully it goes both ways, right?
And finally, one word of thank you to our usual sponsor, fourTheorem. fourTheorem is powering AWS Bites. Thank you for supporting us. So if you're looking to improve the performance, scalability, or cost efficiency of your container workloads on AWS, fourTheorem can help you identify bottlenecks and optimize your architecture. So if you're curious about all of that and what we do at fourTheorem, check out fourtheorem.com. So that's everything for today. Thank you for sticking with us. Thank you for being patient while we were away between this episode and the previous one. Hopefully, we'll be a little bit more consistent going forward. But please send us your feedback and your love, because that's what keeps us motivated. So thank you again, and see you next time.
Use the arrow keys to move between transcript segments, Home or End to jump to the first or last segment, and Enter to seek in the video.
Help us to make this transcription better! If you find an error, please submit a PR with your corrections.