Help us to make this transcription better! If you find an error, please
submit a PR with your corrections.
Eoin: What are the benefits of tags? You've probably seen that you can add tags to almost every resource in AWS, but should you really do it? And if you do it, what is the point? What are the benefits? In today's episode, we'll give you a quick overview about what tags are, give you some examples on how to use them and what kind of benefits you can get from them. Finally, we'll give you a bunch of ideas on how to build a tagging strategy and get value from tags. My name is Eoin, I'm joined by Luciano and this is the AWS Bites podcast. So Luciano, maybe tags are one of those things like flossing or going to the gym that people should probably know they need to do, but they don't do as much as they probably should. But maybe first we could talk about what tags are.
Luciano: Yeah, so it's actually a very simple concept. You can imagine tags as just arbitrary key value waves that you can stick to all sorts of different AWS resources. Now, to be fair, they are not available in every single AWS resource. There are exceptions, but for the most part, you'll be able to use tags for almost every resource. So it's something you should be using and we'll discuss why in this episode.
I just want to give you maybe some examples to make it a little bit more tangible and just to give you a feeling that at the end of the day, it's just metadata. So it's something that needs to make sense to you and to your team and needs to make sense in the context of the application you are building in AWS. And just to give you an example, you could say, for instance, I don't know, who is the owner of a particular resource and that can be the name of the team.
Or you can say, rather than calling it owner, you can call it team, for instance, team e-commerce. You can say something like stage or environment and maybe say this is a production environment rather than a QA environment. Or you can use stock name and say, I don't know, shopping cart, this is an e-commerce and you might have a stock dedicated to the shopping cart or even the application name, maybe, I don't know, you are building an e-commerce that sells organic oranges. So your application tag could be organic oranges shop. So yeah, that should give you an idea of what tags are and how you could use it at a very high level.
Eoin: Okay, so that makes sense. And if you've got the capability to add these kinds of tags and you've given a good few examples there, what kind of use cases would you foresee by having tags? What would you do? What's the value of having the pre-production stage annotated for your EC2 instances or knowing that a Lambda function belongs to your organic oranges shop?
Luciano: Yeah, I think that the first thing that's probably a little bit obvious is that you have additional information attached to your resources. So if you are doing a review of all the resources in your account or in your organization, or I don't know, if you just want to know what is this resource being used for and you have done this kind of tagging consistently across your organization, then that becomes very useful piece of information.
And you can also do search, for instance, you can use tools like Tag Editor to search across all your resources, the one matching particular tags, or even the opposite, you can search for resources that don't have tags. So it becomes an additional way to explore all the resources that you have in your account. But there is probably another use case which is probably the most used or the one for which tags are more commonly used and recommended, which is cost allocation.
Because as soon as you do tags, you can use those tags to group cost. So for instance, you could group cost by team or application, and at that point, you can use that to basically see what kind of things you are paying more on and I don't know, you can see if there are spikes on particular application or particular teams. So that's probably the number one recommendation that you will find online about why you should be using tags is because they will give you much better visibility on your cost.
And you can attach cost to things that make sense for your organization. There are another couple of interesting use cases though. So it's not just about cost and finding resources. You can also use tags for automation and security. And I want to give you a few other examples. For instance, in terms of security, what you could do, you could say, I'm gonna create an IAM policies that limits people from doing certain things if there are certain tags or if there aren't certain tags.
For instance, you could say, I am gonna create a policy for an account that says that nobody can stop, or at least, I don't know, maybe developers cannot stop EC2 instances if they have the tag environment production, for instance. And that's basically something that gives you, using tags, you get that additional context to distinguish between resources that are maybe more sensible in production and resources that are less sensible. And similarly, you can use the same idea for automation. So for instance, you could create, I don't know, let's say a Lambda on a schedule that runs every day at midnight. And you can say, okay, get all the EC2 instances with the tag sandbox true and terminate them because maybe those are things that are supposed not to stay around forever and that somebody is just experimenting with them. So using that tag, you are automatically creating the capability of, don't worry too much. If you forgot about them, they would be automatically killed for you.
Eoin: That's really good. Yeah, so you can use tags to essentially audit what you have. You can use it to get control visibility of costs, also enforce security. So I think that's enough motivation to actually go ahead and create tags. So how do you actually do it? If you've got like a lot of people out there, lots of resources that are untagged, where do you start?
Luciano: Yeah, I suppose the easiest way and probably every tutorial will show you that way for sure is to just go in the web console and go around, wander through the different services. And when you see resources without tags, just use the web UI to add the tags. But of course, the bigger your deployments, the more complicated they will be to actually find all the resources and do it consistently. So of course, there are other tools that allow you to do this in a more automated or more kind of in bulk type of approach rather than just doing it one by one.
And the first thing is you can do it from the CLI. So you can either tag existing resources from the CLI or when you create a new resource, most of the commands will allow you to also specify the tags you want to attach in line when you create that resource. Another thing is that if you are creating or updating resources through infrastructure as code, like CloudFormation, CDK, Terraform, all these tools will also give you the ability to add tags to the resources.
Also interesting enough, with CloudFormation, CloudFormation will also take the liberty of adding a bunch of useful tags out of the box for you to almost every resource. For instance, the stack ID and the stack name, when you deploy with CloudFormation, you will probably notice that those tags will be applied to all the resources. Another interesting thing is that there is a tool called AWS Tag Editor. It's an actual service in AWS, and that one also give you different abilities to search. And I think you can also apply tags in bulk. I haven't used it that much, but something else you could explore and see what are the capabilities. Then there is resource group tagging, which I don't really remember what it does. So if you remember about that, I'll leave that one to you.
Eoin: Yeah, I think this tag editor you already mentioned, it's quite confusing the terminology here, but it's under the umbrella of resource groups, resource groups tag editor, and then you've got an API with that as well. So I think that's the resource group tagging API. And it's interesting, depending on the resource type, not every resource type is supported by each of these things, so you just have to check the documentation.
Unfortunately, it's just the case with tags that consistency isn't there across all the services in terms of the tagging support. So you just have to check the docs and for your costly resources, like EC2 instances, that sort of stuff, it's generally very well, very mature, but for some of the other resources, it's just not there. Yeah. And I like the idea of using infrastructure as code really to enforce it.
I mean, that's, again, it sounds like we're kind of beating the drum about infrastructure as code quite regularly on the podcast, but it's one of the real benefits of using infrastructure as code is that you can define your tags in the root of your repository and enforce it for all of the resources consistently. And you know that the tag names are right, the values are correct, and the casing, because they're case insensitive, and you might want to think about, you know, making sure they're consistent with casing, with infrastructure as code just makes that a lot easier. Mm-hmm.
Luciano: Absolutely, yeah. So now that we understand a little bit more what tags are useful for, is there a set of tags that you would particularly recommend?
Eoin: Yeah, I guess the good idea is to try and keep the number of tags reasonably small. I think you can create up to 50. So the recommended tags name is one that probably people will have used quite a lot. So if you've created an EC2 instance in the console, a lot of people skip that last step where you have to put in the tags, but if you've put the values in, you'd probably put in name at some point because that's recognized by the AWS console when it presents your list of EC2 instances, and it will give you the name column populated for all your instances.
So name is an important one that's actually recognized. Other things, if you're using it for cost control, you might have like a department or a cost center tag, an owner, be that an individual, probably not a great idea to use individuals because they can change department and application, but you can put in the owner, like the name of the team responsible for a resource. As you already mentioned in your example, at the start, you could put in the application or service name, and I really liked the idea of adding like an environment or a stage tag.
So knowing that it's a development environment, a test environment, or a production environment, is that allows you to enforce security. You gave a good example of like disallowing stock terminate instance access, for example, an EC2 instance, if it's a production environment. It's also just good for auditing, right? And also for cost control. You can also say, well, how much of my AWS resource cost is coming from production versus pre-production and test and development environments? So those, I think, cover it all. But the other one that you mentioned earlier was like version, which I really like. Like if you have a Lambda function or an AMI in particular, you could tag it with version numbers so you understand without having to boot up the AMI, and boot an instance based on the AMI, you can understand what version it is and even what operating system version, for example.
Luciano: Yeah, on environment or stage, it might sound a little bit contradictory with something we said in other episodes, where we recommend to have different accounts and structure your organization, so you have dedicated account for different environments. But in reality, if you do, for instance, consolidated billing, then you will have like a high level overview of all your accounts. And then if you use the tags, then it becomes easier to see how you are looking at the different resources, even if they live in different accounts, but then you can use tags to group them logically and see how they affect cost.
Eoin: Yeah, I agree. And we also do things like creating AWS config rules and config, I'm forgetting the term, but there's a way you can collect data from all of your AWS accounts and aggregate it into your AWS, say compliance account, where with AWS config, you can go in and write like SQL queries to query what resources you have in AWS. It's quite powerful. And if you can imagine that, if you've got things tagged with an environment, you could do a select statement that discovers how many Lambda functions you have that are used in development, or how many EC2 instances you've got running in production. So it's quite powerful. So how do you enforce tags then? If you've, we talked a little bit about things like AWS config, but what are the ways that you could say, okay, I'm going to try and get my house in order and put in tagging policies for my organization. How would you start?
Luciano: Yeah, there are a bunch of different ways, and I'm sure that there is more that we haven't discovered yet. So don't take this list as a comprehensive one. Probably we just scratched the surface there. But the first one that comes to mind is IAM, because with IAM, you could create policies, for instance, for, I don't know, developers, that says something like, okay, you can create an EC2 instance, but only, and this is where you specify a condition in the policy that says only if you are adding tags for environment and cost center, for example.
So that that automatically enforces engineers that have that particular role to do that, because otherwise they wouldn't be able to create the EC2 instance in the first place. So this is actually a very powerful one, because it's not retroactive. So this is literally preventing resources from being created if they don't have the right tags in the first place. Another one is service control policy, where it's kind of a similar thing, but I suppose from a different perspective.
So service control policies is something you can apply to entire accounts in your organization, and they can essentially deny certain operation if certain conditions apply. So you could think of the same use case where you want to prevent, for instance, engineers from creating EC2 instances if they are not tagged consistently with your rules, but you have to think it a little bit backward in terms of the policy, because you have to say, I'm gonna deny that operation if I don't see this tag. So the policy is kind of, yeah, it feels like the opposite policy from the one we described before, but effectively you can achieve the same result. And SCP are a little bit more powerful because then you can apply them to an entire sub-tree of organization in your organization. For your whole organization, yeah.
Eoin: Yeah, it's definitely the strictest option, isn't it? It sounds certainly, it's the strictest one I can think of because it's like a preventative control. It's like, no, you can't even, you can't go any further here. You're missing a tag.
Luciano: Exactly, so also use it carefully because maybe you will end up blocking your engineering, your engineer from doing progress because yeah, maybe there might be exceptions that you haven't talked about. And then yeah, people need to go through some sort of a process if they have to go around those particular cases. So other approaches that are a little bit more permissive, but they still give you visibility when something is not compliant with your tagging strategy is for instance, using AWS Config, you can create a rule that checks that your resources are actually tagged correctly. And if you find something that is not compliant, then you can trigger a notification and then you can use that notification in a reactive way, go in and check why those resources are not tagged, if you need to tag them, or maybe you need to create a new rule, or maybe you can make an exception in your rules. So this is a little bit of a more permissive approach and it's more kind of in that idea, yes, let's do compliance, but let's not block engineers from doing progress. Other ways that come to mind to you?
Eoin: Yeah, we mentioned AWS Resource Group. So this is where the Tag Policy Editor came in earlier, and there's also ability there to create tag policies. So this is an account level feature, but it's also an AWS organization feature. So within your AWS organizations in your management account, you can turn on tag policies. So there's different kinds of organization level policies. You mentioned service control policies already.
There's another one called tag policies. And it gives you the ability to create policies saying these tags need to be created. You can specify whether they need to be case sensitive or not, whether the exact casing needs to be specified, and you can also specify the values that should be used. That's optional. It's quite useful. It's also like a detective control, like a permissive option rather than a preventative control.
So it doesn't stop you from creating resources if the policy isn't complied with, but it does allow you to go in and see which resources are failing tag policy. So it's an alternative option. As with a lot of things in AWS, you kind of have multiple ways of doing the same thing. It's very similar to AWS config rules. So it just depends on what method you want to use. If you're already using AWS config rules for lots of other things, it might just make sense to use AWS config and detect compliance that way.
But if you're not, then you might just use AWS resource groups instead. So it's probably worth mentioning as well that with AWS config rules, you can also remediate some of the compliance failures you see if one of your resource, if a resource gets created, a config rule spots that it hasn't got a tag, then you can actually use like a Lambda function, or you can also use system manager automation and go in and actually apply a tag to that. So you can figure out what tag might make most sense for the resource, or you could tag it with some sort of temporary value or just tag it so that you can come along and enforce it better later. Okay, that's a pretty good suggestion.
Luciano: I suppose the more you can automate the better because yeah, it's gonna make your life easier in the long term. One interesting thing I just want to mention very quickly is that there are limits in using tags. We already mentioned that a resource can have at most 50 tags. So this is an interesting one. Again, don't overdo. With tags, 50 is probably more than enough, but I don't know, maybe if you have a very aggressive tagging strategy, you might reach that number.
So it might not be that big as well. Also, every tag key must be unique. That means that you cannot use, I don't know, the tag name twice, right? You can only use name once. Then there is a maximum key length and a maximum value length is 128 characters in Unicode for the key and 256 characters for the value. Again, it should be more than plenty for making sense of your tags, but it's not an endless amount. So don't try to put, I don't know, all the metadata as a JSON into one tag because that's not gonna scale very well. And also there is an interesting one is that tags, the key cannot start with AWS column. That's kind of a reserved one. And I'm not really sure if that's something that AWS uses internally, or if you, in some services, it will be a little bit more visible to you, but you cannot use AWS column yourself as a, for your own custom tags.
Eoin: Okay, that sounds very sensible. And are there some kind of best practices? It sounds like there's a lot of options here and there's plenty of flexibility. So can we simplify it with some kind of best practices and patterns for tagging?
Luciano: Yeah, so, well, the first thing that comes to mind is definitely try to avoid to store personal identifiable information in tags because of course those are not meant to be, I don't know, encrypted or hidden in any way. So just keep it to the metadata that makes sense from the service perspective or the resource perspective. And then the other overall suggestion is try to define a strategy and trying to keep it simple.
And I don't think there is really a bulletproof strategy. So I suppose we can give some hints to people on how to approach that. And for instance, you could ask yourself, which tags are you going to use? So try to come up with a list that says, these are the five or six tags that we want to use consistently. Then what are they going to be used for? It's another question you should ask yourself. Don't just come up with six random tags, but try to find what is the goal of every single tag.
Have a purpose. And like, I don't know, is it a technical tag? Is it related to cost? Is it related to security? Is it useful for some automation? So try to think that way. How are you going to group tags and what are you going to use them for? And the other thing is that if you're doing infrastructure as code, it's easy to think about, okay, I can keep them consistently because that's going to become part of my infrastructure as code.
But ask that anyway. Like try to think, okay, am I going to make this consistent? Can I do something that helps me to keep it consistent? So both proactively and retroactively, try to have ways to simplify the aggression of tags, but also ways to check that your resources are actually compliant with your strategy. And finally, I think about naming conventions, like make sure the casing is correct and used consistently, and maybe have a review process and you can probably even automate some of that using some of the tools we discussed before.
There are some interesting resources, one that I would like to add to the show notes is an example that's from the AWS blog that shows how you could use CrowdTrail to automatically tag resources, based at, for instance, I don't know, the time when it was created, who updated that resource or things like that. Maybe it's not the most amazing example for the use that they present, but I think it's cool to see that you can automate some of this stuff. So maybe you can find a different way of a different classification for tags, I suppose, because I don't think tagging the engineer that created the resources extremely useful, but maybe from CloudTrail, there are other things that you can use to do some of this tagging automatically.
Eoin: And there is also- I can imagine that, sorry, I was just gonna say that came to mind when you mentioned that if you've got a CloudTrail event, because somebody created a resource manually in the console, and you've got their, say, SSO identity as part of that CloudTrail event, then you could look up what department they're in and add the department tag to the resource. So at least when you're doing cost allocation tracking, you've got something to work with, right? And you don't have to go and constantly manually check and remediate.
Luciano: Yeah, absolutely. That's a much better example, I think, rather than just using the name of the engineer, which, yeah, as you said before, people might move around in the company or they might just go away. And it's also personally identifiable information anyway. So I would rather prefer their department name rather than their specific person name. Another interesting one, which is actually archived, but I think there are still some useful suggestion, is a white paper from AWS that was published in, I think, in 2018. So maybe that's the reason why it's archived, but we couldn't find a newer version of that. So I think there are still some useful suggestions there. So we'll put the link in the show notes as well. And I think this is plenty for this episode, but I don't know if you have a final takeaway that we want to use to kind of, yeah, give people something to walk away with.
Eoin: Yeah, I think it's worthwhile covering this, because I haven't seen a lot of commentary about tags in recent years. And I think it's something that could be discussed more and used more because you have to put a little bit of upfront effort in it, but once you've got it, the benefits are there. You can really get a lot of value out of it and it'll really help you to just have a better view and just sleep easy knowing that you've got more control over kind of auditing in terms of compliance, cost, and just visibility into your resources.
Because if you don't know what's out there in your accounts, it can just get very messy over time. So yeah, I think our, both our takes on this is, yeah, use it more, but try to have a strategy, try to be consistent and keep it simple. Start with small number of tags that you know you'll use and have a tier purpose. And once you've got that established, I think you're going to start benefiting from it pretty much right away in terms of cost and visibility into your whole estate. So I'm glad we covered this topic. Let us know what you think and let us know if we missed any other ideas you can consider for tagging AWS resources, any other tools or resources. We'd love to hear from you. So thanks for listening and we'll see you in the next episode. ranch- backdrop-