Grumpy CIO - AWS Cost Explorer Lambda NLP

Founders, startup CIOs, make use of easy tools to manage costs.

We were migrating data through complicated stuff involving NLP (AI thingummy), and using scary stuff with funny names/ acronyms (like "Lambda").

Could have gone pear-shaped. AWS has a Cost Explorer that looks hard but is dead easy to use. Even I can use it.

By taking simple measurements of cost that everyone could understand we were able to decide our path.

Watch my video below to see why and how

Transcript:

Hi, I'm Alistair, the grumpy CIO. What am I grumpy about this week? I'm grumpy about startup founders, startup CIOs not taking advantages of simple tools that they can use to make life easier. For example, AWS Cost Explorer. So what is AWS cost Explorer? And how do you get to it? Well, you go to AWS amazon.com. And you log in, if you haven't gotten an account, you will have to set one up. But it's no more difficult than setting up an account in Google or anywhere else. I happen to use multi factor authentication with a with the authenticator. But again, that's no more difficult to set up and you have MFA anywhere else, you end up going to cost Explorer, you just put it in here in the search bar, just right cost Explorer, and it will show you your costs. Now, a couple of quirks you will find about things to explore. So if I make this go from the first to the 10th, all I did was go to the date picker and do that, you'll see that the costs on this first day are a bit higher. Why is that? What's this big purple lump. It's called tax. And you see how I can get to it, I just hover over it. And it just tells me what it is, there is a doubt there is a breakdown here and I can download this as CSV if I want to take it away and do some analysis in Excel or anywhere else. But this is so simple, you really don't need to. But one quirk you do have to know is that tax all gets paid. At the beginning of the month, you accrue a lot of costs, and then they bill you and you pay the tax at once. So I'm just gonna get out of having the text displayed there. Again, I'm just going to the date picker clicking Apply. And then I can see something that I really quite liked to see, which is the cost of very stable, I don't know what they are, because I can see them. And you can see there's these words down here, elastic container service, ECS relational database, or RDS like a SQL database or something like that. EC2 ELB. What's that? What's an elastic load balancer? And again, you can kind of look at these things and talk about it. So if you don't know all the words are, that's fine. If this is already enough information to go to your tech team and go, what's that number, what's it for and why. And you'll build up a picture of what your costs are really easily. There's lots of other things you can do on here. They give you reports, budgets, there's a cost anomaly detection, you can look at purchase saving plans, which is where they want you to pay up front for 12 months, but they give you a discounts. There are some recommendations, which have never actually been in use for me because we're already doing stuff. And there's all these things you can do on the right hand panel, just just don't this is all if you're a founder of a startup, this is already enough information for you to go to your tech team and go lost that place. One thing you will offer a little quirk in AWS is that I can ask for hourly breakdown as well. The problem is it costs you money. So we don't do that. Because we're a startup, why would we do that. So daily is something you get out of the box when you actually just sign up for AWS cost explorer. But literally, this is all the information you need to go and ask questions. So we had a real life use case to make to make use of AWS cost Explorer, we were monitoring data across we have existing data, we had millions of records millions of files that we wanted to take from an old solution into a new solution. We actually ran this stuff through an NLP pipeline, natural language processing AI, every loves a bit of AI, it's all hot topic. As I'm recording this in May 2023, we've got some lovely a train models that we've used, and it works on some AWS Lambdas. If you don't know what a Lambda is, don't worry, it's basically a computer that fires up as you need it. So it only does things when you need it. Why do we do this? Because we end up with searchable results, news to present integrated with our data. Right? So the problem was, we're going how do we actually know how much it's going to cost to run all this stuff through our through our funky tools? And the answer is just go and look, go and test it. And out of the box, you get this cost Explorer, and it just gives you some results. Now we actually as you said, we don't have the hourly, we have the daily. So we want a daily snapshot, do a test come back next day look at the cost. That's a bit of a pain, because you'd like to do it like moment by moment, or startup. But fortunately, it's a startup, we've got lots of other work to do at the same time. But simply this tool just worked. And we're able to see, look, these lambda costs here you can see them they spiked up on a particular couple of days. And we don't like the look of that. So we actually made a decision based off the simpler information to actually move that particular set of code out of the lambda into another, another place that just ran on continuous compute. So yeah, you can just get simple information from AWS Cost Explorer. You don't have to be digging deep into configuration tools. It just gives you a nice, simple report. You can have a conversation with your tech team, so just do so. Thanks for watching. I am Alistair the grumpy CIO for www.shipshape.bc. We are the free investor search engine come into what your business does and find investors that get you to optimise your investor outreach. Thanks for watching.

Transcribed by https://otter.ai