Agentic AI, Agentic AI, Agentic AI, and other things...

AI used to chat. Now it’s starting to do the work.

In this episode of Optimise to Innovate, Alex and Jason step back from the agentic AI hype and ask what happens when organisations actually start running this stuff at scale, because one agent is interesting, but a thousand agents is an operating model...

We explore the real-world challenges behind agentic AI, from governance, control planes and non-human identities, to trust, explainability, token costs and FinOps.

We also tackle a bigger question - if agents take on more entry-level work, how do we build the experienced people we’ll still need in five or ten years’ time?

A practical, honest and occasionally sci-fi-adjacent conversation about where AI is heading, what organisations need to think about now, and how to separate the signal from the agentic AI noise.

Transcript

Jason 0:01

Welcome to Optimize to Innovate, a show where we help organizations stop wasting money on things that don't add value to their business and understand the technologies that actually will. Join us as we share practical insights into the latest trends and innovations with industry experts across everything from software and FinOps to cloud data and AI. And we decided that for a bit of a change, we wouldn't have any guests today, just Alex and me and a whole bunch of agents. So let's get a second human in the loop and start this week's episode.

Alex 0:34

Over to you, Alex. Funds just keep on rolling, Jason. Love it. It's hard to believe we're already on episode six. It's gone past really quickly. So I just want to probably start by saying thank you very much to everyone who's followed along with us, um, to the fantastic guests we've had as well, who've been sharing their expertise. We've thoroughly enjoyed it. And yeah, we thought we might do something a little bit different this week. Um, and maybe take a look back at a couple of the themes that we've seen from our more recent episodes. Plus, I know that you and I have been on the road a fair bit recently attending some really interesting events, things like the London AI Summit. It was also London AWS Summit. sorry, go on. That was a good one. It was, yeah, and and there was I think it's a really good kind of a litmus test for the world of AI, where it is, you know, what's the zeitgeist, however you want to put it. Uh, and maybe we're gonna pull a few observations out of that. Um, and for me, I think the very first one was at the at the London AI Summit, there was one really major theme, right? Two years ago I attended, and it was all about AI that chats. So it was all about um, you know, I can get all this really useful information or I can get it to write my emails or whatever. And now it's like we've given AI hands. So the theme was absolutely all about AI that works. I mean, is that what you've been seeing?

Jason 2:00

Yeah, the the thing that really struck me at the AWS summit in London this year was compared to last year, last year everyone was talking about model context protocol, right? So they were talking about how agents could interact with with tools, um, with data, effectively giving agents hands. And a lot of the conversations then the focus was around the standards, um, how they were going to be extended, um, what was what else was needed. When I went this year, it was really a focus on this is what we've built, right? This is how we've used MCP to actually make services available to teams to shorten the path to productivity and working with agents, and really to think about building out an ecosystem and taking some of that heavy lifting away from you know teams who are working, building individual agents without that scaffolding there to help them. So, yeah, absolutely. It's it's far more practical. Um, real good examples of things which have been deployed.

Alex 2:53

Yeah. I think there was a really interesting kind of takeaway, I think, from both of these events is a lot of the conversations, there were some brilliant conversations about use cases and genuinely inspiring stuff. Um, but I would say actually more of the conversations were almost around that governance piece. And this is the thing that um, I mean, just kind of harking back to a couple of our previous guests, like Alex and Seva, when they joined us, uh, we were talking about agentic AI, they were saying agentic AI is not really about agents, it's about the governance, your data foundations, your process design, it's all these things. And actually, that was definitely coming through in the themes. Um, I mean, somebody somebody actually said at the event running one agent is is like running one server, um, whereas running a thousand agents is like running a thousand servers. Well, actually, you're running a data center then, and you need to almost have that level of mindset as you expand this into your environment. I think a lot of organizations are running into almost that not a brick wall, but certainly a wooden fence of governance and trying to work out how they're gonna handle it.

Jason 3:57

Yeah, I mean, if you think about an app, if you think about an agent as an app, then you know you're thinking in an organization you might have a couple of hundred apps. Yeah. But but that's the wrong way to think about them, really. That might be how it feels like when they're being exposed if they are user-facing, but if they are agentic processes, then they're gonna be multi-step, there's gonna be lots of them. If they're specialized agents running, you know, in that chain performing particular tasks. The challenge here is the control plane, right? It's having that visibility across all of these things and understanding what are they doing, when are they behaving, you know, in in the normal way, and when are they doing something you don't expect. So when you get to that kind of scale, you've already got to switch to exception reporting. Otherwise, you just you can't keep a handle on things. And I think it's interesting to see that what Microsoft have done, they've released their E7 licensing, um, which is very much focused on the agent control plane. Um, one specific offering, Agent 365, which is not just designed to manage agents within the Microsoft stack, but it's also that being being billed as a way for you to bring third-party agents under control as well. And you see what they're doing with ENTRA, which is their identity provision, and they're giving effectively non-human non-human identities. So we're talking about how do we give an identity to an agent? How can we control its permissions? How can we govern it as we would do a person from the point of view of policies and control? So bringing it back to that idea of control and like you saying governance, that control plane, which is you know, what have I got in my in my technology landscape that's gonna help me to actually get my arms around all of these things, given that they're gonna be coming from different directions. You won't be just working with one vendor, with one platform, you know, you need to somehow think about the flexibility to cover it all.

Alex 5:43

That's do you know what's really interesting you say around the whole enter ID thing? Um, because we are at this kind of point now, aren't we, where if we think of it in that governance perspective, we need to manage these agents in in much the same way as we manage any traditional service. You know, you talked about applications before. So we used to have service accounts and we'd lock them down, we'd give them permissions, we'd allow them access, you know, role-based access to certain services, and then we would track them. We would have, you know, a logging layer in place to actually ensure that we're monitoring and identifying what is actually happening. So that observability element, all of those exact same things we've done with humans with service accounts in the old days, those are the same kind of ideas we're gonna have to have with AI agents. And in fact, I'd go so far as to think of it almost like it's like a logistics supply chain, if you will. You know, you start with that data right at the beginning, and you need to understand, especially given the way that governance is going, or sorry, regulations going in many different parts of the world, you're gonna have to legitimately be able to explain, I think, in the future, and already some of some of these regulations are are describing this, exactly how a particular decision was made, or how did a particular outcome from an agent occur. And you need to be able to roll back through what did the agent do, what did it touch, what sources did it, you know, was it was it utilizing in that kind of logistical supply chain of knowledge, if you will, that then came out with an outcome. So you imagine you're a, I don't know, like a financial services organization, you're doing, say, loan um, you know, loan decisions. Uh, if somebody challenges that loan decision, how do you legitimately say, well, this is the reason why our agent made this decision? And I think that kind of regulation is only going to get stronger. So I think it's gonna be, you know, behoove us, as our good American friends would always say, to have an even stronger internal logistical supply chain, if you will, on that data and the AI agent. Yeah, you could expect that that to come.

Jason 7:41

So if you're not sort of building and thinking with that in mind, I mean, you don't necessarily need to be working in a pharma or you know, really regulated industry like pharma or financial services to for this to be at the top of your list is explainability. You need to think um, how can I ensure from a attack surface point of view, you know, in terms of my cybersecurity, my security posture, that I understand what agents are authorized, which are behaving to norms, which are doing abnormal things. Then also there's the liability aspect, right? So if you've got agents interacting in a supply chain and something is happening which an agent shouldn't be doing, well, what guardrails have you put in place to actually prevent that from happening? And and if you haven't, then is there an issue of liability? So there we we know that the technology capability is as always running way ahead of the legal frameworks, the you know, and actually what we see at the moment is the kind of almost like the operational frameworks are are being built. It's like you imagine the trains running and somebody's laying the track in front of it as it's going along. It's that kind of like, oh come on, guys, we need to we need to build this pathway faster. Absolutely.

Alex 8:48

And interesting you say that, because that was another kind of key theme that I was certainly seeing was around you get lot, you know, every bit of AI-related content you see will often use this phrase human in the loop. Um, and one of the key points that was really coming out in several of the sessions at that London summit was it's not about necessarily the human in the loop, because human in the loop becomes a phrase we can hide behind. So, you know, we say, Oh, this committee over here, they're the ones who are responsible. This group of people, they're responsible for the outcome of this. When actually, if we make something multiple people responsible for something, and we always know that you know humans are humans, nobody's actually responsible for that. And so the human in the loop, I think moving forward, needs to be much more about a named individual, uh, a named person who's in charge of a particular process. You know, they design that process, they see the outcome, the, you know, the sorry, the input and the outputs of that process. Um, and they are the ones who are sense checking, is this actually doing what I expected? Um, but not only that, but actually controlling the costs of that because it's so easy for us to you know utilize this technology to do amazing things. Uh, but sometimes I don't know how how much you've observed this with some of your experiments, but sometimes you find yourself doing something like I did a I did something with Git recently. So I did a very simple update into Git and using actually measuring the cost of it using the API, I saw that a simple push cost me 60p. Now that doesn't sound like a lot, but if you're doing that multiple times a day, that's a very simplistic task, which doesn't cost a human very much, but actually in the end up by doing that using AI costs quite a lot. And so there are plenty of ways that we could optimize the way that we are using AI, which ultimately for me comes down to the responsibility of that human, the human who owns the loop, not the human in the loop, if you will.

Jason 10:49

Yeah, and maybe that's a move from thinking about you know the the the human in the loop or the human owning the loop, the one who's effectively bringing, if you like, the business wisdom to it, you know, the understanding of how a process should run optimally, then that's everything from quality of data to you know the verified outcome an agent's producing, um, but also taking some ownership for the business value created, right? Because you know, if it's costing, you know, 10,000 pounds or $10,000 a month and you look at the business value and it's only half that, then you there's something fundamentally wrong with the with the approach. And you know, when we talk about having an AI operating model and defining an AI strategy, then one of the most important things up front is to look at you know the business value, the business feasibility. And there's an element of cost forecasting that's really important. And I think because so many organizations have been stuck in pilota, I don't think they've had to go through the hardy yards of actually finding out what it costs to deploy and then scale AI. Um, you know, and that's when the real inefficiencies become apparent.

Alex 11:51

Yep. Yeah, AstraZeneca, funnily enough, you say that they were they were referring to that. They termed it pilotitis. Um, the whole challenge there, the whole challenge there. You know, um, you can now develop things so quickly. Um, the development is actually the fastest bit often. Um, it's your rollout of your environments, it's the the trust, the change management, all of those parts in rolling out a proper production system. That's where a lot of the work is lying. I mean, arguably it was always there, but we some people like to avoid that part. Um, but if we don't have a a focus on what are the what are those three really important things to most businesses, right? So what's the return on investment, or what's this kind of cost impact to this, or what's the performance improvement that we're going to bring for our team, then we're probably just going time after time after time, doing endless pilots, not actually gaining any business value. So it's really important to actually pick a, you know, even just a single outcome. Work out what is the ROI on that, work out all the things, the fundamental foundations that you need to get to that and drive that thing to completion rather than just experimenting endlessly with all different, all different options.

Jason 13:05

And I and I remember when Sever, you know, we had Sever and Alex on and they were talking about agentic AI processes, right, and how you build them and how you run them. And you know, they were effectively just saying you designed for failure, right? You designed for failure because if if you don't design for failure, then when it happens, it becomes something that what do you do? When you're not prepared for it, will you throw people into the mix? And I've always had this question in my head at what point does the amount of effort involved when something breaks, whether you set up a great process but it breaks, somebody gets involved trying to fix it. If the cost of trying to fix it is more than the value created for the time it was running, obviously it just doesn't even balance out, you know, financially. Um, but there's also the the the challenge that as organizations step into this world of embracing embracing agentic AI to drive business processes, core business processes, ones that they actually rely on operating. They need that confidence in the tools, in the understandability of the outcomes, that there's visible, there's a visibility and observability of what's happening, and it's explainable, right? So that the teams running it, operating it, understand it. And if you have a process which breaks and people spend 24, 48 hours looking at it and they still can't understand why it's broken, you can imagine the confidence dent that that organization is going to have. It could set them back a long way because you know, fundamentally, the the people who are the the ones making the decisions at the top and signing off the budgets and shaping the agenda for technology, they may steer away from that or it might take them quite a long time to come back to it. So I think it's definitely one of these areas where you want to make sure that you really have understood the risks and that you have you know, like a risk management plan in terms of your operational response to things not going as you expect.

Alex 14:52

Yeah. And and that's an interesting one because again, massive theme that we are seeing all the time is around trust. Um, you know, there's a was it they say trust is built in drips and lost in buckets. Um, and I think we we see that certainly with AI. When it first came along, everybody was a bit like, what is this thing? And you know, just within a couple of years, you see far, far more people almost trusting it beyond where they should, I would argue, in some cases, and not having that level of I'm just gonna make sure this thing is okay before I actually put it in production. So I think that trust element is something that, as again, we go back to human nature, as so many things often are, isn't it? we we start with little trust in something and then we build very quickly often perhaps more trust than we should, or certainly you know, a comfort with a particular technology. Perfect example. Um, if you've ever used an autonomous vehicle, like a Waymo, Wimo, Waymo, I don't know how to pronounce it. but if you've ever been in one of those, you know, the very first time you're in and you're sitting there and you're like, I'm not in control here, and this vehicle is in motion and there's other traffic around me. Um, and then after two or three times, it becomes natural, becomes normal. You forget about those trust issues. and I think the same thing applies when we're using AI. Um, around we understand some of these constraints, and everybody says be careful when you use it. Um, and then you start to get in, you know, you get familiar with it and you start to trust it and you trust the outcomes, but then you maybe you in that case you're forgetting about the guardrails that you should be putting in place mentally speaking, if you will.

Jason 16:34

Yeah, I it's interesting when when when we think about trust and I think about AI, one thing that comes to mind is consistency, right? you think about technology and you know, just think about something simple like an Excel spreadsheet, right? So people have been using Excel or Sheets and Google or whatever for years, right? And you know, if you're using in particular using a cloud-based tool like SAS-based tool, then you know you you enter your data, you expect it to be there, right? You don't expect it to disappear. So we we've become comfortable in working in certain ways and we don't question them. And I think the thing is when if you think about the pace of modern business, is we move at speed. We're all being asked to move at speed. You don't have the chance to question things when you're moving at speed. So very quickly, we move from questioning to just that's just the way it is, it'll work, I'll just accept it, it's part of my norms. I'm thinking about if you know if you're a technology team in an organization and you're effectively trying to build trust, you know, by operating an AI platform, you know, that's doing various things to the business they're starting to rely on. If something happens, then it's not necessarily that everyone will lose faith in AI itself, although that could happen. I've seen an organization where they they had a bad experience with workplace AI and then they just retreated a lot from AI, you know, big picture. But what I'm thinking about here is the is the reputation of the team, right? So, you know, you're building an AI platform, people trust it, or they think it's trustworthy, something goes wrong, you can't fix it. That lack of trust and belief is gonna be focused on the implementation of it by the by the technology team. That's the thing that concerns me, right? And and how you recover from that, I'm not quite sure because everyone at the top of the organization is going to be thinking AI is the way to move forward, but at the same time, you've got this challenge, which is I'm never getting in that Waymo car again. Because it just crashed into a lamppost. Yeah, exactly. Yes.

Alex 18:31

Yeah. We should be clear for the purposes of legal proceedings that we are not aware of any particular Waymo related incidents.

Jason 18:38

no, and you should be safe getting into one, honestly.

Alex 18:40

Trust me, Gov. Yeah. Um, but yeah, so the the the other part though, and when it comes down to that that trust pieces, I think there's a we're we're in a really interesting time right now as we scale this use of AI. Um, because as adoption is growing inside of organizations, um, a lot of that has been on the back of let's call it fixed price AI access up until now. and so I think one of the one of the looming challenges is AI is an incredibly expensive thing in the background, you know, setting up data centers, providing that infrastructure, the research and development that's going into it. I don't think it's as sustainable, let's say, for us to continue to see AI being made available at those same record low costs, if you will, to the consumer or to to certainly smaller businesses.

Jason 19:38

Yeah, so this question of the cost of running AI is an interesting one because first of all, it's it's still such an immature area. So, you know, when when businesses started using you know APIs to drive some of the models to make applications intelligent, one of the struggles that the FinOps Foundation had was that a lot of these um, if you like, levers to execute, they didn't have the kind of financial metrics. They weren't giving you the visibility and observability into what's happening in terms of driving the costs. So the reason FinOps are involved and they have a scope, services scope for AI cost management is because it has a fixed element and a variable element, and the variable element is around the use of tokens, so what you're driving through in terms of inputs and outputs, and also very importantly, not just inputs and outputs, but what the model is doing that you don't see. So the reasoning tokens that it uses, the internal computations that it performs, and they can be really, really high, um, especially if you're doing things like coding because a huge amount of computation being done within the model before you get something back. So the the challenge around the the the economics of and let's call let's let's make that a workable description, let's talk about forecasting, right? So forecasting, tracking, because you mentioned you know being able to track the ROI of using AI. And the only way you can track the ROI is if you understand the value of the problem it's solving and you understand the full cost of what it's actually costing you to solve it.

Alex 21:10

Yep.

Jason 21:11

And the challenge is that the cost of the tokens is not the complete cost to an organization operating AI. In fact, it's anything but the complete cost. But if we think about what's going on in the in in the market in terms of the large players of the frontier organizations like Anthropic and OpenAI, they've got this strange thing going on, which is almost like a structural imbalance. So you've got individual subscriptions, so like you and I might have a you know Gemini pro or yeah, and Anthropic's pro or something like that. Yeah, you know, we're we're paying an amount. I I pay twenty dollars a month for using Google's subscription, right? So I get a certain number of tokens for that, I get a reasonable number. Um, but lots of people who subscribe don't use their allowance, so in a way it's subsidized because they usually don't use a lot of subsidizing those who do. insurance policy model isn't it ultimately yeah exactly exactly exactly but you see that that only makes up a small proportion of what the of the tokens which are being consumed through that company so roughly 15% this is more more of a global stat roughly 15% of tokens are estimated to be going through the consumer tools yeah and about 85% are being consumed through APIs and organizations who effectively plugged you know models and and made them available for intelligent applications and and other things that you might be doing with it such as you know coding developer workflows and things like that so if we think that the majority of the the tokens that are being consumed out there are being consumed by organizations who who are chasing them for business use proper use all kinds of things the cost of those tokens has been dropping and dropping and dropping and dropping and dropping so the cost of operating models and the cost per token has been dropping so so financially you know you could say that the the supply cost for all of the organizations using these things has just been dropping continually and it's the frontier companies who've been driving that cost down yep and the reason they wanted to do that the reason it's become a bit of a pricing war between companies like Anthropic and OpenAI is is they're chasing market share. Yep. It's a land grab yeah a land grab and they've been able to do this because they've been burning funding right they've been burning investment funding and they've been doing what every company does that tries to win by scaling up is spending inefficiently or let's say less efficiently to win customer share mind share because they know that as they mature their products and people get more comfortable with them they'll settle on them. Then as it becomes a if you like a formalized business revenue they've got those people they've won that space. So we've got this strange situation where the majority of tokens are being consumed as a commodity or using commodity pricing. And it's only I think when we see Anthropic and OpenAI go public and they've got shareholder pressure on their returns that we're going to actually see that behavior change. Now that in a sense is what's driving the overall engine of consumption. If we think about tokens a pool globally imagine a huge pool globally of compute there's a certain certain number of a certain amount of inference which is when the model's running on you know chips because the model every model is an algorithm it's software it gets run on compute right so it can you know it's a certain amount of compute out there yes people are trying to build data centers but we know it's taking time for them to line up the extra capacity we see already problems with power with power grids with even the skilled power engineers to help enable this data center scale out that's happening in the US and the UK and other places. Right.

Alex 25:07

But so that's a bit I I I think that's a bit I'm worried about because we talk about the cost coming down but you've just described there we have a limited pool don't we? Right now it's limited and yes the pool will get bigger but that build out will take time some of that build out is going to be on Mother Earth some of that build out if you you know if you follow certain individuals is going to be extraterrestrial right yeah and you and I are both you and I are both sci-fi fans right so exactly we probably shouldn't go down that tangent because that'll be the rest of the episode but that but that capacity whether it be on Earth or extraterrestrial takes a long time. So we given the popularity that we're seeing of AI given the growth explosion that we're seeing in the use of AI my biggest concern is that those tokens become scarcer and scarcer because you have a larger and larger pool of people who are going after the same the same volume of tokens or a very slowly growing number of tokens. And we have that there's that thing called Jevon's paradox in there which is the cheaper you make something actually it's not that people will save money they'll just buy more of it. You know the perfect example is things like light bulbs we move to much lower power and cheaper light bulbs and what happened people put lights everywhere then they put lights into devices and and now there's you know way more light than there ever was and the the actual quantity is going up year on year in terms of power on lighting despite it became being lower power. So now we take that to AI we have this AI explosion and a slowly growing pool of tokens that's where I'm a bit concerned that we're gonna we're gonna run into challenges. What do you think?

Jason 26:51

Yes I think what you've described there is we we could call it token scarcity I'm not for I'm not coining that phrase somebody else has the token scarcity is really relating to the fact that we're we're moving into an era where you'll be wanting to consume something and it may not be there on tap as you've expected it to be right so you hit a cap. And how can we see this happening? Well let's let's think it's really quite simple it we've got companies who are driving demand and they want to because they want to grow market share. So part of that is is they want to grow a bigger piece of an expanding market. We know that AI is a mega trend and as we see agentic AI running business processes the complexity of that AI operation is going to increase what does that mean for tokens well let me let me give you a simple idea humans interacting via an NLP natural language programming chat window typically generate 500 to 2000 tokens per interaction if we think about using something like clawed code or custom enterprise tool sets single command can process between 5000 to 1 million tokens in a in a context block. And the thinking token requirements for agentic processes is also really high and it's the the challenge is it's what you wouldn't see because it's behind the scenes. You'd see the cost but you wouldn't be able to if you were measuring you know from a FinOps point of view tokens in and tokens out you wouldn't necessarily see it. So the the challenge is as we as we move on this on ramp to wider AI adoption which means more token consumption is going to come from business use. Business use is what's going to drive the higher requirement for tokens that's not going to be curbed initially by the costs because the companies are still focused on the land grab. So then you're gonna hit that ceiling and it you know the kind of it the suggestion in a way is that we need to start thinking of tokens as a not a limitless resource. We need to start thinking in terms of how can we make our AI processes token efficient. And actually stepping back from all of that would be do we understand what's in what what's where do we understand where value is being created and do we understand where cost is being incurred because not all tokens are equal right so if you if you're using a really high value or let's say a high very high quality model versus a lower quality one to do a task that doesn't require the high quality you're overpaying for it. Yep. Okay so those tokens are actually costing you a lot more than the ones from the the lower quality one that couldn't give you a very adequate response. And you know I saw a brilliant quote the other day from somebody called Brian Latour who is saying I've seen the same processes operated and you know one person running them using different tooling or different different parts in that process and it costing eight times as much to operate effectively get the same outcome.

Alex 29:57

And this is a challenge because if you think about it from a financial management point of view how how can we understand enough of this to see where the waste is so that's that's sounding awfully familiar Jason I seem to recall many times in the past 20 years having conversations with people about why did I size a virtual machine with eight CPUs when I only needed one? Or why did I put 32 gig of RAM into something when I only needed eight et cetera et cetera this is sounding a very FinOpsy conversation to me.

Jason 30:34

Yeah absolutely absolutely and I think it's interesting you say that because when people sunk investment you know capital into on-premises kit the only the only kind of conversation or argument happens once is getting the purchase sort of signed right whereas when you move to the cloud and it's FinOps then you have people you know coming back and repeatedly asking like okay you know why is this change? What why is this so high? And it it could be down to inefficient usage. It could just be down to growth right so it's trying to understand it. But you're absolutely right. So the FinOps foundation which was formed 2019 has as one of its services a focus on cost forecasting and management around AI workloads but to to put in perspective how important this area of tokenomics is going to be there was an announcement on the 3rd of June that the Linux foundation is now launching something called the tokenomics foundation which is going to work in close partnership with the FinOps Foundation and it makes sense for them to do that. But the the the point or purpose of the tokenomics foundation is to help shape open industry standards, benchmarks and best practices for the economics of AI infrastructure. And this is absolutely needed. It's a Gartner of forecasting that by 2029 50% of cloud workloads will be running AI. So we can see that you know AI is going to dominate both from the point of view of driving business outcomes but also shaping technology platforms and ultimately the cost you know if you think about what's in the IT budget a large proportion of it should be relating to all of the parts of your ecosystem that are enabling AI to drive your business processes.

Alex 32:25

And you that's the same challenge that we've seen again it's a history doesn't repeat itself but it certainly rhymes, doesn't it? We've gone through the same thing as we had the shift towards cloud and let's say the discomfort of financial professionals and organizations with releasing budget for something they're not entirely sure what it's going to cost them. As soon as you move from that you know static CapEx spend to something that's a lot more metered it becomes a challenge and therefore I'm gonna use the word for the second time today it behooves us as IT professionals to to to manage that spend and to actually put those those wrappers and that governance in so I think tokenomics absolutely is something that I think anybody who is touching AI in any way shape or form should be considering and you know who owns the token bill when you are we're running hundreds and hundreds of agents where does the responsibility lie? And it can't just lie with one individual one that doesn't scale and two that leaves no sense of accountability and responsibility in the organization. So I think in just the same way as we saw in FinOps the the idea of tokenomics is going to be a cultural necessity if you will inside of organizations not just a technical necessity. It's not just about observability that's very useful. It's useful to have the data to back up where we're look what we're looking at but it's also fundamentally important that we train and enable people to understand the impact of what they're using. And you know that if people don't understand you don't know what you don't know you don't know what the impact is going to is going to be on token usage is going to be on anything. So I think that's gonna be a really key thing for organizations. And what's possibly more of a challenge there I think for some organizations is they're still trying to wrap their heads around this today in the expert teams, you know, the centers of excellence whatever you want to call them in your organization they're still trying to wrap their heads around this. But if they're not already thinking about how do I enable others in the business, then they're potentially setting themselves up for these you know larger cost implications if you will.

Jason 34:41

Yeah I think there's an element of maybe technology teams are still chasing like running to catch up themselves because you know there's been a lot of focus on trying to understand the the platforms and how you operate them and we talked earlier on about control plane and how you get that governance you know and so we're kind of like you've got this squeeze at both ends. You've got trying to find the business cases that the the right ones the best fit for the models the technology that deliver the value that are feasible based on the data you have, the industry you work in. But then you've also got the other side which is do you really understand all of the financial metrics involved to be able to give somebody a you know that full view of return on investment. And if you don't then you are you know effectively some part of your organization is subsidizing that process and you could be effectively running less efficiently than you were before you put it in place which is a kind of scary thought I guess.

Alex 35:38

Absolutely and and it's interesting we talk about you know education skills etc and one of the other things that this we're seeing in terms of these trends and this was actually brought up at at the summits as well I think both Amazon and the AI Summit they were talking about the same topic which was around the impact on roles and hiring so as it's it's the elephant in the room but the impact of AI on jobs will agents cost jobs so there's been some really interesting studies starting to come out and I think what we'll do is probably post a few links in the show notes about this let people read up and and kind of start to form their own opinions on this but certainly what the data is showing at the moment is in technical knowledge management roles so let's say engineering developers those kinds of things there absolutely has been a reduction in the number of junior positions quite a significant reduction I think it's around eight or nine percent according to one of the large American universities but interestingly the flip side is there's an almost identical increase in the total number of roles in senior development roles and senior engineering roles and so when you look at it from a a technology standpoint or technology role standpoint the balance is not or the trend is not down it's that it's shifted from junior to senior and now this is not going to be the same in every vertical quite clearly there'll be other ones where you know agents are like for like replacing processes and therefore if you think about you know an an organization is really a set of people and collective processes combined and so if an agent can do that process then it will have an inevitable impact on roles but certainly as we say we're we're seeing this more of a a shift from let's call it a top-down you know triangular shaped organization or a pyramid shaped organization with a smaller number of seniors and lots of juniors and a you know higher bottom end intake to more of a you know a a a middle middle yeah i'm thinking of a diamond yeah i'm thinking diamond shaped diamond shape that that's far more polite than what i was describing myself so so i think there's a there's a huge challenge there that we're gonna have to face because where do we get the talent for the next generation okay well you you've touched on a couple of points there so let's let's dig into them more now are are we talking just you know software development developer teams or are we going broad on that in terms of general market well it's either way i mean i'm interested in your opinions on both okay so we I think you you were asking a question you know around the developers you know for example do do does coding do coding skills still matter because if you think about that program of you know juniors on boarding so we've got this challenge which has always been the skills gap and skills transition so when people come out of university they've learned say do software engineering they come they've learned how to code they come out and and they might know 80% 85% of what happens in a coding team in an organization but there's always a gap right there's always that gap of the way we do things of some of the smaller tools and the processes so while they might understand some of the really big building blocks of of the whole software development lifecycle there are some specific things that they're going to learn and develop and shape within the workplace so if we if we talk about what what the what they gain from coming in at their junior level what they gain is they fill in those gaps.

Jason 39:31

What they gain is they get to work with senior colleagues who can coach them who can help them mature they deepen their understanding of processes and they become more efficient more effective also there's that communication aspect as well because no developer works on their own. So working teams communication is a skill you learn that skill by working with other people and you're going to do that most in the workplace. So we think about do we need developers do we need junior developers I think there's there's there's two aspects to this one is do we need them now right in terms of do we need what they could bring in a team coming in to join versus having some agents operating and running some of those development tasks. So I have a colleague who I can't say the company you work for but his measure was if I have got some extra development work I need doing he was looking at whether he would go to an offshore team he said if it's if it's less than six weeks worth of work I won't bother onboarding new people because it takes too long I'll just look to actually start setting some agents in place to do the work. Right. And if it was longer term then he would consider going to an offshore team and and spending time with them, getting them familiar with the you know with the the the project the code base the requirements that type of thing. So he'd already started to work out this equation in his head of when is it not worth bringing the person? If we think about something you said in the past, you know, if we don't have those juniors doing that work and building up that scar tissue of the challenges and and all of the difficult things who is the middle level or let's say the senior developer who has the understanding of all the different aspects of the stack and the software development lifecycle to actually orchestrate those agents to set them up and to run them so that effectively he's got you know 10 10 different tasks running at the same time as if he had you know three or four devs sat there working for him. So there's the productivity at the start which is when the junior comes in they're going to contribute something but you know let's be realistic most people when they join an organization it takes them months to get up to speed and yes ai can help that but then the challenge of not having that pipeline of people coming in is you're not growing the mature competent seniors who you're gonna need in 10 years' time. Exactly exactly and that's and that's an area that we understand well right the area development we you know that you could probably repeat this conversation for legal for scientific you know genetic research or something I don't know the the the point is it's the it is the younger people it's the people who are looking for their first role in work who are really really being impacted back to your point about the diamond it's the bottom of the pyramid that's now becoming the diamond that's the issue.

Alex 42:33

And interestingly Andy Jassy was saying quite publicly made a statement about this just a while back and I don't have the quote in front of me but he was effectively saying that AWS intend to continue to hire developers at the same rate they ever have from those junior roles and arguably from the same perspective, you know, we talked about that Jevon's paradox before well actually if you are now reducing the limits or the the barriers to entry to be able to create code then actually what it does is it will increase demand. So a lot of people are talking about you know there was the SAS pocalypse that that was in the in the news and was impacting share prices and all sorts of things you know a few months back. But what may actually be the reality is that because this becomes so easy because it becomes so cheap to do actually demand continues to increase and so you still need junior and mid-level developers to be driving that you know at the wheel if you will so we could find ourselves in a position where more software demand equals actually still junior as much junior hiring in the future once this thing is settled out.

Jason 43:42

Yeah yeah there could there could be a lull and then there could be a peak actually and I can see that happening because if you think of the power of you know vibe coding and certainly like we were talking with Geo and Yasik about vibe coding with with the scaffolding right so thinking about those really core Skills are defining requirements, um, and you know, then the testing and things like that. It's entirely feasible to believe that you know, um, subject matter experts in line of business functions that we could train them, we could give them the skills to be able to, you know, create 30%, 40% of the value in an application or even more, but it still needs somebody to come in and then look at the code base to work out the inefficiencies, to work out the vulnerabilities, to formally you know take it through a bit of a sheep dip of productionizations. It's safe to let loose in the wild and it can be supported. Um, and you're right, because then that would scale the need for developers. They would be doing a different type of thing. And it actually got me thinking when I was thinking about the episode, you remember like Agile, when Agile first became a way of project managing, you know, there was a role for agile coaches.

Alex 44:52

Oh, yeah.

Jason 44:53

Yeah, and and it was always to scale other people. There's lots of project managers out there. The the challenge was how do we get them all to onboard agile ways of working? How do you bring the, if you like, the the process support around them? That's not just the tools, it's the way we do what we do. It's making sure it's implemented in in the right way because agile is a certain way of doing things. So it's all it's part of it, part of it is a cultural practice. Yep. It's actually, it's actually how you see people do stuff. And I was thinking, well, maybe these, you know, the the kind of senior devs will end up being almost coaches to some people out in the business who are, you know, who are shaping their own applications because they're so close to the need, you know, that if they can think of a really bespoke application that could do something incredible for them, for their team. Um, but you know, we we can't say no, we won't be to say no because the genie will be out of the bottle from the point of view of this is something you genuinely could do. So the question then becomes how do we de-risk it? How do we de-risk people wanting to move at speed and do that?

Alex 45:56

And the the just I I think you you nailed it there. and I just to add one kind of additional almost thought on top is the you mentioned the profile of those juniors, right? I think that's actually the really interesting point here because I I'm gonna I'm gonna put my I don't know, stake in the ground. Uh, and I'm gonna say I think that given the rate and pace of improvement that we've seen in the capability of models, not even just the foundational models, but actually some of the open source, open weight models, etc., um, to deliver high quality code, um, that's only gonna increase to the point where I don't think it will be about the ability to code or even the ability to go in there and sense check and peer check, however you want to put it, the code or peer review, I should say. I think that those juniors, then it becomes far more about them being the problem solver and also the the communicator. So the you you talked about, you know, going out into the business and working with individuals in the business. If you have somebody who's that junior to mid-level developer who has a fundamental understanding of the code, but actually the AI is doing a very large proportion of that, um, then it's up to the junior almost becomes more like an architect. Um, so they're translating those requirements, they're translating the meaning, um, you know, that vertical knowledge, if you will, from the individuals, the subject matter experts in the business. And they are the the conduit, if you will, that helps that to become a reality in the system and be done in a in a way that's sensible. Because as we know, the the larger the context, the more challenging it is for AI to handle. And that's one of the reasons why AI is fantastic at writing you a module, it's less fantastic at writing you an entire software suite. and so then it becomes more of that architectural mindset. And I can tell you from experience, actually, I've run a number of years ago, we actually did a graduate scheme, uh, another organization I was working at, and we brought in people straight out of university and put them into architecture roles, which is sounds crazy when you think about it, um, because they don't have all of that backlog of knowledge, but that also meant they didn't come with a load of baggage. So they didn't come with a load of hang-ups about this is the way we used to do things. And so they were thinking, and they still do, because it's it's fantastic to see you know where they progressed in their careers. Um, they are thinking in a very cloud native fashion rather than say somebody like me who has a mixture of I've done things in data centers and I've done things in cloud. So I think there's also going to be a huge benefit to bringing in these juniors who just have a completely you know, a blank slate, if you will, and a different way of looking at it. And they're gonna be this native generation.

Jason 48:39

Yeah, no, that's that's a really good point. You know, if you introduce somebody to the workplace 15 years from now, there are some things they just will not bother learning, right? Yeah, will have no value to them. And there'll be things that we wouldn't have thought about. Tokonomics, for example, will be, you know, think 10 years from now will be very well defined. Um, in the same way that if you think about FinOps at the moment, you know, FinOps tooling processes, there's so much available. And, you know, to some extent, even the hyperscalers have stepped up to bring their capabilities to try and make it easier, you know, in terms of following open formats and bringing their own tools, improving improving their own tools. So over time, yeah, you it's definitely believable. So maybe as our closing point, I want to finish on something that's a bit of a thought provoker for the audience, because the the data we're seeing suggests the impacts of AI are predominantly on junior employment, um, that it's happening now and it's structural. And you know, we see two things that are affecting the, if you like, the appetite or desire to recruit juniors into organizations. So one is task-level substitution, where we've got people who were doing things within their role, such as modeling, data entry, document drafting, um, preliminary code generation, and and they are the exact capabilities where LLMs can excel. Okay, so it's very easy for an organization to see how they can substitute the role that the junior will be doing, those tasks within their role. And the other is when we think about programs where where organizations bring people in as a junior, um, they rotate around an organization, they progress, they mature, and then they move them up the stack. And we talked about developers there as an example, right? So how people come in and they then start that career path moving up. And because they're seeing a different way of doing that work overall, almost rethinking, re-engineering the way they do the work, they're not seeing the need for that pipeline. So there's both a removal of a need for the somebody to come and do the work right now and be a contributor, and potentially they're thinking we don't need to grow that person for the future. it creates a paradox because if companies do not hire juniors because AI can do their work, then at some point our organizations will face a severe shortage of experienced midland senior managers, right? Whether it's a decade from now, 15 years from now. So to what extent do we think organizations are I was gonna say making short-term decisions, but I'm actually gonna say sitting on the fence because I feel like a lot of organizations, because they're not sure how much AI is going to reshape their future workforce needs, I feel like they've just hit pause because they don't know what decision to make. I'm interested in what people in the in the audience, our our listeners, think to that. That's fascinating.

Alex 51:41

I think that's a that's a fantastic point to end on. actually, if you would like to share your opinion, we would love to hear it. as always, um, you can obviously get in touch with us via our socials. We're at software one just about everywhere. Um, but also we have an email address. So if you want to email in, we would love to hear from you. And perhaps we might even read out a few of those comments on on a future episode. you can email us at o twoi at software1.com. That's oh the number twoi at software1.com. and just before we go, I'm gonna plug an absolute shameless plug. we actually published a blog post literally just a few days ago from the London AI Summit. So I'm gonna drop that into the into the show notes, um, which covers off some of the topics we've been covering today. And also we just didn't have time to get through everything. we had a whole bunch of other things we could have talked about. So definitely encourage you to check that out. Um, and with that, we're gonna wrap up and just say thank you again, everybody, for joining us. Um, and we look forward to the next episode. Please, if you did like this episode, please do hit subscribe on your podcast app. leave us a little review. It absolutely, it really genuinely helps other people to find us. and yeah, as I said, if it's something you want us to cover in the heat in the future, please don't hesitate to reach out. Until next time.