#27: Embedded AI vs bolt-on AI in supply chain planning

Benjamin Obling (00:00):

We've also all received this 20-page Claude-generated PowerPoint with a lot of-

Søren Hammer Pedersen (00:07):

Looks good.

Benjamin Obling (00:07):

Yes, it looks awesome. And there are a lot of figures on it, but you basically don't know which ones are correct and which ones are not, and you can't verify it. So you'll just trust it and then you'll go ahead and shut down that factory.

Søren Hammer Pedersen (00:19):

The embedded AI allow you to have this, let's call it army of agents working before you even start. So everything is pre-prepared when you come in and you start from the conclusions, the recommendations.

Benjamin Obling (00:32):

What is the purely automated AI forecast? What is the forecast with adjustment? So if I just type in Claude generally, "How does the forecast look for next month?" Whatever-

Søren Hammer Pedersen (00:44):

It will show you something.

Benjamin Obling (00:44):

... it thinks it's the forecast. Yes, exactly.

Søren Hammer Pedersen (00:48):

When you meet the planners, supply chain managers, et cetera, how do they view this? Is this a blessing coming in or are you hearing, oh God, here comes another black box in my working life?

(01:10):

Hello everybody. Have you ever gotten the question in your company, "Why don't we just pour all our data into Claude AI and use that as our planning tool?" Well, then this session is definitely for you. Today we are looking at the difference between the embedded AI into dedicated planning tools and the bolt-on AI functionalities that many companies are looking at to see if they can solve the whole puzzle. My name is Soren Hammer Pedersen. In my daily work, I help companies optimize their supply chain planning and I'll be hosting today's session. In the studio, I brought my good friend and colleague Benjamin Obling, the CPO of Perito IBP here at Roima Intelligence. Welcome, Benjamin.

Benjamin Obling (01:52):

Thank you.

Søren Hammer Pedersen (01:54):

This embedded AI versus bolt AI, maybe let's start by setting the scene, what is it, and what is that company's AI experience at the moment?

Benjamin Obling (02:04):

Yeah, so you can see you have different directions and you probably want to go multiple directions, but you could say one way of utilizing AI that almost all companies are doing of course is to have a thing like a Copilot, to have a Claude, for example, OpenAI. You obviously need to be sure that the data is not leaving your company or at least the vendors that you're using, that you're trusting them. So that is you could say the OpenAI model that you can use for different ad hoc tasks that we are all using or almost all using on a daily basis, which are absolutely awesome and are accelerating the way that we all work. And then you have the AI tools that are inside your specific products for various tasks like the integrated business planning, for example, where you have embedded AI agents in that knowing exactly what is the task that you're working on right now in the IPP tool, what are the different rule sets, et cetera, and are handling that.

(03:13):

You could say the LLMs, the agents behind the scene that are running that are actually the same. It's the same setup, but they are guided in different ways and they are enabled in different ways in the software.

Søren Hammer Pedersen (03:26):

So the embedded AI is basically a dedicated AI customized to you working exactly in your planning tool where the LLM is more or less the raw material going into all that?

Benjamin Obling (03:40):

Yeah. You could say you can deploy an LLM in various different ways where the open the ChatGPT as the first mover and then general usage of that of course is one thing and it's using that. But in the tailored into the different tools, the advantage there is really that it knows the contact, where are you in the solution, where are you in the process, what is the governance, which rights do you have to see this and what are you working on right now? And then it's also a lot more likely that you'll be getting some good questions and answers from that. But behind the scenes, the raw material you could say is actually the same or can be the same LLMs.

Søren Hammer Pedersen (04:23):

Yeah. But I understand the question people get in that company of course, when we have Claude or Copilot or ChatGPT or whatever already, why should we look to have other solutions as well? So quite natural. So I think it's important because our statement here is of course that the embedded AI can do different things that you need in your supply chain planning. So let's go to that topic next and say, okay, where is the main differences between the embedded AI and the bolt-on? Where is it that you really either get into trouble or get a lot of benefits in terms of the embedded AI?

Benjamin Obling (05:05):

Yeah. So you could say one thing is how we instruct it. So when you open up and you start a question, if you do it in a general model like OpenAI or Claude, et cetera, you'll need to prompt it to say, "What is it I want to talk about today?" Because it doesn't know whether you want to write an email about postponing a meeting or whether you want to explore a new market opportunity or something like that. So the first part is how it's instructed and what is the context. And having it embedded in the IBP solution, for example, if you are in a forecast, you're working with your forecast, it's quite obvious that then we can instruct the LLM to know that, okay, what you probably want to know when you're looking at your forecast overview is you want to understand, what is the new forecast?

(05:52):

How is it different from the forecast we had from last cycle? What are the different market intelligence, the inputs that we've provided, et cetera, and what are the different dimensions that would be interesting and relevant to analyze? What have you previously found interesting and where could we find those highlights? And there you can see you're a lot more likely in the embedded version here to get a good answer to your question than if you start completely from scratch in prompting your AI. So the instructions is one part of it, what are the instructions? What is it going to do? What is the context? And then also the data, obviously, because you could say in the IBP product, we can make sure that when we talk about a forecast, you want to look at your forecast overview, how is the forecast for next month?

(06:44):

It knows exactly what is the new forecast, what was the last one, and what are the market intelligence changes? If you upload a lot of data and it has access to that in a general LLM or a general AI agent, we'll use AI agent and LLM a bit wise where you could say the LLM is sort of the engine behind it that would then call an AI, and an agent is something that has some tools around that LLM just to frame that. So that, you're a lot more likely to get a very strong answer to the question because it has that context and it knows exactly what is a forecast.

(07:26):

Because if you upload a lot of data unstructured, you would have a lot of different things that would be called forecast, for example. It could be what is the purely automated AI forecast? What is the forecast with adjustment? What is the released forecast? What was last cycle's forecast? What was the forecast made for this month, three months ago, four months ago, already there? A lot of different forecast versions. What are the forecasts we've uploaded to the OP system? What is the supplier forecast? So if I just type in Claude generally, "How does the forecast look for next month?" It will go and pick up whatever-

Søren Hammer Pedersen (08:05):

It will tell you something.

Benjamin Obling (08:08):

... it thinks it's the forecast. Yes, exactly. Whereas in the embedded, it will know exactly when you are on this page, it will know exactly in this step in your governance and your process, what is a forecast here?

Søren Hammer Pedersen (08:21):

Yeah. So basically, the embedded AI is you are starting way ahead already instead of building up. So it's 10 step ahead already and it already knows what to do and what to answer you basically when you start. So there's also, besides coding, getting stronger answers, that's for sure. It's built on best practices and things like that. It also allows you to be much more effective in your company.

Benjamin Obling (08:52):

Absolutely. Absolutely. And you could say there is another element, a more simple element is a cost element. If you throw a very general question and a lot of data at the model, it will burn a lot of tokens which cost money. So that's basically how much energy and water does the LLM need to get to a good answer? That's going to be a lot more costly.

Søren Hammer Pedersen (09:17):

Yeah. I think one thing that is a different that is also I think stopping a lot of bolt-on AI is the matter of security in this. How do you see that? Because there will be issues there, I guess.

Benjamin Obling (09:35):

Yeah, absolutely. You could say if you should be able to in the say general AI, the bolt-on AI to be able to ask any question, then it needs access to all of the data. So then the big question is how do you then make sure, first of all, that that data stays in your company or with your vendor? So you need to be absolutely certain about that of course. But after that also, okay, so who in the company can then look at different forecasts, if we use that as example? So if you are the plant manager or you're the sales manager in Germany, should you be able to see the forecast of Italy? If you're the S&OP manager across, you should probably be able to see everything, you should be able to, and so on. So that's another element that is super critical. So of course making sure that the data stays within a trusted area, that's the bare minimum.

(10:30):

But then the next step is that all the different users link to roles and we can then confine what can they see and what can they do. And that's one of the things we've been spending a lot of time on in embedding the AI into the IBP solution to make sure that you can only ask questions within the data set that you are allowed to see.

Søren Hammer Pedersen (10:52):

Yeah, that makes sense. And I think it will be a big concern, but cost, of course, also what you hear now in the market about, the cost of tokens probably going up quite dramatically now, that is a relevant point as well in all of this.

Benjamin Obling (11:10):

Also because cost is also time. So I would argue it would probably spend a lot more time or you will certainly spend a lot more time analyzing when you just, let's say that you could then download everything or you provide all of the data to the LLM and then you start to ask questions. So what is the forecast? The first analysis will be on the released. Okay, that was not exactly what you wanted. You wanted to know the new forecast. Okay. And then it will look at that. Okay, but then compare it to the released, okay, then it got it wrong and compared it to the last released two cycles, blah, blah, blah. It will take iterations and so on where actually here we can gain a lot if we then prepare that already because there are a lot of the analysis that are actually repeatedly interesting to compare.

(11:55):

For example, understand your new forecast, what has changed compared to the actuals, compared to released in last cycles, compared to the market intelligence and so on. That is interesting every month. So prepare that, have that ready already. So you just press a button in the tool directly, you can see it, you have all your insights, and then you can also go ahead and change it. And that's the most important part because otherwise it's just an analysis for the analysis. We should do something. If we don't change the forecast after that analysis, of course it could be because the forecast is absolutely awesome and that's all good, but you could say you probably need to do something otherwise you're wasting your time.

Søren Hammer Pedersen (12:34):

Yeah. I think I used the very beautiful picture last also when we talked about Aura AI and other things that what the embedded AI here really is also is allow you to have this, let's call it army, that might be a bit much, but of agents working before you even start. So everything is pre-prepared when you come in and you start from the conclusions, the recommendations, the insights to this instead of, okay, now I'm coming in Monday morning, now I need to build this.

Benjamin Obling (13:05):

Yeah, yeah, yeah. Absolutely. Yeah. And then you could also say, I mean there is also actually quite a lot of time that needs to go into defining the analysis. Of course you have the open questions that you can ask inside the solution regarding the data that you are responsible for in the areas, the forecast and so on. But you can say these repeated analysis are actually not trivial to create the instructions for that. So what is interesting when you compare the forecast, for example? Then finding out, okay, it is the released, it is the live forecast, it is blah, blah. What are the different dimensions that we would like to look at? Is it revenue? Is it just quantity? Is it also space? Is it different geography areas? Is it packaging?

(13:53):

And then making that sort of recipe on analysis as something that is really interesting every time. That actually does take some time to create a very good... And you can obviously use AI to do that and you should do that together with the AI, but then make sure that, okay, when you have that, then you can log that and then you'll have that as a repeated analysis on where you have the data governance and so on in place.

Søren Hammer Pedersen (14:18):

How do you see the area of, you can say best practice or supply chain experience coming into play here? Because one thing is my own company, but the embedded AI, as I see it, also offer you access to other areas of what is actually best practice within this planning discipline. What have others done and things like that. How do you see that come into play?

Benjamin Obling (14:47):

Yeah, and that's really interesting because when you have one unified data model behind the IBP platform, then you can also make the instructions, so the recipes for the repeated analysis, you can do that in a uniform way on top of that data model, that means you can copy paste across different companies these analysis. So you can prepare a very interesting analysis you do in one company, you can copy and prepare that and we can then offer that to other companies so that you can quite easily get a lot more interesting analysis because as again, it does take some time to make a prompt for an interesting analysis because if we are just opening completely and say just analyze completely openly, which is also interesting and that's what we do in the Aura AI in Roima. It burns a lot of tokens. We can do that a bit less frequently and there'll be very interesting insights out of that.

(15:52):

But on the other hand, then having these very targeted analysis so you know, okay, I have been looking at my forecast, I have been comparing, and when I say I have, my agent has been looking at all of these different dimensions, comparing the forecast to last round, et cetera, and it has highlighted these four things. But highlighting these four things has changes, for example, in packaging, a brand going up or down. We also know it has also been looking at all of these other 10 dimensions and comparing it, and there was nothing really to highlight. And that's the repeated analysis that you don't know for sure that the open question in Aura AI, for example, will actually do. That will find new insights, and that's what that is really good for. So it's a combination.

Søren Hammer Pedersen (16:38):

Yes. And speaking of combination, I think let's say that I and listener, we buy the premise that we also need this embedded AI in our supply chain planning, but it's not either or. So I think a lot will also be, okay, when do we utilize the embedded AI? And I guess we still need the ad hoc models and tools and so on. When is that the right choice?

Benjamin Obling (17:10):

Yeah. And you can say the ad hoc tools, that's something we wear when we go to work in the morning because that will be handy for a ton of different tasks. And especially of course the ad hoc analysis. So when you have something completely new, you need some inspiration, et cetera, but of course also the repeated task of sending emails, blah, blah. But that's sort of a different area from the integrated business planning. In the integrated business planning, it could be, for example, if you have an ad hoc footprint analysis you would like to do with some different dimensions that you don't have in your existing data set, that would be an area to use the general. And then whenever you have something that you need to know that these 10, 20 different analysis are conducted on a regular basis, then you need to have it inside your IBP solution as one set because then you know that it has been executed.

(18:09):

I understand all the changes in my forecast. Oh, I don't understand it, but I know that everything that is important has been highlighted to me and surfaced by the AI agent. And there, you need to be not explorative, but have a fixed set of instructions. And that makes a lot of sense to have that inside the tool. And then of course the questions in understanding why does the IBP model do what it does? That could be the forecast. So why is the forecast a hundred for this product next month, for example, understanding that. Or when we have a stock policy model that can be pretty complicated and advanced to say, okay, what are the different service levels? What should be on stock? What are the criteria for that? It's a critical component. All of these different criteria that we've automated in the IBP product, but then you need to be able to understand quite fast.

(19:04):

So why does it get 20 on this specific part or item number? And there, the AI insight is super relevant because then you can just ask, okay, so why does it get 20? And that's because it's actually hit by these two different rules. It has the service levels, but it's overwritten because it's critical, blah, blah, and that's why it gets 20, for example.

Søren Hammer Pedersen (19:26):

Yeah. So actually we use the embedded AI just to improve, once again, our IBP process or S&OP process. So you can call it standard process we have. We make that excellent, improve that, use the AI there, and then we have the ad hoc on the side for the daily or very, very exploratory assignments in the process. I like that distinction. But I guess that also means that if we look at a monthly process in a company, you will still have the same steps, but this will basically change who does what in the sense that here comes the AI comes in and takes some of the analysis along the way. You still have the planner, you still have the, but they work in a different way. Same process, but with the better tools, better analysis to the executives. They can be more effective. They can make better decisions.

Benjamin Obling (20:24):

Yeah. And I think there are really two elements of one is the optimization of the content. So that means a better forecast using AI, for example. That's one thing. We improve the accuracy and that's more the content where it's doing actual work, you could say. There could be the inventory optimization, for example, in that doing actual work, improving that. And then the other part is then surfacing everything that we should look at. So okay, now I've improved the forecast. So they're already there. The amount of alerts I get should be less because now it's actually handling more and it's doing a better job in the forecasting. So that is great. But then we still need to surface where do we need human interaction? Where do we need intervention and action? And in the good old days two years ago, then that's the alerts. And we still have alerts, but it's just you could say with power packs on it.

Søren Hammer Pedersen (21:20):

Yeah, flying now.

Benjamin Obling (21:21):

Yeah, exactly. Because that where it was more rule-based saying, okay, if it's more than 20% increase, blah, blah, blah, then you should highlight. And maybe the 20% is actually perfectly fine. It's no problem, but a plus 4% here is a disaster. So that's really also surfacing, that's the other element, optimizing it and then surfacing what we need to pay attention to. And then of course, being able to adjust that in a very fast way.

Søren Hammer Pedersen (21:47):

Yeah. Yeah, because I think if we go back to my initial question or point that if you are considering, and a lot of companies probably are, just to build this yourself in a standard tool, this is the point because we don't disagree with the fact that you should improve or use the AI capabilities in your process. You should definitely do that. I think what I'm hearing, and I'm also myself thinking is the flag we are raising here is that you could actually do more damage than good in the sense that if we just let loose, let's call it a dark AI or a shadow AI into your thing saying, "Okay, just use Claude for this," then you are not in control anymore and you can utilize a lot of time doing a lot of ad hoc things.

(22:39):

Of course, people will try to set up agents or do something, or you can also try to build something yourself in that, but you actually end up being in less control probably, or at least that's a pitfall you could come into. There could be security issues and you won't get the effect that you actually hope. It will give beautiful answers sometimes and other times you are not in control.

Benjamin Obling (23:05):

No, no. And you need to be able to prove the points, the alerts, whatever the AI engine has surfaced to you, you need to be able to prove that it's actually correct. So if it says that the forecast is dropping in this area, I mean, you shouldn't go ahead and then act a lot on that until you know that, okay, is that actually the case? And that's also why the embedded is super important because then when you get that alert and that surface to you that, okay, this is really something you need to pay attention to, you get that together with the dashboard where you can see that it is dropping in Germany, for example. So you know, okay, it is actually dropping. And if you just, as you say, let loose and just let it dive into the ocean of data that we have where we just established that we can have six different versions of what is a forecast.

(23:52):

And that now it's then comparing some forecast is coming up with some super nice PowerPoints and so on with conclusions and actions and blah, blah, and maybe it's all wrong. And we've all tried and then you type, "Sure, because I think that sounds counterintuitive, and it's... Ah, yeah, you're right. That's a good point." Okay, we're just about to close the factory and check for it. So you need to be able to prove it right away. And that's of course also really where the embedded part makes a lot of sense because you have the tool where you can see, okay, is the forecast dropping? Yes, it is. Okay, perfect. Then let's act on it because it is still hallucinating.

(24:32):

And also you could say one thing is hallucinating, but you could also say, for example, with the six different versions of a forecast, well, it's fair enough. I mean, if you started in the company and you were looking at all of that data set and you didn't get anything else but a buzz telling you, okay, so how does the forecast look next month? What would you do? You would probably pick one of these six and it might be right and might be wrong. Instead, let's tell it exactly which one and what is the analysis that you should do.

Søren Hammer Pedersen (25:01):

Yeah, fair point. I think the last thing I want to just quickly also come across is implications for the organization here. When you meet the planners, supply chain managers, et cetera, how do they view this? Is this a blessing coming in or are you hearing, oh God, here comes another black box in my working life?

Benjamin Obling (25:23):

It is if you get a. I guess we've also all received this 20-page Claude-generated PowerPoint with a lot of-

Søren Hammer Pedersen (25:32):

Looks good.

Benjamin Obling (25:33):

Yes, that looks awesome, and there are a lot of figures on it, but you basically don't know which ones are correct and which ones are not. There is way too much information to digest, et cetera. And you can't verify it because you need to go to 20 different places in five different systems to verify that all of the data is correct. You'll never do that. So you'll just trust it and then you'll go ahead and shut down that factory. That's maybe a bit bold, but whereas on the other hand, if you can do that so that it's tailored, you can verify it right away in your dashboards, then it is a blessing. And I think that is how it's received or it is because you could say you run a lot longer on the gallon, your analysis get way deeper and way, way broader and you really surface these things fast.

(26:26):

So that's one thing, the explorative part, like the Aura for example, but also these repeated tasks. Now you just know that it has been through these 10 different dimensions in my forecast and it has serviced all the things that really matters. Something you would like to do, but you don't have the time, you don't have the data scientists for it.

Søren Hammer Pedersen (26:47):

Yeah. So I hear blessing more than fear.

Benjamin Obling (26:51):

Yes, absolutely. If we use it right, but I would certainly advise against these 20 pages of Claude-generated PowerPoint that no human can digest, but it looks good.

Søren Hammer Pedersen (27:02):

Yeah, exactly.

Benjamin Obling (27:03):

But do we get wiser? That's the question.

Søren Hammer Pedersen (27:05):

And I also think there's a learning curve here because that will help with both building the trust in the AI because I think many people have had a quite steep learning curve with AI within the last, let's say 12 months. So we are all getting more used to seeing AI everywhere and also adapting to the fact that AI can help us actually improve. It's not that scary thing out there anymore.

Benjamin Obling (27:32):

No, no, no. And also you can say another thing that we are working on is you could say from the concept that less is more because when you're surfacing things, we can generate reports of the 20 pages just like Claude can do. But the question is, do you get any wiser? Do you have time to digest all of the information that the AI agent can throw at you? No, you don't. So let's keep it at what is really, really important. Keep it short, bullets, et cetera. And you can have a bullshit bingo in these PowerPoints where you just throw a lot of concepts, a lot of data and so on. And the question is, did we get any bit wiser other than just spending time looking at something that looks awesome and then we actually don't make a better decision? Because that's what it's all about. That correction we do in the forecast, is that a better forecast after we did that, after we looked at Germany that dropped by blah, blah, X percent because it was surfaced? That's great.

Søren Hammer Pedersen (28:34):

Yeah. I think that is an excellent last sentence in this podcast. Did we make better decision? That is really what it's all about. Benjamin, thank you for your input here today. I thought it was very great. I hope also the listeners can really see that we are not saying either or, but we are saying you will need something to improve still your IBP or your S&OP process. You should use it, but you should root it in the right way and avoid the pitfalls that we have described here today. Thank you very much for your input.

Benjamin Obling (29:07):

Thank you.

Søren Hammer Pedersen (29:07):

And also a huge thanks to all you listening in out there. We really appreciate you dialing into our S&OP masterclasses here from Roima. And of course, as always, if you have interesting topics that you want us to have a look at, please feel to reach out. Also, if you want to know more about what Roima does or what we do in Perito IBP, we are here to help. Otherwise than that, hope you all have a nice day out there and see you again next time.