Microsoft Foundry Platform Foundations

0:00Welcome to the content on Azure AI and Microsoft Foundry, and welcome to the AI 103. In this exam and in this particular topic, we are introducing the Azure AI platform specifically with Microsoft Foundry. Foundry is kind of the workstation where all of your AI designs and workloads can run in Azure, whether that's document retrieval and RAG or building your own agent to answer questions that are based on top of your entire document database.

0:31Whatever your internal usage is, you can build AI and host it within Azure, and then leverage it between your other Azure services or make it public facing in part of your own application. Now, in order to get going with this, you just need to understand what your options even are. So what we're focused in on in this set of videos is just introducing concepts, like what Azure AI really is and what Foundry really is. What's the difference between a model and an agent?

1:03What is context and where does it come from? That's what we're focused in on here. So that's what we're doing first. We're diving in to the Azure AI platform, clicking around and just understanding the lay of the land. So without further ado, let's get started with Azure AI and Microsoft Foundry.

0:00We are going to begin our journey by talking about Foundry and Azure AI. Now, here on the screen, I am logged into the good old-fashioned Azure portal. And you can eventually make your way to Foundry by searching for Foundry at the top. And I promise this is going to make a lot more sense why Foundry is so important. And then from here, you can click on the link that takes you to Foundry. Or you can just go to ai.azure.com. You should think about Foundry as a separate portal and a separate developer experience from Azure.

0:36Almost like if you're familiar with Microsoft Fabric. Microsoft Fabric is like a completely separate portal with a separate set of tools from what you can see in Azure. Now, you can deploy Fabric from within Azure, but you don't really access Fabric from within Azure. You go to fabric.microsoft.com or whatever it is. Well, the same thing exists for Foundry, and that's at ai.azure.com. So what is Foundry and why do we care? Well, it's kind of the place to do your AI development work within Azure and Microsoft's ecosystem right now.

1:16If you are a Microsoft shop, if you already have Microsoft 365 and Azure dependencies, and that's where you are currently working or your developers are currently working, then it's likely you want to keep your same users and permissions and sign-in experience all within the same ecosystem. Foundry is the way to do your AI work within that Microsoft ecosystem.

1:46It is your AI workbench. So when people talk about Azure AI, quote-unquote Azure AI, 90% of the time they're talking about Microsoft Foundry. Now, that doesn't mean that it's 100% of the time though. What are the other things? Well, if you actually look over here, you can kind of see there's something called AI Hubs. There's Azure's partnership with OpenAI. So if you really are only in it to use OpenAI services, you can get to OpenAI through your Azure subscription using that.

2:20And then there's AI Search, which is kind of being absorbed by Foundry right now into something called Foundry IQ. So by the time you watch this, AI Search might be gone and it might be replaced with something called Foundry IQ. So just keep that in mind. For the most part, you're going to live in this exam in ai.azure.com in the Foundry portal. Now, if you're watching old content online, this used to be called Azure AI Studio, then it was rebranded into Azure AI Foundry, and now it's Microsoft Foundry.

2:57So what do you come here to do? You come here to build, test, deploy, secure, and monitor AI applications. Custom built AI applications or AI applications that allow you to leverage your own Microsoft subscription to get to the backend models. For instance, you may download an app and that app may allow you to bring your own keys, your own API key, so that instead of accessing the apps models, we want to access our own models and pay it on our own Azure subscription.

3:47That's why they put the API key right here front and center. So this app is running on your local laptop, but it's leveraging your own custom built models or maybe something like a well-known model like Opus 5 or GPT 5.6, but on your own subscription instead of the apps subscription. Or maybe you're building the app and that app leverages these AI models and agents and everything like that. This is what it's all for.

4:18It's for building around this particular framework. And the time that I'm recording this, it's late 2026. And I think most people now understand or have some exposure to AI at some point in their life. In fact, it's estimated that quite literally over 50% of the world right now has some exposure or capacity to use AI. We're talking four-ish billion people are using AI at some point in a given 30-day period.

4:50Why? Well, because it works, because it answers plain English questions. I mean, not 100% perfect, but it's pretty darn close. So imagine now if the tools or the apps that you've worked with for the past decade or more all of a sudden had some intelligent co-pilot built into it that you could ask plain English questions in. Now imagine that you're a business that sells apps. Wouldn't it be nice to give this offering to your customers and say, if you get stuck, just ask a question.

5:23It's there to help. That's the idea. That's why AI-powered applications are different and useful. And that's why pretty much the whole economy is built around it right now. So if you're kind of drawing it up, how does it look right now? You've got some end user out there that is leveraging or using your application, some application that you built. And that application has AI built into it, or a user is asking a plain English question to it.

5:56This is, again, there's more to it than just chatting with an AI agent. There's a lot of different things that you can do, and we're gonna talk about that in this set of videos. But right now, I think the thing that everybody's familiar with to some capacity is AI agents and chat agents. So we're gonna pretend that that's just what we all know, and that's what we're leveraging. So they ask a question, then what happens? This gets sent via the API to Foundry using something like the API key and the project endpoint, or if we're leveraging OpenAI, the OpenAI endpoint.

6:35Then from there, it's probably talking to something like an agent. This is an application that has tools available to it to perform tasks. Underneath the agent is the model. This is the brains of it all. You can kind of think about the model as the brain, and the agent is the body, the thing that does what the brain tells it to do. So if you could imagine a model driving an agent, and the agent uses tools to accomplish a task that can then reply back with an answer, back to the user, like, hey, I queried the database and found this information.

7:20This is probably relevant to you, and here's why. That's what the solution is here. And it's kind of important to understand how these all break apart. Now, in the next video, we're gonna dwell on the difference between an agent and a model.

0:00In this video, we are comparing the difference between models and agents. And we kind of talked a little bit about them. Actually, we talked a decent amount about them in the last video. The model is the brains of your operations. The agent is its body. It's the app that you build, typically, is the agent. And the agent understands what tools or data or context is available to it in order to accomplish whatever task it needs to do.

0:39But the agent doesn't really do anything until the model has thought about it and decided what tools it should use to accomplish this objective or performance task or answer its question. So the question then becomes, okay, well, it sounds like since the model is the brains of the operation, it's the quarterback, it might be the most important part of what we build. And of course, that is, you know, objective. It's up to, you know, beauty's in the eye of the beholder, but it's not a stretch to understand that a model is one of the most important things you decide.

1:15Now, models are broken down into a few different categories, but the one that everybody knows is an LLM, a large language model. And the large language model is ultimately responsible for receiving a question in some language. I'm gonna put English because I'm speaking English and the majority of the people who are listening to this are understanding in English, but of course you could be watching this with subtitles. But the idea is that we say, you ask a model a plain English question and it gives you a plain English response.

1:54So it's all about giving and receiving information in English and returning information in English. This is what we are familiar with working with when we work with chatbots like chat GPT or Claude like so. Now, there's a gajillion different models that are out there today. And we're gonna talk a little bit more about them in just a second. There is also such a thing as a small language model. And this is a language model. It's still working with language and English, but it's hyper specialized in performing very specific niche tasks.

2:33So why have that? Because it feels like, hey, I know, I follow along with everything and GPT 5.6 SOL is perfect or Claude Opus 5 is great, but Fable 5 is even better. Why don't I just use Fable 5 for everything or GPT 5.6 SOL for everything? It all comes down to the almighty dollar because those things are expensive. What makes them expensive? This is where we get into the real talking point about what a model is.

3:06A model is a store of parameters. And what is a parameter? A parameter is usually one parameter. I'm gonna put one param is usually one 16 bit piece of data. And in that piece of data, some of those bits, one of those bits or more of those bits is a weight that basically says how important is this parameter?

3:46So when you think about how a thought might traverse your brain and your synapses and your neural network, it might start here, then go down here, then over here, then over here, then maybe over this way and that way. And then it finally gets to the core memory or thought that you were trying to get to in your brain. The weight in these parameters is what influences that same path, that neural pathway.

4:20So when you start thinking about it, the more parameters you have, the more data and information the model has. And the larger the bits are or the weights are, the more likely it is to take a path through all of those parameters. So when we start looking at it, you might see things like GPT-56 Sol and Opus 5. And they don't publish what their weights are.

4:52When we talk about open source software, we're talking about the code. When we talk about open source models, we talk about the weights. Because we can give something, we can give two models, take up something 27 billion parameters, for example. But what makes them different is the weights that are attached to those 27 billion parameters. That's what influences how it gets to an answer and what path it takes to find that answer and how accurate it is at finding these answers.

5:29So the more parameters you have is important, but also how the weights are tuned is important. Now again, these items right here, the closed source cloud frontier models, like Opus 5 and GPT-56 Sol, we don't know what their weights are or even how many parameters they have. But it is estimated that they have between 2.5 trillion to 5 trillion parameters.

5:59To have that many parameters, you would need something like terabytes of VRAM, aka graphics cards, in order to run one copy of Opus 5 or GPT-56 Sol. Again, these are not hard numbers. These are purely speculative numbers that come from the internet. Nobody knows, I'm not revealing any trade secret. But I'm telling you right here, it costs millions of dollars in infrastructure, let alone the networking backbone to bring all these devices together, let alone the electricity to run this many graphics cards, to run one instance of Opus 5 or something even bigger, a 5 trillion parameter model.

6:55Now compare that to something like an open weight model, like QWIN 3.827b that just released. Yes, we are talking a tiny fraction of parameters compared to something, again, that we're speculating would be Opus 5 or GPT-56. But QWIN 3.827b could potentially fit on an RTX 5090 graphics card. And it fits very comfortably on an RTX 6000 Pro Blackwell graphics card.

7:2827 billion parameters might potentially be enough for your workload, and it doesn't require nearly the investment in order to get that up and running in your environment. What matters then, of course, is the weights. What path does the neural pathway go down in QWIN 3.827b? And what becomes really, really cool and really, really fun with these open weight models is that you can change the weights.

8:02This is through training and fine tuning. You can fork QWIN 3.8 in so many words and retrain the model to shift the weights towards your domain of expertise on your own data. So perhaps one of the ideas that you might have is to create your own model that is trained on your data, derived from one of these open weight models.

8:37But then it gets even deeper. Once you've got this model, you can also train the model about the agent and specifically the agent's tools that it can use. The tools are also things that you can custom build. You can say you're allowed to browse the internet in this way, or you're allowed to run these commands in this way, or you're allowed to use Azure in this way. And you can then use those tools as part of your training so your custom built model can be custom tuned to work for your agent on your own data in your own information.

9:19So I know this is a big, big topic that we're talking about is how do large language models come to be? And the fact of the matter is, is there are such things as small language models that are significantly well smaller. They are usually 1 billion parameters or less, meaning you could see something like an 800 million parameter. These usually translate to something like one to two gigabytes in size, meaning there's even a potential to be able to run these on a laptop that doesn't have a graphics card at all.

9:55These would perform extremely well on something like a MacBook Air that uses unified memory, but just doesn't have a lot of it. Shout out to Cisco for giving us one of these that we could explore right here called Cisco Antares. This is a, from anywhere from 350 million to 1 billion as an open weight model, and it does one thing. It is a code security review item. Code review is looking for patterns, pattern matching. It's not a tremendous breadth of information like network engineering would be.

10:30So it can be condensed down into small models that are hyper-focused on doing one task, which is code security review. And the truth is, is you find most models this day and age on a website called Hugging Face. Hugging Face is where open source, open weight models go to live. And it's what we're looking at right here. Here's Quinn 3.8 Flash, for example. You've got the read me section that tells you all about these models and as well as what kind of benchmark scores you can expect to get from this.

11:04You can also see what the weights are under the files and versions item right here is. Think about this as GitHub for open weight models. So when you become interested in which models you want to use, jump into testing the playground, click on the dropdown and choose some models that you might be interested. Now, Microsoft has a partnership with OpenAI. So you're going to see GPT everywhere. But if you hit browse more models, you'll see everything else too. It'll just take a second to load.

11:36So when I see these things and I see GPT 5.6 Sol, it checks for capacity right here. And then I say open in playground. And now I can start asking questions or just saying things like hello. And we see what kind of responses we get. We got a bad request. Okay. That's because my subscription is still being initialized in the background. Well, trust me, once your subscription is initialized in the Foundry environment, then it will work. So do this, try it out, try and challenge it to think through some certain things to determine which model might work best for you.

0:00So while we're picking up, I want you to see that this did end up working. I did end up getting this chat to work by just letting it sit for a couple days like it suggested we do. So you can see, I said hello right here and GPT chat latest said, hello, how can I help? So now that you understand like it does work, you just have to let it sit and warm your subscription up for a few hours. In my case, I just let it sit for a couple of days. Now it works. So now what we're gonna talk about is how does knowledge, memory and context work within the idea of agents and models.

0:41Like we said, we've got a brain here, right? If you kind of imagine a brain, nope, that's not a brain. I was trying to do a little cerebellum in the stem right here and all the lines and everything. If you imagine a brain here, a very important part of your brain is memory, remembering things and understanding what is going on in a given situation. And memory in AI isn't just like remembering things in the ether. It needs help remembering things in different ways.

1:12There are different ways to remember things. Ultimately, when you're having a conversation with AI, you want it to have context, meaning it needs to be able to understand your specific use case or your specific problem. I mean, yes, you can ask it things like, why is the sky blue? And it just knows these things by having a huge breadth of knowledge, right? And just a huge depth of knowledge too.

1:42But it might not know things like how your business works and does this one specific workflow. It might not understand why you configure your corporate firewall in the specific way that you did. And this is giving it context. Now, in a chat, I can say, I configured the corporate firewall in this specific way. And then go on to describe the firewall, then ask it a question. That would be giving it context.

2:15You, in your prompt, which you are writing right here, is what you're going to prompt it to do, to think about, to analyze. And ultimately that's it. You're just asking plain English questions to a chatbot. But how it gets context can come from so many different angles. Starting with your prompt. If you suspect that a model doesn't have the latest and greatest information or the knowledge about your specific task or problem or situation, you will need to give it detailed information so that it knows how to give you an accurate answer.

2:56It can't just infer things. I mean, they're getting better at guessing and reading between the lines, but it's still just guessing. And it's really just guessing based on a database of parameters and not your specific human use case. So context, context is incredibly important for an AI to understand your specific use case. Now in Foundry, we look at these as kind of three different layers of where context could come from.

3:31Tools could be one of the places that they come from. You can create a tool to go research something. For instance, if I usually tell these models, like, hey, go research the web and find the best Italian restaurant in Louisiana in July, 2026, it's not gonna know that just off memory. It will need a tool to go find this information and gather context about Italian restaurants in Louisiana in July, 2026.

4:07A tool can help it gather context. The next thing is a context database itself, specific knowledge. Now this gets into a bigger conversation about things like RAG pipelines, retrieval augmentation, or excuse me, retrieval augmented generation. Kind of says it in the name, but you kind of do also have to see it in action to really get it. It retrieves information and augments it and generates a response based on that information.

4:42You should think about this as your corporate database. Whatever that database contains, whether it's sales figures or processes, a diagram of how data moves through the corporation, what the corporation is currently building internally, your corporate firewall policies, you can store this in a database and provide it as a context to your model.

5:13And again, lots of times we see this in the form of RAG. Sometimes you might see this also, oh, excuse me, I'm running in memory right now. Sometimes you might also see this as MCP. Now MCP is an entirely different thing called model context protocol. And it kind of lives in the name too. The model, hey, that's what I'm talking to. The context information that I wanted to have, a protocol that has been widely established and streamlined and standardized so that models know how to talk to this tool to gather context.

5:51So like we were saying here, there could be something called a RAG pipeline, which is all about querying your corporate database. We could give this to a model using MCP. The model sees this as a tool and it knows I can request information from this tool. The MCP server runs a RAG pipeline and returns the context back. Again, you should think about this as your internal corporate knowledge that there is absolutely no way that a general purpose model like GPT knows anything about.

6:31But I can use MCP and RAG pipelines to present this corporate database and allow the model to figure it out with my own corporate data. We're providing these specific to me information to the model. Then there's memories. Memories are what models can write for themselves. Have you seen the movie Memento? If you haven't, it's a Christopher Nolan movie. And the idea here is that he had some sort of accident and he can only, his brain resets every night when he sleeps.

7:06He has no memory of what happened in the last 24 hours. But he's trying to solve this big problem. I think his wife got kidnapped or something. And I forgot, it's been a long time since I've seen it. But what he does is as he progresses through trying to find his wife, knowing that his brain is going to reset the next time he goes to sleep and he's not gonna remember what he did yesterday, he tattoos what he did on this day on his body so that when he wakes up, he sees, oh, this is what I did yesterday.

7:39This is exactly what memories are in the AI space. They leave notes for themselves of what they worked on and how they solved their own problems and what happened in the last chat session that we had when we were working through problems. They leave memories for themselves. That way, when we start a new session and I ask like, hey, let's pick back up on where we were, it can recall memories that said, this is what we were working on, this is where we were, and these are the tools that we used to get there.

8:15And those tools might be an MCP server or some other tool. And it'll say, it'll even say things like, if you need to recall this data, run this query on this MCP server to get this data back and it'll refresh. It literally leaves itself notes with memories. So now we understand context. What are tools? What are prompts? Where does knowledge and information and history of our chat sessions come from?

0:00Now what we're focusing in on is where context and search comes from. And it's going to be a lot of understanding. There's a lot of moving parts and quite frankly, the platform itself is changing. The features, the options, the names, the tools are in flight. So there's an okay chance that the buttons that you click on today will not be there, you know, a year from now when you might be watching this content. But the concepts are the same. So what I want you to first understand is as of today, Azure makes their platform for managing all search operations that your AI will use. It is known as Azure AI Search.

0:44And it tells you something like Azure AI Search is Foundry IQ. Like I said, these are kind of merging moving products. Technically, Foundry IQ is built on top of Azure AI Search, because Azure AI Search includes setting up all the databases themselves and the files and the data stores that can be accessed. And then Azure AI Search also importantly includes something called indexing. Indexing is where you don't just upload files and call it a day.

1:22It allows your Azure environment to mark which pieces of data, it's kind of like tagging data and saying these relevant items are all here. That's what the indexing portion does. It can say for this particular topic, look in these files or for that particular topic, look in the vector database or a hybrid of both. But like I say, also with Azure AI Search is setting up the data stores and documents themselves. Now, again, we're not configuring anything in this set of videos. We're just understanding. But you can see there's a data set section, a file section, a data generation section, which is in preview and stored completions are deprecated. They might be gone by the time you're watching this. So the idea with data sets, files and more is that we are building a database of context for these agents to use.

2:20Now, especially with data sets or even files, but especially with data sets, it is common to upload files that get converted into something called a vector database. Let's say you have a 50 page employee handbook and you may want to use an A.I. agent to ask questions about the employee handbook, pull out relevant information for specific topics of the employee handbook, or be able to refer to it for any other thing. And you want the A.I. to have access to it.

2:52A 50 page employee handbook is not ideal to have an A.I. searching the entire 50 pages for a very specific clause or subset. Like an acceptable use Internet policy that's in basically every employee handbook ever. So the way this works is vector databases. I'm going to put V.E.C.T.O.R. Vector databases were built to help A.I. agents and A.I. assistants get to the specific clauses that you're talking about. What you can think about it is, and this is the highest, easiest summary that I can give you, is that an A.I. agent will ingest these 50 pages and then dice them up into chunks. You can sort of think about it as taking sections out of this employee handbook and breaking them up into chunks. Then mathematically identifying which sections of these chunks are relevant to each other. So if you're extrapolating this, just kind of close your eyes and brainstorm here. An A.I. now needs to retrieve data out of your vector database.

4:07It can use mathematics to find the relevant related sections that are relevant to the prompt it's given. So it can quickly get to the part, the chunk of the employee handbook that answers the question. They can then pull it back out and do something with it. This is the R in RAG, the retrieval in RAG. The augmentation and generation part of it are what comes after. Based on the information that is retrieved from the vector database, the model can then augment its reply response decision as it generates this response and sends it back to the customer.

4:55A single RAG pipeline for something like the employee handbook is particularly useful if you have an A.I. agent that only handles HR issues. You can have a specific scoped A.I. agent that has a scoped database back in. Where FoundryIQ comes in is what happens if you need a CEO level agent operating for you that knows everything in your environment and every department and every access to every database or can retrieve data from all of these disparate data sources. This is what FoundryIQ does. FoundryIQ is your comprehensive A.I. search tool for all the datasets, multiple datasets that you want it to have access to. Like I say, it is built on top of Azure A.I. search.

5:53Azure A.I. search is the broad spectrum of the entire A.I. search functionality, whether that's setting up the back-end datasets, indexing the data, building a RAG pipeline that an agent will use. That's Azure A.I. search. FoundryIQ is a, I don't want to say specialized because it's not, it's actually just a super powerful A.I. search for bigger, more complicated search tasks. So now we have an idea of where we're going when it comes to setting up context and RAG where a vector database comes into play and how FoundryIQ is slightly different from Azure A.I. search.

0:00Now we're going to focus in on Foundry Agents. We've already talked a little bit about the difference between a model and an agent, but it's worth reinforcing this particular topic. The model, again, is the brains of the operation. You can ask it questions, it can answer. But it can't do anything because it needs tools. It needs context and knowledge. It needs instructions on how to do what it's supposed to do. And then it needs state. It needs to understand where things currently are. And then if I do this, what change will it make? These things could be simple.

0:38It could be as simple as resetting a customer's password. The difference between a model again would be, here's how you can reset your own password. Whereas an agent is, I just reset your password. Same thing could go refunding a customer. Here's how you can log into the billing system and refund a customer. Versus I refunded the customer. That's the difference between a model and an agent. Now when we click on the start building button right here under build an agent, you got to give this agent a name. I'll call this my demo agent. In fact, you know what? I'm actually going to show you something a little bit different in just a second.

1:14And I'll click create. And it's going to scaffold something here after a second or two. Do, do, do, do, do, do, do. I'll pause it. Oh, nope. Here it comes. Here it comes. All right. So now you've got a playground agent. We can give our agent instructions. We can give our agents tools. We can give our agents knowledge and memories. We can also give our agents guardrails, the do not do this type of things. Do not violate the terms of service of this contract.

1:54And steps to fail fast. How to iterate as it goes through a loop trying to solve whatever problems you have. This is kind of a very nice way of deciding how we can build an agent. Now this is a web based, you know, GUI based driven system. You can absolutely build an agent from code too and just ship the code in and it works. And when you're done, you can save and publish this agent for use, whether internal or public. Now let me show you some other ways that you can get the brainstorm going. I'll hit back right here under the agents section. And look at this. There are actually templates where you can kind of scaffold an agent to get started. I highly recommend clicking on this to start brainstorming the types of things that you can do. I mean, really these agents, if they have access to data and tools, they can really do anything that you explain or that you can give it and explain that they can really do anything you want it to do.

2:57You just explain how it's supposed to do it and it goes and does it. You can even build the tools or the MCP server for it to use. So look at things like release notes generator, knowledge base Q&A assistant, web fact checker, document or spec sheet summarizer, a meeting notes summarizer, a blog post drafter. Although look, when it comes to blog posts, like if you're using AI to, this is just personal, this is just my personal thing.

3:31If you're using AI to generate blog posts, then it's really only useful for other AI agents who are crawling your web to learn from whatever blog post you're posting. It's not going to be useful for humans. I mean, it will, humans can read it and they can absorb the information, but it's going to be blatantly obvious that it's for humans. If you really want humans to be reading your blog post, then humans need to write it. That's what I'm trying to say here. Again, no shade on using blog posts to write certain blog posts. I've done it before because I may have a very technical blog post that I want another AI agent like Grok or OpenAI or Anthropic to crawl, consume and add to their training database that there are reasons to have AI draft your blog posts. Scroll on down some more and there's some other really useful ones. Data quality auditor. I mean, that's great. Anytime you give me a troubleshooter, I'm going to use it.

4:30If you can find the root cause faster than me, which is almost always because these things are really fast. It's not that I'm unskilled. It's just that these things are really fast and I would like for something else to find the root cause quick and report to me what happened. These are the types of things that we can work on. So when I look at something like repo audit and report builder, I can give this a click and it shows you what you're going to need to create this agent, what tools it uses and how to use this agent. What is it supposed to do?

5:02And then all you have to do right here is click create. Ultimately, the tools are a GitHub connector, a code interpreter and a file search. And this was created on June 15th of 2026. You can see how new or old it is. So this is introducing what Foundry agent builder is going to be like.

0:00Let's recap what we've learned about Azure AI and Foundry fundamentals. So to kick things off, a company needs to classify millions of short customer comments while minimizing cost and latency. So if we're thinking about short customer comments, we want to be able to mark them or classify them as a positive comment or a negative comment or a feedback comment or a suggestion comment. That's the idea here. So which model type is the best starting point?

0:31Hmm. So we need to classify millions of short customer comments while minimizing cost and latency. Is it multimodal, small language model, large language model, or document search? Well, the answer, and this isn't super obvious. If you struggle with this one, don't worry. The answer here is a small language model. The first key thing is they want to minimize cost and latency. Well, being a small model means you can run it on significantly smaller hardware. You don't have as many parameters, so you don't have to load a large, humongous tens or hundreds of billions or trillions of parameters into graphics cards.

1:13No, we could be talking 800 million or something like that. So we can run it on a single RTX 5090. And we have a specifically scoped task to do one thing, classify millions of short customer comments. We're just classifying comments. So a small language model here is the thing that I would start with first and make sure it is tuned to doing the exact task that we want. A company wants to build a custom vector and hybrid index search over thousands of internal documents.

1:47Now, we did talk about vector searches and how it's all about grouping chunks of related information together. And an AI agent can do a good job of a vector search. But we didn't really talk about hybrid searches. And it's not relevant to this question, but I'll tell you what a hybrid search is. It combines a vector search with the keywords that the user provided in their prompt. So if they're saying like, hey, I need you to search for this specific thing, it's gonna grab the keywords and then perform the vector search with those keywords.

2:22That is the hybrid, the combination of a vector and a keyword search through the index. So they're building a custom vector and hybrid search index over thousands of internal documents. Which service should it use? Azure AI Speech, Azure AI Search, Foundry Agent Service, or Foundry Memory? Well, the answer here is Azure AI Search. That is the building place for all of your search needs, especially when we hear the word indexing or vector database.

2:58An IT support agent must create a ServiceNow incident after it determines that a user's problem requires escalation. What capability does the agent need? So we've got an agent, it's gotta do something in the ServiceNow after it determines that this needs to be escalated to either another agent who is specialized in that work or a human for review. So what tool does the capability need? Does it need more memory?

3:28Does it need a deeper knowledge base? Does it need an embedding model? Or does it need the ServiceNow API tool? What it is right there, the ServiceNow API tool is what it needs. That way it has access to interacting specifically with the ServiceNow environment to do the task it requires. So this has been reinforcing what we've learned about Microsoft Foundry in the Azure AI ecosystem.