Explain How Data Leaks Through AI Apps

0:00Hello and welcome. My name is Jonathan Barrios and I'm excited that you're here. In this course, Data Leakage Prevention with LLMs, we're going to explore how to build, deploy, secure, and cover data governance and evaluations that use LLMs and need to understand one very practical question. How do we stop sensitive data from leaking through an AI system? But before we get into it, a little bit about me. Again, my name is Jonathan Barrios and I teach data science, machine learning, and AI engineering here at CBT Nuggets. And I'm also the founder of Barrios AI and co-founder of Gitari. And this is where I build production agentic systems.

0:48I'm also a veteran trainer. I've taught at top online platforms like Treehouse, Thinkful, and Chegg. And I'm now happy to be here with CBT Nuggets. You can also find me at LinkedIn. So at Jonathan-Barrios-AI or on my website, JonathanBarrios.com, where I write different articles about machine learning, AI, and also music. And then finally here, AI underscore data underscore science. And you might be wondering, well, why are you on X?

1:19And the simple answer is this. That is where all of the leading minds of AI, machine learning, deep learning are. And they're posting amazing things all of the time. And so I try to curate that information. I'm not doing it as much as I used to, but if you really want to understand what's happening and to stay current with AI, this is the easiest place to do it. What I do suggest, and this is my own personal opinion, I think social media is a waste of time and it is going to try to steal your time. So what you want to do is just follow the people that are talking about AI and ignore everything else if you can. If you want to follow me and see who I'm following, that might help you sort of curate a list. Now you might be asking, well, who is this course for and what are we going to cover? So I'd like to just take a couple minutes to talk about that. So the question that we're answering, let me write that up here.

2:20How do we stop sensitive data from leaking through an AI system? So primarily this is going to be for AI engineers, but that also includes more often than not software engineers, machine learning engineers, also data scientists and security engineers. And those of you in the security field, which means that I would say summarize the security part as maybe people moving into AI security and anything that has to do with responsible AI roles or responsibility.

2:56But that can also include application developers, technical architects, platform engineers, and anyone who is working with data governance. And it also works well for experienced IT or cybersecurity professionals who understand traditional systems, but need to understand where these LLM applications are creating new data boundaries. So all of these different roles would take this course because an LLM application is much more than a prompt and a model.

3:29Sensitive information can move through prompts, uploaded files, conversation history, RAG system, vector stores, agents, tools, model outputs, logs, traces, and monitoring systems. And that is a lot. So we're going to cover all of these boundaries in this course so that you have a good understanding of first of all, how does AI work generally speaking? So you should have a foundational understanding, but the way that I'm going to frame this is how I like to do this at the beginning of these courses is to talk about software 1.0, 2.0, and 3.0. Because software has evolved. And if you don't know that, it's going to be tricky to take this course. But if you do know that in some intuitive way, I at least like to make this very explicit. And Andrei Karpathy's way of explaining this is software 1.0, 2.0, and 3.0, which makes it really easy to build mental models as we progress through this course. So I keep saying sensitive data. So here's the question that we're going to answer, but let's talk about the sensitive data. Also, I just noticed that I didn't update this. If that's confusing, let me fix that. All right, much better. That was actually

4:46the last course, which is a great companion course. And I will get to that in just a second. But first, let's talk about the sensitive data. So we are definitely going to talk about security principles, but we're really going to focus on progressively building controls based on these security principles. You might hear this word security by design, and that really does apply to AI engineering, especially in the context of this course. So what are we talking about as far as the data? So first, we need to be able to detect sensitive data. And then we need to classify the sensitivity that's more specific. And then once you do that, so what is it? Well, now you need to first understand your organization and the different policies. So you're working with data governance here for your organization. And once you classify that sensitivity, then you may redact or block the model, right? You want to do that before the model, I should say.

5:51Let me write that here, before the model. And that is exactly the controls that we're going to engineer so that we can really implement these security principles. But that also includes, let me just say here, this would be maybe file uploads. And whenever we're using retrieval augmented generation, that means proprietary information inside of your organization, that also counts. So RAG, retrieval, that could be sensitive information. And then we also have to focus on authorization and protect multi-turn conversations, handle detection from prompt injections. And that also includes the agents. So we have to protect agent tools and tool results.

6:38And we have to filter out things like PII. I'll write that here, personally identifiable information. We're also needing to protect logs and observability and combine the controls into one architecture. But also we need to respond when a leak actually happens. And we're going to design and test a complete protected LLM system together. All right. So by now you should have a pretty good idea of what we're going to do in this course. And we're going to do this skill by skill. And we're going to iteratively build more and more controls as we identify the different boundaries for this sensitive data. And you might be asking, well, why are we doing all of this? Well, here is the fundamental issue. AI applications move data across many more boundaries than people initially realized. And so traditional application security might focus on databases, APIs, authentication systems, and maybe just the code itself. But an LLM system can introduce additional flows, such as users, prompts, applications, RAG, the model itself, tools, other models that are associated with tools like agents, outputs, and logs. And each one of these can become a data leakage boundary.

7:59So simply saying the model is secure doesn't really solve that problem. The application still decides what gets retrieved. And that's why we added RAG here for retrieval. And also what gets sent to the model. That's why we added file uploads. And there's other opportunities to explore there. And what tools can access for the agents. And what gets returned to the user. And what gets recorded. That is really the central lesson of this entire course. So let me summarize it here.

8:32I'm just going to add a star so you remember this. We're going to explore how to design LLM applications so sensitive data is controlled at every boundary from the user input. Let me write that here. All the way to retrieval tools, model output ability, and an instant response. So you don't need to secure an LLM by protecting only the model. You secure the entire path that data takes through the AI system. How can data leak through an AI application? That's what we're going to answer. And we're going to start by doing that in the first skill where we're going to explain how data leaks through AI applications. All right. So here we are in Adept. I'm going to make this a little bit larger. And we go down here to author and scroll down. So you see my name right here and click on that. All right. So now we have a bunch of courses that I've authored.

9:37And so here is the Cloud Certified Architect Foundations course, which is a great course to take after this one. But I would also suggest the Introduction to Machine Learning and AI Engineering. This is really the best of both worlds. I highly recommend this course if you don't have a background in machine learning and if you're still fuzzy on AI engineering. And this is the course that I just finished, Secure AI Adoption and Deployment, which really gets into the adoption part and the deployment part with all the controls in between.

10:11So again, if you don't have a foundation on adopting and deploying applications, I would suggest this course, maybe this course, and then the course that we're taking now. But if you don't have a machine learning background and you're fuzzy on AI engineering, these are the more foundational principles. I definitely require or recommend this course. All right. On that note, I will see you in the very first video of this skill. See you there.

0:00Welcome back. In this first video, we're going to start pretty much how we're going to start almost all of the skills in this course. And that's why I want to cover this together. So we're going to do a couple of things. So usually I start up with a description explaining what the course is going to cover. And I like to start with a question that we're going to answer for the whole skill. And then we're going to look at this. This is the what we're going to cover section, which we're going to get into in just a second. And then we'll go to Google Colab.

0:32And this is what we're going to use for our development environment. So we're not going to get too deep into this and you don't need to execute this code, but I will show you how you can follow along if that is something that you'd like to do. There are two notebooks here. So we have a start notebook, which is going to be usually blank or just have some very basic boilerplate at the very beginning. And then I will start with the same boilerplate here and code it out and do that throughout the course. If you do want to code along, that's why we have a start completed notebook that allows you to do that so that you can store it into the long-term memory.

1:11Because repetition is what really does that. So doing it five times. So hearing me talk about it, writing it, looking at it, executing it, looking at the output. Once it gets to about five times, it takes to get it into long-term memory. And I know that because I read a lot of studies on neuroscience and a lot of studies and books on education, and they mix these two domains and it's called neuroeducation. And that is pretty much kind of the consensus of how to store things in long-term memory.

1:49There's all kinds of educational approaches to help to do that. And this is one of them, but I am really always thinking about this. So that's one of the reasons that I have created all these start notebooks and completed notebooks. But if you don't want to do that, that is totally fine. We all learn different ways. So if you want to just follow along, then you can just open the completed notebook at the very end of the skill. You'll see it at the bottom there. So that's for you to do that, whatever is best for you. And we are going to require an API if you do want to follow along.

2:22So if you go over here to secrets and Colab, you can have these different key value pairs. And this is my anthropic key right here. And then the value is right here. And all you do is turn it on and then gives access and then you add a little code. So that's what we're going to be doing in this whole course. But this is the first time we're going to do that. And so you might need to go over here and go to triclod. Let's go over there together now. And you don't really need this part. You really need the API here.

2:54So that's all you need for this course. If you have everything else, that's totally fine. It's helpful to talk to different models. It doesn't have to be Claude. It can be any model. But the API here that I'm going to be using is this one here. You can click on start building, log in, and maybe add five or $10 to this. You might have to add 20. I don't remember exactly how much you have to add to get this API working, but it's not going to use a lot of API calls in this course.

3:26This is just for educational purposes. But on top of that, I'm building something called a fixture. And so that means that you don't even need the API. So the fixture is going to run and create some mock data, like as if though you did call the model and then you got an answer back. So I'm going to give you that option too. So you don't really need an API key. I didn't want to make that a requirement. It's only a requirement if you want to follow along in the start notebooks and then add your own key and then kind of follow along like that.

4:01It's not going to work unless you have an API key. So that's the only reason that I'm sort of suggesting that. But you can also decline and just say live API equals false in the boilerplate that I'm going to give you. And then you don't need any of that. All right. So let's get started. All right. So here is the boilerplate code. I didn't want to write it all out in front of you, so I just wrote it out now and then now I'm going to explain it. So first of all, we are pinning this version and that's because these models are changing very, very quickly.

4:37And so what happened is that they are deprecating temperature and older models, but it should still work with Claude 4.6, but the SDK doesn't make that possible. So the only way we can get Claude 4.6 to use temperature, and if you don't know what temperature is, don't worry about it. It's just something that we're going to get into later that really changes how, let's say, stable the output is going to be. If you want a more creative or unusual next token prediction, you would change the temperature, but you can't do that with newer models.

5:14So it's changed, but the SDK, they updated it. So it doesn't matter. You can't use it for Claude 4.6 unless you pin it. So that's why we're doing that. So let's go ahead and run this together. While this is running, you do need a Google account. And once you do have that, you can run these. But if you don't have a Google account, I believe you can also execute these, but you can't save any changes. So that's something to consider. And you may not have access to Gemini here, which is nice that you can quickly ask questions and to help you with any debugging as you progress with this course.

5:49So here is the fixture. So we're going to leave this to false, but actually I'm going to say true so that any opportunity to make an API call, I'm going to make sure that this is selected and then it's going to call that model. So make an API call. Otherwise it's going to use fixture code. So like mock data. So that's just another way to make sure that you don't need an API key. All right. So we have that SDK installed. And then here for the user data, the way that we're working with this is to do this line right here.

6:25So we're using user data. So user data dot get, and then the secret that's right here. And I'm just using this variable API key. And then this is just check to see if we have that. If not, it's going to raise a value error. And then if we do, then it's going to print something. I'll show you exactly what that is. Let me close this. And you see that we have live API with fixture fallback. Now if I make this false, it's just going to say fixture, right? So I'm going to go back to true.

6:57So I'm going to fall back with the fixture, but I am going to execute live API calls whenever possible. And in this skill, there's one at the very end. All right. So what are we going to cover in this skill? There's so much to talk about in the first skill because we're kicking off the course, but then in every subsequent skill, we're going to start right here and it'll make a lot of sense. So first we're going to explain the difference between data movement and exposure and leakage. And then we're going to talk about software and how it evolved through the lens of software 1.0, 2.0, and 3.0.

7:33So this is an easy way to understand traditional software, machine learning and deep learning, and now generative AI. We'll get into that. And then we're going to explain why the LLM is just one component. So an LLM is really just one component when we're considering the entirety of all of the leak surfaces. And then we're going to identify all of those leakage surfaces. So identify surfaces. And the way that we're going to do that is that we're going to follow one synthetic sensitive value.

8:06We're going to trace sensitive data through multiple application components. And then we're going to explain why a clean final answer doesn't prove that the entire system handled the data safely. So this is going to be about false sense of security. And then we're going to explore a simple data flow. And then we have a challenge. And so the idea here in this challenge is really to make sure that you have a good understanding of all the concepts in this skill, because we build iteratively.

8:42That means that if you have any missing gaps in this skill, it's going to make it harder in the next skill. And as a result, it's less fun and you're going to not learn as well. So we are designing this so that we have these small learning curves like this. And if you are successful in each skill, and the challenge is how we measure that, then you're going to have a good time and you're going to learn. But here's where the trouble starts. Let's say you have a knowledge gap and then this happens and then it happens again.

9:15These are the points where it becomes a much larger learning gap. And that's when learning suffers and also it's not as fun. So there's a flow state to learning. If you're excited and you're having a good time, that's optimal. That's how you learn, because this is actually very exciting. So for me, learning this and teaching this and keeping up to date with everything is exciting. So it's easy for me to put this into long term memory. And the way that I'm ensuring that this is easy to learn is by really focusing on these small iterative learning curves.

9:50All right. And that's what the challenge is for. So I will see you in this first video here where we talk about the data movement and the different ways of or not the different ways, the differences between data movement, exposure and leakage. See you there.

0:01Welcome back. In this video, we're going to kick it off with software 1.0, 2.0, and 3.0, because that is really the foundation for everything that follows, even before data leakage. And the reason why is that it's going to allow us to build a mental model that explains why modern AI systems create unusual data paths. And again, Andrej Karpathy is the one who came up with this. It's a brilliant way to talk about it. Andrej Karpathy is like one of the founders of OpenAI and responsible for pretty much the self-driving at Tesla and is now at Anthropic leading the charge there.

0:37He's one of the best minds in the field and really a great way to explain this. So how does this work? So we have software 1.0 here, and then let's draw software 2.0 over here, and then software 3.0 over here. And I do go into much more depth in the Secure AI Adoption and Deployment course, but this is really just all we need for this course. So software 1.0, we write the rules.

1:07And this is really just traditional programming, validation, permissions, allow, or block. And you can think of it as this. So we have an if statement here, and there's some kind of condition here. So I'm just going to put a condition like a question mark, and then this can go to yes, and this can go to no. And so yes would be allow, and no would be block. But the more AI capability that we have, the more places that data will move.

1:40So we're going to talk about data at rest and data in motion to describe the state of the data, but movement, exposure, and leakage describe what is happening to the data from a security perspective. Okay, so you might think about, oh yeah, we're talking about data at rest and in motion, sort of. So let me go to software 2.0 now. Okay, so here's where software evolves. So now the models, they learn from the data. They don't need us to do any else, nothing.

2:13All they need are a bunch of examples or different kinds of data that we feed into the model. So I'm going to draw a big brain here. Let's call this, it's an artificial neural network because we're talking about deep learning. So I'm just going to draw a brain. So this is an artificial brain. So the model learns from the data using these examples or the data that flows into the model, and it can be statistical, also known as statistical learning, and machine learning or deep learning. So software 2.0 learns from the data.

2:44There's a history there. But just to make it really simple, it can learn patterns from statistics or it can learn patterns using a neural network, and that's what deep learning is about. It's software modeled after the brain. They use mathematical representations of neurons and then create a whole network of those, and so that's an artificial neural network. And when you add many layers, then it becomes deep, and that's why it's called deep learning. So that's a very simplistic way to think about it, but very useful for our example in this course.

3:19And then when you are done with these, this is trained data, and then this is actual training the model, then you get prediction. So this is inference. So this is the output, right? So this is a prediction. So you can say spam or not spam, or you can say that's an apple or that's a hamburger or that's a hot dog or whatever. So now we're going into software 3.0. So what is this? Well, software 1.0, this is traditional programming.

3:49Software 2.0, the models are learning from the data themselves. So we don't need to write the rules, but what's this? So software 3.0, this is where language becomes part of the inference, right? So let's call this generative AI for now, and let's call this machine learning and deep learning. And this is traditional programming. Okay, so generative AI. So these are prompts, documents, retrieved texts, tool results, conversation, history, all that good stuff.

4:21And it can include prompts, logs, files, rag, outputs, tools, and agents. So here, what's interesting is that we're using plain language to create an output. So we're really programming with plain language. We don't have to write the rules like in software 1.0. We don't have to build a model to do anything. In fact, we use deep learning to build generative AI models. So you use deep learning to, deep learning is a combination of 1.0 and 2.0.

4:58But 3.0 is really just built with 2.0. And in order to build 2.0, you need 1.0. So they're all kind of interconnected. But the simplest way to think of this is that generative AI receives a prompt in plain language and produces an output. So let's say we're trying to perform a classification, right? So we give it some text and we say, is this spam or is it not spam? And it can make that call. Now with machine learning, you'd have to build a model and give it a ton of training data, train that model, and then you could ask it, is this spam or not spam?

5:32So it's a lot of work. With generative AI and these frontier models, you just say, is this spam or is it not spam? And it'll tell you. But it hallucinates. That's the problem. So software 2.0, that's where the hallucination comes from. And that's because we're using all of this training data to predict the most plausible next thing or whatever it is, a token or pixel or whatever it's predicting. And by doing that, it will create the most plausible answer regardless of whether it's right or wrong.

6:05And that carries right into 3.0. And that's why we're in this course. Basically, if that algorithm didn't create these hallucinations, we would not have this problem. So, yeah, that's what we're doing here. Taking care of the hallucination problem inside of software 2.0 and 3.0. Software 1.0 does not have this problem. And when we think of data movement, that doesn't mean that it is automatically some kind of leakage, right? So let's talk about this.

6:36So data at rest and in motion describes the state of data. Movement, exposure, and leakage describe what is happening to the data from a security perspective. All right, so the leakage happens when predicted information reaches an unauthorized user, model context, tool, log, external service, or other destination. So at rest is like in the database. In motion is when it's retrieved.

7:06In use is when it's placed into that model context. But none of those automatically mean leakage. So it does not mean leakage. All right, so what do we mean by that? So let's talk about data movement. All right, so here data movement means that data travels between components. All right, so we have some kind of a database, or it could be a vector store. And then it goes into some kind of document. Then it goes into a cloud or a model, right? Now, that's just movement.

7:37It's traveling between components. But data exposure, that's when data is present somewhere it can be observed. So we have some data here. But we have this creepy eye over here looking at it, all right? Meaning that data can be observed. So that's data exposure. And then we usually build some kind of control to mitigate that problem. And that is going to be maybe, let's say, unauthorized disclosure like that.

8:10So unauthorized disclosure, that's where we detected data reaches the wrong destination. So we can illustrate that here with a document that has a padlock on it here, all right? So it somehow meets this guy in a hoodie, bad. All right, so unauthorized disclosure, again, means protected data reaches the wrong destination. In this case, this like happy hacker. So the first thing we want to do is map where the data exists, then decide which movements are authorized.

8:47So this is the first step. And then we can talk about things like the prompt boundary, meaning that once the application accepts a prompt, the data has entered a request path. So let's talk about that using Google Colab. All right, so here we have our live API with fixture fallback. It's up to you if you want to make this true or false. So it doesn't matter if you don't have an API key, this must be false. All right, so let's go to this first series of code cells here.

9:17So what I'm doing here is I'm using fake sensitive data, so a fake synthetic secret key, right? And we're going to be able to instantly recognize it because it says demo key, blah, blah, blah, do not use, right? So this is unlocking nothing. It's just internal value, meaning a known test value that's deliberately placed into the system so that we can trace where it appears. And this gives us a safe way to study leakage behavior without using a real password key, customer record, or some kind of, I don't know, confidential document.

9:51So let's run that. So there it is. That's our sensitive value, our fake demo key. And we're storing it as synthetic secret here, okay? So that's what we're going to be tracing. So this one variable gives us this controlled test fixture. All right, and so here for user prompt, this is just some kind of ordinary sentence, right? Help me summarize this deployment note. The temporary API key, never do that, but that's what's in that prompt.

10:23So then we're going to have this request, which packages that prompt with that fake user ID here. So again, this is just some fake user ID. So what we're going to get here is the request that contains a secret inside of that prompt field. So notice that we still have not called an LLM. That's on purpose. The sensitive value already exists in application memory. And if the application logs, stores, copies, or forwards this request, the secret can continue moving even before inference occurs.

10:56Super important. All right, so now we're going to talk about another surface that's important, which is the conversation history, and we'll do that in the next video. See you there.

0:00Welcome back. So now let's look at this conversation history. So let's think of it as three different turns. So here the conversation history carries all data forward. So here is the first one, right here. So the user is basically giving content that has that demo key in it because it's the user prompt, right? You can see it right here, all right? And then the assistant gets it, that's turn two, and it gives a normal response. I can help summarize the deployment node. And then turn three, right here, this is where the user asks a new question, but this time it's clean. So what is the main action item, right? That's benign. But from this one here, the next model request, the entire conversation history is now sent together. So even here, if this is a clean sort of prompt with no secret, the secret from turn one still travels forward. And that's how this works. So the newest message can be clean while the conversation state or the history still contains sensitive data. And then here for this function, what we're doing is serializing any Python object into JSON text. And then we're checking whether the exact sentinel value is

1:20present anywhere inside it. This is not production secret scanner code, it just only works because we already know what the exact test value is. So this is just for tracing example, right? So let's do this. Okay, so secret appears in newest prompt false, secret appears somewhere in the conversation history true. And that's because the newest message here is clean. And that's why we get a false here. But the second check is true, because the older turn still contains a secret that conversation history is always going to be concatenated, put together, because otherwise the model won't have the context. And that's how it has context. And if you know, the term context window, it gets bigger and bigger and bigger. So that's why it's still in there. Alright, so now let's go to the same thing with an uploaded file. So here we have sensitive data that does not have to be typed into a chat box. So modern AI applications that can accept PDFs, CSVs, configuration files, source code, I don't know what images, internal documents. So from a security perspective, an upload is just another ingress path. Ingress simply means a way for data to enter the system.

2:42And the application might extract the text store the file converted into like, let's say model input, or maybe an index for retrieval later. So the question is, what data did we accept? And where will it go next? So here for the upload file, this is a simple dictionary representing a file after text extraction contains the file name and the extracted content. Let's go ahead and run this. Alright, and you can see right there. So the output shows the extracted file content and Sentinel key. Again, presence does not automatically mean a breach, it just means that the file processing layer now handles sensitive data and must eventually be included in security by design. Alright, now we have rag. This is another one. So retrieval augmented generation gives them all information from documents it was not trained on, right. And if you know what don't know what rag is, let's say we have a list of trainers at CBT nuggets, but the foundation model doesn't know it does know, but let's say it doesn't. And as a result, we say we need to have that information in the chatbot. So let's create our own chatbot and add some documents for the trainers, so that the model knows who they are. That's pretty much rag,

4:00you're adding some extra content that can be retrieved, and then retrieve augmented and then generated in the output that the user gets. So that's a rough explanation of rag. So we're doing here is we have these retrieved chunk, they contain two fake document chunks. So document one is ordinary deployment information. And number two is that synth has that synthetic key that we don't want in there. And then rag contacts that just joins the chunk, text it to one block, right. So the user did not type the secret, the model did not invent a secret, the application retrieved it. So this is why semantic relevance is, and authorizations are totally different problems, a chunk be the perfect semantic match, and still be the data that the current user is not permitted to see. Meaning if the retrieval chunk contains confidential information, the application may place that confidential information directly into the model request.

5:02And that's what I'm talking about those two different problems. So let's go ahead and run this. Deployment testing is scheduled for Friday, temporary integration key, boom. Okay, so you can see that the rag contacts includes the synthetic key from document two. And this demonstrates a core idea for the later skills, which is that rag can introduce sensitive data into the model context, even when the user never supplied that data directly. So secure rag, therefore, needs both good retrieval and permission aware retrieval. Alright, so now let's get into the assembled context. So a real application is going to combine several sources before inference, it's pretty much a given. So that could be conversation history, uploaded content, retrieve documents, the latest questions, maybe system instructions, those sources do not all have the same origin or trust level, but they can eventually be serialized into one model request.

6:05And that's why provenance matters. And if we lose track of provenance, then it becomes much harder to decide which policy applies to what piece of context. So here we have assembled context right up here. And this groups and conversation history, upload files and retrieve chunks into one structure. And then here, we're going to print that. Oh, but before we do, we do have this. So that this function value appears is checking whether the synthetic secret exists anywhere in that combined object. And this is what lets us see several independent ingress paths converging into a single model context assembly step. So let me show you exactly what's happening here.

6:50So it says, the secret appears somewhere in the symbol context true. So remember that check from earlier. So the point here is not that the secret is present is that it arrived through three different paths, the old conversation state, uploaded content, and retrieval. And those paths will need different controls later in the course. So we're setting this up. But how about tool calls and agents, the same thing, we're going to follow the secret through actions in outputs. And tools are no exception, they introduce another direction, right? So far, we've just watched data flow into context. But here we're getting into tools because a tool can move data out of the AI applications reasoning context, and into another system, such as a ticketing platform, CRM, database, email service, or external API. And that makes tools an egress path. So egress just means the data leaves one boundary and is sent to another destination. So let's go ahead and run this.

7:59Oh, yeah. So let me explain what's happening here before we do that. So if a model proposes a tool called containing sensitive arguments, and then the host application executes it, that means that the secret is going to probably be copied into that downstream system. This is why it has to have valid syntax, and it can still be unsafe. So here is our tool call, right, that's our request. And it contains a tool name and an arguments directory, but look at this, we still have that synthetic secret. And then here for notes, let's say that we have a host application that sent this object to a real support system without inspection, then that secret would just be written into that external platform. And you can see that right here. And the same thing with the agent. So if we run this, you can see that we have the content here. And then we have this one here. And that means that the final answer is clean, but the trajectory contains that secret in both the observation and the tool call. And this is exactly why agent security evaluation has to inspect internal behavior, not just the final text return to the user, it may be hidden. All right, so let's go ahead and we have this really big long one here, I'm just going to run these two and show

9:19you what's happening here. So we followed the same synthetic value through several components. And now the habit is to turn that observation into some kind of inventory. So if we go back up here, this fixture model response is a deterministic function, and it accepts the prompt so that it has the same conceptual input as the live path. And it's going to return a fixed summary that does not contain a synthetic key. And then here, this is the execution wrapper. So the prompt parameter here represents the text that would cross the model boundary. And that first if branch is going to ask whether the live mode was requested. And we do know this is set to true for me, but it could be set false for you if you don't have an API key. So just remember, this is at the very top of the notebook. So if this is false, then we're going to use a fixture. Otherwise, we're going to create a real API call. So let me just talk to you about what's actually happening here.

10:24So here we can see whether it's a live API or a fixture. And then the model called true, the fallback false and secret in the final output is false. But the secret is somewhere in the application log is true. And this gives us the canonical teaching result, meaning this is the visible response, meaning it's clean, but the application still contains a sensitive value internally right here can be tricky. And so this is the all surfaces, I should say, user prompt conversation history uploaded file rag tool call agent trajectory model output and application log. So these are the or these are every application artifact that we created. And then the loop is going to check each surface for the exact Sentinel value and record the result as a boolean true or false. Alright, so that we're converting the results into a pandas data frame down here. That's what we're doing here. So you can see which is pretty amazing that most internal services should show true while the sanitized model output should be false. And you can see that here. So don't interpret true as a breach happened, right interpret it as this surface contains the

11:42protected test value. I think that's a good place to be. Alright, so we're just setting things up. It's okay, if some of that was not totally clear. We're just going to do this through this whole course. So I was just giving you a taste of like all the different surfaces, how it can be tricky. And maybe you don't see it in the output of the model. So it's safe, right? Nope. It's going to be hidden in all of these surface layers that have different behaviors. And there's just so many moving parts that in the whole rest of the course, skill by skill, we're going to identify these one by one, and we're going to take it super slow. So if this was a little bit fast, that was on purpose, because it was kind of like a tour of all of the concepts at once. And now we're going to get into this step by step. So until then, I hope this has been informative and like to thank you for viewing.