Explain VXLAN and EVPN

0:00[MUSIC PLAYING] Welcome to explain VXLAN and EVPN. VXLAN or the Virtual Extensible Local Area Network is an extension to VLANs. It's going to help us scale outer layer 2 data centers. And Ethernet VPN or EVPN, that's going to help us know where all of the MAC addresses in our data center live. It actually combines with VXLAN. Because what we're going to find is that VXLAN is a great data plan protocol, but it doesn't really have anything in the way of a control plane. And that's where EVPN is going to come to the rescue.

0:33So throughout this skill, we're going to focus primarily on how VXLAN operates. And then at the end, we'll see how EVPN comes in and really complements it well. Come on and have a lot of fun with me exploring how exactly VXLAN is going to work for us inside our data centers. With that, I'll see you in the next video.

0:00[MUSIC PLAYING] So why are we using VXLANs? And what are these Virtual Network Identifiers, or VNIs, that we keep hearing about? In this video, we're going to unpack what exactly VXLAN does for us in the data center and how it accomplishes it. And then also learning more about that Virtual Network Identifier, or that VXLAN tag. Let's take a look. If we're going to talk about a new technology then we better find out ways that that new technology is making our life easier. And so let's start with the problems that we have. In a data center today, we're running out of VLANs.

0:33When you think about modern enterprise, service provider data centers, a lot of times we deliver VLANs per customer, maybe a set of VLANs per customer. But we only have 4096, so let's called that 4K VLANs, we're going to run out pretty fast, especially in the modern era. Furthermore, we're still scaling with Spanning Tree Protocol. And not to mention Spanning Tree Protocol, but also first hop redundancy protocols, which are often limiting us to, let's say two aggregation switches, in a particular architecture. And so we do throw things in there like virtual port channels and such, but it's really just patchwork.

1:07We're still dealing with archaic technology like Ethernet, broadcast storms, Spanning Tree Protocol, again. And we've got to get away from that in our data centers, if at all possible. And lastly, when we expand our data centers via these Layer 3 pods, we introduce Layer 3 boundaries. And layer 3 boundaries that causes problems because we like Layer 2 in our data center. That's why have all of these problems, really, is because we are so obsessed with Layer 2 inside of our data centers, to keep our applications on the same subnets, or for that matter, dealing with storage traffic and live migration and such.

1:38So this is where VXLAN is going to come to the rescue. VXLAN is going to enable us to better scale our networks, and it's going to accomplish that in two primary ways. First of all, it's going to increase the number of broadcast domains that we have available to us. So when we're talking about the number of VLANs for example, we don't need to worry about that anymore. We can way beyond 4,000 with the VXLAN. And second of all, VXLAN is going to enable us to punch through our Layer 3 domains. It's going to extend Layer 2 over Layer 3, which means we're no longer going to be bound by this Layer 3 pod architecture that we've been building our data centers with.

2:10So the way this works is, when we take our data center switches, we're still going to be connecting them downstream to our 4,000 VLANs. That part isn't going to change. However, when we enable VXLAN on the switch, it becomes what's known as a VXLAN Tunneling Endpoint, or VTEP. Now one VLAN is going to get mapped, in most cases, to a single Virtual Network Identifier, or VNI. We might have a couple of VLANs coming in, they each get mapped to different VNIs. And then each one of those VNIs is going to be able to be extended through a Layer 3 boundary.

2:42Let's say we have now this Layer 3 architecture, we're sending it through a Layer 3 network, or fabric, and then we connect to another VTEP on the other side. Well, that VTEP is going to receive those VNIs, that encapsulated traffic that VXLAN has enabled us to send, and it's going to translate that back into the individual VLANs that we're originally connecting with. That means that we have effectively extended this Layer 2 network all the way across through this Layer 3 boundary. Furthermore, we might have a lot of different VTEPs out here. We'd have dozens, or even hundreds, of VTEPs on this Layer 3 network.

3:14Because, hey, Layer 3 allows us to scale. So we don't need to worry about a whole lot of our limitations anymore, at least as far as network segmentation is concerned. And so we can connect to as many of these VTEPs as we desire, and each connection is going to be a tunneled interface that's going to be from loopack interface to loopback interface. Now in order to accomplish this, we're going to use a VXLAN header. We're going to break down the packet here in the next video. But for now, the VXLAN header is an 8 byte header, that's got 1 byte dedicated for flags, 3 bytes dedicated to this Virtual Network Identifier.

3:44And if we do quick math there, that's 24 bits. That means we have 16 million different Virtual Network Identifiers. Technically by the way, it's 16,777,216. So I would say that's closer to 17 million relative to 16 million. But, hey, everyone likes to say it's 16 million and that's fine. And then that leaves 4 bytes left that are simply reserved. So we're not going to use those inside the data center for the most part, however for example, we do see Cisco take advantage of some of that space in their Software Defined Access, or SDA solution, that exists in the Campus environment.

4:19Now something we need to be aware of with these VNIs, is that they can be scoped differently depending on our desire. We have two primary ways of scoping the VNIs. First of all is a network scope. Network wide scope means that if we're going to apply, let's say this VNI, let's just call this VNI, I don't know, 100,272. Why not. So if we were to apply that VNI to a particular VLAN in here, maybe it's VLAN 10, the question we need to ask is whether this identifier is significant only to the local switch, or to the entire network.

4:51So again, we have the idea of scoping it via the whole network, and then we also have the idea of scoping it locally. So what's the difference here? Well if it's scoped network wide, then that means that 100,272 on this switch also from presents that VLAN 10. At least that same broadcast domain, even if we'd use a different VLAN identifier on the other side. But that's true also for all of these different VTEPs that are on the network. So across dozens, and maybe even hundreds, or thousands of VTEPs on our network, we have to reserve that VNI everywhere.

5:22Even if that broadcast domain doesn't exist on all of those switches. Now on the one hand that's inefficient, on the other hand, we have 16 million of these, so maybe it doesn't particularly matter. And so that's why the benefit to a network scope is that it's simple. We don't need to worry or stress out about that VNI changing anywhere, it's going to be the same throughout the entire network. The downside is management, because we have to worry about the fact that this VNI is reserved everywhere. We can't repurpose that VNI, by accident, anywhere in the environment. And on the one hand, yes, again, 16 million is a lot of values, but on the other hand, we can start to overlap pretty quickly, especially in very large scale data centers.

5:59This is where we might want to consider a locally scoped VNI. A locally scoped VNI is going to behave a little bit like MPLS, for those who have studied MPLS tags. And the way this works is, this 100,272 VNI might come in on this switch and be received on a different VNI. It might be received on 256,771. Now the same policy gets applied, it still gets mapped to the same broadcast domain on both sides as VLAN 10 example that we're using. But the advantage to this is that we don't have to worry about overlapping VNIs anywhere else in the data center, we only need to worry about it on a local switch.

6:35100,272 only needs to apply to this specific VLAN on this specific VTEP. So the advantage of this is, from a management perspective it can be a little bit easier, but the downside is that it's less readable from a human perspective. I can't just look at this VNI over here, 256,771, and know intuitively that it maps over here to this VNI. So as we can see, VXLANs are a game changer in the data center space. They're going to vastly increase our scalability from a Layer 2 perspective. And they're primarily going to handle that in two different ways.

7:08First of all, we have those 16 million different VNIs at our disposal, that's an incredible amount of broadcast domains that we can now support. And second of all, we can now tunnel our traffic, our Layer 2 traffic, through a Layer 3 fabric. That's going to change the way that we build our data centers, as we're going to continue to see throughout the skill. Now lastly, when it comes to those VNIs, we need to keep in mind that we have two different scopes. We have the network based scope, that means that we are going to have one VNI represent a particular broadcast domain everywhere throughout the entire data center. And in some cases, smaller data centers especially, it's going to make a lot of sense.

7:41But then we also have this concept of locally scoped. Locally scoped means the VNI is only significant to the local switch, and those tags change as we traverse the network. And so that's going to matter more for large scale deployments and large scale data centers, as opposed to the smaller ones. I hope this has been informative for you and I'd like to thank you for viewing.

0:05By now, you've probably heard about leaf-spine architectures, this crazy new way of connecting all of our data center switches together inside of these larger fabrics. If you've been working your way through the DC Core Course, then we've mentioned this a couple of times. And maybe, again, you've seen this already in the real world. But the reality is that leaf-spine is only doable because we have VXLAN. VXLAN gives us the layer 2 scaling across layer 3 fabrics, and hey, guess what? That totally enables us to deploy leaf-spine architectures. So in this video, we're going to take a look at how VXLAN and the leaf-spine come together in order to deliver these large-scale deployments.

0:39Let's take a look. As mentioned, leaf-spine is becoming very popular these days. Leaf-spine can also be occasionally referred to as a Clos fabric. And this is named after Charles Clos, which maybe I'm saying his name wrong. I don't know. But either way, the Clos fabric or the leaf-spine architecture has been around for a very long time. So the question is, why is it becoming popular now? And the short answer is VXLAN. Let's go ahead and draw this out. And what we find here is that we've got a spine layer and a leaf layer. Let me label these. We've got spines up here, and we've got leaf switches down here.

1:13And at first glance, this is going to look pretty similar to an aggregation access layer, where we might have the network core up here-- and we call this the Agg layer-- and the access layer down here. But the similarities end pretty quickly, because right away, we see that we connect these switches very differently. Specifically, we connect spine switches to leaf switches. So every leaf to every spine, every spine to every leaf. And that's it. We do not connect spines to spines. We do not connect leaf switches to leaf switches. And the differences don't end there, because we're accustomed to the intelligence of the network living higher up on these higher layers, but not so with spine switches.

1:49All of the intelligence actually lives down here at the leaf layer. The spine switch has one job-- to bring traffic in and send it right back out. We want to switch traffic between our leaf switches as fast as possible. That is all the spine switch is designed to do. Furthermore, our external connections don't come into the spines. Our internet block, for example, is going to come in at the leaf layer, as is the campus architecture. So when we connect back to the campus or possibly the network core, that's going to come in at the leaf layer, as well. The leaf-spine is designed for scaling.

2:21We absolutely need to be able to scale out to very large layer 2 fabrics in our data centers today. We've made that comment a couple of times now. And we talked about how VXLAN helps us to scale, but now we need an architecture that scales, as well. So let's look at how leaf-spine scales. First of all, we have the independent scaling of different resources in the data center. Specifically, we have our backbone layer. And so when we think about our backbone layer, when we think about that, we might think about packets per second and how many packets we can process in a moment, as well as the bandwidth.

2:52And so if we need additional switches for processing additional packets, well, guess what? We just expand the spine layer. We add another spine, connect that spine into all the different leaves, and our load balancing takes care of the rest. Furthermore, if we need more bandwidth, we can simply expand the number of connections that we have between our leaf and spine switches. Adding a connection to a port channel is not disruptive. Neither, by the way, is adding a spine switch into the network. And so we can scale out our network backbone far more simply than we can with an aggregation layer. And furthermore, if we need ports, we need additional connectivity at the edge, well, we simply add leaf switches in.

3:26So we install a new rack into the data center. We can throw a couple new leaf switches at the top of that rack, connect them into the spines, and voila. We have expanded our data center footprint. The second way that we improve scaling with leaf-spine-- and I love this one-- is this concept of declaring a two-hop behavior for all traffic in the data center. In other words, when a packet arrives on a leaf switch, I guarantee you that, regardless of where it's going in the data center, it's going to get there not in more than two hops and not in less than two hops. It is precisely two hops to every destination.

3:59I always go to one spine switch and that spine switch always sends me down to the appropriate leaf switch, whichever leaf switch is my destination. That's incredible, not just from a latency perspective, but also from a jitter perspective. And I know when we talk about network jitter, we tend to talk about voice over IP packets. And maybe we don't have a lot of that going on in the data center. But all applications are affected by jitter. And so if we can deliver a consistent experience in the data center, then that's going to make all of our applications behave better. Furthermore, we can make our network services available very seamlessly throughout the infrastructure.

4:31In other words, we bring our services into a pair of switches. Maybe we've got a dedicated pair-- I'm trying to figure out where to draw this in this diagram. Let's put them up here-- a dedicated pair of leaf switches that are coming into these spine switches here, and these leaf switches have all of our network services. We think about things like network load balancers, and cloud gateways, and next-generation firewalls. And because of our two-hop behavior, we have very seamless access to all of those services. Furthermore, we have other services, such as anycast gateways, that we can deploy out onto the network that reduce our routing behavior from an overhead perspective.

5:07So we can deploy a gateway to every one of these leaf switches, meaning that we can get routed locally. We don't have to get forwarded up to the spine, or formerly the aggregation layer, in order to get routed. And by delivering our services everywhere throughout the data center and making it a maximum of two hops away, we've enabled those services to be accessible for everyone. So what's going on with VXLAN? Why is it so important that it enables this architecture? Well, the ideal situation with the leaf-spine architecture is that these connections in here are layer 3.

5:37If we want this architecture to scale out, we have to leverage layer 3 at some point. Imagine having dozens of leaf switches and several different spine switches and all of the connections are layer 2. And imagine the spanning tree nightmare that we're going to have in that situation. Imagine having our broadcast storms going everywhere. We need layer 3 scalability. But at the same time, we need a host on this leaf switch and a host on this leaf switch to be on the same subnet, because we need layer 2 services in the data center, as well. This is why VXLAN is such a beautiful solution, because now we can have these subnets stretched out, such that these hosts can be on the same VLAN, and yet, our fabric is layer 3.

6:16We use VXLAN to tunnel through the layer 3 fabric from one leaf switch to another and stretch that layer 2 boundary for us. So it enables a layer 3 backbone, but then furthermore, again, we've got these 16 million different segment IDs. So now we have virtually no limit to the amount of broadcast domains that we can have inside a single data center. So even if we were to make leaf-spine work on traditional VLANs, we'd be limited to that 4,000 broadcast domains inside the data center, and that would only be so effective. So yes, we can now take advantage of far more broadcast domains by leveraging VXLAN on top of this architecture.

6:50One more extension to this architecture we'd want to consider is the idea of having a virtual host connecting into one of these leaf switches, and maybe a couple of these leaf switches. And the idea here would be that we have a virtual switch inside of this virtual host. And Cisco makes these virtual switches-- the Nexus 9000v, for example, and way back in the day, the Nexus 1000v. And these vSwitches can run VXLAN. That said, they are functionally a VTEP. And by deploying a VTEP into our virtual space, well, now we need to be able to tunnel straight to them.

7:21Well, in a leaf-spine, we can do that. We can essentially turn this into a virtual leaf switch, and therefore extend our VXLAN tunneling policies down into that virtual space. This further increases the scale of our environment to extend down into the virtual domain. So yes, we can even deploy virtual leaf switches into our infrastructure. And that's not just a Cisco technology, that exists with a lot of the different hypervisor manufacturers. So regardless of whether we're scaling out physically or virtually into the virtual space, either way, VXLAN on top of leaf-spine architecture really gives us the scaling that we desperately need in the data center.

7:55So as we see here, leaf-spine is built for scaling layer 2 fabrics. It does the same thing as VXLAN, except where VXLAN does it from a network logical layer perspective, spine-leaf does it from a physical perspective. For example, we see that spines are only going to connect to leaf switches and vice-versa. We're not connecting leaves to leaves. We're not connecting spines to spines. The only thing we're doing is enabling the backbone connection, essentially, via the spine. So a spine switch cares primarily about bringing traffic in from one leaf and forwarding it out to the next leaf as fast as possible. And lastly, we see that leaf-spine connections are built as layer 3.

8:28We're building a layer 3 fabric. And VXLAN is the magic that gives us the ability to move layer 2 packets across this layer 3 boundary. I hope this has been informative for you, and I'd like to thank you for viewing.

0:00[MUSIC PLAYING] Now that we know what VXLAN is, and we know the architecture we're deploying it on, it's time to investigate how it actually behaves. In VXLAN, at its routes is very much a data plane protocol. We're going to find out that we need to throw some enhancements around, potentially, to make the control plane work. But for now, let's first start with the data plane, and see how exactly it functions when we're talking about tunneling these packets through this Layer 3 backbone. Let's take a look. So we know that one of VXLANs primary functions is to extend Layer 2 connections over Layer 3 fabrics.

0:33Let's try this out here. We've got a switch on the left, and we call this a VXLAN Tunneling End Point, VTEP. And we're going to connect that via a Layer 3 connection to another VTEP. And maybe there's another bunch of switches in here. Again, maybe there's a spine switch in here that connects us between these two leaf switches. However this comes together doesn't really matter, what we're trying to accomplish is the same. So we know that we've got a host on the left here and it's, let's just say, on VLAN 10. We've got another host on the right, that's also in VLAN 10. So how exactly is VXLAN going to extend connectivity between these two different devices?

1:08Well, the way we're to do this is with encapsulation. We are going to tunnel our traffic from one VTEP over to another VTEP. And so the way this is going to work is this frame, this Layer 2 frame, is going to come into the VTEP on the left. And so let's just draw this out. And let's show that we have now A Layer 2 frame. This Layer 2 frame by the way, has an Ethernet header, it has an IP header, it has a Layer 4 header in there. And we are going to encapsulate that into a VXLAN header. Now this VXLAN header is 8 bytes in size. And as we discussed, this 8 bytes is going to include a 3 byte, in other words a 24 bit, Virtual Network Identifier.

1:45That 24 bit Virtual Network Identifier gives us-- You remember now-- 16 million different network segments that we can use. Now we're not done because we still need to get from point A to point B, and the VXLAN header isn't going to include information on how to accomplish that. As such, we're going to, first of all encapsulate this into a UDP header. And this UDP header is going to have a destination port of 4,789. And finally, we still need our IP addresses in here. So we're going to throw an outer IP header on there, that's going to have the source IP address of the VTEP on the left, and the destination IP address of the VTEP up on the right.

2:20Now any time we talk about tunneling, we're going to have to talk about the idea of MTU. And so we need to figure out how exactly our Layer 3 infrastructure needs to be adjusted for the new MTU to support VXLAN. And so if we draw a dividing line right here, and we say, OK, well this is the original packet, and so we know that we're good here. Well look at everything to the left of that line. We need to account for all of that when it comes to our MTU. So that Ethernet header is 14 bytes, for example. The VXLAN header, we've said, is 8. UDP is another 8 bytes. And the IP header that we're adding on to the front, is another 20 bytes.

2:54So we add this all together, 20 plus 8 plus 8, that's 36. Plus 14, that gives us 50 bytes. We need to increase our MTU by 50 bytes. And it's worth noting by the way, that we do technically have a VXLAN option to include the Dot1q header. We're assuming, with 14 bytes, that we don't have a VLAN tag in there. If, however, we do have a VLAN tag, then we need to account for another 4 bytes. And if we were to include that, then we'd need to increase our MTU by 54 bytes in actuality. Now fortunately, in most cases, we're not going to be doing this, so let's not worry too much about it.

3:26Just have it in the back of your mind that that's technically an option in real world scenarios. For the sake of what we're going to be looking at, from a DC Core perspective, from a Cisco perspective, we're likely going to be talking about 50 bytes as the MTU increase that we need to add into our Layer 3 fabrics. Now one other thing worth mentioning is that fragmentation is not actually supported by VXLAN. Now what's interesting about this is that there's a Do Not Fragment bit that we can set in the IP header. And that's not actually configured with VXLAN, we leave that alone. So in theory, from a technical perspective, it can be fragmented.

3:58However, it's not supported, and it does result in some weird behavior if we forget to increase the MTU. Let's just increase the MTU and be done with it and not worry about fragmentation. Now when we consider the VTEP, we're going to divide it into two different types of interfaces. We have the IP interfaces, that face the Layer 3 network. And we have LAN interfaces that are going to face, well, the downstream LAN, the Layer 2 domain. Now let's explore how VXLAN works from a data plane perspective. Let's say that we have this Layer 2 packet, or Layer 2 frame, that's showing up on the VTEP. So we have this. We have a destination Mac address of, let's say Mac address B, over here hanging out on VLAN 10 on the right side.

4:34So this switch is going to check its CAM table, just like it would always do from a Layer 2 perspective. It's going to look to see, hey, do I know where destination-- I guess I didn't write this-- destination Mac address of B. So do I know where B lives. Now in a typical Layer 2 environment, B would probably be attached to an interface. However, in VXLAN, we have the possibility of attaching B to an IP address. And specifically, this would be the IP address of the VTEP on the other side. If that's the case, we're going to encapsulate that packet and send it across as a tunneled connection through that Layer 3 fabric.

5:07The VTEP on the other side is going to de-encapsulate it, check its own CAM table, realize that Mac address B lives out port, whatever that is, 1/1 we'll call it. So it just forwards that 1/1, and it makes it towards its destination. So that's it. From a data plane perspective, VXLAN is pretty straightforward. But now we need to have a conversation, right? Because how exactly did the VTEP on the left know where Mac address B lived? And there are a couple of different ways we are going to explore in this skill, starting with the Flood and Learn methodology in the next video. So as we see here VXLAN is going to use a MAC-in-IP/UDP encapsulation.

5:41That means that we've got the IP address, the source of the VTEP that's sending it, the destination on the other side. And then from a UDP perspective, we use port 4789. Next we got to consider MTU, always, with any tunneling mechanism. And with VXLAN, when we add it up, that IP header, the UDP header, the VXLAN header, and then that Ethernet header that we're encapsulating, we've got to take that into account as well. We add all those up, it comes out to 50 bytes that we need to increase our MTU in the Layer 3 fabric. And lastly, VTEPs are going to bring a packet in, realize, hey, in my CAM table I've got it destined for a VTEP, so I'm going to encapsulate that, send it across the Layer 3 network, and allow the VTEP on the other side to de-encapsulate it, and then from there, check its own CAM table to figure out what to do with it.

6:24I hope this has been informative for you and I'd like to thank you for viewing.

0:00[MUSIC PLAYING] As mentioned, VXLAN is primarily a data plane protocol. It's not an expert at figuring out where all of the Mac addresses live in the domain. But fortunately for us, there is a built in mechanism to handle this, in case we don't have a control plane protocol that we're deploying otherwise. So in this video, we're going to explore how exactly VXLAN leverages Flood and Learn, as its means of learning where all of those Mac addresses live. Let's dive in. So let's draw out an architecture here, let's do spine and leaf. And let's just assume that we've got one spine here, just for simplicity's sake.

0:34We'll draw four different switches. We've got connectivity to all four of those. Hopefully we'd never actually deploy a single spine architecture into a data center. And as we think about this concept of Flood and Learn, we should be thinking about another protocol, specifically Ethernet. Think about the way that Ethernet learns where Mac addresses are. We don't have a control plane protocol running, we're not running OSPF for Mac addresses, for example. Instead, when we have a switch and we receive a packet, and we do not know where the destination lives, what does the switch do with it? Well it floods it out.

1:05It floods it out by sending it out all of the interfaces that it has on it, other than the interface that it received it on. And by flooding it out, eventually, the theory is, that packet will land where it's supposed to land. And ideally that host will send something back to us, and when that packet arrives we're going to learn where exactly that hosts lives. So maybe it lived out this interface. And so next time we get a packet in, instead of flooding it out, we simply send it out the interface that now we know where it lives. So eventually we achieve sort of a steady state operation.

1:36I mean, certainly we're going to be learning new Mac addresses always, and we're going to be learning that Mac address move, and such. So we get as close to a steady state operation as possible, and we do it all without a control plane protocol. This is a Flood and Learn technology. And so we're going to do something very similar with VXLAN. However, with VXLAN we don't have the luxury of flooding out via a Layer 2 broadcast, because we have this Layer 3 network segmentation in place. And so we're going to need to come up with a new way of doing this. And so the way we're going to accomplish this is, yes, with multicast.

2:09Now multicast is going to create that flooding environment, that ability to send packets out and get them everywhere in the network, and it's going to do it as efficiently as possible. Let's dive right into an example. Let's say we have a VNI, a Virtual Network Identifier, that maps to VLAN 20, it doesn't matter what the VNI is in this case. And so what we're going to do is we're going to map a multicast address to it. Let's use 239.1.1.1. Furthermore, we're going to have a second VNI. Let's map this to VLAN 30, and we'll create a multicast address of 239.1.1.2 for this VNI.

2:44What we're going to do is we're going to form a multicast tree through the spine architecture. So let's say VLAN 20 exists on the first three switches, and VLAN 30 is going to exist on the last three switches. So we've got a little bit of an overlap here. We've got switches that support both VLANs 20 and 30, and we have switches that only support one or the other. What going to do is we're going to build a multicast tree through the spines. And so this multicast tree is going to be built according to each one of the addresses on the left. So what I just drew here is the multicast tree for VNI 20.

3:16And so now we can draw a new multicast tree that maps out to the VNI that maps to VLAN 30. The way multicast trees work is that when a packet arrives destined for the multicast address, in the green case of 239.1.1.1, it's going to forward it out all of the interested paths. Basically, we're going to forward it out the other branches on that tree. And so that gives us an avenue to flood traffic for that particular VNI. Let's see this in action. Let's say I have a host with a Mac address of A and it's attached to this switch here on the left.

3:46Meanwhile, we have a host on switch 3 that's also attached to the same VNI, the same VLAN, the same broadcast domain, ultimately. And we'll call that Mac address B. Now we are sending a packet onto this network with a destination Mac address of B. And the switch here on the left, it doesn't know where B is. So what are we going to do with this frame? Well for all we know Mac address B might live out one of our own interfaces. And so we're going to flood it out all of the interfaces according to our typical Ethernet Layer 2 flooding rules.

4:17However, because VXLANs in play, we know that we have ports out all of these other switches that have connections to hosts that are in the same broadcast domain. And so we need to give them the opportunity to flood this packet out all of their interfaces as well. This is where we're going to send it via this multicast stream. We're going to send it up with a destination address of whatever our multicast address is, in this case 239.1.1.1. That tells the spine switch to send it down to all of the VTEPs in the environment that want to receive that particular traffic for that VNI.

4:49At this point, both of these other VTEPs are going to receive this traffic and flood it out all of their interfaces. So, yes, on some of these VTEPs it doesn't go anywhere. It just lands on the switch and it floods out all the interfaces, just like in a traditional Layer 2 network. However, over here on all right, we know that we are going to flood it out all of our interfaces, and one of these packets is actually going to arrive on the destination. Now the first thing that happens over here on the right is that our CAM table gets updated. Our CAM table now shows that Mac address A lives out VTEP, we'll just call that VTEP 1.

5:22Let's say we have 1, maybe it's a dot 1 interface. Dot 2, dot 3, and dot 4, on these four different leaf switches. So we'll say that it's actually attached to IP address dot one. So now this host decides to respond, which is usually what happens in a network stream, and so it's sending traffic with a destination of Mac address A. Well this switch knows where Mac address A lives, it lives out VTEP 1. And so it's not going to take advantage of the multicast streams, instead it's going to bundle that up and send it via unicast through the spine switch down to the VTEP on the left.

5:57The VTP will de-encapsulate it, and the first thing it's going to do, it's going to update its own CAM table. It's going to see that Mac address B now lives out VTEP-- We'll just call it the IP address of dot 3. And then we send that Layer 2 frame down towards the host. So just like a traditional Ethernet environment, by flooding the traffic and learning it as the return traffic comes back, we are eventually going to achieve as close to a steady state operation as we can get. Again, learning new Mac addresses always, learning Mac addresses that move around, potentially, all of that's still going to happen in update, just as it would in a traditional Ethernet environment.

6:32However, one thing to be aware of is that we're going to have to continue to deal with what we call BUM traffic. That would be Broadcast Unknown Unicast and Multicast traffic. All three of these traffic types are going to continue to use the multicast tree. So if I have a broadcast packet that needs to go everywhere, well, guess what, I've got a multicast tree that's going to enable me to do that. That broadcast has to fit into one of these VNIs, because each VNI is a broadcast domain. And so I will use the appropriate multicast address to send that out. I'll receive the broadcast in, I'll forward it out all of my local interfaces, just like I normally would.

7:04And I will also send it up that multicast tree. The multicast tree will turn around and send it down to all of the appropriate VTEPs that need to then flood it out all of its associated interfaces. The same thing with unknown unicast. The only difference with multicast, is that our multicast streams might look a little bit different, because we only need to send to the VTEPs that have interested parties. So if we do nothing else and simply deploy VXLAN as it was built, then it has this control plane functionality built in, and we'll be just fine. However, we need to understand that we can add control plane functionality on top of VXLAN native operation.

7:38Now we have several different options for that. First of all, we do have a concept called Ingress Replication. Ingress Replication is when we have a unicast backplane. Ideally, this never happens, but in the event that we can't actually send multicast traffic across our spine switches, or whatever our Layer 3 core is, we're going to have to use ingress replication. Which basically uses the same methodology, except it's going to require a lot of manually mapping VNIs to VTEPs, because we need to understand which VNIs live where, we can no longer automatically discover that via multicast.

8:10And furthermore, we're going to have to replicate a lot of this traffic, hence Ingress Replication. So if I need to send a packet to four different VTEPs, or 40 different VTEPs, or 400 different VTEPs, I'm going to have to replicate that and send it unicast to every single VTEP that requires it. Now beyond this, we have an option that we're going to study in more depth in the next video, and that is known as BGP Ethernet VPN, or EVPN. This is going to proactively tell us where the Mac addresses are, and again, we're going to be covering that in more detail here shortly. On top of that, when we look at prepackage solutions, like Cisco SDA.

8:43Software Defined Access leverages VXLAN and it relies on LISP at the control plane to advertise where the Mac addresses are. And we have something similar with Cisco Application Centric Infrastructure, ACI, here in the data center, where we are leveraging a Cisco created protocol called the Council of Oracle's Protocol, or COOP. But fortunately, even though we seem to have a lot of different options available to us, including, by the way simply relying on Flooding and Learning. We are starting to see the industry converge around a standard, and that would be BGP EVPN. So as we summarize this, VXLAN lacks a native dedicated control plane.

9:15Instead, we're going to rely on Flood and Learn, very similar to how Ethernet functions. So the way we do this is that we're going to have to flood it out to all of the different VTEPs. But the difference between VXLAN and Ethernet, is that VXLAN we can't just flood it out from broadcast perspective, we have to get it across that Layer 3 boundary through the spine switches. And so what we're going to do is we're going to form these multicast trees and send our traffic across those multicast connections. Lastly, once we do learn where that destination is-- the first part's the flood part, now we have the learn part.

9:46So fortunately, just like with Ethernet, when a packet comes back and we see, hey, that particular destination Mac address lives out a particular VTEP. Well now I can put that into my CAM table, and I can make sure that I send that via unicast, just to the VTEP that needs it, when a unicast packet next arrives. I hope this has been informative for you and I'd like to thank you for viewing.

0:00[MUSIC PLAYING] Now that we understand how exactly VXLAN operates outside of a control plane protocol, now let's introduce one. We're going to be talking about the Ethernet VPN, or EVPN technology, that's going to serve as a control plane for our VXLAN environment. Let's dive in and figure out how exactly this works. Ethernet VPN is going to rely on the Border Gateway Protocol, or BGP. Now what I like to say about BGP is that it's not actually a routing protocol.

0:30We think about OSPF, and EIGRP, and such, and those are true routing protocols, they can only carry one piece of information, and that would be an IP subnet. However, BGP is more of an information carrying protocol. We can use it to carry all kinds of things. And when we do, we call it Multiprotocol BGP, or MPBGP. Now when we think about Multiprotocol BGP, oftentimes we're thinking about IP version 6, and NPLS, and such. However, why couldn't we use BGP to carry Mac address information, along with Virtual Network Identifiers, and such.

1:02And that's exactly what we're going to do. We're going to leverage Multiprotocol BGP, carry this information, and we're going to call it the Ethernet VPN, otherwise known as EVPN. Well let's draw out a topology here, let's give ourselves, again, a single spine switch, just for simplicity of conversation sake, not that we'd ever deploy a single spine switch. Now what we're going to do is we're going to configure these devices for Multiprotocol BGP. We put BGP on all of the leaf devices, and we put BGP up here on the spine. Now specifically on the spine, we're going to make this a BGP route reflector.

1:36If you've studied BGP then you probably know what a route reflector is. If not, the gist of it is that BGP peers require a full mesh of connectivity in a normal environment. And so rather than form a full mesh of peers among all of these, we can use a route reflector in order to, more or less, turn it into what we'd expect with EIGRP or OSPF, or simply have direct relationships with the upstream device. Now I forgot to include my physical connectivity, so let me get that in there. So the way this works is we're going to abandon our Flood and Learn methodology.

2:07We no longer need it, because we're going to proactively advertise our connections to one another. And so let's say we've got a particular IP and Mac address combination hanging off of this VTEP, we know that it's attached to a particular VNI. And so now we can advertise that upstream to say, hey, I've got an IP address, I've got a Mac address, it's part of this VNI. And that route reflector is going to push that advertisement down to the other VTEPs. So when we consider the CAM table here on the right, we're going to have our Mac address that we just learned, and we're going to know which VTEP it's attached to.

2:39Maybe we have VTEPs 1, 2, and 3, labeled here again. So in our case, this Mac address is attached to VTEP 1. That means, right from the get go, if I get a packet in that's destined for that particular Mac address, well, I can unicast that over to VTEP 1 without having to flood it. That works great from a unicast tree perspective. And meanwhile, by the way, we continue to rely on multicast, or the BUM traffic that we talked about. So the Broadcast Unknown Unicast and Multicast, it still needs to get sent to multiple VTEPs at once, and so we're still going to rely on those multicast trees in order to make that happen.

3:10Now as if that's not good enough, we get a couple of other benefits by leveraging BGP. Specifically, we get a true sense of peering relationships. We're actually forming BGP peers with one another, which allows us to do things like authenticate those relationships. We no longer need to worry about rogue devices showing up in our VXLAN domain. Furthermore, we can take advantage of ARP suppression. If you're working your way through the DC Course, do you remember we talked about this concept with OTV? And the idea here is simply, if I get an ARP request, let's say I've got a device down here, it sends an ARP request for that IP address earlier that we talked about.

3:43Well that is a broadcast packet, and so normally I would take advantage of the multicast tree, I would forward that out to all of my VTEPs. But the reality is that I already know what the Mac address is for that IP address, because it was advertised to me via EVPN. And because I know that this IP address belongs to this Mac address, I can simply send down an ARP response without having to forward that into the rest of the network. ARP traffic is not an insignificant part of my data center busyness. And so just the fact that I can prevent flooding ARP requests throughout the entire domain, that's a pretty big benefit to deploying EVPN.

4:17So let's look at an example here. Let's say I'm attaching a brand new host to VTEP 2. It's got an IP address of A. It's got a Mac address, well, also of A. And we'll say it's part of VLAN 20. Now we know that VLAN 20 is going to get mapped to a particular VNI upstream, it doesn't particularly matter what that VNI is. Well soon as we send any kind of traffic into that VTEP, the VTP is going to learn several things. It's going to learn the IP address, it's going to learn the Mac address, it's going to learn its interface, it's going to understand, based on the interface, what VNI it's attached to.

4:48It's going to take all of that information, advertise it via BGP. That means that this information is going to get distributed to the other VTEPs in the environment. Now let's go and clean up this drawing here just to make this more clear. So now the other two VTEPs both know that Mac address A, and IP address A for that matter, belong to VTEP 2. So A belongs to 2 here on the left, A belongs to 2 here on the right. And now when a packet arrives on the left here, let's say, and its destination is Mac address A, well I already know where that is, it's part of my CAM table.

5:19And so I don't need to flood it anymore. I can encapsulate that into a VXLAN. I can tunnel it through the spine switch, and it will arrive on the appropriate VTEP via unicast. Meanwhile also, if an ARP request comes in for that particular IP address. So an ARP request saying, hey, where is IP address A on the network. Well I can get that ARP response back directly from the VTEP. It doesn't have to get flooded into the rest of the network, instead I will get a response back instantly from the upstream VTEP, and the rest of the network never needed to hear about it. So that's a best of both worlds scenario.

5:52Now as we can see, EVPN is going to do us a large favor. It's going to leverage Multicast Protocol BGP, or MPBGP, to send, proactively, IP addresses, and Mac addresses, and VNIs, to the other VTEPs. What that allows is that it makes it so we don't have to use flooding behavior anymore to learn where everything is. Instead, for unicast at least, we no longer need to flood traffic, we can simply send it right where it needs to go because our upstream VTEP already knows where it goes. Now keep in mind that throughout all of this, the BUM traffic, the Broadcast Unknown Unicast and Multicast traffic, that's still going to rely on those multicast trees because we still need to deliver that to multiple VTEPs at once.

6:28However, these Multiprotocol BGP peerings bring with them some enhancements. Like for example, we can authenticate our peering relationships. And ARP suppression. ARP suppression is fantastic in a data center, it's just one set of broadcast traffic that we don't need to deal with anymore inside our data centers. I hope this has been informative for you and I'd like to thank you for viewing.

0:00[MUSIC PLAYING] We have reached the end of another skill, which means it's time to review what we've learned by taking a quiz. First up out of five questions. Describe the VXLAN header, breaking it down by bytes. Well the VXLAN header is 8 bytes in size, and so as we break that down, we have 1 byte that's dedicated to flags, and we have 3 bytes that are dedicated to the Virtual Network Identifier. That Virtual Network Identifier, remember, 3 bytes, 24 bits, that gives us 16 million different network segments that we can use with VXLAN.

0:37And lastly, we have 4 bytes remaining. Those are reserved, we're not actually doing anything with them. So for now it's really just all about those first 4 bytes. If we can remember this and break it down this way, we can not only remember the size of the VXLAN header, but also the components. And as well, coming back to that MTU conversation later, we can remember what we need to do there. Next up. Where do the default gateways live in a leaf spine architecture? We touched on this briefly. Recall that we have spine switches, we have leaf switches.

1:09And one of the advantages to leaf Spine architectures, is that we can distribute the default gateway out to the individual leaf nodes. We can have the default gateway here, but we can also have the default gateway over here. This is known as an anycast gateway, meaning that we have the gateway located at multiple locations. It all has the exact same virtual IP address, but we're not running first hop redundancy protocols. The answer here is A, leaf switches. Question three. What MTU increase is required on the VXLAN underlay network?

1:43Well let's think back to that first question. We recall that the VXLAN header itself has 8 bytes, but we also know that VXLAN is embedded into IP and UDP. Now an IP header is 20 bytes and a UDP header is 8 bytes. However, that's not the end of the story. Because when we're encapsulating the entire layer 2 frame, well that layer 2 frame not only has the original packet, which, hey, that could be up to 1,500 bytes, but also has the Ethernet header. And when you take that Ethernet header into account. Now that Ethernet header by default is 14 bytes, and so we add up 20 plus 8 plus 8 plus 14, well that equals 50 bytes.

2:19So B is our answer. However, we might recall that there is that corner case scenario, where we could actually include the VLAN tag inside that Ethernet header. VXLAN does support that, and if we do decide to go that route for whatever reason, then we would need to account for the extra 4 bytes that the VLAN tag provides. However in most cases, we're going to be focused on that 50 byte header increase. So B, 50 bytes, that is our answer. Question four. How does VXLAN natively perform control plane operations?

2:50Well I had to hide the source, because the source gives it away. The answer is Flooding and Learning. Remember, VXLAN is focused on data plane operations, so we don't have a dedicated control plane protocol as part of VXLAN. Instead, we're going to have a control plane methodology, similar to Ethernet, where we're flooding traffic out to all of the VTEPs. And when we receive traffic back, that's when we are going to learn where Mac address actually lives. At that point, we update our CAM tables, and now we know where to send it via unicast next time we have a message for that host. Last question.

3:20It's an open ended one. What BGP configuration is added to the spine switches in a leaf spine EVPN architecture? We might recall that we have leaf switches, and then we have spine switches. And in BGP, we have to have a full mesh-- Whoops, there we go-- We have to have a full mesh of relationships. So if we have five different BGP nodes, then they all have to have relationship pairings with one another. Well, fortunately for us BGP gives us the option of route reflectors.

3:51So we can configure the spines to be route reflectors, which is our answer, and that allows us to only form relationships between the leaves and the spines directly. So it's important that we remember that BGP route reflectors are going to be part of our EVPN designs. Well that wraps up this quiz. If you find yourself running on empty for any of these questions, then be sure to go back and watch the appropriate videos in order to fully gas up. Otherwise, congratulations on completing Explain VXLAN and EVPN. I hope this has been informative for you, and I'd like to thank you for viewing.