Machine Learning Algorithms: Classification Trees

0:00Hi, Sean here and welcome to this skill where we're going to be taking a look at another machine learning technique. And this one is trees. All right. Now, specifically, we're going to be taking a look at classification trees. And these are just a special type of decision tree that's used to classify data points, similarly to what we've seen with neural networks, K nearest neighbor, things like that.

0:31All right. Now, the basic idea of classification trees is what if we could reduce the problem of classifying an unknown data point down to simply a series of questions that would lead us to the right result, right? So let's just say that we're trying to write a program that will predict whether someone will qualify for a loan, perhaps, right? Well, you might ask something like, you know, is there income greater than a certain amount?

1:05All right. Here, I'll just write that as income greater than X. And you would have a specific amount here. So if that's true, then you'd go in one direction. If it's not true, you'd go in another direction. And at the next steps, you would ask a different question. Now, this question could be, you know, different in either of these cases. If the income is greater than X, for example, you might want to know if their debts are greater than a certain amount. And over here, you might want to know, you know, if their income is greater than X, you might want to know if they qualify for down payment assistance, for example.

1:44All right. We'll just say qualify for assistance. And in each of those cases, you'd split off further and ask different questions in each of those until you reach a specific answer, right? So maybe down here, you get yes, no. And maybe down here, you get yes and no like so, all right? So that's the basic idea of a classification tree is again, just by asking a specific series of questions that's tailored to the previous answer, right?

2:19We're only asking about debts if the previous answer to this is yes, for example. In many cases, we can achieve a very high degree of accuracy in labeling unknown data points. Now, the problem, of course, with classification trees like this comes down to how do we decide what questions to ask in the first place, right? Why did we ask about income first instead of debts first? Or why do we ask about debts over here instead of qualifying for assistance, right?

2:51In other words, how do we organize these questions into a classification tree so that they'll lead us to the highest possible accuracy, right? That's really the big question here in working with these classification trees. And it all comes down to something called entropy and a similar concept here called information gain. I'll just write that as info gain, but these are two concepts that when combined, we can use in order to construct our classification trees in the most beneficial way.

3:34All right, so let's discuss what each of these things means. We'll start off with entropy. And while you might be familiar with entropy in a physics context, this entropy is a little bit different. Although you will definitely see some similarities. So up here at the top of this, I've basically just run the same code that we've been using in previous skills to create some blobs, which we've assigned group A, B, and C . And we've displayed them in a scatter plot, right?

4:05Now essentially what the questions in a decision tree allow us to do is draw lines either vertically or horizontally along a specific axis. Now the only reason that vertical and horizontal are the only two options here is just because we're working with only two attributes, but in higher dimensional data sets, you can obviously draw different lines than these. But anyway, let's say that in order to try and tell whether something is in class A or not with a classification tree, we might start off by asking, is the X value greater than, let's say, negative six.

4:52I drew that line a little bit slanted, but you get the idea. And then we might ask if the answer is yes, is why greater than, I don't know about one or so, right? So if the answer to both those questions is yes, then the output is yes, it's in group A, otherwise the output is no. So a decision tree or a classification tree for this situation would look like this. We would start off by asking, is X greater than negative six?

5:25If so, then we would ask, well, really in this situation, if the answer is no, the answer would just always be no, all right, you can have decision trees or classification trees that just have that, you know, that sort of short circuit like that and just lead directly to no. But if yes, we would ask another question, right? We would ask, is why greater than, we'll just say 1.0 there, right? If the answer is yes here, then the ultimate answer to that would be yes.

6:00All right, otherwise the answer would be no. So this would be a decision tree here. But again, we just constructed that by looking at a graph. In most cases, the dimensionality of the data set, right, the number of attributes that a data set has is too high for us to, you know, be able to just construct it by looking at a scatter plot. So here's where entropy comes in. Let's say that we draw a line right here, all right, and we say that everything over here on the right hand side is a, and we say that everything over here on the left hand side is not a.

6:43Well what entropy refers to is how well that division actually divides that data set, right? So we can see in this case over here with not a, that's 100% correct for all of the points inside of here. So this has a very low entropy. Entropy refers to how mixed the members of a group are, whereas over here on the right hand side of this division, we have, we can see that we basically have almost even numbers of elements that are in A and that are not in A in this group.

7:23So this group has very high entropy. Now the basic idea here, we can actually use this concept to know exactly where to draw the correct lines. What we want to do is draw the lines in just the right places so that we minimize the entropy of our groups. All right, and this makes sense if you think about it because if we were to have different groups, maybe groups that are easier to divide and we have one group here that 's maybe group A and we have group B over here, right?

7:58That's just sort of a cluster of points like that. Well if we were to draw a line like this, that obviously reduces the entropy of the overall group because the new split, both of these groups have very low entropy because there's no mixture of different points in each group. However, if we were to draw that line in a slightly different way, right? If we were to draw that line through here, both those groups have very high entropy because they're a mix of elements from different groups.

8:33And in fact, almost an even mix, which as we saw is the highest entropy. So this idea of dividing our data set based on what reduces the entropy the most, that act of reducing the entropy of our data set is referred to as information gain, right? So basically what we want to do is choose our divisions based on which division gives us the highest information gain.

9:07So anyway, that's the basic idea of working with classification trees and some of the fundamental concepts around constructing these classification trees. So the next thing that we're going to do is we're actually going to jump in and implement this algorithm ourselves. I understand that this might seem a little bit strange, but one thing that I love about classification trees is that this is an algorithm that's actually very easy or relatively easy, I suppose, to implement on our own.

9:38And I think it will help you to understand these concepts on a much deeper level. So anyway, that's what we're going to be doing next is implementing a classification tree algorithm, right? One that actually will build classification trees for us inside Jupiter. So I hope this has been informative for you and I'd like to thank you for viewing.

0:00All right, so now that we're familiar with the basics of how these classification trees work and have some idea of how to create these, let's get started creating this algorithm that will create decision trees for us. So the first thing that we're going to do here, and again, this is where it will really help us get a deeper understanding of this process by implementing it ourselves , the first thing that we're going to do is figure out how to loop through all the possible ways that we could split our data.

0:35It's pretty obvious when we have, let's say, an attribute that has a yes or no value, such as drinks coffee, for example, if you're looking at a data set of people, it's pretty easy to choose where to split it. There's only one place you could split it, and that is have all of the people whose answer is yes in one group, and all of the people whose answer is no in another group. However, as you can see, even just from this simple blob data that we generated here, the attributes can sometimes be more complex than this, in particular with numeric attributes, right, such as X and Y here, these have a range, and that means that there's a lot of places that we could draw vertical or horizontal lines here, right?

1:25So how do we decide, you know, what the possibilities even are? Well, when working with numeric data, it's pretty common to try the split along each of the specific values, right, each of the unique values in each dimension, right? So we might try splitting along the X axis at all the unique X values and along the Y axis at all the unique Y values. And this probably seems like a lot of potential splits.

1:57So in some cases, you know, you might want to reduce that number down a little bit, but in our case here, it's not going to be too difficult to actually implement that . And it's not going to be too computationally expensive to run it. So here's what this is going to look like. First of all, what we're going to do is we're going to define a function here, and we'll call that function something like find best split. Okay, and what this is going to do is it's going to take all of the X values as well as all of the Y labels, which we'll just call capital X and lowercase Y.

2:36And we're going to assume that this X, by the way, is a data frame. That'll just allow us to perform some nice operations that'll make it a little bit easier to implement this. So what we're going to do here is we're going to start off by looking through all of the potential columns that are in this data frame. Right? So in this case, with our blobs, that's pretty easy. That's just going to be the X and Y columns. But again, because we're assuming this is a data frame, we don't even have to know that much.

3:07We're going to do this before, and then we'll say feature, or you could call it attribute in X dot columns. What we want to do there is we simply want to determine how many different values there are in a specific column. Right? So if we're looking at this X axis feature, for example, how many different X axis values are there? Right? So we're going to have a lot of different points we might want to split our data set at.

3:40Now, again, data frames have a pretty nice way of doing this. And that is we can just say possible splits, we'll call this equals. And then what we're going to do is say X capital X, that is feature. Right? So we're looking at that column, and then we're going to say dot unique, right? You may recognize this method from a lot of other previous skills. This gives us all of the unique values that that column contains.

4:10All right. So now that we know all of the possible columns and values in those columns that we might want to split on, all we have to do is really test it out to see how each of those splits improves or worsens the entropy. So here's what this is going to look like. First of all, we'll try splitting by that value that we just got for that column. So what that's going to look like is we're going to say here, we'll just say four, and we'll call this threshold in possible splits.

4:46All right. So we're going to try out each of those possible split points in this column. What we're going to do is we'll start off by just dividing up the groups based on that threshold. So we'll call this group one, and we'll say equals, and we'll use another nice feature here of NumPy and pandas. We're just going to say X and use that feature. And we're going to test whether or not it's less than or equal to that threshold.

5:17Okay. And then we'll say group two, and that's just going to be sort of the opposite of that. So we'll say X feature is greater than whatever that threshold was. Cool. So now that we have our two groups, and by the way, this won't give us the groups themselves, we need to actually say X and then have that X feature thing in there. All right. So we'll say X, there we go, X feature, and then put that in square brackets. So now that we have our two groups, the next thing that we're going to do is we 're going to calculate the entropy of both of those groups and see what that gives us.

5:58So here's what that's going to look like. We're going to say entropy one, and I'll talk about the entropy equation in more detail a little bit later. But for now, we're just going to say something like get entropy, or we'll call that calculate entropy. Okay. And we're just going to pass group one into there, and then we'll say entropy two equals calculate entropy of group two.

6:30All right. So I'm going to just define this as a stub method here. We'll say define calculate entropy up here, and then we'll say just group. And for now I'm just going to return something like zero, right? Or here, we'll return something like one, perhaps. All right. Cool. So now that we've done that, right, we have the entropy of both of those groups . So now we need to combine those two entropies to get the full entropy of that division, right?

7:06So the way that this works is we basically just need to multiply the individual entropies by the size of each of those splits, right? If one group just has one item in it, we don't want that to have a huge influence on the overall entropy versus if the two groups are having even number roughly of elements. So here's what that's going to look like. We're just going to say, and we'll call this something like total entropy. And what that's going to look like is we're just going to say equals, and we'll get the relative sizes by saying group or a length of group one, that is, divided by the size of the total group.

7:47So that's just going to be the length of why there. And in fact, I think I did something wrong up here. I think this should be why and why because we want the labels instead of the actual rows themselves. All right. So we're going to, we have the relative size of group one. So we're going to multiply that by entropy one. All right. So entropy one. And then we're going to add that to the same thing, but for the second group. So I'm just going to copy this actually, and we'll paste it over here.

8:19And this is just going to be the size, the relative size that is of group two times the entropy of group two. That'll give us the total entropy. And what we want to do now is check to see how that total entropy compares to the other options that we have available. Right. So you can probably imagine that as we go through all of these columns and all the possible splits, some of those splits are going to be much better than others, right? A split that goes like right here is much better than a split that goes like right through here, right?

8:55That one doesn't really do much of anything for us. So what we need to do now is just compare the entropies across multiple features and possible splits within that feature to find the best one. And that's what we're going to do next as well as see how to actually calculate the entropy of a group in the first place. So I hope this has been informative for you, and I'd like to thank you for viewing. [BLANK_AUDIO]

0:00All right, so at this point, we're going through all of the columns and all of the possible split values that we might be able to use in each of those columns. So the next thing that we're going to need to do is compare the relative entrop ies of all of those different options to find out which one is the best one. Now really the starting point for making this happen is going to be to actually implement this calculate entropy function.

0:31And really the only thing you need to know about this is that there's a specific equation for determining the entropy of a given division. And that looks like this. Don't worry. I'll show you how to implement this in Python in just a minute if you don't understand the mathematical syntax. But the equation here is that entropy is equal to the negative sum from i equals 1 to however many groups we have.

1:03And then under that sum sign, we want to know how many samples or how many data points are in each group. So we want to know, you know, this PI thing is the proportion of data points belonging to whatever group i is. And then we multiply that proportion by log base two of PI. All right. So that's just the equation for entropy.

1:34Don't worry too much about that equation. Just know that by implementing this calculate entropy function with that equation, that will allow us to make the best decisions with regards to where to split our data set. So here's what this is going to look like. We're going to change this calculate entropy thing from return one, which obviously isn't what we want to. And we're going to say groups and counts equals, and then we're going to say NP dot unique.

2:07And we're going to say group, right. And this group thing here is a potential division. All right. And that should give us all of the unique groups there. But we also want the counts, which we can get nicely enough by saying return counts as a keyword arg equals true. All right. So at this point, we have the different groups that are in that division, as well as how many data points are in each group. So that's really all that we need in order to implement that equation there.

2:41So what we're going to do is we're just going to say probabilities equals, we 'll say counts divided by length of group. And then what we're going to do is we're going to say return negative and we'll use the NP dot sum function. That's very similar to Python's built in some function, but it's optimized for working with NumPy arrays. So we're going to say NP dot sum, and then we'll say probabilities times NP dot log two.

3:14And then we're going to say probabilities. And we'll say plus, and you need to do this in order to avoid calling log two on the value zero. We just need to add a very small value here, IE to the negative ninth there that will prevent that from happening. All right. Cool. So that's how we calculate the entropy. That's just the function that we need there, which implements that equation. So don't worry too much about the details there. If that doesn't make sense, that's honestly the least fun part of all of this.

3:45But now that we have that, we're actually calculating the entropy for those two groups and combining that into a total entropy. So this really becomes a pretty elementary programming exercise. We just need to keep track of what the minimum entropy is, and you know, what feature and split give us that. So here's what that's going to look like. We're going to say up here at the top of this function, we'll say something like best feature equals none. Right. We don't really know what the best feature is yet.

4:18Then we're going to say best threshold equals none because we don't know that either yet. And then we're going to say best entropy and we'll set that to infinity, but we 're going to convert that to a float here just so that, you know, because infinite entropy is obviously going to be the starting point because anything will be less than that. And then what we're going to do is we're going to say number of features and that's going to be equal to X dot shape. Oops, let's try that again.

4:48There we go. Index one. Right. That'll give us the number of features that are in this data set. Cool. So all we need to do now is after we've calculated the weighted entropy, right, or the total entropy here, we called it, we just need to compare that to the current best entropy and replace the best feature and best threshold if it's better, right, if it's less than the best entropy that we have currently. Here's what this is going to look like, we're going to say if total entropy, all right, is less than the current best entropy, then what we want to do is we want to say best entropy equals total entropy, right, we're replacing that former best number with the new best.

5:38And then we're going to say best feature equals feature. And then we'll say best threshold equals threshold. And that is all we need to do there. So at the end of this, after the for loop completes, we just want to return the best feature and the best threshold. Okay. So that should be all we need to do. Let's test this function out now, what we're going to do is we're going to just call find best split now on the blobs that we created up here.

6:15This will be super informative, but later on, we'll see how to apply this to a slightly more involved data set. But first what we need to do is we need to actually convert the positions here into a data frame. So what that's going to look like is this before we call find best split, we're going to need to say df equals pd dot data frame. And we do need to import pandas up here at the top. So let's just do that import pandas as pd.

6:45And if we go back down now, we're going to pass the positions in there. And we'll just have the columns that are in there. Here we'll say columns equals. And then what we'll do is we'll say, we'll say feature one. Right, that's going to be the X there. And we'll say feature two. And that should be all we need to do there. So now that we have that as a data frame, oh, and here we can say data target or df target rather, if we want to equals, and we can set the labels there if we want.

7:21So let's just let's actually start off by printing that data frame out. We'll just say df dot head there. And sure enough, that gives us a lot of the points from our from our plot up here, along with the groups that they belong to. So we can call this groups instead if we wanted to, I suppose we'll just we'll just say group. There we go. And that gives us all we need. Cool. So let's try this out now. Now that we have that as an actual data frame, we're going to say find best split.

7:54And we just need to pass the data frame itself there. Okay, that's going to be the positions. And then we'll have the labels as the Y argument there. And actually, we don't want to add this group there because we don't want our algorithm there to try and split on this group column, which it will if we don't remove that. So we'll just remove that like so leave the labels intact. So let's just try running that. And sure enough, we see that feature two apparently has the best entropy when we draw the line at negative 0.2.

8:30So let's actually see what that means. We're going to go up to here. And that is around here ish, I believe, right? So feature two is this axis here. So it's saying that the best place we can divide it is right about there, which I would I think I would agree with. All right. Well, whether I agree with it or not, the math has spoken. So anyway, that is a function for finding the best split. So you might wonder where we're going to be going from here.

9:01Well, the next thing that we need to actually do is run this find best split thing repeatedly on smaller and smaller groups. So we know the best first split here is this line, but this leaves us now with two different regions that we have to determine how to split up further, right? So in other words, if we draw the line there, that gives us a place in the decision tree where we know that if something is greater than this line, it belongs to group A, just based on the, you know, just based on the current data that we have here.

9:41But we still need to try and divide up the B and C groups here by drawing another line, right? So that's the next step in this algorithm is actually finding what that next line is and keeping track of those usually in something like a Python list. So that's what we're going to see how to do next. So I hope this has been informative for you, and I'd like to thank you for viewing.

0:00All right, so now that we've finished this find best split function, as I said previously, the next thing we need to do is actually repeatedly apply this find best split function to smaller and smaller groups until we reach the point where we can successfully identify each and every point in this graph. Now in practice, that's probably not the best idea, right? As you can see, there's quite a bit of overlap here.

0:30So in order to really create a decision tree that would identify each of the points in this graph correctly, we would need to do quite a bit of what's known as over fitting, right? In other words, you'd start to see that the lines would be drawn sort of like this here in a way that's not really very useful and isn't very general, right? We want it to be kind of like, you know, maybe like that at best where it separates them kind of crudely, but simply cool.

1:02So anyway, here's what this is going to look like. We just need to create a recursive function here and we're going to call this something like we'll call it create classification tree. But what this is going to do is it's going to take the data that we want to create classification tree for. So this is going to just take x and y as we had before as arguments. And what it's going to do is it's going to again, recursively call this find best split function in order to find splits on smaller and smaller groups.

1:44However, before we can do that, we need to actually decide what we want this decision tree to look like. In theory, it would be possible to represent this thing fully programmatically, right? As a series of functions or something like that. But what we want here really is something reusable so that we could use this this classification tree over and over again after a single training. So what I'm going to recommend here is that we have our classification tree look a little bit like this.

2:15We're going to have it represented by a nested Python dictionary that has type entries. And this will tell us whether this current part of the tree is a leaf which contains an answer to what group that element should be in. Right. So we might have type leaf. And in that case, we would have another entry called something like group. And that might be, you know, a, for example, or we might have type equal to, we 'll just call it node.

2:51I suppose you could call it branch as well. This is, you know, a part of the decision tree where we have another question that we want to ask or another split that we want to perform. So in that case, that might look like this, we might have type node. But in this case, we'll have the feature we want to test, right? So we'll call that feature here and that'll contain the name of the feature. So maybe X or Y or feature one or whatever your feature is called.

3:22And then we'll have the threshold there as well. So maybe feature one, right, as we just got here, or we'll say feature two, the threshold for that is negative zero point two, seven, nine, blah, blah, blah. Okay. And finally, depending on the answer to that, we'll want to have a left and right, you know, subtree. So we'll say left and that will actually be a nested structure containing either a leaf or another node like this. All right.

3:53So that's how we're going to represent our tree here. So we just need to make it so that this function will construct that for us using this find best split thing over and over again. So here's what this is going to look like. We're going to start off with some of the so-called base cases or stopping points in our recursion. So the first one here is if all of the items in the group are the same, then we want to return a leaf node because there's no way we can possibly split that anymore, right?

4:26So we'll say if and the way we test this is by checking the length of NP dot unique and we're going to test why here, those are all the possible labels in this sub group. If that's equal to one, then we want to return a type leaf, right? Because we know, oops, leaf not left, because we know what class this should belong to. So we'll say group and that will be whatever that group happens to be.

4:56So that's going to be why and in order to actually get the label, we need to say dot I, loc, zero in square brackets. So all right. So that's really the main stopping point for our recursion here. What we're going to do next is we're going to say feature and threshold and we 're going to use this find best split function that we created up above in order to split it further. So here's what that's going to look like. We'll say feature and threshold equals find best split.

5:27And that's going to be X and Y there that we're going to pass through. And then we're going to take that feature and threshold and actually split the data that we have here, according to it. So this is going to look pretty familiar to what we did up here when we did something like this, we're going to say, and then we'll actually call this left mask and right mask, because we'll be using this a few times here, we'll say X feature less than or equal to threshold.

5:57All right. So we're getting the sort of the left side of that split. And then we'll get the right side of that split by saying right mask equals X feature is greater than threshold. And now that we have those two things, we're going to build the sub trees, right? We know that because this stopping point wasn't true, that we actually need to have a node that produces a split. So what we're going to do is we're going to say left tree equals, and we're going to call this create classification tree thing recursively.

6:33So create classification tree. And we're going to call that with X. And we're going to pass the left mask to it there. And then we're going to say Y left mask to it. So we're just passing the sort of X's and Y's for the subgroup here. And then we're going to do the same thing for our right tree. We're just going to say create classification. There we go tree. And that's going to be X right mask and Y right mask.

7:10And finally, the actual Python dictionary that we want to create for this, we 're going to say return. The type for that will be node. And then we'll say feature, that's going to be whatever feature we decided was the best one. We also need the threshold. Just like I said earlier, that's going to be whatever threshold we decided on there. And then we're going to have the left and right sub trees there by saying left, that's going to be our left tree that we created recursively.

7:44And right, that's going to be our right tree that we created recursively. And that should be all we need to do there. So that should create our tree for us recursively. Let's give this thing a try instead of saying find best split for our data frame and labels, we're going to say create classification tree and call that on there. Let's give this a try. And oops, it looks like something was wrong there. So what I'm going to do, we need to actually convert the labels here into a data frame.

8:15What I'm actually going to do is I'm going to say, D F and we'll say labels like that. And then I'm going to say equals labels. All right, that will create a data frame column for those. And then I'm just going to say labels equals D F labels. And finally, down here, we're just going to pass that data frame without that labels column by saying D F dot drop. And we're going to drop the labels column there when we pass it in again so that it doesn't try and split it on that labels column, which would kind of ruin the point.

8:52So let's try that again. And oops, I need to add axis equals one to this dot drop thing there. Okay, that should work better. And sure enough, we see that that creates our decision tree. Now, I'm only going to walk through the first few decisions here, right? First of all, we see that feature two is up at the top. So that's the first line that's going to be drawn there. And recall that that gives us a line right about there. In fact, it goes basically through this uppermost orange point.

9:25Since again, we use the actual y values of real points in order to decide where to do the split. But let's take a look at the next one here. We see that the next split is on feature two as well. And that is negative six point zero five. So let's just go take a look at where that is exactly. But sure enough, we see that that's right along here. So it's trying to draw a horizontal line through groups B and C in order to divide them.

9:57That's just mathematically what the best split was. All right, so you can continue on with that. And I encourage you to take a look at that full decision tree that was generated. But one thing that you might notice is that really the, oh, and here you see that the, you know, the, the rightmost thing from the first node says that it belongs to group zero, which is group A. Because again, once you've drawn that horizontal line that divides A from B and C, you know that anything above that is in group A according to the decision tree or classification tree that we've created.

10:33As I was saying though, you'll notice that this decision tree is actually quite deep. And it's because of this that many times people will set a maximum depth on the decision tree that can be constructed, right? That helps us to avoid overfitting. And that's something that I'm actually going to give you in the challenge. So I hope this has been informative for you and I'd like to thank you for viewing. [BLANK_AUDIO]

0:00All right, so now that we've seen the basics of recursively building a classification tree It's time for you to do a challenge that will significantly improve in many cases the performance of the classification tree that you build and that is as I said previously adding a maximum depth parameter to this create classification tree Function so what we would like to be able to do is specify how deep it's possible for the tree to go So that you don't get ridiculous little Choices like what we had down here. So let me try that again. There we go You know if you look at some of these things down here, you can see that it's really at the point of identifying single points in Some sort of boundary right and usually while things like that will lead to better accuracy on the training set They won't actually improve the real world performance of the tree that you build So we usually want to set some sort of maximum depth so that we don't go that far now Your task here is going to be to Stop at a maximum depth and in that case what you'll want to do is simply return

1:15whatever the majority is of the items that are in that group right so in other words if you're left with a group that has a lot of Elements from both B and C in it right if you have a group here like this Then you're just going to want to return whatever Group has the largest number of data points in that area right so in that case or in this case here That would be C right because there's four C's and three B's So that's really all you need to do is add that case and you should be able to add another Argument or parameter to this function as well you can call that something like max depth if you want to and Well, that is your challenge so feel free to give this a try Maybe give it you know five to ten minutes to complete and once you've done that you can feel free to move on to the next video We're I'll walk you through the solution So I hope this has been informative for you, and I'd like to thank you for viewing [BLANK_AUDIO]

0:00All right, well, hopefully you gave this challenge a try. Let's take a look at the solution. So the solution here was pretty straightforward. All we had to do, again, was add this little max depth thing, or max depth argument to our function. So here's what that looked like. I just said max depth. Oops, there we go. And I actually set the default value for that keyword arg to none so that we could say, we could just leave that off if we didn't want to have a maximum depth.

0:32From there, all we really had to do was add another stopping case to our recursive create classification tree function. And that looked like this. We could just say if max depth is not none and then-- actually, we had to add another argument here called depth. And I said, and depth is greater than or equal to max depth. Well, in that case, once we've reached the maximum depth, what we want to do is just return a leaf node with whatever or whatever the most common label is for the elements in that group.

1:16So here's what that's going to look like. We're just going to say return type leaf. And then we'll say group. And that's going to be, in this case, y dot mode index 0. That mode thing gets the most common thing in there. But you could have done that with a for loop as well, I suppose, if you wanted. Cool. So that is all you really needed to do there, except, of course, when we recurse, we want to make sure that we're passing that depth argument at the right place.

1:47So let's actually flip these things around. We'll say x, y, and depth. And we'll set the initial value for that to 0. Since we're starting at depth 0, and then we'll increment that as we recurse. And then all we need to do is when we recurse down here, we're just going to say depth plus 1 for that depth argument. So depth plus 1 there as well. And then we need to pass that maximum depth for maximum depth. So we'll say max depth, max depth.

2:19And that should be all we need to do there. So let's maybe try setting a maximum depth here of, say, 3 or 4, something fairly reasonable. And we can do that by saying create classification tree. We'll start off by saying max depth. And we're going to set that equal to-- we'll try 4 here. Actually, we'll try 3, just so that it's easier to see. So let's try running this. And what we should see is that sure enough, much smaller classification tree, you can adjust this too.

2:51If you wanted to set that to max depth equals 2, which actually would get you pretty good performance. That would be where we have a tree that's only two decisions deep. And you could also set this to 4 if you wanted to. And that would give you a significantly larger tree. So feel free to play around with that. But that is the solution to the challenge. And in case you're wondering how to take this classification tree that we've created and actually apply it to some data, I was going to leave this up to you as a further challenge.

3:23But I'll show you it because it's pretty straightforward. What we're going to do is we're just going to say something like define apply tree. That's going to take a classification tree as an argument, as well as a data point that we want to apply it to. And what this is going to look like is we're simply going to say if tree type is equal to leaf. Well, in that case, we just want to return whatever the group was for that. So we'll say return tree group.

3:56Otherwise, we want to test the threshold. So we're going to say if data point, and then we'll say tree, and we're going to have whatever the feature is here. So we'll say feature like so is less than or equal to the corresponding threshold. So we're going to say tree threshold there. Then in that case, what we're going to do is we're going to return and we're actually going to call this thing recursively by saying apply tree. And we're going to call it on the data point with that subtree by saying tree.

4:30And this would be the left subtree there. And we can do the same thing-- oops, here I forgot a closing parentheses there. And we can do the same thing here for the right subtree. We just need to say if data point-- and actually, you could just say else here. That would be easier, I suppose. So we'll say else. In that case, we would want to apply the right subtree. And that should be all you need to do. So I'll leave it up to you to test this thing out and make any further changes or adjustments to it. But that is the basic solution to the challenge.

5:03So I hope this has been informative for you, and I'd like to thank you for viewing.