Monitor & Measure Resources Usage

0:00[MUSIC PLAYING] Hey, guys, and welcome to the LPIC-2 course here on CBT Nuggets. So before we begin actually digging into the details of what we actually have to learn for this examination, let's just briefly take a word about the exam itself. Now, the first thing to state about this examination is that you will have heard of the LPIC-1 examination if you are interested in taking the LPIC-2 exam no doubt.

0:38That is because the LPIC-2 actually builds upon the LPIC-1 exam. Pretty much what you learn in LPIC-1 is a more foundational level of information, and then you take that information and build upon it on LPIC-2. Now, the LPIC-2 examination ultimately attempts to validate the candidate's ability to administer small to medium networks. Now, when you're getting to administer medium size networks, for example, you really do have to be fairly comfortable with the Linux system.

1:14Now, one thing to note about the LPIC-2 examination is that any candidates seeking to take this exam must actually have an LPIC-1 certification-- an active certification. So that means that your LPIC-1 certification cannot be expired. I do believe the validity period for the LPIC-1 certification is five years. So within five years of passing the LPIC-1 exam, you will be able to take the LPIC-2 exam.

1:44Now, another thing to note is that just like with the LPIC-1 exam, the LPIC-2 certification actually is comprised of two separate exams. The first exam is known as the 201 exam, and the second exam, this is the 202 exam. Now, as it transpires, what you can do is take these examinations completely separately. So you could just focus on the content of the 201 exam, pass that, and then focus your attention on the 202.

2:18This is often the recommended way to undertake the LPIC-2 examination. And just like what the LPIC-1 examination, when you happen to pass your LPIC-2 exam, you will have this certification valid for five years unless, of course, you happen to gain an even higher level of certification-- say, for example, an LPIC-3 exam. Now, one thing to really note about the LPIC-2 examination is that there is going to be such a common theme with the contents of what you learned within LPIC-1.

2:52Like I say, it does heavily build upon that knowledge. That means that you can expect to see many of the same tools that you've used within LPIC-1 again revisited in LPIC-2, and very often actually expanded upon-- i.e., we dig a little bit deeper into these tools. But as a high level overview, some of the things that you should be expected to be able to do when you go to sit the LPIC-2 certification exam is that you should be able to perform fairly advanced system administration tasks.

3:26And that should be things like relating to the Arch Linux kernel as well as the system startup and its maintenance. Now, not only that, you should also be able to perform fairly advanced networking and authentication as well as, by the way, being able to handle and manage particular security features such as dealing with firewalls or Virtual Private Networks, otherwise known as VPNs. Now, with respect to talking about networking, some of the things that you can expect to be able to have to know for this examination is being able to be comfortable with configuring things like DHCP-- this is the Dynamic Host Configuration Protocol.

4:11Now, again, if you happen to be a little bit hazy on this from what you learned in the LPIC-1 examination, do not worry. You will get to cover the concepts once again and, like I say, dive a little bit deeper. But we'll get to see things like DHCP, like I say, as well as DNS, which is Domain Name System which we use to resolve our domain names to particular IP addresses, as well as dealing with things such as Secure Shell for secure, encrypted remote management of servers.

4:42As well as dealing with things like web servers and fail servers, using things such as FTP-- the File Transfer Protocol-- as well as dealing with email delivery and things such as Samba. And as always, with Linux, we also want to dip our toe into a little bit of automated management-- things like being able to use scripting to help us manage our systems. So really, what I'm saying is that the LPIC-2 examination really is a fun one. There's a lot of things it again too.

5:12And if you have recently passed the LPIC-1 examination, so much of what we're going to cover should be fresh in your head. And that will make it much, much easier for you to take that fresh knowledge and build upon it within this course in LPIC-2. But like I say, if you happen to be a little bit rusty on that information, do not worry. We will absolutely cover all the essentials and core fundamentals we are bringing along with us from the LPIC-1, and make sure we are definitely up to speed and where we need to be.

5:43So that really is our little introduction into the LPIC-2 certification exam. How about now we actually dig in to some of the details of what we have to know? The first subject I want to talk to you about is all about monitoring CPU and memory utilization. And, well, that is what we're going to be covering in the very next Nugget. So I hope this has been informative for you, and I'd like to thank you for viewing.

0:12Hey, guys. And welcome back. So with the LPIC-2 examination, naturally it follows we do have to have access to some type of Linux-based system. In my case, I actually have a Windows machine. But the way I can access a Linux machine is that I just happen to use some type of virtualization software such as VMware, or in my case right here, VirtualBox. So within my own VirtualBox, I have a whole ton of different Linux distributions. And this case right now I happen to be using Xubuntu.

0:45Now if you were following along from the LPIC-1 examination, I actually went through how we could install things like VMware and VirtualBox. And how we could install within those tools things like Xubuntu or Red Hat Rocky Linux. So what I'm saying is that if you don't have a virtual machine up and running, then I definitely would recommend you do that just now to make sure you're up and running. My preference here is to be using the Xubuntu Distribution, which is a lightweight version of the Ubuntu Distribution.

1:17But if you would prefer to use something like, say, Ubuntu itself, then that is absolutely OK. So with that little word at the beginning, just to get us up and running, now let's begin and log into my system. So right here I have my virtual machine. What I'm going to do is I'm going to log in to this virtual machine. And here I am right here. So this should be a familiar sight. I have my Linux Terminal. In fact, let me just open this up afresh. So we can see here, just go with the Terminal Emulator, and this is all I'm doing.

1:47So let's begin talking about our subject then. So what we've planned to do is to talk about how we can monitor CPU and memory utilization. Now why would that be important? Well, think about it. What would be the number one complaint that your users would happen to mention when looking for help with their computers? Pretty much-- and I feel fairly safe in saying this-- is that people are probably going to be complaining that for whatever reason, their computer, their system, their network happens to be slow.

2:20And nobody likes a slow, sluggish system. It just really brings down productivity. This is something that we as system administrators would want to be able to solve quite quickly. So very often what you would have to do would be to identify the actual Bottleneck going on here. So what do I mean when I say the Bottleneck? Well, I'm talking about the part of the system, the component that is actually slowing things down. So you might just have, let's see a computer here, and you're connected to a router or a router, if you prefer.

2:54And that connects to your ISP, which is a massive network. And then you can go to then internet. Now this is obviously a masterpiece of artwork right here. Clearly my drawing skills have not really improved. So if we happen to experience some real sluggish behavior on our machine right here, we would want to be able to determine is there something, because the server we're reaching over the internet is very slow. Or maybe your ISP is having some issues, maybe our local router is experiencing some issues, maybe there's too many users within the local network actually using this router and overloading it.

3:33Or perhaps the problem lies directly on our system itself. Maybe we are taxing the CPU too much, maybe we are taxing the memory too much. So ultimately this is something we would want to be able to investigate. Now, we will actually look at things such as network input, outputs, and how we can test such things. And that may be, indeed, it is our local network itself that is slow. But let's just talk from now about our own system, our own local computer. Let's dive in and try to monitor the CPU usage, as well as the memory usage.

4:08Now, check this out, what I'm going to do here-- so let me just full screen this for now. Now the first thing I will say is that on this little virtual machine, I'm not really taxing my computer too hard right now. So in order to be able to simulate such problems whereby my memory is being really harshly used or abused or my CPU is really overloaded, what I want to do is to install the stress package. This is going to allow to simulate such conditions. So what I'll do is I will say sudo apt install stress.

4:39Now if I hit Enter, I'll have to type in my administrator password, and hit Enter again. And this package is now going to begin installing. Cool. So I just wanted to get that installed on the system because we're going to be using it very, very shortly. But before we do that, one tool I want to talk to you about straightaway is one called VM Start. This is going to allow us to report our virtual memory statistics. And this will give us information about processes, about memory, about paging CPU activity, so on and so forth.

5:12So what I'll do is I'll clear the screen, and I will just issue the command vmstart. If I hit Enter here, we can actually see some information right now. And at the very top, we can see our headings, we can see processes, we can see memory, we can see information relating to swap, as well as input, output system, and our CPU. If we check out the memory here, we can see our swap memory, as well as our free memory, our buffer and our cache.

5:43So when we see swap memory, what we're talking about is when the hard drive is being used as a source of memory. Now this is a concept we did talk about in the LPIC-1 examination. But if you don't remember, what happens when we use so much memory, i.e. we have so much demands on a RAM, that a RAM is fully exerted. What the system could do in that case, it could just crash if you try to open another program, because there is no more memory left.

6:14But, really, that is not a good solution. Instead of crashing, what we can do is use this swap memory. So this is going to allow us to temporarily dump that information, which would automatically be held in RAM, and use the hard drive instead. Now really, this is not a good sign for your system. If you happen to be seeing your swap memory being used, this tends to mean that your system is being taxed quite heavily when it comes to memory. I.e. There are some conditions whereby you're actually using more memory than you do have available on your system.

6:50This is something to watch out for. But we can also see information relating to our free memory, as well as memory cache. Now another command we could use to get similar outputs is the free command. So what I'll do is I'll say free. And, again, we're seeing similar information. We can see our memory that is used, how much memory is free, as well as what is available for our cache. Similarly, we can see our swap memory, which is coming from a hard drive. And we can actually see how much swap memory has been used.

7:21Now, if we want to happen to see this information, perhaps, in a little bit more readable format, we could do free -m to get these values and megabytes. This can make things a little bit more readable. But, honestly, one of my favorite commands to use, with respect to monitoring things such as CPU utilization and memory utilization, is the top command. This is going to allow you to see your running processes, and even more. So what I'll do is I'll just clear my screen, and just use the command top.

7:54So check this out. We actually get a lot of cool information right here. We can see the number of tasks we have, 176, two are running, that has now changed to one on the fly. We have 175 sleeping, and so on. Other things we can see, though, are these statistics right here. Check this out right there. We can see our CPU percentage that is being used, as well as the percentage of memory that is actually being used. And we can actually correlate that with a particular process.

8:25So the process ID on the left, we can actually correlate that to the CPU usage or the memory usage. And what it can actually do here is use the greater than sign, which is going to allow me to sort by the memory that is being used. You see that the very top, you see the process with the most memory or, rather, that is demanding the most memory, should I say, which happens to be update-manager. And then we have xorg, and then xfdesktop. And if I press the less than symbol, that's going to flip once again.

8:59So we're actually going to see it by CPU utilization, the top being the process using the most CPU. So the top command really is very useful. And the respect that it allows you to drill in and see the processes themselves, that happen to be swallowing your CPU and swallowing your memory if they happen to be getting taxed really quite heavily. So check this out right now, I have the top command running. What I'm going to do is I'm going to actually create a new terminal. So I'll just open another terminal window right here for now.

9:31And actually let me just temporarily full screen this. And what I will say is I will say stress --help. This is the till we just happen to install. So we can see here some information relating to how we can actually use this tool. Say, for example, I want to put some stress on the CPU. I can use the -c flag or --cpu. And this is going to spawn some CPU processes. So check this out then, I should clear the screen. I will say stress -c, and then 1.

10:03I.e. Spin up one CPU process. And hit Enter, so we're now dispatching hogs, as it says. We are going to be increasing the CPU demands. If I go back to my other terminal right here, notice the CPU right here is being utilized at 100%. You see that at the very top, CPU. And we can tell straight away that it happens to be the process ID of 14603. And the command is actually using all the CPU as the stress command. Now you might not be able to hear this because of the microphone filters that I do happen to be using, but my computer is beginning to work.

10:40That is the fans coming on because the demands on the computer is beginning to heat this computer up. So right now what I'll do is I'll go back to my other terminal, press ControL-C to stop the stress command. And if I go back to top, notice we are no longer utilizing 100% of the CPU. Now if I wanted to simulate stressing some of the memory, what I could do is I could go back here. And, again, if we look at the man page, or rather, not even the man page, just go back to the help page, rather, we can use the -m flag, which will allow us to tax our memory.

11:17So let's try this then. So I'll say clear, and I'll see stress -m, let's just give it a value of, maybe, say 4. So we're running 4 VM. If I go back to the top output, right now we are seeing what is consuming the most CPU at the very top, which is this process right here. But we can, again, press the greater than sign. So that would be the right arrow in my case. Now we are actually seeing what is causing or consuming the most memory at the top. So in this case, it is this process for now, consuming 9.9%.

11:51And it has just changed to 7.5. This is obviously very fluid output and always changing in real time. But, nevertheless, you're always going to see, at least in this view, what is consuming the most memory in real time. In this case right now we can see it as the stress command. So I'll go back here, I'll press Control-C, clear the screen. And, in fact-- let me just close this terminal. So as we can see here, we do have different tools available to us that we can use to interrogate our CPU usage, our memory usage. Things like the free command, the VM stack command, as well as my favorite, the top command.

12:27And it's using these types of tools that is going to allow you as a Linux administrator to be able to diagnose when you have a CPU problem or a memory problem. And as you can see here, particularly from the top command, you can actually identify the process that is actually causing that problem or contributing to that problem. And you can take the appropriate action needed to begin to stabilize your system. So that is all for our little introduction into monitoring CPU and memory. The next thing I want to talk to you about is monitoring desk and put and outputs.

13:02And, well, that is what we're going to be talking about in the very next Nugget. So I hope this has been informative for you, and I'd like to thank you for viewing.

0:12Hey, guys. And welcome back. So in the previous Nugget, we explored some solutions that we could use in order to monitor our CPU utilization, as well as our memory utilization. Now, in this Nugget right here, we're going to be looking at some additional tools that we can use with respect to monitoring. And we'll also focus on Disk I/O, i.e. Disk Input, Output monitoring. Now, the first command which I just want to touch upon is the uptime command.

0:43So if we go into the man page here, we can see here, this is actually going to allow us to see how long our system has been running. It's also going to tell us our load averages. And we're going to see three different values within our output with the uptime command, and these three different values are going to correlate to three different time spans. The first one is 1 minute, the second one is 5 minutes, and the last value you correlates through the system load average over the last 15 minutes.

1:18Now, we're actually talking about the system load averages right here, it says here this is the average number of processes that are either runnable or in uninterruptible state. Now ultimately what we're really talking about here is CPU usage. So a runnable state is when the CPU is being used or a process is waiting to use a CPU. Now, with respect to an uninterruptible state, this is when the process is waiting for input, output access.

1:52I.e. The hard--disk may be fully utilized, therefore, a particular process has to wait before it can access the disk in order to execute a particular task. Now, these load averages we are seeing within this uptime command relates to this type of information right here. Now, one thing to note about the output with respect to the uptime command-- and let me just press Q to clear my screen. Now, when we're talking about our three different values of 1 minute, 5 minutes, and 15 minutes, we're actually going to see the values denoted a particular way.

2:29Now if we happen to see the value 0.5, what this tells us is that the CPU is ultimately half used, OK. Kind of makes sense, 0.5, half used. If the CPU is at the value of 1, that means that it's being fully utilized. Whereas if it's at 0.-- let's say maybe 0.25, then only a quarter of the CPU is being used. And, again, you can see this at different time intervals.

3:00Now, a few important points to note is that you may actually see a value such as 1.5. This is going to tell you that your CPU really is being taxed. Because whilst it may not make sense that how can 1.5 of the CPU being used if 1 is the entire CPU being used. What this is actually indicating to us is that some processes are having to wait. And as such, the CPU is in effect being overloaded. So if we get a value greater than 1, well, we may have problems to address.

3:35Now, one thing I just want to point out here is that the values I'm giving you relates to if you happen to have one CPU available. If you happen to have, let's say for example, four CPUs on your system, this actually changes the values. So think about this, if I happen to see the value from the uptime command, and the value is at 1. So 1.0, you may automatically think that the CPU is at full utilization.

4:05In actuality, what this means in the context of having four CPUs, only 25% of the CPU is being used. Ultimately, what you're doing is you're dividing this number here by the number of CPU. So it's 1 divided by 4 CPUs is actually 0.25. So only 25% is being used. Again, if we happen to see something like, say, 0.5 being used over 4 CPUs, the CPUs are actually not at half utilization.

4:38Instead it's going to be 0.5 divided by 4, instead. This is only going to come out as 12.5% utilization. Now if we actually happen to run this command here, uptime, we can see these values right here, 0.09, 0.07, and 0.01. Again, these are values over 1 minute, in this case here, 5 minutes in this case, and 15 minutes in this case. So the uptime command really is a valuable command that we can use to assess these types of values.

5:14It really does give us insight into our system. Now the next thing I want to talk to you about is relating to I/O Wait, i.e. Input, Outputs, Wait. So where am I going with this? So what I/O Wait actually is, it's ultimately a percentage of CPU actions that are awaiting for disk access. So like I happen to say, when we talk about I/O waiting, if you had, maybe say, a program, perhaps might be a browser, and your system is already using 100% of the disk, then your browser, which needs disk access in order to perform its actual task, this process, i.e. the process being used for the browser opening, this is actually going to go into a state of I/O Blocked.

6:05Input, output has been blocked. In real terms, what this means for us as the user is that the system is overloaded, at least in respect to the disk utilization. So when you try to open that browser, you double click that icon, the process may actually hang, i.e. the browser is just not going to pop up on your screen. You may be waiting a while because the process really does need to wait until the resource is free. And here's the thing, the actual percentage of CPU processes that have to wait, that is referred to as the I/O wait percentage.

6:44Now one of the ways we can interrogate this type of information comes from my favorite command, we just talked about it in the previous Nugget. That happens to be the top command. Let's try and check this out and see the information that we can actually derive. So what I'll do here is I'll just clear my screen right now, and I'll use that good old command, top. So check this out. There is a particular piece of information we can focus in relating to our I/O wait. If you can spot it here, it is this one right here, which is-- in fact, let me get my drawing pen because it's not showing quite so well.

7:20This one right here, wa. So this number is ultimately a percentage, and right now it is at 0.0. That is good, meaning that we don't have any CPU processes actually waiting. So we know we actually have a tool downloaded on our system which we can use to stress our system, to simulate this type of problem. So that commands is going to be the stress command. So what I will do actually is I'll go to my options, and I will open a new terminal.

7:53So we have a new terminal right here, what I'm going to do is use the stress command. If I happen to say man stress, we can see we have the option of -d or --hdd. This is going to spawn the number of workers spearing on rates. Basically this is going to allow us to simulate a lot of disk utilization. So what I'll do here then is I'll press Q to quite, clear the screen, and obviously stress --hdd. And I'll choose the number 14, one hard-drive process.

8:26If I hit enter-- in fact, before I actually do this, let me minimize this-- keep an eye on this value here, the wa value, OK. So if I hit Enter now, that is going to actually spin up a process. And you can automatically see, or rather, instantly see, should I say, the waiting value, the percentages are actually beginning to speak. That is due to the fact that we're actually stressing our hard-disk, so to speak. We are really using that resource. And as such, if I actually try to go and maybe say open a browser, click here, it may take a little bit of time to actually open up.

9:03It's not going to be quite so snappy. That is because a lot of my processes are actually having to wait right now. They're not getting access to the disk, therefore they can't operate as they ordinarily would. So right now we can see here we're still waiting on this browser, and waiting and waiting and waiting. Sadly, our browser is kind of at the back of the queue, it can't access this resource as being hogged. So if I actually just stop the stress by pressing Control-C, now straight away the browser is going to load up because we are no longer overloading the hard-disk.

9:37Therefore, the process could access the resource, and lo and behold, we can open up the browser. But, again, think about what we're actually seeing here. In this case, we had a problem whereby our browser was very, very slow to load. If we just happened to be looking at the top commands, we would have seen that the I/O wait value was around 80% or 90%. And we could actually tell or infer that the hard-disk was being particularly tax. So straight away with our troubleshooting mind on, we would have quite a good idea of where to go to solve that problem, or the type of action we would need to take in order to solve that problem.

10:16Now, honestly, as helpful as top is and as very helpful, with respect to input, output, it is a little bit limited. So what I want to do is to actually show you some additional tools that we can use. So if I press QT quits-- I'll just close this terminal for now, and I'll also close my browser for now. I'll clear the screen. And what I'm going to do is install something called iotop. This is like top but up for input, output. So I'll say sudo apt install io-- if we can get it-- iotop.

10:49Hit Enter, type in the password, this is going to begin installation. Just give a little minute or so. Now if I try to run the command iotop, watch what happens, operation not permitted. You can probably guess where I'm going with this. The way to actually use this command requires superuser privileges. So I'll clear the screen, and I'll say sudo iotop, and I will hit Enter. So what we're seeing is top like information. But we are really focusing on this right here, the IO utilization, and what processes happen to be using a lot of disk resources.

11:24So what I can actually do here is I can go and open up a new terminal. And we'll do a similar thing to what we saw before, we'll use the stress command to actually put some stress on the disk. So I'll say stress --hdd, and I'll just say 1. And now the stress command should actually be putting some pretty hard work on the IO. But look at this, we're actually getting to see the input, output, and getting to see what is taking up a lot of that. So we can see the stress hdd1 taking up quite a lot of the resources.

11:55And, again, if we go back here, I just press Control-C to stop this. Instantly that's going to come back down, and the iotop command shows us that. So a really valuable tool right here, as opposed to just using the regular top command. Now, what I will do here is I'll press Q to quite this, clear the screen, I'll now say sudo apt install. And the package I want to install is called sysstat. So if I hit Enter here, this is going to install the sysstat package. Just give it a little minute or so.

12:27So now I should have access to a command called iostart. If I go into the page here, this is going to report CPU statistics for input, output. So this is going to give us, again, just like with iotop, IO related information and statistics. Now this is, in my opinion, not quite as good as iotop. But, nevertheless, the examination does mention iostart within the exam objectives. So it does actually make sense that we are actually familiar with it.

12:58So I'll clear the screen, and I'll just say iostat. And what this is ultimately doing is giving us a snapshot. So we're getting to see a lot of information about data being written to disk and read from disk. This is ultimately what is taxing the particular disk, and we can see the iowait percentage right here, which in this case is 1.77, quite low. But like I say, this as really just a snapshot. Whereas if you recall, the iotop command was giving us real time updates, and much more expansive. So I can clear the screen.

13:29Another command we want to be able to be familiar with, again, not quite as good as just using iotop, but the exam does require we are familiar. If we go in to the man page for the sar command, this is going to allow you to collect, report, or save system activity information. So you can scroll right through this man page, and you will see plenty of options and plenty descriptions of what this command can actually do. That is quite a lot here. For now I'll press Q. What I'm going to do is I'm just going to say sar 1, and then 5.

14:03So what this command is going to do, it's going to allow us to take five different data points, one second apart. So this is quite good to check how things progress over a time frame, in this case here, 5 seconds, because we have five recordings every single second. So it's a little bit better of a snapshot than iostart gives us. We have more data points, for example, but it's not quite, like I say, as good as iotop, giving us that streaming information and continually updating.

14:34But, nevertheless, let's hit Enter and see how we go. We're going to get five pieces of information. So we start at 27 seconds, we get our first data point at 28. So that's number 1, then 2, 3, 4, and 5. So 5 data points one second apart, we can see the percentage. So we get statistics over a whole bunch of different types of information, such as system or idle. But really the one we are focusing on right now is iowait. Now as it transpires, we are not actually taxing the io, so the values right here all happen to be 0.

15:10But as you can expect, we could, like we say, actually stress the --hdd once again, and we can rerun this command by hitting Enter. We get five different values right here. And at the very bottom, we get the average of those five values. And, again, we could change these values to be maybe 2 and 3, meaning we will get three data points two seconds apart. If we hit Enter, we can see here the actual times. The first one, on 00, the next one on 02, the next one on 04.

15:43And at the bottom, we get the average of those three over that time period. So, again, another really cool command to be aware of. Now what I'll do here is I'll just press Control-C to stop this stress test. In fact, let me just close this terminal right here, and I'll clear the screen. Now, another command I want to show you is one called LSOF. This is going to allow us to last our open file. That should be one you will recall from the LPIC-1 examination, no doubt. And like I say, it's going to allow us to track the open files on our system, being all the open files.

16:17Now, if we just happen to use this command on its own LSOF, we're going to be bombarded with an absolute megaton of information, just far too much to actually read. But we can do things such as filtering to make this information much more palatable. So what I'll do actually-- let me just quickly open terminal, and we'll just rerun that stress command. What I can actually do here is I can run this command, OK, that would just show me absolutely everything. But what if I happen to grep for stress itself?

16:50Now, we can actually see here what files are being opened by the stress program. So if we happen to note a particular process or a particular command is causing a lot of IO issues, we can actually correlate what actual files are being opened or being used by that particular command or process. So this just gives us more granular information that we can use to interrogate what is exactly going on. Now, one point I should actually add-- and, in fact, again, let me just stop this right here.

17:22One point I should add is that with respect to Disk I/O problems, one of the ways, of course, we can stop such a thing is by not running so many processes that are actually requiring that resource. But as you know in the world of enterprise computing, it's not quite so easy just to make sure that everyone is giving up processes that they may actually need in case we may need better solutions altogether. Now, one of those solutions may be that the rate and reading of the hard-disk is kind of slow and inefficient.

17:56And this can actually cause things to be bogged down. Because the longer it takes a particular process to write and to read, the longer it's going to engage that resource, i.e. the more of a backlog on that resource is going to happen. One of the best ways you can actually ease this type of problem is by changing things from a hard-disk to an SSD, a solid state drive. This has got far, far, far, faster and more efficient read and write operations. And this will actually alleviate a lot of the problems that you are seeing if you do happen to be encountering a lot of iowait on your system.

18:34Okey doke, so that is all for our introduction into being able to monitor your Disk I/O utilization. Like I say, we saw a handful of different commands, some more useful than others. As we know in these exams, very often you still have to be able to be familiar with perhaps some less efficient and older commands, such as what we saw with respect to the iostart command, for example. So, really, what I'm seeing is don't just skip over the commands, it may not be so obviously useful.

19:05Definitely dig in and be familiar with them, at least just for the purposes of the examination. Okey doke, so that is all for monitoring disk input, output. The next thing we have to tackle is network input, output, and how we can actually monitor that too. And, well, that is what we're going to be doing in the very next Nugget. So I hope this has been informative for you, and I'd like to thank you for viewing.

0:12Hey, guys. And welcome back. Now what I want to talk to you about is how we can actually use tools to monitor our Network I/O. One of the primary tools that we want to use in order to monitor such a thing is the Netstat command. So let's go and check this out then. So what I will do is I will open up a terminal right here. OK, so if I do it man netstat, we'll actually see the manual page. It's going to allow us to print network connections, as well as things such as routing tables, interface statistics, and others.

0:46So this command really is very, very useful. Now, one of the best switches to use when using the netstat command is the -s option. This is going to give you a nice summary of network statistics based on a particular protocol. So what I will do here is I'll say netstat-s. And check this out, you can actually see per protocol our network statistics. We can see our IP packets right here, the amount received, the amount forwarding, ICMP, which we use for ping messages, TCP, UDP, so on and so forth.

1:22So think about this. If you happen to be maybe experiencing some poor connections, with respect to Video, what you may know is that Video, just like Voice, is going to use UDP. So straight away by using netstat-s, we can target return and have a look at this then. We can actually see are our UDP packets being received, are they being sent, are we receiving a lot of packet errors. So we've been able to use this command and neatly get a summary of the particular statistics, such as the amount of packets received and the amount of errors received or sent, and been able to pass out this information per protocol.

2:02This can be really invaluable troubleshooting command. And we also have additional switches, of course, that we can use. Let's clear the screen. Say, for example, we can say netstat -l. So we scroll on up, we can see all of these sockets and the state of Listen. So if we happen to be having a server that should be accepting connections, one of the ways we could check that it actually is listening for connections, and there that may be we have connectivity issues, is to log in and to check that that server does in fact-- or rather, is in fact, should I say, listening for a particular connection.

2:39So this filter right here, the netstat -l, this is going to give us that information. But, again, we can go even further yet again. As we can see here, we can see we are listening for UDP connections. If I wanted to filter just for let's say, maybe, TCP connections, I could actually change this filter. I could actually say -lt to see TCP connections that are listening. So if I hit Enter now, notice on the left hand side, the protocol is only TCP-based connections, as opposed to UDP.

3:11And, of course, we can actually invert that logic by saying lu, and we just see the UDP connections that happen to be listening. Now, one cool thing we can actually do also is to say netstat -c. What this is going to do, this is going to allow us to get real time information. And we'll actually see, as we saw there with the stream changing, continual updates of the netstat display. So when you want to actively be seeing what is happening right now, this is going to give you a window into what is actually traversing your networks.

3:43So a really, really valuable tool. Now, if you happen to experience some type of network connectivity issue, maybe you can't reach a particular server or you can't reach the internet, one of the commands you can do it is say netstat -r. Which is actually going to allow you to see your routing table. So if you happen to have some type of error configured, we don't have a gateway so that we actually can't reach external networks such as the internet, this netstat -r command will give you an insight into that by giving you the routing table.

4:15So you can actually check that these entries do exist and are correctly configured. So the Netstat command really is one of the main commands that I would use in order to monitor and troubleshoot any network issues. But as always, I would recommend you go through the main page and explore the different options that this command has available to us, and really familiarize yourself with them. Okey doke, so I hope this has been informative for you, and I'd like to thank you for viewing.

0:12Hey, guys. And welcome back. So now what I want to briefly talk to you about is the concept of being able to predict future resource needs. Now, to begin one of the first tools I want to talk to you about is something called collectd. Now, what this is is ultimately a daemon. And this daemon has the purpose of collecting system statistics. Now, the way collectd actually does this is that it uses something called plug-ins.

0:42And these plug-ins actually determine what collectd will actually collect. Now one thing to note about the collectd daemon is that there are some limitations here. This is not an all purpose tool. For example, the purpose is really limited to collecting. And it would help if I could spell that correctly, collecting data. That means that it's not going to display the data for you and in a graphical form or anything like that, or analyze the data.

1:13Instead, collectd is just going to, like I say, collect that data, and you can then feed that data, if you so wish, into another program. Now collectd typically is not installed by default on most Linux systems. So we do actually have to run through the installation. So what I will see here on my Xubuntu machine, I can just use the app package manager and install collectd. Just type in my password. And hit Y, and Enter. OK, perfect. So now that we have collectd installed-- let me just clear the screen-- one of the things that we want to be aware of, for the purposes of the examination, is where we can actually configure the collectd daemon.

1:54Now we are going to be able to do this by modifying a file in a particular location. So let me show you what this location is. Is if I go into the etc directory, and then go into collectd, If I do an ls, we should now see this collectd.conf file. So try to remember that path, etc collectd, collectd.conf. If I go in and modify this, let's say nano, for example, and hit Enter, this as the main configuration file. Now within here, we can do many cool things.

2:26We can actually determine how often we should be able to collect particular data. Say, for example, here we can see we have this value here, interval 10. This means that every 10 seconds we will have a new query collecting more data. Now this is the default value. If we want to happen to change this, we would uncomment this configuration by removing the what is now called the hashtag. And we can see the color change. And we could change this interval to maybe say every 5 seconds or every 15 seconds, however you choose.

3:01But for now I'm actually not too balled with this, so I'll just comment this out, so it's inactive. And we'll still have the default of 10. Now if we scroll on down, another really important thing to notice is this Plugin section right here. This is going to allow you to specify which features we actually want to collect information on. Now if you happen to scroll down, you'll notice that most of these plugins are commented out. The ones which are actually on by default are the battery, the CPU, the disk, entropy, interfaces, memory, so on and so forth.

3:35But the vast majority, like I say, are commented out. But if we wanted to decide that, hey, we actually want to collect information on our network, we could go to the plugin network, and we could just remove the hashtag to uncomment out this configuration. And this would allows us to activate this configuration. Say, for example, I could just save this. Oh, and notice, I should have pointed this out. This is a global configuration file within the etc directory, therefore I'm going to have to use superuser privileges.

4:07So I would say sudo nano collectd.conf. And scroll on down. Now I could remove and save the changes. And in order for these changes to take effect, we would actually have to reload the service. So could say sudo systemctl restart collectd.service. If I hit Enter, that should now reload the service. And the new configurations, i.e. the changes we have just made, will take effect.

4:37And we can actually use collectd to monitor our network statistics. So, really, for the purposes of the examination, understand what collectd does and what it does not do. And really try to remember the location of the configuration file, and understand that we have these concepts of plugins. And if we want to activate a particular plugin so that we can measure a particular resource, such as a CPU or what is going on in on network, we can just simply uncomment that plugin so that it will be active after we reload the configuration file.

5:12Now another thing I want to just briefly touch upon as the subject of monitoring solutions. There are some solutions we have to have an awareness of, at least at a high level for the purposes of the examination. But like I say, this is just a high level topic. Meaning that you don't have to really concern yourself with how to configure these tools specifically, just more have an awareness of what they actually do. So the first monitoring solution I want to point out is one called Nagios or Nagios.

5:45I've heard it said both ways, I'm not quite sure the correct way. Now Nagios is ultimately a family of projects. So we could have a Nagios log server or a Nagios network analyzer or a Nagios incident manager, so on and so forth. Now the cool thing about Nagios is that it can provide a plugin to interact with collectd. We can actually use collectd-nagios, and that can actually be used to query the collectd daemon for information.

6:16So the information that is collected by collectd, we can feed it into Nagios, and use the Nagios network analyzer to analyze the network traffic or what statistics, should I say, that we have gathered using collectd. Another solution we have is one called MRTG. This is the Multi Router Traffic Grapher. And this is really used to monitor network traffic loads. It basically reads information from your router and your logs.

6:47And you can actually take this information and graph it. So this can provide a nice visual interpretation of that information to make it much more digestible for you as a system administrator. Another one we want to be aware of is one called cacti. This too, like MRTG, also focuses on network monitoring. Now this tool is basically a front end interface for something called RRD tool. This stands for the Round Robin Database tool.

7:20So simply put, information collected by the Round Robin Database tool can be ingested by cacti. And cacti will provide to you a nice, easy interface so that you can display information or graph particular data received via this tool. Now the key thing to note about all of these types of tools is that we can take this information, monitor our data, and we can use that information to observe and use for capacity planning. Say, for example, you happen to note on your graph you're having a very large and prolonged spike in, maybe say, network traffic.

7:57You may think, hey, we're actually getting much more network use, maybe it's now time to increase our bandwidth. Because if we keep following this trajectory of network use just getting higher and higher and higher, suddenly we're going to get a bottleneck, and the users are going to suffer. And we definitely do not want that. So having this type of monitoring information allows us to plan ahead. And we can see trends on the horizon as they are approaching. So really these tools altogether is going to help you identify resource exhaustion, and allow you to predict growth, and even diagnose particular problems.

8:37If you happen to see a spike in some particular resource, CPU, maybe it is network traffic, you can then use that information to potentially identify a particular problem on your network. So the ability to collect data, to be able to feed that data into a tool which can graph that data or analyze that data, this is obviously very valuable for anyone administering and managing the systems on our network. Now one thing to say about these tools just as a disclaimer is that there is no way you can get 100% accuracy with these tools.

9:13You may be able to graph a particular trend, but there is no way for you to know that that trend is going to continue. Say, for example, you've got a nice graph which is showing a very steady rate of growth over a particular period of time, there is no way for you to know that suddenly you're going to encounter an abnormal spike, OK. So just be aware that these tools definitely have a use, they definitely provide valuable information, but there are limitations to their abilities. Okey doke, so I hope this has been informative for you, and I'd like to thank you for viewing.