Deploy Terraform Templates with GitHub Actions

0:11Hi, guys. My name is Trevor Sullivan. And welcome to another CBT nugget skill. In this skill, we are going to be exploring how to integrate a few different popular technologies in the DevOps ecosystem. Now, one of those tools that we're going to predominantly be focusing on in this bill is GitHub Actions here. And GitHub Actions is a managed workflow engine that allows you to perform really any type of automation task that you might want to. Now, most people, when they hear of GitHub Actions, are probably going to think of continuous builds, continuous integration testing, continuous deployment.

0:47And we're going to be focusing kind of more on the continuous deployment side of things here. So our objective here is for our little development team right down here to basically be able to push infrastructure code changes up to GitHub Actions, or up to, I should say, a GitHub repository. Not specifically actions, of course. But basically, write code locally. Use Git as a local tool on your dev system to basically commit changes. And then push those changes up into a GitHub repository.

1:18And then once those changes have been pushed, the GitHub actions service, which is the workflow engine and automation engine, basically will pick up on those code changes in the Git repository and then will publish those to the Amazon Web Services cloud platform. Now, Amazon Web Services has a declarative configuration management service, known as AWS CloudFormation. And CloudFormation basically, it's a managed service that allows you to take a template of resources.

1:49And then you can basically create what's known as a stack of resources and push that up to CloudFormation to create a stack. And then the CloudFormation service basically behind the scenes will go ahead and provision and make any configuration changes to cloud resources that were requested inside of that template file. Now, what's unique about the CloudFormation service in AWS is that CloudFormation actually manages the back end state for you.

2:19And state management, when you're talking about declarative resource provisioning and management, is very important. And so essentially, any time that you create a resource in your cloud vendor, such as Amazon Web Services, take a EC2 instance for example, which is essentially just a virtual machine. That EC2 instance, let me just kind of find a little EC2 logo right here and bring that in. So how about this guy right here. And so once you provision EC2 instance or virtual machine into your AWS account, that EC2 instance will automatically be assigned a unique identifier, also known as the instance ID.

3:01And any types of management tasks or API calls against that particular EC2 instance are going to require that you manage it by using the instance ID as an input parameter. So if you want to delete the EC2 instance, if you want to stop or start the EC2 instance, or make any configuration changes, like resizing that EC2 instance to a different instance type with a different number of virtual CPUs and the memory assigned to it, then any of those changes would have to pass in the instance ID that you are wanting to affect.

3:38Now, the Terraform utility here that we are looking at is very similar in some sense to a CloudFormation in the sense that it allows you to deploy resources up to Amazon Web Services. However, what's different about Terraform is that it doesn't handle the state management for you. So when Terraform provisions the EC2 instance in this particular example. Whereas Cloud formation will record that unique instance ID and keep track of it over time, the Terraform utility doesn't inherently do that for you.

4:11I mean, it does technically, but it actually stores that state as a separate file that you as a developer or a DevOps engineer are ultimately responsible for managing. And they call that the Terraform state file. So any time that you deploy a Terraform template out to your cloud vendor, like AWS, then that is going to generate a state file to your Terraform state. And then you're going to have to maintain that file in order to maintain continuity in managing your stack of resources.

4:44So the nice thing about CloudFormation is that it takes care of that state management aspect for you, whereas Terraform doesn't. However, it does give you a lot more flexibility. One of the disadvantages of using CloudFormation, which is native to AWS, is that it is a proprietary service within the ecosystem. So there are some hacky ways that you could use CloudFormation to deploy technically to other cloud vendors. But all things considered, the AWS CloudFormation is really designed to deploy AWS cloud resources.

5:17Whereas Terraform is actually a cloud agnostic tool. So you can use Terraform to deploy resources to a variety of different services. AWS just happens to be one of those. But it also supports things like Google Cloud and Microsoft Azure, as well as several other providers out there. So it's kind of got a plug-in based model, where Terraform can support a number of different cloud providers. So we're going to be doing in this case is actually using the GitHub action service right over here to trigger a deployment of a Terraform template.

5:54And that Terraform utility is actually going to run within the context of GitHub actions. But we'll call out to the Terraform CLI utility and deploy a template that is sourced from the GitHub repository here and executed through GitHub Actions. So any time that our dev team is going to make an infrastructure change to the Terraform template file, those changes will automatically be picked up on by the GitHub action service. It'll trigger the Terraform deployment utility.

6:24And then it will publish those changes out to AWS. Now, the other thing that we're going to have to do as well, as any time we're dealing with GitHub actions here, is we're going to have to set up cloud credentials in order to ensure that Terraform has the necessary AWS credentials and permissions to publish changes and deploy and manage resources in our account. And to do that, we're going to be using AWS identity and access management. Let me just draw a little bit of a line right over there.

6:57So AWS IAM is going to provide the access credentials for our Terraform utility, which is, again, running inside the context of the GitHub Action service to deploy those changes out to our AWS account. So there's a few moving parts here, but we're going to build this solution step by step here. And we're going to start by just kind of setting up our connection between GitHub Actions and AWS. Then we will go ahead and add in a Terraform template file. We'll set up a GitHub actions workflow that actually invokes Terraform, runs the Terraform plan command.

7:32And then followed by a Terraform apply command to actually apply those changes to the AWS account. So why don't we jump into the next video and actually start building out this solution. I hope this has been informative for you, and I'd like to thank you for viewing.

0:11Hey, guys, and welcome back. So in this video, what we're going to do is start out by setting up our integration between GitHub actions and the AWS Identity and Access Management service to ensure that the GitHub actions' runner, which is essentially just a virtual machine that is executing our GitHub actions workflows, has the necessary access to publish and manage-- provision and manage resources in our AWS account. So let's jump into our AWS Management over here, for starters, and we'll navigate down to the IAM service.

0:46And the first thing that we're going to want to do is to set up our identity provider here. And this is basically making AWS IAM aware of our GitHub repository, that we want to use GitHub as a source to trigger different actions inside of our account. And there is some good documentation on GitHub around setting up OpenID Connect, and that document is called "Configuring OpenID Connect in Amazon Web Services." And so I actually have some separate training here at CBT Nuggets that focuses on this as well.

1:20So we're going to speed through this a little bit. So if you get lost, go ahead and watch my other skill that talks about connecting GitHub actions to AWS, and it'll talk about your different authentication options here. So for starters, we're just going to go ahead and create this identity provider, which I already have here. And then once that identity provider has been created, I'm going to go ahead and create a role. And a role in AWS is essentially an application identity that doesn't have any static credentials associated with it. In fact, you dynamically obtain credentials from the service when you need them.

1:56And so that is actually a more secure option than storing any static credentials. And then I'm going to choose web identity over here because we're going to be using our GitHub identity provider in order to obtain credentials for this role. And then on the Permissions section here, we need to make sure that we provide Terraform any permissions to manage our account. And for starters, I'm just going to go ahead and give it to VPCFullAccess so that we can manage different VPC resources inside of our account. But if you are going to provision any other types of resources, maybe EC2 instances or virtual machines, you might want to attach EC2FullAccess there as well.

2:35I'll go ahead and actually just check that off. And then we'll go ahead and create this role. I'll call this github-actions-terraform2 and hit Create Role. And then once this role has been created, one of the typical things you need to do when you're setting up OpenID Connect is to edit the trust relationship on the role. And we will basically need to add in a condition here to ensure that our GitHub repository has access to obtain credentials using this role identity.

3:08So to do that, we're actually going to need to go and create a new repository. So I'm going to go into my GitHub organization which is github.com/cbttrevor. There's two T's in there, cbttrevor. And then I'm going to click on New. And we'll give our GitHub repository a name, like github-workflow-terraform, and we'll go ahead and make that a public repository and click on Create. So this will give us an empty repository to work with.

3:40And I am going to go ahead and just create a file here. I'm just going to call this asdf so that I have an empty file in here. And it's kind of weird, but if you don't have any files in your repository, you can't use this Codespaces feature. So I am actually going to go ahead and provision a codespace so that I have a nice browser-based editing environment in order to author the different files, like the workflow files as well as the terraform files and things like that. So let's give this codespace a second to build, and then it'll launch me into my VS Code environment right here and I can start working inside of my browser here without having to install VS Code.

4:21So this is a really nice feature. If you haven't tried out GitHub Codespaces, then I would definitely encourage you to check it out. But in any case, now that we have a GitHub repository created, we now need to basically take our organization, cbttrevor, and our repository name right here and plug that into this condition. And right here in this Configuring OpenID Connect, there is a condition right here that we're going to copy over into our role.

4:52I'm going to go ahead and come into Condition, StringEquals and then just replace the octo-org with my org, cbttrevor, and then go ahead and replace octo-repo with my repository name right here. I'll paste that in and fix the formatting. And then the branch here, I'm going to replace octo-branch with just the main branch because that's just the only branch we're going to be dealing with here. And let me just make sure there's no extra space in there. Yeah. So it's just line-wrapping there a little bit. So nothing to worry about there. So we'll hit Update Trust Policy, and now our GitHub repository will have access to use this role identity.

5:30Cool. So let's go ahead and start out by configuring a base workflow that is just going to set up our AWS credentials. And to do that, if we go to the Actions Marketplace-- let's head over to the GitHub Actions Marketplace here-- and search for AWS Credentials, and there is an action here called Configure AWS Credentials that's going to allow us to retrieve our session credentials using that IAM role, we basically just need to specify the role to assume and then plug-in the ARN to our role.

6:06I'm going to come into my Codespaces editor here and create a .github directory and then a workflows directory, and this is a special directory path where GitHub is going to search for workflow files. And then I'm going to create a new file called terraform-deploy.yml. And inside of this file, we are going to plug-in our On section so that we can trigger this. And we'll say workflow_dispatch manual triggers, and then we'll also do a push event as well.

6:37Every time we push code changes, it will automatically pick up on those and then attempt to run our workflow. And then down here, I'm going to go ahead and do jobs and set up a job called terraform-deploy. And we'll say runs-on ubuntu-latest. And for our steps here, I'm going to go ahead and add in the first step, which is to configure AWS credentials. And so let's go ahead and just give this a name, and I'll just say name, Configure AWS Credentials.

7:14All right, so what we need to do now is add in our "with" section here because we do need to specify the role that we are going to assume. So under the "with" property right here-- I'm referring back to the documentation here on the GitHub Actions Marketplace-- I'm going to plug-in role-to-assume in our file here, and I need to grab the ARN, or Amazon Resource Name, of this role. So I'll copy that to my clipboard and paste it in. And then the other thing we can do here is set our default AWS region. I'll just do a aws-region and set it to us-west-2 but feel free to use whichever region is appropriate for your particular use case.

7:50If you're in a different part of the world and you want to get better performance, just find a region that's close to you. All right, so let's go ahead and save this file, and then we'll go ahead and commit and push this to our repository. So we'll say "Adding workflow file" then do sync changes to push that up to our Git repository. And then we'll go over to Actions here, and we should see it attempt to trigger. Now, there is one other thing that I need to do as well, and that is going to be to set the id-token permission to write.

8:28Otherwise, we won't be able to obtain our credentials. So, sure enough, you can see that it failed right here. And it says "Credentials could not be loaded, please check your inputs," and that's because we actually need to add in this permission to the GitHub workflow. So permissions, id-token is write, save that, and re-commit this and say "Add write permission for token." And then we'll do sync changes once more, and then we should have another invocation of our action here.

9:01And basically it's going to run really fast because the only thing it's doing is going out and obtaining credentials. Let's give this a second to run here and just make sure that our credentials are retrieved correctly. And this time, sure enough, it works. So if you're missing that permission, go ahead and just add that into your workflow file here, and you should be good to go. All right, so now that we have set up our role in IAM and we've also set up our basic structure for our GitHub workflow here, the next thing that we need to do is go ahead and add a Terraform template file into our project here.

9:35I hope this has been informative for you, and I'd like to thank you for viewing.

0:00[MUSIC PLAYING] Hey, guys, and welcome back. So in the last video, we did the basic kind of infrastructure set up to get GitHub actions connected over to Amazon Web Services using the OpenID Connect method, which I do generally prefer, although you can, of course, generate static credentials with an IAM user instead of an IAM role if you would prefer to do that, and then store those values as secrets.

0:34That's certainly an option that you have. But what we need to do now is come into our repository, and add in a Terraform state file-- sorry, not a state file, but a Terraform template file. So I'm going to head over to the Terraform docs. I'm just going to search for Terraform docs, and head over to the home page if I can click on the link without Google moving elements around on me. And basically the first thing that we need to do in Terraform here is take a look at the configuration language. And there are a few different providers.

1:06And a provider is basically defining a cloud service that we want to publish to. And so there is an AWS provider. Actually, I'm just going to head over to the registry, here the Terraform registry. And if we search for AWS the provider, click on a provider's right here, and then drill into the AWS provider, it will show us how we can use the AWS provider to interact with the service.

1:37So this is a module here. Trying to look for where the actual provider itself is. So let's drill into AWS again here. And I'm actually going to click on the Terraform provider right here. And this will take me over to GitHub here, and hopefully they have an example right here. We'll click on using the provider. And this is actually what I'm looking for right here. So basically, we just need to specify the provider is going to be AWS, and then we want to specify the region that we want to be working in.

2:15So I'll be using US West 2, but feel free to use a different region if you would like to. So let's copy the provider here into our editor. And we're going to create a new file called, let's say, trevorterraform.tf. You can call that whatever you want to, of course. And then it will change this region to West 2. And I'm just going to delete this little placeholder file that I created earlier, because I don't need it anymore, because I've got my code space provisioned.

2:46And now that we've got the provider defined here, and we've got the credentials that are coming from the GitHub workflow itself, I can now start to add in different resources. And there's a couple of different ways that you can do that. One way is to use something known as a module. And you can kind of think of a module in Terraform as being kind of a building block that consists of several different resources. So if you're familiar with AWS, and you're building out things like a VPC, for example, a VPC, a software-defined network in the Cloud, essentially, is basically just a CIDR block.

3:20But you actually have to, in order to actually deploy different resources into the VPC, you first have to subnet it. You also have to attach things like an internet gateway, or a NAT gateway, depending on what you're going to be using it for. If you want EC2 instances to be public or private, or if you're going to be using things like RDS, Amazon RDS provisioning database services into that VPC, there's a lot of options there. You've also got things like your route tables, your security groups, as well, to allow inbound or outbound access to different services, and things like that.

3:53So modules are kind of a composition of several different types of resources. A VPC, a subnet, security group, and things like that allow you to construct an entire resource. Whereas if you are interested in just building with the base building blocks themselves, that is what we know as a resource. So a resource could be like a VPC, or a subnet, or a security group. And so rather than using a module that ties all those concepts together, you can instead use those building blocks individually instead.

4:23You can kind of think of them like LEGO pieces. Rather than having a LEGO piece that's already been constructed for you out of several different pieces, you can actually take those individual pieces and then piece them together. But there is actually a resource-- not a resource, a module right over here on the registry-- let me come back to my registry tab here-- and there are modules for things like VPC. We've got a module here for security groups.

4:54We've got modules for IAM resources and things like that. But I'm going to go ahead and grab this module right here for VPC. And as you can see, this shows us an example of what a VPC module would look like. So inside of a VPC module, it actually allows us to define private subnets and public subnets. So we can actually subnet everything from within a single module rather than having to define all of these individual resources separately, and then stitch them together manually.

5:25So our VPC could have an optional name. Of course, it has to have a CIDR block. That's one of the essential properties of a VPC, is that you have to define your supernet so that you can then subnet it into smaller environments. You also have the ability to map your different subnets to availability zones in AWS. So if there's specific availability zones that you want to use in this VPC, you can specify those as an array, as well. You can also choose if you would like to enable a NAT gateway, if you want to have private resources on private subnets, have outbound access to the internet.

6:00You can also switch on the VPN gateway, rather than having to provision that as a totally separate resource, as well. So this is kind of a convenience module that allows you to construct several different resources, individual resources, in AWS just by using this module instead. So I am going to go ahead and construct one of these VPCs inside of my Terraform template here. So let's go ahead and do module. And we'll call this trevorvpc.

6:30And then we'll go ahead and set the source of our configuration here to use the Terraform modules/vpc/aws. We'll go ahead and bring that in here and just paste that in. And then we need to set the CIDR block. So let's go ahead and set the CIDR block parameter here. We'll say cidr equals 10.70.0/16, for example. And then I'll just go ahead and save this, because I don't think any of these other properties are mandatory.

7:03I could be wrong about that. But we'll find out when we actually attempt to deploy it. And then the idea here is that later on, once we start adding more content to this Terraform template, we'll actually automatically deploy those changes through GitHub actions using that push event that we've already configured. So now that we've added in a base Terraform template file here, let's go ahead and actually run the necessary commands inside of our GitHub actions workflow to actually test and do a plan, and then a deploy on this particular template.

7:37I hope this has been informative for you, and I'd like to thank you for viewing.

0:11Hi, guys, and welcome back. So in this video, we are going to take this Terraform template file, the very, very basic file here with just a simple VPC configuration, and we are going to go ahead and deploy this out to AWS as part of our GitHub actions workflow right here. So go ahead and open up your workflow file. You don't have to name it Terraform dash deploy if you don't want to. You can just name it whatever you'd like to, as long as it conforms to the GitHub actions expected schema here. So you need on section, you need the job section, and things like that in order to define a very basic workflow.

0:46But as long as you have a valid workflow file, you can name it whatever you want to. And now down here under the steps, under our Terraform deployed job, right now the only thing that we have going for us is that we are configuring our AWS credentials and setting a default to AWS region as part of that process. So at this point, we need to actually run the Terraform tool. Now, normally you would have to install Terraform. However, thanks to GitHub actions and the software that they pre-install onto the service, onto their images that they use to spin up these virtual machines, Terraform is actually included.

1:21Now, how do we know that? Well, we can search for GitHub actions included software environment, and check out this documentation here for GitHub actions virtual environments. This will take you over to a GitHub repository that contains information about the different images. And if you scroll down to available environments, these are the different operating system based images that you can run a GitHub actions workflow based off of. So I am using Ubuntu latest right here, which is a Ubuntu 20.04 LTS, long term support, which is focal fossa.

1:57That's the code name for it. And if you click on the link for included software right here this will show you basically this huge list of all the different programs that are pre-installed on the base image for Ubuntu focal fossa. And this is all provided by GitHub. So we've got things like our different runtimes and package managers. We've got the pip package manager. We've got NPM. We've got a Homebrew for Linux. We've got Helm if we want to deploy Kubernetes helm charts.

2:28We've got yarn, if you prefer yarn over NPM for JavaScript projects. We've got a whole bunch of other tools down here, like Docker Compose is pre-installed. And if we scroll down just a little bit here under Tools, under the Tools heading right under project management, we have Terraform version 1.1.3. So this is really nice because we don't have to add an extra step into our GitHub actions workflow to install Terraform. Unless, that is, you need a specific version of Terraform.

2:59If Terraform 1.1.3 doesn't work for you for some reason, then sure you could go ahead and install your own version over top of that instead. But I'm just going to rely on the built in version of it for the time being. So I'm going to go ahead and close that tab right over here, come back to my editor, and we are going to go ahead and add a run step in here. And there is a great document I like to refer back to for GitHub workflows called "workflow syntax for GitHub actions."

3:30And there is a run option that you have under steps. So if you take a look at jobs and then steps.run, you can add in a run step and it will basically invoke a shell script, or just a command line program. And by default it will use the bash shell, I believe. However, you can actually override the shell that it's going to use if you would like to. So you can actually add in the shell option right here on your step configuration, and that will change the shell that is being used by the run command here.

4:07So if you wanted to run PowerShell, for example, which is also pre-installed in the GitHub actions Ubuntu focal fossa image, then you could certainly do that as well. But for the time being I don't really need to do any PowerShell scripting, so I'm just going to use the default shell. And let's do run. And then the first thing I need to do is do a Terraform initialization to initialize the provider. We'll do a Terraform init, and I'll go ahead and just give this step a name. We'll say run Terraform init command.

4:41And then we'll add in another command here. So I'll go ahead and just select these and then do Alt Shift down arrow, and that will copy those two lines down. And we'll do a run Terraform plan. I'll say run Terraform plan command. And then we'll do Terraform plan. And I'm going to go ahead and just save this for now and test this before we actually attempt to deploy it just to make sure that plan step actually runs. So let's go ahead and add in our template file here. And then we'll add in our updated workflow file and also add the change here where I'm deleting that ASDF placeholder file.

5:18And I'll say adding Terraform template and update workflow. Control Enter to commit, and then we'll do sync changes, hit OK. That'll push the changes up to GitHub and kick off a invocation of our workflow. So we'll go over to Actions right here. And as you can see, we've got a new invocation right here. And if we drill into that, we've got our Terraform deployed job. And it looks like we are missing some configuration options here.

5:53So let's go ahead and take a look at this. So we've got no configuration files. It says it can't find any configuration files. So it's looking for TF files. And of course, the reason for this is because we did not check out our source code. Sometimes I just forget to do that. So with GitHub actions one of the things that you actually have to do is check out your source code. And if we head over to the GitHub actions marketplace right here, and go to Actions, and then search for checkout, this is the action that we can use inside of our workflows in order to check out our source code onto the GitHub runner system, the virtual machine that's actually running our workflow.

6:38Now, there's a bunch of options that you can specify under the with property here. But we don't really need to specify any of these unless you are using some kind of advanced scenario where maybe you need to provide a custom SSH key to provide access, or you want to clone it to a different path on your local file system, or if you want to specify the repository that you want to clone. But by default, it's just going to clone the current repository. So I'm just going to grab this uses statement right here and come back into my configuration on my workflow here.

7:13And before we do anything else, we'll go ahead and just add in this checkout step. And I'll give it a name. We'll say name and say Clone, source code, and save this. And then we'll go ahead and commit that change. We'll say add checkout step to workflow, and do another sync changes, and click OK. And then we will monitor this action again. So here it is. Here's our new invocation currently in progress.

7:44We'll drill into the job and see what we get for output. All right, so this seems to have gone much better this time. And so what you can see is right here the very first step after this built in setup job-- we don't have to define the setup job. That's just built into GitHub actions. But then the very first thing that's defined in our actual workflow is the clone source code here. And so this is basically cloning our Git repository into the local file system. And so that's going to provide us access to the files that are inside of our repository.

8:17Next, we have configure AWS credentials. We already know that that's working just fine because we tested it previously. Next, we have the Terraform init command. And this is basically going to take a look at our Terraform template file. And it is going to see that we have referenced the AWS provider. So if we come back in here, we defined our provider AWS right up here, though it is going to download the necessary provider plug-in for AWS. And so it's downloading version 3.72.0 right here.

8:49And that's going to make sure that we can use that provider in our actual execution of our template deployment. And then after that, we've got run Terraform plan right down here. And at this point, it's going to show us any changes that Terraform is planning to make to our account. So we've basically defined this new resource called an AWS VPC. This is actually part of the module that we've defined. But basically, it's going to go ahead and just define a new VPC. The little plus sign here just indicates that it's a new resource that it's going to be provisioning.

9:23That's this little legend right here. So it's going to create a VPC, and it's going to set our CIDR block to the property that we set in that high level module previously. So at this point, it says planning to add one resource. There's zero resources to change and zero to destroy. So at this point, nothing has actually been executed. We have simply kind of planned out what our state changes will actually look like if we were to execute it. So let's go ahead and jump into the next video where we will customize our workflow to actually deploy this change.

10:00I hope this has been informative for you, and I'd like to thank you for viewing.

0:00[MUSIC PLAYING] Hey, guys, and welcome back. In this video, we are going to take a look at how to actually apply our changes as part of our GitHub actions workflow. And then we'll also take a look at how to make some changes to the template a little bit later on so that we can see the automation in action as we are making changes to our source code, which basically just consists of our Terraform template at this point.

0:35So in the last video, we got our Terraform init and Terraform plan commands working just fine. We of course, had to add in the step to clone our source code here, as well. So we have access to our Terraform template right here in the root. But let's go ahead and switch over to our editor here. And we are now going to add in another step after we run the plan step here. And we'll just go ahead and select these lines, do Alt-Shift down arrow again, nice VS code shortcut. And then we're going to run Terraform Apply or Deploy.

1:07You can think of it like deploying your changes or applying changes, right? And so what we're going to do is run Terraform Apply. However, if we just run that by itself, things are going to fail. Do you know why that is? Well, I'll give you a hint. So it's because, if we run Terraform apply, we are making production changes, right? We're actually making resource changes to our AWS account. And if this was a production account with a business critical application, then the changes that are being planned could potentially have a significant impact, potentially a negative impact if misconfigured to our production application.

1:45Now, that production application we don't want to go down. We don't want to affect the end users of our business applications negatively. And so by default, the Terraform Apply command actually will prompt you and say, hey, are you sure that you actually want to apply these changes? It's basically just an interactive sanity check to make sure that you actually are going to be OK with the changes that have been proposed by the Terraform plan command. So in order to work around that, we have a parameter for the Terraform CLI.

2:17So if I come back to the Terraform documentation here and just go back a level up here to the Terraform Intro, go to the Terraform CLI, there is a document. Let's see. We've got alphabetical list of commands. We have the Apply command right here. And then we have an optional parameter on here called -auto-approve. So the first parameter right here you can see, auto approve, it says skips interactive approval of plan before applying.

2:48Now, this is really important when you are running the Terraform utility in any kind of background service that you don't have console access to, right? We're not actually typing Terraform Apply into our terminal here. We're not firing up PowerShell locally or iTerm2 locally if you're on Mac OS, for example, and actually running that command. Because it's running in the background on a GitHub actions virtual machine or a runner, as they call it, we need to make sure that we are not being prompted for any kind of input. Otherwise, that will simply cause our workflow to fail.

3:22So what we need to do is go ahead and add in the -auto-approve argument there to the Terraform Apply command. And that should apply any changes to our environment, to our AWS account, right? And so we saw previously in the proposed output from the plan step that the plan step is going to add in a new VPC resource right here based on that module, the VPC module that we're using, which is a high-level construct over here in the Terraform Registry.

3:59So all we did is we specified the CIDR block. We didn't specify any private subnets. We didn't specify public subnets or azs that we want to use for those subnets. We also didn't choose enabling a NAT gateway. So the service is really not going to do anything for us there. But we will come back and then add these things in a little bit later on. So if I go ahead and just save this file and do a commit-- we'll say adding Terraform Apply step to workflow, Control-Enter, say, yes, we want to stage and commit all changes, and then let's do a sync to push those changes to our repository, and then that's going to automatically, thanks to the push action, kick off a new execution here.

4:44And we're going to see an issue here. But I'm not going to reveal it quite yet here. [LAUGHS] So let's just watch this run. So we've got cloned source code configure AWS credentials, Terraform init-- that's going to download the AWS provider onto the runner, and that has to happen each time. And then we run the Terraform plan command, which again, is going to say this is the planned changes to add in this VPC. And now we're running the Terraform Apply, which is actually going to make those configuration changes. Now, remember, the IM role that we configured here in the Configure AWS Credentials step, the role named GitHub actions Terraform 2-- that role we assigned the policy to that role in AWS IM to allow it access to the VPC APIs.

5:31And so that's why this succeeded here. If we had not given it the permissions policy in AWS IM to ensure that the role has access to the VPC APIs, the Apply step would have actually failed, because the permissions were not there to call the necessary create to VPC REST API in the AWS REST API for the VPC Service in order to create that resource. But because we assigned that permission, we do have a successful execution here. So now let's go over to the AWS Management Console.

6:05Take a look at the VPC Service here and see what we've got. So if we go into Your VPCs, sure enough, you can see right here we've got a VPC with 092ba as the ID, and it has a CIDR block of 10.7/16, which is exactly what we specified in our Terraform template file. Now, I also have a couple of other ones here. I have my default VPC and just another one that I created here previously.

6:36I'm going to go ahead and clean up this old one really quick just so we don't run out of VPCs. I think there's a limit of five by default, or something like that. But let's go ahead and try to run this again. So I'm actually going to come back into my Actions here. And I'm not going to make any changes. So nothing has changed at this point with our Terraform template. It still just has this little module in it right here with a CIDR block set. And if we run this again, what do you think is going to happen? Do you think it's just going to look at the CIDR block and say, OK, well, the CIDR block is already set properly, so I'm not going to make any changes, right?

7:13So if we do a Terraform plan, it's basically going to say there's no changes to apply, right? Let's find out. Then I'm going to come back into actions here. I'm going to take this last execution that was successful. I'm going to click on these little dots over here. Or actually, let me drill into it first and then click on these dots right up here-- or sorry, this button right here. [LAUGHS] It's right in front of me, and I can't even see it. So re-run all jobs, and so that's going to trigger the exact same execution of the workflow to run again.

7:44And if we drill into the job and check out the logs here, let's see what happens. We're setting up our credentials, running a Terraform init, running the plan command. And let's see what the plan command says. So when we run the plan command, look what it says right here. It says, let's create. It's going to create a VPC-- sorry, it automatically closed that on me. So it says I'm going to create a new resource, a VPC, and I'm going to give it a CIDR block of 10.7/16.

8:20So what's wrong here? Well, think about this. We've already created our VPC, right? And our VPC had the correct CIDR block. We didn't make any changes to the template file. But Terraform plan is telling us that it's going to create a new resource. And if we scroll down a little bit down here to Apply, it says Terraform will do this action. Once again, it's basically just doing another plan right before it does the Apply. And sure enough, it says creation complete after three seconds.

8:50So it actually added a new resource. It says that right here-- apply complete! Resources-- 1 added. So what happened here? Well, this is where the state management comes into play with Terraform. And Terraform supports different state management back ends. Now, by default, the back end where your state will be stored is stored in a local file. But when the GitHub runner is terminated, so after our workflow is finished, all finished running, the entire file system and that virtual machine that is running our workflow is discarded.

9:28So we don't have access to that state file anymore. And so what's essentially happening here, if we come back into the VPC console, is that it actually created a duplicate VPC. And of course, we don't get to specify the ID. The ID is automatically generated by the AWS VPC Service. But what's happened here is Terraform has made us another VPC. So if we were to run that workflow a third time, then Terraform is actually going to give us a third VPC with the same CIDR block. But that's not what we want to happen, right?

10:00What we want to actually happen is we want to just have a single VPC with that CIDR block. And if we are to make any changes to the configuration, like perhaps changing the name tag, for example, then we want Terraform to update the existing VPC. We don't want it to create entirely new VPCs, because that really defeats the whole point of doing declarative configuration management, right? So basically, if we take a look at the Terraform Docs here and then drill into-- let's go back here, go into the configuration language, and take a look at state.

10:38The state section really deep-dives into the different state storage back ends. But essentially, if we go into the overview here, the state file, by default, is called Terraform-- let me find the file name right here. So we have Terraform import. Here it is. So it's called terraform.tfstate. And so that file is actually getting created by the Terraform CLI when we run the Terraform Apply command.

11:10And it's going to store the state, the unique ID of the VPC that gets created by AWS. When we call, or when Terraform calls, the Create VPC API for us, it's going to actually store that ID, that unique identifier, for that VPC into this state file so that, in the future, if we make any additional changes to the configuration of that particular VPC, Terraform will be able to look up the existing VPC and then make changes to it rather than duplicating that resource.

11:41So how do we deal with this? Well, I'm not going to get too deep into the weeds in the back end state management. I'm going to show you a hacky way to do this. And so the hacky way to do this is to actually commit the Terraform state file to your Git repository. And this is not the recommended way to do this. However, it will kind of show you what that file looks like with fairly minimal effort. Now, if we go back over to our workflow here, after we run the Terraform apply command here, let's say that we want to pick up that terraform.tfstate file that's generated by this Terraform apply command and then add that to our Git repository and push it back.

12:26Well, there's a pretty easy way to do that. And again, this is not recommended for production state management, but it is kind of the easiest way to just get access to that file and kind of extract it out of the GitHub runner. So what we're going to do is you go back to the GitHub Actions marketplace here and search for an action called commit and push, I think-- commit and push. And I'm going to sort by most installed.

12:57And there's a bunch of different actions that we can use here. The most popular one, by far, is this one right here by Stefan Zweifel. I hope I'm saying that right. And what we're going to do is come in here and grab this step. And this is basically a commit and push action. So it will add in local files that have been changed in the context of the Git repository that was cloned locally by the checkout step, that very first step that we added. And then it will commit those changes on the local file system on the GitHub runner.

13:30And then it'll push those changes back to our repository. Now, we have the ability to specify a file pattern. So I am going to specify a file pattern that simply includes the name of Terraform active state. So let's go ahead and say Commit Terraform State File as the name of this step. And then we'll go ahead and do uses. Or actually, I already have uses in my clipboard there, so I'll go ahead and just paste this in. And then for the with section, where we can add in input parameters to this action, I'm going to specify the file pattern.

14:07And we'll go ahead and just do file pattern, and then we'll say terraform.tfstate. So that's the only file that will get added, committed, and pushed to our Git repository. And then the other thing that I can do is set a commit message, as well. So I'll say commit message and just say adding Terraform state file, or maybe updating Terraform state, because it could always change in the future. And so now we're going to push this back to our code repository.

14:37However, we need to, because we're now writing back to our repository, we actually need to add in another permission. And this permission is called contents. And you can look this up in the GitHub workflow syntax documentation if you want to. But if we are going to commit and push back to our Git repository on GitHub, we need to add in this contents permission and give it write access. So we'll go ahead and save this, and then we will commit our change to our workflow file and say adding Terraform state commit step.

15:10And we'll do Control-Enter to commit and then push our change back to our Git repository. And if we go back to Actions, we should have a new execution pop in right here. And we'll see what happens. All right, so here's our new execution. And as you can see right over here, we currently have two VPCs. And I think it's actually going to create a third one, because we haven't done any state management yet. And so the first time it runs after we add in the state management component to it, it's going to actually have to create a whole fresh one.

15:45So in just a second here, it'll pop probably pop in with a third VPC that has a 10.7 CIDR block. So it looks like everything just completed here. Let's do a quick refresh-a-roo here. And sure enough, as I just expected, it created a third VPC. Now, which one did it create? I think it's-- want to say it's the one with 009, but let's just check over here. Go down to the Run Terraform Apply, and it'll tell us right here at the end.

16:20It looks like it's actually the 0c2 one. So it tells us what the ID of that resource it just created is. And so basically we've got this 0c2. And so now, if we were to make any changes to our Terraform template file, hopefully those changes would get made to this particular resource. So we'll give that a shot in the next video. I hope this has been informative for you, and I'd like to thank you for viewing.

0:11Hey, guys, and welcome back. So in this video, we are going to make a change to our Terraform template. And we basically just want to test and verify that our Terraform state that we committed back to our Git repository is actually taking effect any time that we run the Terraform apply command, and periodically getting updated every time that the workflow executes. All right, so I've got these two defunct VPC IDs here. So I'm just going to go ahead and just clean these up really quick. The two that were created before we did the state management stuff.

0:45The 0c2 is the one I want to keep. So I'm going to go ahead and delete this one as well. And I'm just going to leave my default VPC with the 172 CIDR block in place. All right, so now we're down to just the VPC that we have the state management on. So let's take a look at the state management file for starters. So if we come over to our project here, go to code, you can now see that we have this commit message saying updating Terraform state file. And it's called Terraform dot TF state.

1:16All right, so now that we've got this file in our repository here, we can explore it. And this is what the Terraform state file actually looks like. So we've got a version. We've got the Terraform version that was used to deploy. We've got this huge resources section that kind of describes the state of our resources. And you'll notice that under our resources we have this instances section right here. And then under attributes of a specific instance of a resource we have the ID attribute here or property.

1:52And the ID property sure enough is this VPC ID that was generated by AWS. So now, because I'm storing the state file in Terraform inside of my Git repository side by side with my Terraform template file itself, now I have information about the VPC that is in my AWS account. And Any changes that I make to my Terraform template file are going to get applied to this VPC rather than creating a whole new one like we saw happening before.

2:25So let's go ahead and take a look at this module over here. And as we could see previously when we first explored this, we have the ability to provide different subnets. And we can make those either private subnets or public subnets by simply creating an array called private subnets that the VPC module here is expecting or public subnets. And then we can also tell it which access AZs, or availability zones, we want it to use. And because we're working in the US West 2 region, if we were to create a subnet, for example, I believe we just have four different AZs.

3:05You can just come into the create subnet here, and then take a look at your availability zones. So we've got 2A, 2B, 2C, and 2D. And so we could basically limit and say, I only want subnets in 2A or 2B. And so we could limit that by using that property. And then once we make those changes, those will be applied back to our VPC. Now the other thing I wanted to mention to you here, just a general observation, is that in our subnets here-- I don't have any filters applied, so I'm looking at all the subnets.

3:36The only subnets that I have are under my default VPC for this region, which is this one 172.31/20. And so right now, we have a VPC with the ID 0c2, but it doesn't have any subnets at all. It's not subnetted in any capacity. We don't have any private subnets. We don't have any public subnets. In fact, I don't think we even have an internet gateway either. Yeah, see if we go to internet gateways, we don't even have an internet gateway attached probably because we don't have any public subnets.

4:08So it says, well, if you don't have any public subnets, you don't need an internet gateway. Because that's the whole point of the internet gateway is to expose your resources directly to the public internet, like EC2 instances or anything that has an Elastic IP address, for example, attached to it. So what we're going to do is actually edit here in our code spaces environment. We're going to open up Trevor Terraform dot TF. And in this module here, we're actually going to change the configuration. And one of the things that we'll do is add in public subnets.

4:42And so we'll pick a couple of CIDR blocks to assign to that. They'll say public subnets equals array. And I'll say 10.7.100.0/24. And I'll also add in, let's say 10.7.200.0/24 and four AZs. If you recall, we have this AZs property that is also going to an array of AZs inside of this region that we want to support.

5:14And so I'm going to say, well, I only want you to create these subnets in US West 2A or US West 2, let's say, C. All right, so basically we're saying we want two public subnets and we want these AZs supported. So now at this point, as a DevOps engineer, because we've set up our fully automated pipeline in GitHub actions, I don't have to worry about manually deploying these changes. I can literally just make these changes to my Terraform template file right here, and then commit those changes, push them to my GitHub repository, and the GitHub action service is automatically going to pick up that change thanks to the push event that we defined right here.

5:56So every time we've been pushing code changes, it's automatically getting triggered. And so we don't have to do anything else. That's literally all we have to do to make changes to our environment is to save this file, come into source control, and commit the change, say adding public subnets and AZ config to Terraform template, and we'll say sync changes, and it is complaining. Why is it complaining?

6:26Let's see, oh, that's probably because when we executed our GitHub workflow up here, it automatically committed this Terraform state file. However, our code spaces environment, our client machine that we're basically working on with VS Code here, doesn't know about that change yet. So I actually need to synchronize that change locally. So I'm going to do a Git pull. Of course, it's not going to like that. So I need to do a quick Git reset. So let me do that. We will do F1 in our editor here and we'll say, undo last commit.

7:02And so that will basically just restage our change here. And so now we can go ahead and do a Git pull. And so that is going to download this Terraform state file from our GitHub repository onto this local dev system that's running in code spaces. But even though it's running in GitHub code spaces, it's actually just a client machine that's separate from GitHub. It's not actually synchronized directly with this unless we actually do a Git pull operation on it. So that's why we saw that inconsistency there.

7:32So now I'll go ahead and just recommit this file now that we are synchronized, and then push that change up to GitHub. And that, of course, is going to trigger our actions here to execute. And we'll drill into the logs here and see what the Terraform planning output looks like. So it'll basically say, OK, I'm going to add these subnets, and I think that's pretty much it. Actually, it'll probably add an internet gateway too since we made a public subnets, because otherwise they wouldn't be public.

8:04So there's the Terraform initialization. We're doing a Terraform plan now. And in the Terraform plan output, sure enough, we have a bunch of stuff here. So we've got plus subnet and 7.100.0/24, that's going to be put in 2A. Then if we scroll down, we're going to see and 10.7.200.0/24 US West 2C. So neither of the subnets that we added are getting placed into the 2B or 2D availability zones.

8:37Because, again, in our module configuration, we told Terraform that we did not want to use either of those availability zones. And then the other thing that you'll notice here-- sorry, it keeps closing this on me-- is that we have an internet gateway that's going to be added as well. And it'll be attached to our VPC. So you can see right down here it's going to attach that internet gateway to our VPC for us. And then down here, it's actually doing some routing setup. So it's going to create an AWS route table, a VPC route table.

9:08And it's going to create that for the internet gateway. So that's going to be a routing table rule in our route table. And then down here is the actual route table itself. Though the route table contains routes, and this is a route, and this is the route table. And then down here, we have a route table association, which basically takes the route table which contains the rule that goes to the internet gateway, and it's going to associate that route table with both of our subnets. And so it's basically saying that it's going to associate the two subnets that we have down here with the route table.

9:45So we have two separate associations right there and the subnets, and that's it. So if we come over to the VPC console now and take a look at subnets, now you can see right here under VPC, I can go ahead and just filter things down to that VPC ID. Let's choose the 0c2 one just to clean things up a little bit in this view. So now you can see we've got two subnets. We've got them associated with the 0c2 VPC.

10:16And then the CIDR block that we have right here is what we specified in our Terraform file. You can see our availability zones right here. That looks good. A and C, that excludes B and D, which we didn't include in our file. And then we've got things like our route table here. So we've got this public route table. We should now have an internet gateway sure enough, which is attached to our VPC. And what else do we have? Do we have any security groups? We should have a default security group, I think, at least. Yeah, they've got a default security group right here 0c2, on the VPC named 0c2 as well.

10:54So that's what things look like. That's how we can set up an end-to-end workflow using GitHub actions, using Terraform, and then very rapidly being able to make changes by simply committing those changes right up here to our GitHub repository. And we did all that pretty much with really just two files, our Terraform template file, which is really not that big, and then our workflow right here, which is really doing all the heavy lifting as well as our AWS IM configuration to allow our GitHub actions runner environment to actually obtain roll credentials from IM.

11:32I hope this has been informative for you. And I'd like to thank you for viewing.