Automating .NET Framework deployments with AWS CodePipeline to Elastic Beanstalk

When it comes to Windows CI/CD pipeline people immediately start thinking about tools like Jenkins, Octopus, or Azure DevOps, and don’t get me wrong because those are still great tools to deal with CI/CD complexities. However, today I will be explaining how to implement a simpler .NET Framework (Windows) CI/CD pipeline that will deploy two applications (API and Worker) to two different environments using GitHub, CodePipeline, CodeBuild (Cross-region), and Elastic Beanstalk.

Requirements

• AWS Account
• GitLab repository with a .NET Framework blueprint application
• Existing AWS Elastic Beanstalk Application and Environment

CodePipeline setup

Let’s create and configure a new CodePipeline, associating an existing GitHub repository via CodeStar connections, and linking it with an Elastic Beanstalk environment.

First, let’s jump into AWS Console and go to CodePipeline.

Once in the Codepipeline screen, let’s click on Create Pipeline button.

This will start the multi-step screen to set up our CodePipeline.

Step 1: Choose pipeline setting

Please enter all required information as needed and click Next.

Step 2: Add source stage

Now let’s associate our GitHub repository using CodeStar connections.

For Source Provider we will use the new GitHub version 2 (app-based) action.

If you already have GitHub connected with your AWS account via CodeStar connection, you only need to select your GitHub repository name and branch. Otherwise, let’s click on Connect to GitHub button.

Once at the Create a connection screen, let’s give it a name and click on Connect to GitHub button.

AWS will ask you to give permission, so it can connect with your GitHub repository.

Once you finish connecting AWS with GitHub, select the repository you want to set up a CI/CD by searching for its name.

The main branch we’ll use to trigger our pipeline will be main as a common practice, but you can choose a different one you prefer.

For the Change detection options, we’ll select Start the pipeline on source code change, so whenever we merge code or push directly to the main branch, it will trigger the pipeline.

Click Next.

Step 3: Add build stage

This step is the one we will generate both source bundle artifacts used to deploy both our API and Worker (Windows Service application) to Elastic Beanstalk.

We will also need to use a Cross-region action here due to CodeBuild limitations regarding Windows builds as stated by AWS on this link.

Windows builds are available in US East (N. Virginia), US West (Oregon), EU (Ireland) and US East (Ohio). For a full list of AWS Regions where AWS CodeBuild is available, please visit our region table.

⚠️ Note: Windows builds usually take around 10 to 15 minutes to complete due to the size of the Microsoft docker image (~8GB).

At this point, if you try to change the Region using the select option, the Create project button will disappear, so for now, let’s just click on Create project button and we can change the region in the following screen. And, please, make sure to select one of the regions where Windows builds are available.

Once you’ve selected a region where Windows builds are available, you can start entering all the required information for your build.

For the Environment section, we need to select the Custom image option, choose Windows 2019 as our Environment type, then select Other registry and add the Microsoft Docker image registry URL (mcr.microsoft.com/dotnet/framework/sdk:4.8) to the External registry URL.

Buildspec config can be left as default.

I highly recommend you to have a look at AWS docs Build specification reference for CodeBuild If you don’t know what a buildspec file is. Here is a brief description extracted from AWS documentation.

A buildspec is a collection of build commands and related settings, in YAML format, that CodeBuild uses to run a build. You can include a buildspec as part of the source code or you can define a buildspec when you create a build project. For information about how a build spec works, see How CodeBuild works.

Let’s have a look at our Buildspec file.

version: 0.2

env:
  variables:
    SOLUTION: DotNetFrameworkApp.sln
    DOTNET_FRAMEWORK: 4.8
    PACKAGE_DIRECTORY: .\packages

phases:
  install:
    commands:      
      - echo "Use this phase to install any dependency that your application may need before building it."
  pre_build:
    commands:
      - nuget restore $env:SOLUTION -PackagesDirectory $env:PACKAGE_DIRECTORY
  build:
    commands:
      - msbuild .\DotNetFrameworkApp.API\DotNetFrameworkApp.API.csproj /t:package /p:TargetFrameworkVersion=v$env:DOTNET_FRAMEWORK /p:Configuration=Release
      - msbuild .\DotNetFrameworkApp.Worker.WebApp\DotNetFrameworkApp.Worker.WebApp.csproj /t:package /p:TargetFrameworkVersion=v$env:DOTNET_FRAMEWORK /p:Configuration=Release
      - msbuild .\DotNetFrameworkApp.Worker\DotNetFrameworkApp.Worker.csproj /t:build /p:TargetFrameworkVersion=v$env:DOTNET_FRAMEWORK /p:Configuration=Release
  post_build:
    commands:
      - echo "Preparing API Source bundle artifacts"
      - $publishApiFolder = ".\publish\workspace\api"; mkdir $publishApiFolder
      - cp .\DotNetFrameworkApp.API\obj\Release\Package\DotNetFrameworkApp.API.zip $publishApiFolder\DotNetFrameworkApp.API.zip
      - cp .\SetupScripts\InstallDependencies.ps1 $publishApiFolder\InstallDependencies.ps1
      - cp .\DotNetFrameworkApp.API\aws-windows-deployment-manifest.json $publishApiFolder\aws-windows-deployment-manifest.json
      - cp -r .\DotNetFrameworkApp.API\.ebextensions $publishApiFolder
      - echo "Preparing Worker Source bundle artifacts"
      - $publishWorkerFolder = ".\publish\workspace\worker"; mkdir $publishWorkerFolder
      - cp .\DotNetFrameworkApp.Worker.WebApp\obj\Release\Package\DotNetFrameworkApp.Worker.WebApp.zip $publishWorkerFolder\DotNetFrameworkApp.Worker.WebApp.zip
      - cp -r .\DotNetFrameworkApp.Worker\bin\Release\ $publishWorkerFolder\DotNetFrameworkApp.Worker
      - cp .\SetupScripts\InstallWorker.ps1 $publishWorkerFolder\InstallWorker.ps1
      - cp .\DotNetFrameworkApp.Worker.WebApp\aws-windows-deployment-manifest.json $publishWorkerFolder\aws-windows-deployment-manifest.json
      - cp -r .\DotNetFrameworkApp.Worker.WebApp\.ebextensions $publishWorkerFolder

artifacts:
  files:
    - '**/*'
  secondary-artifacts:
    api:
      name: api
      base-directory: $publishApiFolder
      files:
        - '**/*'
    worker:
      name: worker
      base-directory: $publishWorkerFolder
      files:
        - '**/*'

As you can see, we have a few different phases in our build spec file.

1. install: Can be used, as its names suggest, to install any build dependencies that are required by your application and not listed as a NuGet package.
2. pre_build: That’s a good place to restore all NuGet packages.
3. build: Here’s where we will build our applications. In this example, we are building and packing all our 3 applications.
   1. msbuild .\DemoProject.API\DemoProject.API.csproj /t:package /p:TargetFrameworkVersion=v$env:DOTNET_FRAMEWORK /p:Configuration=Release
      1. msbuild: The Microsoft Build Engine is a platform for building applications.
      2. **\DemoProject.API.csproj: The web application we are targeting in our build.
      3. /t:Package: This is the MSBuild Target named Package which we have defined as part of the implementation of the Web Packaging infrastructure.
      4. /p:TargetFrameworkVersion=v$env:DOTNET_FRAMEWORK: A target framework is the particular version of the .NET Framework that your project is built to run on.
      5. /p:Configuration=Release: The configuration that you are building, generally Debug or Release, but configurable at the solution and project levels.
   2. For .NET Core/5+ we use the .NET command-line interface (CLI), which is a cross-platform toolchain for developing, building, running, and publishing .NET applications.
   3. Last but not least, we have our Worker or Windows Service application build. One of the differences here is the absence of the parameter MSBuild Target parameter /t:build when compared with our DemoProject.API.csproj web API project. Another difference is the folder where all binaries will be published.
4. post_build: After all applications have been built we need to prepare the source bundle artifacts for Elastic Beanstalk. At the end of this phase, CodeBuild will prepare two source bundles which will be referred to by the artifacts section.
   1. In the first part of this phase, we are creating two workspace folders, one for our API and another for our Worker.
      1. Next, we are copying a few files to our API workspace.
         1. **\DemoProject.API.zip: This is the Web API package generated by MSBuild.
         2. **\InstallDependencies.ps1: Optional PowerShell script file that can be used to install, uninstall or even prepare anything you need in your host instance before your application starts running.
         3. aws-windows-deployment-manifest.json: A deployment manifest file is simply a set of instructions that tells AWS Elastic Beanstalk how a deployment bundle should be installed. The deployment manifest file must be named aws-windows-deployment-manifest.json.

1. 1. Our Worker’s source bundle is prepared in the second part of this phase and it contains 2 applications:
      1. One is just an almost empty .NET core Web application required by Beanstalk that we are using as a health check.
      2. The second one is our actual Worker in form of a Windows Service Application.
   2. **\InstallWorker.ps1: Here’s a sample of a PowerShell script used to execute our Worker installer.

aws-windows-deployment-manifest.json: Very similar to the previous one, this file, the only difference is that now we have a specific script containing instructions to install our service in the host machine.

In the artifacts section, CodeBuild will output two source bundles (API and Worker), which will be used as an input for the deploy stage.

Once you finish configuring your CodeBuild project, click on the Continue to CodePipeline button.

Now back to CodePipeline, select the region you created a CodeBuild project, then select it from the Project name dropdown. Feel free to add environment variables if you need them.

Click Next.

Step 4: Add deploy stage

We are now moving to our last CodePipeline step, the deployment stage. This step is to decide where our code is going to be deployed, or what AWS service we’re going to use to get our code to work on.

⚠️ Note: You will notice that we don’t have a way to configure two different deployments, so at this time you can either skip the deploy stage or set up only one application, then fix it later on. I will choose the latter option for now.

Select AWS Elastic Beanstalk for our Deploy provider.

Choose the Region that your Elastic Beanstalk is deployed under.

Then, search and select an Application name under that region or create an application in the AWS Elastic Beanstalk console and then return to this task.

⚠️ Note: If you don’t see your application name, double-check that you are in the correct region in the top right of your AWS Console. If you aren’t you will need to select that region and perhaps start this process again from the beginning.

Search and select the Environment name from your application.

Click Next.

Review

Now it’s time to review the entire settings of our pipeline to confirm before creating.

Once you are done with the review step, click on Create pipeline.

Pipeline Initiated

After the pipeline is created, the process will automatically pull your code from the GitHub repository and then deploy it directly to Elastic Beanstalk.

Let’s customize our Pipeline

First, we need to change our Build step to output two artifacts as stated in the build spec file.

In the new Pipeline, let’s click on the Edit button.

Click on Edit stage button located in the “Edit: Build” section.

Let’s edit our build.

Let’s specify our Output artifacts according to our build spec file. Then, click on Done.

Now, let’s click on the Edit stage button located in the “Edit: Deploy” section.

Here we will edit our current Elastic Beanstalk, then we will add a second one.

Let’s edit our current Elastic Beanstalk deployment first.

Change the action name to something more unique for your application, then select “api” in the Input artifacts dropdown and click on Done.

Let’s add a new action.

Add an Action name, like DeployWorker for instance.

Select AWS Elastic Beanstalk in the Action provider dropdown.

Choose the Region that your Elastic Beanstalk is located.

Select “worker” in the Input artifacts dropdown.

Then, select your Application and Environment name, and click on Done.

Save your changes.

Now we have both of our applications covered by our pipeline.

Confirm Deployment

If we go to AWS Console and access the new Elastic Beanstalk app, we should see the service starting to deploy and then transition to deployed successfully.

⚠️ Note: If you, as in this application repository demo, are creating an AWS WAF, your deployment will fail if the CodePipeline role doesn’t have the right permission to create it.

Let’s fix it!

On AWS Console, navigate to IAM > Roles under IAM dashboard, and find and edit the role used by your CodePipeline by giving the right set of permissions required to CodePipeline be able to create a WAF.

Go back to your CodePipeline and click on Retry.

That will trigger the deploy step again and if you go to your Elastic Beanstalk app, you will see the service starting to deploy and then transition to deployed successfully.

After a few seconds/minutes, the service will transition to deployed successfully.

If we access the app URL, we should see our health check working.

See deployment in action

This next part is to make a change to our GitHub repository and see the change automatically deployed.

Demo application

You can use your repository, but for this part, we’ll be utilizing this one.

Here’s the current project structure.

DotNetFrameworkApp repository contains 3 applications (API, Worker, and a WebApp for the Worker) created with .NET Framework 4.8.

We are also adding an extra security layer using a Web Application Firewall (WAF) to protect our Application Load Balancer, created by Elastic Beanstalk, against attacks from known unwanted hosts.

Code change

Make any change you need in your repository and either commit and push directly to main or create a new pull request and then merge that request to the main branch.

Once pushed or merged, you can take a look at the CodePipeline automatically pull and deploy this new code.

What’s next?

The next step would be to introduce Terraform, have everything we have built here as code, have an automatic way to pass additional environment variables, and introduce logging.

Final Thoughts

AWS CodePipeline when combined with other services can be a very powerful tool you can use to modernize and automatize your Windows workloads. This is just a first step, and you definitely should start planning to have: automated tests, environment variables, and even a better way to have Observability on your application.

DNX has the solutions and experience you need. Contact us today for a blueprint of your journey towards data engineering.