This series of posts is about how to use Azure DevOps and Pipelines to set up a basic web application, app service infrastructure, and CI/CD pipelines – with everything saved as code in our source code repository.

reference-architecture-diagram-2

Previously I’ve written about how to configure multi-stage pipelines in YAML – I created a simple skeleton of different stages for creating infrastructure and deploying a website to integration, test and demonstration environments. However, the skeleton YAML had no actual build/deploy functionality.

This time I’ll cover a couple of different topics:

  • Creating resource groups for each environment in our overall solution (integration, testing and demonstration).
  • Reducing code duplication by parameterising YAML templates, so that the main pipeline can call smaller YAML templates. This simplifies the overall structure.

As always, my code is available on GitHub here.

A simple YAML template

In the previous post, I created a series of skeleton stages in my pipeline that simply wrote text to the standard output using the echo command. I’d like to start including code in my stages that now does something useful, and the first step in each stage is to create a resource group that can hold all the logically related pieces of infrastructure for each of the 3 environments (integration, test and demo).

It was pretty obvious that there was some similarity in the stages:

  • Build integration infrastructure and deploy the website to this
  • Build test infrastructure and deploy the website to this
  • Build demo infrastructure and deploy the website to this

This kind of repeating pattern is an obvious signal that the code could be parameterised and broken out into a re-usable block.

I’ve created a sample below of how I could do this – the block of code has 4 parameters:

  • name, for the name of the job
  • vmImage, for the type of vm I want my agent to be (e.g. windows, ubuntu etc)
  • displayName, to customise what I want the step to display in my pipeline,
  • resourceGroupName, obviously for the name of the resource group.

Because I just want to change one thing at a time, the code below still just echoes text to standard output.

parameters:
  name: '' 
  vmImage: ''
  displayName: ''
  resourceGroupName: ''

jobs:
- job: ${{ parameters.name }}
  pool: 
    vmImage: ${{ parameters.vmImage }}
  steps:
  - script: echo create resource group for ${{ parameters.resourceGroupName }}
    displayName: ${{ parameters.displayName }}

And I can use this in my main pipeline file – azure-pipelines.yml – as shown below. It’s pretty intuitive – the “jobs” section has a “template” which is just the path to where I’ve saved the template in my source code repository, and then I list out the parameters I want to use for this job.

- stage: build_integration
  displayName: 'Build the integration environment'
  dependsOn: build
  jobs:
  - template: ci/templates/create-resource-group.yml
    parameters:
      name: create_infrastructure
      displayName: 'First step in building the integration infrastructure'
      vmImage: 'Ubuntu-16.04'
      resourceGroupName: integration

Using Tasks in Azure Pipelines – the Azure CLI task

I’ve updated my template in the code below – instead of simply writing what it’s supposed to do, it now uses an Azure CLI task to create a resource group.

The Azure CLI makes this kind of thing quite simple – you just need to tell it the name of your resource group, and where you want to host it. So if I wanted to give my resource group the name”development-rg” and host it in the “UK South” location, the command would be:

az group create -n development-rg -l uksouth

Open the pipeline in Azure DevOps as shown below, and place the cursor where you want to add the task. The “tasks” sidebar is open by default on the right hand side.

task1

In the “tasks” sidebar there’s a search box. I want to add an Azure CLI task so that’s the text I search for – it automatically filters and I’m left with only one type of task.

task2

After clicking on the “Azure CLI” task, the sidebar refreshes to show me what information I need to supply for this task:

  • the Azure subscription to use,
  • what kind of script to run (inline or a file), and
  • the script details (as shown below, I just put the simple resource group creation script inline)

task3

Finally when I click the “Add” at the bottom right of the screen, the YAML corresponding to the task is inserted to my pipeline.

It might not be positioned quite correctly – sometimes I have to hit tab to move it to the correct position. But the Azure DevOps editor uses the red wavy line under any elements which it thinks are in the wrong place to help out.

task4

I find this technique quite useful when I want to generate the YAML for a task, as it makes it easier for me to modify and parameterise it. I’d find it quite hard to write YAML for DevOps tasks from scratch, so I’ll take whatever help I can get.

I’ve tweaked the generated code to parameterise it, and included it as a task in my template below (highlighted in red). I always want to deploy my code to the “UK South” location so I’ve left that as an unparameterised value in the template.

parameters:
  name: '' 
  vmImage: ''
  displayName: ''
  resourceGroupName: ''
  subscription: ''

jobs:
- job: ${{ parameters.name }}
  pool: 
    vmImage: ${{ parameters.vmImage }}
  steps:
  - task: AzureCLI@1
    displayName: ${{ parameters.displayName }}
    inputs:
      azureSubscription: ${{ parameters.subscription }}
      scriptLocation: 'inlineScript'
      inlineScript: az group create -n ${{ parameters.resourceGroupName }}-rg -l uksouth

And as shown earlier, I can call it from my stage by just referencing the template location, and passing in all the parameters that need to be populated for the job to run.

trigger:
- master

variables:
  subscription: 'Visual Studio Professional with MSDN'

stages:

# previous stages...

- stage: build_integration
  displayName: 'Build the integration environment'
  dependsOn: build
  jobs:
  - template: ci/templates/create-resource-group.yml
    parameters:
      name: create_infrastructure
      displayName: 'First step in building the integration infrastructure'
      vmImage: 'Ubuntu-16.04'
      resourceGroupName: integration
      subscription: $(subscription)

# next stages...

And now we have a pipeline that builds the architecture shown below. Still no app service or website, but I’ll write about those in future posts.

resource-group-architecture-diagram

Ironically, there’s now more lines of code in the solution than there would have been if I had just duplicated all the code! But that’s just because this is still pretty early skeleton code. As I build out the YAML to generate infrastructure and do more complex things, the value will become more apparent.

Service connections

Last note – if you take a fork of my repository and try to run this yourself, it’ll probably not run because my Azure subscription is called “Visual Studio Professional with MSDN” and your subscription is probably called something else. You could change the text of the variable in the azure-pipelines.yml file in your fork, or you could also set up a Service Connection with this name.

Setting one up is quite straightforward – hit “Project settings” which lives at the bottom left of your Azure DevOps project, which opens up a window with a list of settings. Select “Service connections” as shown below, and select the “Azure Resource Manager” option after clicking on “New service connection”.

serviceconnection1

This will open a dialog where you can specify the connection name (I chose to call my “Visual Studio Professional with MSDN” but you could call it whatever you want), and you can also select your subscription from the dropdown level. I didn’t need to specify a resource group, and just hit ok. You might be asked for your Azure login and password during this process.

serviceconnection2

Whatever you decided to call your service connection will be the text you can use in your pipeline to specify the subscription Azure should use.

Wrapping up

There’s been a lot of words and pictures here to achieve only a little progress, but it’s a big foundational piece – instead of having a huge YAML pipeline which could become unmaintainable, we can break it up into logical parameterised code blocks. We’ve used the Azure CLI task as an example of how to do this. Next time I’ll write deploying some ARM templates to create an Azure App Service in each of the three environments.

2 thoughts on “Everything as Code with Azure DevOps Pipelines: C#, ARM, and YAML: Part #3, resource groups and YAML templates

Comments are closed.