3 min read
Saving runtime variables to Azure DevOps Variable Groups
We use ARM outputs quite extensively in our Azure DevOps pipelines. At times you may need to get outputs from previous steps and store them as...
3 min read
Bernie White has a Powershell Module (PSDocs) that can generate mark down files (*.md) and Stefan Stranger’s blog post shows us how to upload these to Azure DevOps Wiki. We started investigating this as we saw this being a great feature to automate the creation and maintenance of our README.md files within our IaC Templates. The only issue is that our README.md files live side by side with our ARM Templates in the Azure DevOps Repositories and not in the Wiki section that Stefan’s post updates. So the challenge is, how do we make our Azure Pipelines write back the README.md files it dynamically creates on the build agent to the repository?
The solution was to use Git within the pipeline to commit the new or updated README file to the repository.
Requirements
First thing we found was this Microsoft documentation which specifies additional access requirements for the Project Collection Build Service. However, we found that our pipelines were using the <Project>'s Build Service (<Organisation>) account instead and therefore we had to assign the additional access to that account.
Note: Permissions “Read” and “Create Tag” where already Inherited so not need to reassign again.
Create ReadMe Template and Calling Script
Next, we created our own PowerShell template file called ReadMe.doc.ps1 using Bernie’s ARM Template example as a base. For more information regarding how to create the template PowerShell script have a look at Bernie’s example.
# # Azure Resource Manager documentation definitions # # A function to break out parameters from an ARM template function GetTemplateParameter { param ( [Parameter(Mandatory = $True)] [String]$Path ) process { $template = Get-Content $Path | ConvertFrom-Json; foreach ($property in $template.parameters.PSObject.Properties) { [PSCustomObject]@{ Name = $property.Name Description = $property.Value.metadata.description } } } } # A function to import metadata function GetTemplateMetadata { param ( [Parameter(Mandatory = $True)] [String]$Path ) process { $metadata = Get-Content $Path | ConvertFrom-Json; return $metadata; } } # Description: A definition to generate markdown for an ARM template document 'README' { # Read JSON files $metadata = GetTemplateMetadata -Path $InputObject.MetadataFile; $parameters = GetTemplateParameter -Path $InputObject.ARMTemplate; "[?branchName=Dev)](https://dev.azure.com/#######/IaC/_build/latest?definitionId=$($InputObject.PipelineID)&branchName=Dev)" # Set document title Title $metadata.itemDisplayName # Write opening line $metadata.Description # Add each parameter to a table Section 'Parameters' { $parameters | Table -Property @{ Name = 'Parameter name'; Expression = { $_.Name }},Description } # Generate example command line Section 'Use the template' { Section 'PowerShell' { 'New-AzResourceGroupDeployment -Name <deployment-name> -ResourceGroupName <resource-group-name> -TemplateFile <path-to-template>' | Code powershell } Section 'Azure CLI' { 'az group deployment create --name <deployment-name> --resource-group <resource-group-name> --template-file <path-to-template>' | Code text } Section 'Azure Custom Template' { "[](https://portal.azure.com/#create/Microsoft.Template/uri/$($InputObject.LinkedTemplateURI))" } } } We will also create a PowerShell script called CreateReadMe.ps1 that will call the ReadMe.doc.ps1 and pass on the required parameters like our ARM template file and metadafile.
param ( [Parameter(Mandatory=$true)] [string]$WorkingDir, [Parameter(Mandatory=$true)] [string]$PipelineName, [Parameter(Mandatory=$true)] [string]$PipelineID, [string]$DocTemplatePath = '000-Scripts\ReadMe.doc.ps1', [string]$LinkedTemplatePath = 'https://<storageaccount>.blob.core.windows.net/arm/' ) Install-Module -Name PSDocs -Force $templatefile = "$WorkingDir\$PipelineName\azuredeploy.json" $metadatafile = "$WorkingDir\$PipelineName\metadata.json" $PSDocsInputObject = New-Object PsObject -property @{ 'MetadataFile' = $metadatafile 'ARMTemplate' = $templatefile 'LinkedTemplateURI' = [uri]::EscapeDataString($LinkedTemplatePath + $PipelineName +"/azuredeploy.json") 'PipelineID' = $PipelineID 'PipelineName' = $PipelineName } Invoke-PSDocument -Path "$WorkingDir\$DocTemplatePath" -InputObject $PSDocsInputObject -OutputPath "$WorkingDir\$PipelineName" -Instance README Create a BASH script to run the git commands and commit the new README file to the repository
Last we also need to create a BASH script called GITCommitReadMeFile.sh. This will run the necessary git commands needed to upload our new README file to our repository.
BRANCH=$1
FILETOCOMMIT=$2
BUILDNAME=$3
ECHO "Setting git config..."
git config --global user.email "IaCBuildServiceAccount@<domain>"
git config --global user.name "IaC Build Service Account"
ECHO "CHECK GIT STATUS..."
git status
ECHO "CHECK OUT BRANCH..."
git checkout -b $BRANCH
ECHO "GIT ADD..."
git add $FILETOCOMMIT
ECHO "Commiting the changes..."
git commit -m "ReadMe Update from Build $BUILDNAME"
ECHO "Pushing the changes..."
git push -u origin $BRANCH
ECHO "CHECK GIT STATUS..."
git status
Note: We are using
git add $FILETOCOMMITto only commit the README.md file to minimise the risk of this process completely destroying our repository.
Creating a Pipeline Job
Let’s put everything together into a pipeline job. The first task will call the Readme.doc.ps1 file with the required parameters as mentioned above, the second task will add and commit the new file to our repository.
- job: UpdateReadMe
displayName: "Update Template ReadMe File"
steps:
- checkout: self
persistCredentials: true
- task: PowerShell@2
name: 'CreateReadMe'
displayName: 'Create ReadMe File'
inputs:
filePath: '$(System.DefaultWorkingDirectory)\000-Scripts\CreateReadMe.ps1'
arguments: '-WorkingDir $(System.DefaultWorkingDirectory) -PipelineName $(Build.DefinitionName) -PipelineID $(System.DefinitionId) -DocTemplatePath 000-Scripts\ReadMe.doc.ps1 -LinkedTemplatePath $(LinkedTemplatePrefix)'
errorActionPreference: 'silentlyContinue'
- task: Bash@3
name: 'CommitReadMe'
displayName: 'Upload ReadMe File'
inputs:
filePath: '$(System.DefaultWorkingDirectory)\000-Scripts\GITCommitReadMeFile.sh'
arguments: '$(Build.SourceBranchName) "$(Build.DefinitionName)/README.md" $(Build.BuildNumber)'
Running the pipeline
Now we kick off our pipeline and have a look at the results. 
We can see that after the pipeline completes the README.md file has been updated. 
2 min read
Setting an Azure AD group to Azure SQL Database with ARM templates
I was recently looking at a way to automatically set an Azure AD group as the SQL admin for our Azure SQL databases with ARM tempplates. We use SQL...
3 min read
Use Azure DevOps Self Hosted agents with Azure App Service access restrictions
By default, when you deploy a new Azure WebApp, Function app or API app it will be publicly available to the internet. For the current customer I’m...