Chendrayan Venkatesan

4 minute read

Introduction

Of late, I got an exciting task. Yes, it’s not challenging but requires a pinch of concentration. The requirement is to copy files from the source storage account to multiple destinations (storage accounts). Why? Infrastructure is designed and developed to sync few JSON files from master (management) subscriptions to all other subscriptions. Yes, it is not easy to follow the theory. The picture below depicts the functionality

Functional flow

I hope it’s not confusing! In short, the ask is to copy the POLICY.JSON file to individual subscriptions (NP – Non-Prod and P - Prod), respectively.

Solution

Indeed, you can try alternative solutions. In this blog post, I cover the solution I built using the event grid trigger. Yes, I need to keep my solution neat and straightforward. The main reason is flexibility, and knowing the workplace culture, I didn’t opt for another solution.

Note: The focus is to demo the event grid trigger. Do not ask why not management groups for inheriting policies? Or alternate solutions.

Before moving to the code, let me write down the requirements in few steps

  1. A change in the MGMT policy file should backup the destinations subscription file (Current)
  2. Replace the new file from MGMT.
  3. The event should trigger for NON-PROD and PROD (Depends on the change)

Step 1 - Create Azure Event Grid Subscritpion

#region - Generate a bearer token
$AzureRmProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
$CurrentAzureContext = Get-AzContext
$ProfileClient = New-Object Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient($azureRmProfile)
$Token = $profileClient.AcquireAccessToken($currentAzureContext.Subscription.TenantId)
$Headers = @{Authorization = "Bearer {0}" -f ($Token.AccessToken) }
#endregion

#region - REST API to Create Azure Event Grid Subscription
$SubscriptionId = ""
$ResourceGroup = ""
$ResourceId = ""
$SubjectBeginsWith = ""
$SubjectEndsWith = ""
$FunctionApp = ""
$Function = ""
$Body = [pscustomobject]@{
    name       = "ForDemo"
    properties = [pscustomobject]@{
        topic               = "/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroup)/providers/Microsoft.Storage/StorageAccounts/storageaccountname"
        destination         = [pscustomobject]@{
            endpointType                    = "AzureFunction"
            properties                      = [pscustomobject]@{
                resourceId                    = "/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroup)/providers/Microsoft.Web/sites/$($FunctionName)/functions/$($Function)"
                maxEventsPerBatch             = 1
                preferredBatchSizeInKilobytes = 64
            }
            filter                          = [pscustomobject]@{
                subjectBeginsWith  = $SubjectBeginsWith
                subjectEndswith    = $SubjectEndsWith
                includedEventTypes = @('Microsoft.Storage.BlobCreated', 'Microsoft.Storage.BlobDeleted')
            }
            
            enableAdvancedFilteringOnArrays = $true
        }
        eventDeliverySchema = "EventGridSchema"
    }
} | ConvertTo-Json -Depth 10
$Uri = "https://management.azure.com/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroup)/providers/Microsoft.Storage/StorageAccounts/storageaccountname/providers/Microsoft.EventGrid/eventSubscriptions/ForDemo?api-version=2020-10-15-preview"
Invoke-RestMethod -Uri $Uri -Headers $Headers -Method Put -Body $body -ContentType 'application/json'
#endregion

Step 2 - Event Grid (Function.Json)

{
  "bindings": [
    {
      "type": "eventGridTrigger",
      "name": "eventGridEvent",
      "direction": "in"
    }
  ]
}

Step 3 - Event Grid Function (run.ps1)

param($eventGridEvent, $TriggerMetadata)
$Result = $eventGridEvent | ConvertTo-Json | ConvertFrom-Json

#region - AUTHENTICATION 
$TenantID = "$ENV:TenantID"
$ClientID = "$ENV:ClientID"
$ClientSecret = "$ENV:ClientSecret"
$Credential = [pscredential]::new($ClientID , ($ClientSecret | ConvertTo-SecureString -AsPlainText -Force))
Connect-AzAccount -Tenant $TenantID -Credential $Credential -ServicePrincipal
#endregion

#region
$Query = "Resources 
| where type =~ 'Microsoft.Storage/storageAccounts' and name matches regex '.*NAMINGCONVENTION'
| project id, name, location, subscriptionId, resourceGroup, environment = case (
    tostring(name) contains 'prod', 'PROD',
    'NON-PROD'
)"
$PageSize = 1000
$Iteration = 0
$SearchParams = @{
    Query = $($Query)
    First = $PageSize
}
[System.Collections.ArrayList]$StorageAccounts = @()
do {
    $Iteration += 1
    $PageResults = Search-AzGraph @SearchParams -Verbose
    $SearchParams.Skip += $PageResults.Count
    $StorageAccounts.AddRange($PageResults)
} while ($PageResults.Count -eq $PageSize)
#endregion

#region - Switch Case
switch ($Result.subject) {
    '/blobServices/default/containers/policies/blobs/Blob-Name/Policies.json' {
        $Params = @{
            Uri             = 'https://aka.ms/downloadazcopy-v10-windows'
            OutFile         = 'AzCopy.zip'
            UseBasicParsing = $true
            Verbose         = $true
        }
        Invoke-RestMethod @Params
        Expand-Archive ./AzCopy.zip ./AzCopy -Force -Verbose
        Get-ChildItem ./AzCopy/*/azcopy.exe | Copy-Item -Destination "./AzCopy.exe" -Verbose
        ./AzCopy.exe copy 'https://storageaccountname.blob.core.windows.net/policies/Blob-Name/Policies.json' '.\Policies.json'
        $Exemption = (Get-Content ".\Function-Name\exemption.json" | ConvertFrom-Json)
        # $Items = $StorageAccounts | Where-Object { ($_.subscriptionId -notin @($Exemption)) -and ( { $_.environment -eq 'PROD' }) } 
        $Items = $StorageAccounts | Where-Object { ($($_.environment -eq 'PROD')) -and $($_.subscriptionId -notin $($Exemption)) } 
        foreach ($Item in $Items) {
            try {
                Set-AzContext -Subscription $($Item.subscriptionId)
                $Context = (Get-AzStorageAccount -ResourceGroupName $($Item.resourceGroup) -Name $($Item.name)).Context
                Get-AzStorageBlobContent -Container 'policies' -Blob 'Policies.json' -Destination ".\$($Item.name)-Policies.json" -Context $Context -Force -Confirm:$false
                Set-AzStorageBlobContent -Container 'policies' -Blob "Backup/$($Item.name)-Policies.json" -File ".\$($Item.name)-Policies.json" -Context $Context -Force
                Set-AzStorageBlobContent -Container 'policies' -Blob 'Policies.json' -File ".\Policies.json" -Context $Context -Force -Confirm:$false
            }
            catch {
                $_.Exception.Message
            }
        }
    }
    '/blobServices/default/containers/policies/blobs/Blob-Name-01/Policies.json' {
        $Params = @{
            Uri             = 'https://aka.ms/downloadazcopy-v10-windows'
            OutFile         = 'AzCopy.zip'
            UseBasicParsing = $true
            Verbose         = $true
        }
        Invoke-RestMethod @Params
        Expand-Archive ./AzCopy.zip ./AzCopy -Force -Verbose
        Get-ChildItem ./AzCopy/*/azcopy.exe | Copy-Item -Destination "./AzCopy.exe" -Verbose
        ./AzCopy.exe copy 'https://storageaccountname.blob.core.windows.net/policies/Blob-Name-01/Policies.json' '.\Policies.json'
        $Exemption = (Get-Content ".\Function-Name\exemption.json" | ConvertFrom-Json)
        # $Items = $StorageAccounts | Where-Object { ($_.subscriptionId -notin @($Exemption)) -and ( { $_.environment -eq 'NON-PROD' }) } 
        $Items = $StorageAccounts | Where-Object { ($($_.environment -eq 'NON-PROD')) -and $($_.subscriptionId -notin $($Exemption)) } 
        foreach ($Item in $Items) {
            try {
                Set-AzContext -Subscription $($Item.subscriptionId)
                $Context = (Get-AzStorageAccount -ResourceGroupName $($Item.resourceGroup) -Name $($Item.name)).Context
                Get-AzStorageBlobContent -Container 'policies' -Blob 'Policies.json' -Destination ".\$($Item.name)-Policies.json" -Context $Context -Force -Confirm:$false
                Set-AzStorageBlobContent -Container 'policies' -Blob "Backup/$($Item.name)-Policies.json" -File ".\$($Item.name)-Policies.json" -Context $Context -Force
                Set-AzStorageBlobContent -Container 'policies' -Blob 'Policies.json' -File ".\Policies.json" -Context $Context -Force -Confirm:$false
            }
            catch {
                $_.Exception.Message
            }
        }
    }
    default {
        "No Action Required"
    }
}
#endregion

Summary

Whenever the file policies.json is changed in PROD or NON-PROD, the application subscription gets changed with a backup. Yes, it’s possible to download the AzCopy executable and use it in Function. So, for a demo, I used both Az cmdlet and AzCopy CLI. Why Exemtion.JSON? To skip few subscriptions, which requires an exemption from the standard governance policies.

comments powered by Disqus