“Modern enterprises are engaged in near-constant development, testing, and deploying software applications, which necessitates a flexible infrastructure that can respond quickly and securely to ever-changing user demands!” – FORBES
The accelerated cloud adoption has ushered in Infrastructure as Code, which has tremendous speed and agility benefits for deploying, changing, and tearing down services for virtual infrastructure. It has in fact enabled enterprises to interact with infrastructure in a programmatic way to automate lifecycle management.
To be specific, in the past few years, organizations have switched to a data-driven approach for making accurate business decisions.
“Much of the impetus for this shift has been the advent of cloud-enabled analytics solutions (like Azure Synapse Analytics) which empowers enterprises to harness their data to derive data insights.”
By shifting to a robust data platform like Azure Synapse Analytics, your organization can leverage a lot of benefits! For instance, you can get…
- Clear & consistent feedback from your data
- Enable them to identify the most successful business practices
- Address specific needs that can help drive up conversion rates
- Discover prime opportunities for improvement
- Optimize your marketing spends
- Streamline supply chains
- Bolster security through anomaly detection and
You can achieve a lot more!
Continue Reading
When shifting to a better data platform like Azure Synapse Analytics has a lot of benefits, the process of transition should also be a hassle-free one! This is where Azure Resource Management (ARM ) templates step in to help you automate your deployment with the power of infrastructure as a code.
Being industry leaders in data strategy and digital transformation, here in this blog, we have explained everything about automating infrastructure deployments, Azure Resource Manager templates, its benefits, and how we are enhancing and accelerating Azure Synapse Analytics adoption using these templates!
Without much ado, let’s plunge in and explore!
Why Infrastructure as a Code for deployments?
Today, enterprises need to repeatedly deploy and redeploy their solutions to cloud and ensure their infrastructure is in a reliable state. As infrastructure has become a part of the iterative process, the division between operations and development teams has disappeared and organizations are in desperate need of a solution that will help them manage infrastructure codes in a unified process.
This is where (IaC), infrastructure as a code, steps in to overcome all the technical impediments that modern enterprises are facing today. By embracing an IaC model, you can easily define the infrastructure that needs to be deployed. This infrastructure code will become a part of your project. You can also store the infrastructure code in a repository, so that anyone on your team can run the code if they want to deploy similar environments.
If you are finding ways to reduce the burden of multiple systems and create a single architecture foundation, your enterprise must switch to a best-in-class data analytics platform like Azure Synapse Analytics.
But deploying this platform using outdated and manual techniques will increase the time for deployment, and also, you will not be able to manage and upscale the platform in an orchestrated manner.
Looking for a solution to speed up the deployment and increase the scalability of your Azure Synapse Analytics platform?
Well, Azure Resource Manager (ARM) template has got your back!
An ARM template is a JavaScript Object Notation (JSON) file that will help you define the infrastructure and configuration for your Azure Synapse Analytics platform. As these templates leverage a declarative syntax, you can just state what you want to deploy without having to write a sequence of programming commands. It is just as simple as specifying the resources you need to deploy and the properties for those resources. Deploying your Azure Synapse Analytics platform using ARM templates will deliver a lot of immediate and long-term benefits!
Some of the common benefits of leveraging ARM templates are…
- Declarative Syntax – With ARM template, you can create and declare your entire Azure infrastructure declaratively. For instance, apart from virtual machines, you can also deploy network infrastructure, storage systems, and any other resources you may require.
- Repeatable Results – ARM gives you the confidence that your templates are deployed in a consistent manner. As the templates are idempotent you can deploy the same template as many times and get the same resource type in the same state. Apart from that, if you come across any errors or failures, you can quickly redeploy your infrastructure with ease.
- Reusability– ARM templates are highly reusable. This aspect of ARM will enhance productivity and decide how quickly you would be able to get your Azure Synapse Analytics platform. With ARM templates you can deploy your complete infrastructure at a speed and quality no human being can match.
- Enhanced Orchestration – Even if you are deploying your resources in an interdependent manner, the Azure Resource Manager will orchestrate your deployment in the correct order. If it’s feasible ARM will also deploy your resources in parallel, so you can accelerate and speed up your deployment process. Instead of giving multiple commands, with ARM, you can make the deployment happen with just one command!
- Flexibility – With Azure Resource Manager, you have the flexibility to break the templates into smaller reusable components, and if required you can link them together at deployment time.
- Extensibility – During deployment, if you wish to extend your ability to set up resources, you can easily do it with the help of deployment scripts. Your end-to-end environment setup can be completed in a single ARM template, with deployment scripts.
- Simplifies Testing & Validation– When you are performing automated deployment using ARM templates, you can check whether the template is going by the recommended guidelines by using the ARM template toolkit. Even before deployment, you can check the preview of changes by using a what-if operation. Before initiating the deployment, ARM’s built-in validation will check the templates to make sure that the deployment will turn out to be a success.
- Tracked Deployments – When you are deploying Azure Synapse Analytics workspace, you can easily get visibility into the deployment history and get information about the template deployment.
- CI/CD Integration – By leveraging Azure DevOps and Azure Resource Manager template tasks, you can harness the benefits of Azure Pipelines to continuously build and deploy the resources in Azure Synapse Analytics.
Note: ARM templates can be leveraged for automating deployment for pretty much everything. By leveraging IaC you can automate deployment for Synapse, dedicated SQL pools, storage accounts, virtual networks, virtual machines, key vaults, and logic apps.
Having discussed the benefits, let us now see how ARM templates are leveraged for automating the deployment of resources in Azure Synapse Platform.
The below infrastructure as a code (IaC) deploys Azure Log Analytics, which is a monitoring tool that enables users to monitor the logs of resources that are configured in it. This will be kept as a centralized resource where each Azure resource will be connected and also the respective logs can be viewed here!
“type”: “Microsoft.Blueprint/blueprints/artifacts”,
“properties”: {
“displayName”: “Azure Activity Log – Subscription Level Monitoring”,
“description”: “”,
“dependsOn”: [
“01a-decentralizedMonitorLogsWorkspace-template”
],
“template”: {
“$schema”: “https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#”,
“contentVersion”: “1.0.0.0”,
“parameters”: {
“localBU”: {
“type”: “string”,
“metadata”: {
“description”: “BU for the resource names deployed in this template”
}
},
“monitoringApproach”: {
“type”: “string”,
“metadata”: {
“description”: “Organizational monitoring approach. For more info: https://docs.microsoft.com/en-us/azure/azure-monitor/platform/design-logs-deployment#important-considerations-for-an-access-control-strategy”
}
},
“decentralizedLogAnalytics_workspaceId”: {
“type”: “string”,
“metadata”: {
“description”: “ResourceID of the Decentralized of Functional approach Log Analytics workspace in which resource logs should be saved.”
}
},
“centralizedLogAnalytics_workspaceId”: {
“type”: “string”,
“metadata”: {
“description”: “ResourceID of the Centralized Log Analytics workspace in which resource logs should be saved.”
}
},
“monitorLogRetention”: {
“type”: “int”,
“metadata”: {
“description”: “Days to retain diagnostic logs”
}
},
“deploymentEnvironment”: {
“type”: “string”,
“metadata”: {
“description”: “Deployment environment. Production, QA & Development provide enhanced security and monitoring. Sanbox is meant for POC and short lived exploration only. See the environment mapping to understand more details.”
}
}
},
“variables”: {
“settingName”: “[guid(concat(parameters(‘localBU’),’subDiagnostics’))]”,
“environmentConfig”: {
“p”: {
“AdvancedSecurity”: “Yes”,
“diagnostics”: “Yes”
},
“d”: {
“AdvancedSecurity”: “Yes”,
“diagnostics”: “Yes”
},
“s”: {
“AdvancedSecurity”: “Yes”,
“diagnostics”: “Yes”
},
“x”: {
“AdvancedSecurity”: “No”,
“diagnostics”: “No”
},
“dr”: {
“AdvancedSecurity”: “Yes”,
“diagnostics”: “Yes”
},
“q”: {
“AdvancedSecurity”: “Yes”,
“diagnostics”: “Yes”
}
},
“workspaceId”: “[if(equals(parameters(‘monitoringApproach’), ‘Decentralized’), parameters(‘decentralizedLogAnalytics_workspaceId’),parameters(‘centralizedLogAnalytics_workspaceId’))]”
},
“resources”: [
{
“condition”: “[equals(variables(‘environmentConfig’)[parameters(‘deploymentEnvironment’)].diagnostics,’Yes’)]”,
“type”: “Microsoft.Insights/diagnosticSettings”,
“apiVersion”: “2017-05-01-preview”,
“name”: “[variables(‘settingName’)]”,
“location”: “global”,
“properties”: {
“workspaceId”: “[variables(‘workspaceId’)]”,
“logs”: [
{
“category”: “Administrative”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
},
{
“category”: “Security”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
},
{
“category”: “ServiceHealth”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
},
{
“category”: “Alert”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
},
{
“category”: “Recommendation”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
},
{
“category”: “Policy”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
},
{
“category”: “Autoscale”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
},
{
“category”: “ResourceHealth”,
“enabled”: true,
“retentionPolicy”: {
“days”: “[parameters(‘monitorLogRetention’)]”,
“enabled”: true
}
}
]
}
}
]
},
“parameters”: {
“localBU”: {
“value”: “[parameters(‘resourceNameBU’)]”
},
“monitoringApproach”: {
“value”: “[parameters(‘monitoringApproach’)]”
},
“decentralizedLogAnalytics_workspaceId”: {
“value”: “[artifacts(’01a-decentralizedMonitorLogsWorkspace-template’).outputs.decentralizedLogAnalytics_workspaceId]”
},
“centralizedLogAnalytics_workspaceId”: {
“value”: “[parameters(‘centralizedLogAnalytics_workspaceId’)]”
},
“deploymentEnvironment”: {
“value”: “[span(‘deploymentEnvironment’)]”
},
“monitorLogRetention”: {
“value”: “[parameters(‘monitorLogRetention’)]”
Once adding all the values and template generation, the template can be deployed using Azure portal or any of your preferred deployment options. The deployment history, the parameter values passed on and the other output values can be reviewed using the Azure portal.
As you can see, setting up your Azure Synapse Analytics platform using ARM templates is super easy! You can provision your desired infrastructure in a snap and move forward creating more value out of your data.
Not just the infrastructure set up! By leveraging ARM templates, you can also promote code from lower environment to higher environment in a completely hassle-free manner.
Now, we will share an instance of how we automate the deployment of Azure Synapse Analytics Objects by promoting the code from the lower environment to higher environments.
First, our data engineers will publish/create the required Azure Synapse Analytics objects in the Development environment. Now with ARM, the objects will be turned into templates and moved to the ‘Testing’ environment using CI/CD piplelines.
Once moved to QA the code will be deployed as objects and again a similar process will be followed in the ‘Production’ environment. Finally, all the Azure Synapse Analytics objects will be deployed successfully! And as you know, with codes your deployment would be performed in a hassle-free manner!
Want to streamline your deployment in a clearly defined workflow? Our DataOps experts are all geared up to spin as many as you want! We are Microsoft Gold partners! And we have automated deployment for various Azure resources.
