Build your Cloud Analytics Solution from an Azure Template

Build your Cloud Analytics Solution from an Azure Template

Cloud computing presents a unique platform for generating complex analytics solutions quickly. With the use of pre-build solution templates, Azure provides an easy mechanism to get started.

If you are beginning the journey to Cloud Computing in Microsoft Azure for analytics, one of the best ways that I can recommend to you to get started is with the use of Microsoft pre-built solution templates. The common tool used for these solutions is the Microsoft Cloud BI tool Power BI which is part of Office 365.

The specific solution templates for Power BI are available here. I'm going to focus on 1 of those templates that I use often for customer POCs: Campaign Brand Management for Twitter. That's because that template includes the back-end Azure cloud functionality to call into Twitter to pull Tweets, process them and mash them up with the Microsoft Azure Cognitive Services for Sentiment Analysis.

The entire concept of template-based solutions is a key pillar of Cloud computing. In leveraging the Cloud, you will want to take advantage of the concepts of quickly deploying and tearing-down environments, eliminating the lengthy and costly exercise of building new physical environments that remain running when not in use. Fail early, fail fast.

In Azure, we use JSON templates to save, store, manage and rebuild environments all the time on projects. Since we can save entire architecture definitions as JSON files, we can also then manage change and deployments using Visual Studio and version control mechanisms very easily.

This is also very well exemplified in the Microsoft Cortana Intelligence Gallery set of Azure solution templates. In this template portfolio, I am going to focus on the DW and Data Science template which will set-up a working Azure Data Warehouse environment with sample data that integrates with Apache Spark in HDInsight. You can essentially stand-up end-to-end Big Data Analytics solutions in Azure with the Cortana Intelligence solutions gallery.

Both of these templates establish Microsoft Azure Cloud architectures for you in minutes using your Azure account. You will need to have either a trial or existing Azure subscription in order to build out these backend data platforms. In both cases, when you are simply investigating or playing around with a template solution, make sure to monitor the billing usage of the solution on your Azure account and pause, shut down, or delete the individual architecture pieces in the Resource Group in the Azure Portal or delete the entire Resource Group if you are done with your work.

For example, below is what is left in my SolutionTemplate Resource Group in my Azure subscriptions. I've taken out the Azure SQL DB because I've already loaded up my Power BI model with data, so I can now remove these pieces if I do not wish to incur continued on-going metered costs in Azure:

Power BI Brand Management for Twitter

Click the "Install Now" button and you will walk through a wizard. Because this solution will take Tweets and run the messages against the Microsoft Sentiment Analysis Cognitive Service API, you will need to use both your Azure subscription here and either a Cognitive Services account from the Azure portal or a free Cognitive Services account.

Notice that when your deployment is complete, you will have an architecture which is depicted above with an Azure App Service account which will call into the Twitter API for Tweet queries. You will specify the query handles in the wizard and a subsequent Azure Function will send the Tweet text to the Sentiment API and store the results in an Azure SQL Database. You are then able to use the pre-built Power BI reports to examine the Tweet analytics, incorporating sentiment for a brand and campaign management scenario.

I have used this solution template many times to simply perform any Twitter analysis, not necessarily tied to a product marketing use case. Simply adjust the reports and dashboards to fit your particular need. Lastly, when I operationalize a Big Data Analytics solution like this in production for customers, I would add in Event Hubs in Azure to capture Tweets closer to real time and bind that with an Azure Function to evaluate sentiment and store the results either in SQL DB as this template provides, or consider placing the results as JSON in Blob Storage or Azure DocumentDB, all depending upon the throughput requirements of the customer's solution.

Here is an example where I just simply used the SQL CAT Twitter feed to run a Twitter analysis, using the Twitter template from the Power BI Solution template, but having nothing to do with campaign or brand analysis:


Data Warehouse with Spark Analytics

The Cortana Intelligence Gallery solutions are less Power BI-centric than the template that I touched on first. Instead, these focus on the Microsoft Azure Analytics platform, branded as Cortana Intelligence Suite. The sample that I show above will install a sample end-to-end Data Warehouse and Hadoop based big data analytics solution in Azure for in within minutes. Data movement is handled with Azure Data Factory, interactive data analytics is provided via Apache Spark on an Azure HDInsight managed Hadoop instance, Azure SQL Data Warehouse is spun-up and sample data from the Million Song dataset is also included.

When you click on the Deploy button, you will be prompted to answer a series of questions related to the Azure subscription that you will use to install the solution. The same rules apply to the above Power BI solution template in terms of billing and using an existing Azure account. One of the really nice additional benefits of the Cortana Intelligence Gallery solutions is that you can also manage your existing solution deployments in your Azure Resource Groups from the website. Just click on the "Manage" link to see your existing deployments. 

When you are all done, you will have a working solution that demonstrates cloud big data analytics in both Jupyter notebooks and Power BI to explore and visualize the raw data and processed results residing in Azure HDInsight and Azure SQL Data Warehouse.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.