It seems fitting that the SQL Saturday Silicon Valley on the Microsoft campus in Mountain View, CA., on April 9 took place on the anniversary of the signing of the ENIAC (Electrical Numerical Integrator and Computer) contract in 1943 in the facility where that foundational device resides in present day. Though it’s a far cry from today’s calculating power, ENIAC was cutting edge for the time - capable of 5,000 additions per second.
It was in the spirit of looking back that SQL Saturday Silicon Valley keynote speaker Ross Mistry, Senior Director for Technology and Innovation, Microsoft Technology Center Silicon Valley, gave the audience of more 500 data professionals a preview of the soon-to-be-released SQL Server 2016 and how the new features will assist in the democratization of data.
He opened the keynote by noting that modern computing has its roots in math, and flashed back to 1200 AD when people had to count on their fingers to enable financial transactions and math was the province of the elite. Flash-forward to today when data is the new currency, and Microsoft is a leader in making complex computing systems, really advanced decision-making tools, for anyone who wants them. The SQL Server 2016 features coming in the next version of Microsoft’s Data Platform tools will allow users to save and spend that new currency freely and easily.
“Databases of the past were about recording and retrieving data. Databases of the future are about recording and retrieving intelligence. If you look back at databases of the past, indexing and querying mechanisms were to officially retrieve data. We look to databases of the future (and) we want to create models and algorithms to predict information from our data” asserted Mistry.
SQL Server 2016: Born in the Cloud
SQL Server 2016 has, in a sense, already been released. It’s essentially been running in Microsoft Azure for quite some time. That’s why Microsoft expects the most-stable release of SQL Server to date and why there are fewer service packs for SQL Server releases than years past. The “box” product has been stress-tested in real-world use through over 1.4 million new databases created in Azure over the past year. This is a critical improvement for the product as today’s Database Administrator is no longer simply responsible for typical DBA tasks of years gone by. Today they deal with administration, development, engineering, data science, business intelligence, business analytics and more. They aid in critical decision making processes or, with the maturing of Cortana Intelligence (formerly Cortana Analytics), engineer machine learning processes to derive intelligent decisions for enterprises, small businesses, organizations, governments and academic institutions.
A Learning Path for the Next Iteration of DBAs
Today, Mistry said, every data professional needs to have what he calls a “challenger mindset” and personal learning roadmap to make sense of the scale-out of features and applications that comprise the Microsoft Data Platform. In the past that platform was comprised of SQL Server alone, but that is no longer the case. Excel is a key component of the new Microsoft data strategy and from SQL Server and Excel has sprung Power BI and the suite of Azure Data Platform tools such as Cortana Intelligence, Azure SQL Database, HDInsight, Azure Machine Learning, Data Lake Analytics, Stream Analytics, SandDance and so many more. (We’ll get to SandDance in a bit.) This challenger mindset requires data platform professionals to continually learn and expand skillsets.
Highlighted Data Platform Features
From the abundance of options, Mistry chose to highlight real-time mission critical intelligence, Always Encrypted, Stretch Database, integrated R language support in SQL Server, Cortana Intelligence, and the SandDance data visualization tool.
The overhead in terms of storage, compute, staffing, development, and everything else that goes into supporting a separately architected data warehouse and the lag in populating that data warehouse from the core transactional environment can cripple an institution both financially and competitively. Historically it was necessary to choose between fast and smart when it came to engineered decision-making processes: fast and you could get near-like decisions derived from less-current subsets of your data; smart and you could collect more data – data from more recent transactional collection – but in doing so you played the waiting game and possibly timed yourself out of decisions while your competitors beat you to the punch. Now you don’t have to choose.
Real-Time Decision-Making and In-Memory Columnstore
There is a need for real-time decision-making constructs and Microsoft allows you to build those using tools such as In-Memory Columnstore to derive intelligence directly from the transactional database without impacting the database with costly queries that can impact performance to the point of crippling your SQL Server instance. SQL Server 2016 and In-Memory Columnstore team to provide up to 30x faster online transactional processing (OLTP) and 100x faster analytics. Mistry offered a real world example of a company that uses SQL to collect extensive elevator traffic data. With that data, they’re able to determine with a high degree of accuracy when stores will shut their doors simply by analyzing traffic patterns in the elevators against their real-time transactional database without impacting overall performance.
Mistry stated “You have the ability (now) to do end-to-end mobile BI. If you look at that again (this is) a platform, not just a database. High performance data warehouse capability, the ability to do hybrid transactions whether on premises or taking advantage of the cloud; making it a seamless experience whether it’s on premises, through the cloud, hybrid, and high speed mission critical OLTP.”
Always Encrypted
SQL Server 2016 provides functionality to encrypt data at all states: at rest, in transit and even in the buffer pool. You’re also easily able to encrypt the data from visibility by those with the broadest level of security on your SQL Servers: the DBA. Row level security – only allowing those records to be viewed by end users that should be able to see that data -- is finally implemented in SQL Server 2016 as well. The overhead for these features is easily implemented and supported with the minimal amount of impact currently possible.
Stretch Database
Maintaining archival “cold” data in your production databases can be costly in terms of storage, maintenance, performance, and pure dollar spend to provide resources to support the constantly growing amount of archival data mandated through governmental, regulatory, or internal corporate policy. Stretch Database in SQL Server 2016 and Microsoft Azure provides a viable solution towards reducing the amount of cold data kept in your core transactional databases. Using Stretch Database one can trickle cold rows – or even complete archival tables – to their Azure implementation where storage options are affordable. There is no need for complicated sharing architectures or re-writing application or t-sql code to support this either. How SQL Server presents this data is agnostic to the other components of your data solutions. Additionally, if your queries don’t need to hit this cold data you don’t incur any latency from the implementation of a Cloud data source. At the same time if you have a rare query that only acts upon this cold, Azure data your local resources for tempdb and buffer pool don’t get touched. Since much of this cold data is substantial in size queries that hit this archive data can often “blow out” the buffer pool and tempdb alike. Jarek Kazmierczak, Principal Enterprise Strategist at Microsoft Canada, demoed Stretch Database for the audience, showing just how easy it is to set this up from end to end.
Integrated R Language Support and Cortana Intelligence
Data science and business analytics are critical components to the modern corporate and governmental entities. Integrating R, the powerful statistical language used by data scientists and mathematicians and evolving Cortana Intelligence is vital to support these trends. The Internet of Things has resulted in an explosion of sensors, and devices collecting petabytes of data. Kazmierczak discussed the concept of the AI Winter, a period of time following a stretch of over-hyped technical advancements where innovation tends to stop or stagnate. He contends we’ve exited that period and what should now follow is what has come historically to be a period of higher funding and higher expectations – plus media fear about such advancements.
According to Kazmierczak: “We can argue we are now facing the (next) industrial revolution and the pundits would claim this will be the biggest one, the one which may make the biggest changes to social and political systems and the quality of life – both negative and positive. This industrial revolution will be driven by four factors: data, massive growth of data, artificial intelligence, and cloud which makes it all possible. ”
We’re seeing this now in spades because what the Cloud offers: limitless computing power and the ability to collect unlimited data and drive extremely accurate decisions is here and thanks to Orwellian themes that can be perceived pessimistically as much as it can optimistically. Cortana Intelligence is central to these optimistic advancements in intelligent decisions.
What Cortana Intelligence allows is the ability to compute any data (text, video, image) and compute on a Cloud scale with a convergence of new processes such as Azure Machine Learning and core BI constructs to make powerfully accurate decisions. Cortana Intelligence is really a set of services in the Cloud – many coming from open source technologies like Spark and Hadoop. This is one of the fundamental reasons you see Microsoft embracing open source so heavily. It’s important to be part of this open source ecosystem to advance the ability to derive intelligent decisions from data.
Finally, SandDance
Mistry and Kazmierczak closed the keynote with a popcorn-worthy demonstration of Microsoft’s latest visualization capability: SandDance. SandDance allows you the ability to visualize all your data points in a wide range of formats: standard bar and line charts but also geospatially. While this may not sound like anything new or exciting, think again. This is not aggregating results from your data points and visualizing these aggregations. This is plotting the individual data points in these visualizations and then allowing you the ability to rapidly move these points from visual presentation to the next or to drill into the individual points at varying levels of detail. (Microsoft’s official announcement of SandDance is here.)
SQLSaturday and PASS
The PASS SQLSaturdays are one-day training events for SQL Server professionals that focus on local speakers, providing a variety of high-quality technical sessions, and making it all happen through the efforts of volunteers. Approximately 100 events are held globally across the world annually providing content for today’s Data Professional. They are hosted by PASS, an independent, not-for-profit organization run by and for the community.
