Microsoft Ignites the Connected Enterprise

Microsoft's annual conference for IT pros took place this week. As such, this sprawling monolith of enterprise computing had a lot to announce about a wide range of products and services. To commemorate the occasion, Microsoft even published an 88 page e-book listing all the announcements. Long gone are the days when Windows and Office dominated these events, although the resurrection of the previously deprecated OneNote 2016 is very welcome.

Instead, it is Azure which dominates the Book of News, with well over half the announcements related to Microsoft's public cloud platform in one shape or other. The flagship announcements from the show are Azure Arc, a hybrid cloud version of Azure, and Azure Synapse Analytics, which is being pitched as a next generation data warehouse. From the Office side of the house, there is also Project Cortex which is a self-organising knowledge base. The common theme linking these announcements is AI. Microsoft have given an AI facelift to key components of two of their most successful products: SQL Server and SharePoint.

Azure Synapse: A Connected Database

The pitch for Azure Synapse is based around scale. Microsoft's database products are hugely popular both on-premise and in the cloud, but they have a reputation for being not quite as scalable as their competitors. Oracle have built their business on being the database that can handle the largest data volumes, whilst Google are seeing increasing adoption of their BigQuery data warehouse service. Synapse is Microsoft's answer to the latter threat. The traditional Enterprise Data Warehouse (EDW) still has a place in the technology stack of every large business, but its central role has been usurped by other platforms. Data analysts are increasingly using technologies such as data lakes to capture and query data for their analyses because they are much more flexible than the EDW.

Synapse is intended to be much more flexible than an EDW, with advanced data transformation capabilities and built-in integration with Power BI. In effect, what Microsoft have done is combine a data warehouse and a data lake into a single product, before adding a set of analysis tools and a Machine Learning engine on top. The AI engine allows predictive data models to be run on the data in Synapse to normalise the data using full text analysis, spot trends or to calculate lead scores and other derived profile information. These are all key function of a modern data warehouse, where the scalability and rigidity of a data warehouse is still an advantage.

By adding the best capabilities of a data lake, the disadvantages of a traditional data warehouse are mitigated. The modern enterprise has thousands of different data sources that need to be fed into the warehouse. These need to be converted into the structure of the EDW before they can be ingested. Synapse avoids this by storing the raw data format, carrying out the required data transformations in the database itself and then copying it into the desired format. This allows data analysts to work with both the raw data and the standardised version of the data as required. The built-in data analysis tools enable analysts and data scientists to carry out this work directly in the platform using their preferred coding language or using WYSIWYG tools. The results of the query can then be streamed into Power BI for visualisation and presentation to the business. This all-in-one has significant advantages which will allow Microsoft to protect their position as a leading database vendor in a fast moving market.

Project Cortex: Enterprise Knowledge Network

Enhanced Platform Integration also forms the foundation for Project Cortex, a new Office 365 application that organises SharePoint and related content into a topic based knowledge base. The pitch is that solves the content discovery problem, automatically delivering the relevant content at the right time using AI to classify all the content stored across the Office 365 portfolio and beyond. It does this using metatags, both those added to files in SharePoint by users as well as automatically generated metatags extracted by content using AI based rich text analysis. However, the scope of Project Cortex expands well beyond SharePoint to conversations in Teams, emails in Outlook and records stored in integrated systems.

As with any AI based tool, Cortex can be trained to scan particular document types and extract the critical pieces of information in a structured way. This can include extracting contract details from commercial documents, as well as text recognition from whiteboard images or request details from PDF forms. The real power is the ability to extract keywords and phrases to derive topic information from written material such as documents and presentations.

Tagged documents and resources are then added to automatically generated topic pages that display not just relevant topics but subject matter experts within the business and related topics. All this is built on top of SharePoint and the key topics and themes identified by the business. As such, it integrates with the rest of the Office 365 application suite so that topic pages and content can be linked to and will appear in Outlook and Teams. A whole host of third party services such as Salesforce are also supported by the service.

The value of Project Cortex lies in its ability to automate one of the most frequently neglected tasks in any business: maintenance of the corporate knowledge base. The value of a knowledge base is directly proportional to the amount of effort put into maintaining it. Even the best designed repository will fail if content is missing or out of date, and all too often people don't have time to put in the necessary effort to maintain the content. What makes it worse, is that the missing content generally exists elsewhere in the business for some other purpose and just needs copying into the relevant location or format. Finding that content is a full time job that few organisations can afford to invest in when other forms of enablement are much more important. That is where the value of AI lines: in the ability to automate labour intensive tasks that are otherwise essential to the business.