It is a comprehensive solution that encompass business intelligence capabilities and advanced analytics. Microsoft Cortana Intelligence offers a suite of services that are good at automating the process of providing intelligence to business. Having all tools working together makes possible to deliver insights to decision makers. This can be used as a wider solution that cover complete BI Solution.
Cortana intelligence is a grouping of different tools. Each group provides specialized functionality that allows us to manage data flow. There are 5 distinct groups of functionality that we can distinguish.
If you have plans to get benefits from data in your organization then implementing Cortana Intelligence Suite will greatly simplify and benefit your business. Think of this as a bundle that delivers
In this group we have tools to manage data gathering and integration from different sources. We can read data from multiple systems, IoT devices, applications or databases. This tool set allows to organize and clean data during import. You can orchestrate the whole pipeline with various activities required to prepare data for next steps.
Now we have access to second generation of data factory with some very useful features. It has connectivity to plethora of sources so you won’t be disappointed. There are over 70 different data connectors, that enable access to variety of sources. You can use Hadoop, Spark and Hive. The good news for those who use SSIS on premise, you can run your SSIS packages in the cloud without much effort. This will allow to manage data movement and integration from system to system. You can transform it the way you want.
This is the first step in implementing Cortana Intelligence Suite getting a grasp of data flows between systems. Making sure all sources are integrated and point to one location where analysts will do their magic. Having all data ready in one place will provide instant benefits to business people. Why? because it will be easy to visualize and watch how to solve problems.
In any organization there is so many different sources of data hence there is a need to tidy them up. Creating catalog allows to keep all meta-data in one place. You can group all relevant information into controlled environment. This provides benefits of standardization. Making sure business is using the right data for right purpose. Every piece of information can be detailed. Now your business has one place where you control the source of data for each system. This kind of standardization improves quality in organization.
Azure gives you fully managed services to control catalog in a way you want. Your analysts, managers or data scientist can use their tools to interact with catalog in a secure manner. This provides benefits to business and simplifies collaboration between users. Everyone is on the same page when discussing a particular system. This works as a crowdsourcing model, where relevant people can contribute to keep data source up to date with constantly changing business rules.
Now where you created Data Catalog your business can take advantage of Cortana Intelligence Suite. It’s time to promote catalog to wider business audience to start using it and sharing the knowledge about systems and data.
Once we invested lots of time and energy into building systems and connecting hundreds of IoT devices it’s time to ingest them to our data pipelines. These pipelines allow to get insights and manage data flow in real time. This is fully managed service, which I personally like because I don’t have to worry too much about admin work.
As its name suggests Event Hub is good at dealing with events. Whether do you have distributed systems or devices you can control millions of events in real time. You can apply Big Data tools like Apache Kafka to analyze incoming data.
You can send data to event bus at speed of 1 MB per second and consume at 2 MB per second. Each message coming in is partitioned in order of arrival, which has performance benefits. In addition, we can decide on the data retention, you can keep up to 7 days of messages in memory. One of the many useful features is the ability to create consumer groups. Each group can have a separate view on the processed messages in the partition. This gives you the flexibility to manage the way messages are viewed and consumed.
As many of the services dealing with data allows to create georecovery in different region where data is replicated.
Big Data Stores
Cortana intelligence has not forgot about needs for the most demanding and data hungry businesses. As its name suggests Big Data Store have services that will allow to manage huge data volumes commonly known as Big Data. Analysts with such powerful tool can think of processing petabytes of data, which were only available to the limited number of big enterprises. The whole concept of Big Data lies in sending the code to the data. To makes analysis possible data is stored on thousands of machines and actual processing happens on each of them. This is real magic.
Data Lake Store
It is a storage designed to cope with Big Data needs. Fully capable to manage huge volumes of data in fact you can store any amount of data, it has unlimited storage capacity, there is no maximum data limit. There are two generations of Data Lake Storage, Gen1 and Gen2.
The most important feature is HDFS API, which allows to use Big Data tools and create map reduce jobs. Data in this store can be encrypted and hide behind firewall, making it very secure. In addition administrator can set up catalog permission on the Active Directory. It is flexible if you want to have different read or write permissions for each catalog. You can store different data types files, text, video or audio.
The generation V2 allows to take advantage of both worlds of blob storage and data lake storage Gen1. You can treat them as distinct object and set various tiers of hot or cold storage. For very hungry business people it is the right tool that can easily take advantage of Data Lake Analytics.
SQL Data Warehouse
This is a cloud version of Enterprise Data Warehouse. You can treat this as an engine for parallel processing of massive volumes of data. Querying petabytes of data becomes achievable with minimal effort. Its a key component of big data solutions. Design for analytics in mind, and parallel data processing. SQL Data warehouse stores data in tables with columnar storage. The additional benefits is near real time analytics by taking advantage of Azure Databricks streams of data. When you want to know how much it cost to operate, you should know what is DWU – Data warehouse unit: an abstraction of required computation to perform calculation. It is a aggregation of CPU, RAM, IO resources and operations that were used to deliver information.
Machine Learning and Analytics
When you think about predicting future events or advances analytics this service is definitely for you. With Cortana Intelligence you can use machine learning to build predictive models.
Cortana intelligence has included the most powerful tools in Artificial Intelligence box.
It is a full stack of tools to take advantage of Artificial Intelligence. Machine learning is an algorithm that is used to train a
Azure machine learning studio is a SAAS graphical environment to manage ML algorithms. With this tool you can clean data and prepare it for training and further processing. You have ability to select specific data model processing like regressions or clustering.
Azure Machine Learning Compute
This is new service, its predecessor Azure Batch AI is decommissioned.
As a user you have ability to manage all environments and every machine in a cluster. Service will make sure that model runs correctly on the right machines. Assign correct cluster size that will match experiment needs. It has support for many libraries and frameworks as well as many programming languages. You can use your favorite tool like Scikit-learn, Tensorflow, Pytorch or many other open-source technologies. Process can be automated by using scripts describing all relevant properties for the job.
Data Lake Analytics
Cortana Intelligence would not be intelligent if there was no tool for analyzing large data sets. So we now have the
The most important thing for you is to create scripts with logic that Azure service will apply against your data. There is no need to for you to manage the environment Data Lake Analytics automatically provisions resources. From the cost standpoint you pay for what you use when the job is running.
Use the tools that your developers are familiar with – these days Visual Studio is very popular so there will be no need to learn new tools. There is new and powerful language U-SQL easy and powerful that will improve time to implementation.
If you like using open source tools like Apache Hadoop, Spark or Kafka, HBase, Hive, Storm and ML Services. This is
Like in Data Lake Analytics service this one is ready for big data needs. You don’t need a lot of time to set up big data clusters and scale them to your needs. Additional benefits lies in simple integration with other Azure services – Cortana Intelligence tools like Data Factory or Data Lake Storage. If you will have a need you can always add extra tool to take advantage of it.
To make Azure even more powerful Cortana Intelligence bundle have real-time data analytics service. If you have IoT or non-IoT streams of data you can use SQL like language to run different scenarios answering your questions.
You don’t need to worry if you have a lot of devices or sensors data, with this service you can transform them into useful insights. The easiness lies in the SQL like language, which is simple to write and modify. This makes service easy to deploy. Since there is not much hardware to manage your team can start the process in no time. Paying for processing for each job allows to monitor costs easily.
Once you set up basics there is time to crate visuals, for this Power BI is handy. Fully functional dashboards will appear in front of you.
Cortana Intelligence consists of services, which are perfect match for machine learning. It gives them extra power as if they were on steroids. The biggest value that will you get, is the ability to inject AI into your applications.
Cognition is the ability to acquire knowledge and process it.
This service has 5 main categories:
Vision – advanced image processing and classification
Speech – from recognition to translation of voice
Language – spell check and grammar, sentiment analysis, language detection
Search – linked to Bing and allowing various ways of searching Internet
Knowledge – custom decision service with contextual decision making
Cortana is a personal assistant with ability to understand and analyze what human want.
It provides functionality to create smart bots. Cortana sits at the back of your bot working silently as the brain giving you answers. It offers channels that bot can access and use.
Bot Service is a tool set to build and manage smart bots. Bots can use speech, understand natural language and event handle questions and answers. They are good with simple, repetitive tasks that can be triggered by using just voice. The experience for user should be similar to dealing with a person or at least intelligent robot. You may want to introduce this to your customers and create system that process customer requests.
Dashboard and visualizations
Last but not least Cortana Intelligence Suite provides visualization tool. Power BI these days is a business standard. Widely used in many organizations. Its benefits lie in simplicity and flexibility to connect to almost any data source.
Power BI is for everyone, ordinary people like me or you can learn it in no time. It’s a drag and drop tool for that will provide benefits straight away. Return on investment is guaranteed. It connects to large number of sources like Database, analytics tools …. Actually just counted them and there are 96, list below. I am slightly disappointed that there is not 100 to create a nice even number, Microsoft please add few more just to make a nice even number.