Azure Synapse Analytics - Highlights

Paraphrased notes from Microsoft documentation:

Azure Synapse Analytics is an integrated cloud-based platform for big data processing and analysis.

You can use it to build descriptive, diagnostic, predictive, and prescriptive analytics solutions.

Azure Synapse Analytics combines a centralized service for data storage and processing with an extensible architecture through which linked services enable you to integrate commonly used data stores, processing platforms, and visualization tools.

A Synapse Analytics workspace defines an instance of the Synapse Analytics service in which you can manage the services and data resources needed for your analytics solution. 

After creating a Synapse Analytics workspace, you can manage the services in it and perform data analytics tasks with them by using Synapse Studio; a web-based portal for Azure Synapse Analytics.

One of the core resources in a Synapse Analytics workspace is a data lake, in  which data files can be stored and processed at scale. A workspace typically has a default data lake, which is implemented as a linked service to an Azure Data Lake Storage Gen2 container. 

The Azure Synapse SQL system uses a distributed query processing model to parallelize SQL operations, resulting in a highly scalable solution for relational data processing. Azure Synapse Analytics supports SQL-based data querying and manipulation through two kinds of SQL pool that are based on the SQL Server relational database engine:

  • A built-in serverless pool that is optimized for using relational SQL semantics to query file based data in a data lake.
  • Custom dedicated SQL pools that host relational data warehouses.

Apache Spark is an open source platform for big data analytics. In Azure Synapse Analytics, you can create one or more Spark pools and use interactive notebooks to combine code and notes as you build solutions for data analytics, machine learning, and data visualization.

A notebook enables you to interactively run Python code in an Apache Spark pool and embed notes using Markdown.

Azure Synapse Analytics includes built-in support for creating, running, and managing pipelines that orchestrate the activities necessary to retrieve data from a range of sources, transform the data as required, and load the resulting transformed data into an analytical store. Pipelines in Azure Synapse Analytics are based on the same underlying technology as Azure Data Factory. 

Azure Synapse Data Explorer is a data processing engine in Azure Synapse Analytics that is based on the Azure Data Explorer service. Data Explorer uses an intuitive query syntax named Kusto Query Language (KQL) to enable high performance, low-latency analysis of batch and streaming data.

Azure Synapse Analytics can be integrated with other Azure data services for end-to-end analytics solutions like:

  • Azure Synapse Link 
  • Microsoft Power BI 
  • Microsoft Purview 
  • Azure Machine Learning
Common use cases for Azure Synapse Analytics are:
  • Large-scale data warehousing
  • Advanced analytics
  • Data exploration and discovery
  • Real time analytics
  • Data integration
  • Integrated analytics

Comments

Post a Comment

Popular posts from this blog

"Data Prep & Exploratory Data Analysis" course by Maven Analytics

Image
The Maven Analytics course on " Data Prep & Exploratory Data Analysis " by Alice Zhao is a beginner-friendly introduction to machine learning with Python. It covers the key steps of the data science workflow, including: Gathering Data Scoping a Project Modeling Data Sharing Insights Cleaning Data Exploring Data Modeling Data  Sharing Insights The presentation deck which is available as a download crisply summarizes the essential details and is a great reference to keep going back to. The course is well-suited for aspiring data scientists looking to get hands-on experience with the data prep and EDA stages of the machine learning workflow using Python and Jupyter Notebook. 
Read more

Maven Crash Course - Learn Power Query, Power Pivot & DAX in 15 Minutes

Image
In the Maven Crash Course " Learn Power Query, Power Pivot & DAX in 15 Minutes ", Chris Dutton walks through a hypothetical business case while showcasing the power of modern Excel tools like Power Query, Power Pivot and DAX. Power Query is used to extract, transform, and load data from external sources like SQL databases, PDF and CSV files. A relational model is created to join the tables without writing a single formula. Exploratory analysis is done using Power Pivot and Data Analysis Expressions (DAX). Data Analysis Expressions is the language that we can use to define new calculated columns and measures to enhance our Data Model instead of the traditional cell formulas that we'd typically use if our data was stored in worksheets. Pivot Charts and slicers are used to design a simple interactive report that the sales managers can use to analyze revenue trends and product performance.
Read more

Gartner Magic Quadrant for Strategic Cloud Platform Services

Image
Gartner defines strategic cloud platform services (SCPS) as standardized, automated, public cloud offerings integrating infrastructure services (e.g., computing, network and storage), platform services (e.g., application, data and value-added services such as AI/ML) and transformation services (resources to help customers adopt cloud-oriented IT delivery models). Image - Gartner Insights I found interesting from the Gartner Magic Quadrant for Strategic Cloud Platform Services report: The eight SCPS providers & global hyperscalers featured in this Magic Quadrant control over 97% of the total CIPS market. While AWS has maintained profitability and a leading global infrastructure as a service (IaaS) and platform as a service (PaaS) market share of over 42%, its growth rate relative to its key competitors slowed in 2023. Azure public cloud market share is 26%. GCP's overall market share stands at 11.5%. Oracle's strategy of positioning OCI as the top cloud for Oracle software i
Read more