Azure Databricks Training

Azure Databricks Course:
Databricks is an Apache Spark-based analytics platform. Microsoft has optimized Databricks for Azure cloud services platform. This service is available by the name of Azure Dataricks. Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace. Azure Databricks enables collaboration between data scientists, data engineers, and business analysts. Python Programming and Fundamental SQL & databases are the prerequisites of Azure Databricks training. This Azure Databricks course starts with the concepts of the big data ecosystem and Azure Databricks. It then covers internal details of Spark, RDD, Dataframes, workspace, Jobs, Kafka, Streaming and various data sources for Azure Databricks. Advanced concepts of Azure Databricks such as Caching and REST API development is covered in this training. This Azure Databricks training is delivered by our experts at our Bangalore office, as well as at client locations across India and South-East Asia in cities such as Bangalore, Chennai, Hyderabad, Pune, Mumbai, Delhi, Noida, Gurgaon, Singapore, Hong Kong and many more.

Azure Databricks Course Curriculum

1. Introduction to Azure Databricks

Azure Databricks Concepts

Workspace

Interaction Management

Data Management

Computation Management

Model Management

Authentication and Authorization

2. Introduction to Big Data

Big Data

Understanding distributed computing

Introduction to Hadoop

HDFS, YARN, MapReduce

Limitations of Hadoop

Introduction to Spark

Introduction to Kafka

Introduction to NiFi

Hive, Cassandra

3. Internal Details of Spark

Driver

Executors

Partitions

Jobs

Stages

Tasks

Resilient Distributed Data-structure

DataFrames as a High Level Datastructure

4. Foundations of Spark using RDD

Basics of Distributed Computing

Resilient Distributed Dataset

Simple Transformers - map,filter,groupby

Actions - Collect, count, foreach

Complex api - combinebykey

Caching, Debugging

Important Configuration

5. Data Wrangling using DataFrames

Creating DataFrames from collections

Creating a DataFrame from csv,json etc.

DataFrame Row

DataFrame Column

Creating tables from dataframe

SQL query

DataFrame Grouping

DataFrame Functions

User Defined Functions (UDF)

6. Workspace

Python to Folders

Objects - move, delete, restore

Recently used objects

Search the Workspace

7. Jobs

View jobs

Create a job

View job details

Run a job

Run a job with different parameters

Notebook job tips

JAR job tips

View job run details

Export job run results

Edit a job

Delete a job

Library dependencies

Job options

Control access to jobs

8. Spark Machine Learning

Overview of Machine Learning

Transformer

Estimators

Classification

Regression

Clustering

Pipeline

Predicting the chances of infant survival with ML

Loading the data

Creating transformers

Creating an estimator

Creating a pipeline

Fitting the model

Evaluating the performance of the model

Saving the model

Parameter hyper-tuning

Grid search

Train-validation splitting

Other features of PySpark ML in action

Feature extraction

Discretizing continuous variables

Standardizing continuous variables

Classification

Clustering

9. Serverless

Display Pools

Create a Pool

Pool Configurations

Pool Size and Auto Termination

Instance Types

AWS Configurations

Pool Tags

Edit a Pool

Delete a Pool

Use a Pool

Requirements

Attach a cluster to a pool

Inherited configuration

10. Getting started with Kafka

Publish/Subscribe Messaging

Enter Kafka

Why Kafka?

The Data Ecosystem

Kafka’s Origin

Getting Started with Kafka

11. Spark Structured Streaming

What is Spark Streaming?

Why do we need Spark Streaming?

What is the Spark Streaming application data flow?

A quick primer on global aggregations

Introducing Structured Streaming

Kafka Integration with Spark

Streaming with Databricks Labs - Kafka

12. Data Sources with databricks

Connecting to SQL Databases using JDBC

Azure Blob Storage

Azure Data Lake Storage

Azure Cosmos DB

Azure SQL Data Warehouse

Binary Files

Import Hive Tables

MongoDB

Avro Files

CSV Files

JSON Files

LZO Compressed Files

Parquet Files

Zip Files

13. Caching with databricks

Delta and Apache Spark caching comparison

Delta cache consistency

Use Delta caching

Cache a subset of the data

Monitor the Delta cache

Configure the Delta cache

14. REST API with databricks

REST API 2.0

API Examples

Authentication

Clusters API

DBFS API

Groups API

Instance Pools API

Instance Profiles API

Jobs API

Libraries API

MLflow API

Secrets API

Token API

Workspace API

Frequently Asked Questions

What are the modes of the Azure Databricks training?

We provide classroom-based as well as online training. Since this is a hand-on training so batches generally does not contain more than 4 people.

Will I be provided with any study material during the Azure Databricks training?

We will provide web services specific study material as the course progresses. You will have lifetime access to all the code and basic settings needed for these Azure Databricks training through our github account and the study material that we share with you. You can use that for quick reference.