Emerging tech - Big Data Trunk

https://project.bigdatatrunk.com Quality Corporate and Classroom Training in Bay Area CA Tue, 28 Oct 2025 12:18:15 +0000 en-US hourly 1 https://wordpress.org/?v=7.0 Kickstart DBT for Snowflake in a Day https://project.bigdatatrunk.com/courses/kickstart-dbt-for-snowflake-in-a-day/ https://project.bigdatatrunk.com/courses/kickstart-dbt-for-snowflake-in-a-day/#respond Thu, 07 Sep 2023 06:34:07 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=43402 In this one-day course, students will dive into core concepts of data build tool (DBT) and learn how to streamline data engineering pipelines for Snowflake.

The post Kickstart DBT for Snowflake in a Day first appeared on Big Data Trunk.

]]>

Deprecated: Creation of dynamic property OMAPI_Elementor_Widget::$base is deprecated in /home2/schooli5/public_html/project/wp-content/plugins/optinmonster/OMAPI/Elementor/Widget.php on line 41

Overview
Prerequisites
Audience
Curriculum

Description:

In this one-day course, students will dive into core concepts of data build tool (DBT) and learn how to streamline data engineering pipelines for Snowflake. From understanding models and materialization to exploring source freshness and using advanced techniques like macros and hooks, students will gain a solid foundation of using data build tools (DBT) effectively. Students will get practical hands-on experience using the data build tool with Snowflake.

Duration: 1 Day

Course Code: BDT302

Learning Objectives:

After this course, you will be able to:

Introduction Data Build Tool (DBT)
Understanding DBT models
Using DBT tests to ensure quality of DBT models
Explore DBT materializations to optimize performance & scalability of DBT models
Integrating seeds and sources in DBT project during data ingestion
Dive into DBT Hooks to integrate external scripts and actions into your DBT workflows

Basic understanding of Snowflake and SQL

This course is designed for Analytics Engineers, Data Analysts, BI Professionals, Data Scientists, Data Engineers, DevOps Engineers, and Architects

1. Introduction to Data Build Tool (DBT)

Introduction to Data Warehouse (Snowflake)
ETL v/s ELT
DBT introduction
DBT installation
DBT cloud introduction
Lab: Getting started with DBT

2. Understanding DBT Models

What are DBT models?
Creating DBT table
Using DBT schema
DBT project organization
Lab: Project organization

3. Using DBT Tests

What is DBT schema?
What is a DBT Macro?
Understanding DBT test types
Lab: Generic and Singular Tests

4. Exploring DBT Materialization

What are materializations in DBT?
Default materialization in DBT
Using Config Block for materialization
Lab: Setting materialization

5. Integrating DBT Seeds and Sources

Seeds and Sources overview
Adding sources in DBT
What is source freshness?
Labs: Adding source freshness check in DBT

6. DBT Hooks

What are DBT Hooks?
Understanding pre-hook, post-hook, on-run-start, on-run-end hooks
Implementing DBT Hook

Training material provided: Yes (Digital format)

Hands-on Lab: Instructions will be provided to set up a free tier snowflake account. Also, students will be provided on how to install DBT tool on Windows/Mac

The post Kickstart DBT for Snowflake in a Day first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/kickstart-dbt-for-snowflake-in-a-day/feed/ 0 Mastering Data Build Tool (DBT) for Snowflake https://project.bigdatatrunk.com/courses/mastering-data-build-tool-dbt-for-snowflake/ https://project.bigdatatrunk.com/courses/mastering-data-build-tool-dbt-for-snowflake/#respond Thu, 07 Sep 2023 05:53:45 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=43416 In the data-driven landscape, the ability to efficiently manage, transform, and materialize data is crucial.

The post Mastering Data Build Tool (DBT) for Snowflake first appeared on Big Data Trunk.

]]>

Overview
Prerequisites
Audience
Curriculum

Description:

In the data-driven landscape, the ability to efficiently manage, transform, and materialize data is crucial. In this comprehensive hands-on course, students will dive deep into the world of DBT and learn how to leverage its power to build robust data transformation pipelines. From foundational concepts to advanced techniques, students will gain hands-on experience working with key components: models, materialization, seeds, snapshots, source freshness, macro, and hooks. By the end of this course, students will be equipped with skills to build efficient data pipelines using modern data build tools.

Duration: 2 days

Course Code: BDT301

Learning Objectives:

After this course, you will be able to:

Introduction Data Build Tool (DBT)
Understanding DBT models
Using DBT tests to ensure quality of DBT models
Explore DBT materializations to optimize performance & scalability of DBT models
Integrating seeds and sources in DBT project during data ingestion
Enhance data loading capabilities and create custom macros
Using DBT snapshots learn how to capture historical versions of data for auditing and analysis process
Dive into DBT Hooks to integrate external scripts and actions into your DBT workflows

Basic understanding of Snowflake and SQL

This course is designed for Analytics Engineers, Data Analysts, BI Professionals, Data Scientists, Data Engineers, DevOps Engineers, and Architects

1. Introduction to Data Build Tool (DBT)

Introduction to Data Warehouse (Snowflake)
ETL v/s ELT
DBT introduction
DBT installation
DBT cloud introduction
Lab: Getting started with DBT

2. Understanding DBT Models

What are DBT models?
Creating DBT table
Using DBT schema
DBT project organization
Lab: Project organization

3. Using DBT Tests

What is DBT schema?
What is a DBT Macro?
Understanding DBT test types
Lab: Generic and Singular Tests

4. Exploring DBT Materialization

What are materializations in DBT?
Default materialization in DBT
Using Config Block for materialization
Lab: Setting materialization

5. Integrating DBT Seeds and Sources

Seeds and Sources overview
Adding sources in DBT
What is source freshness?
Labs: Adding source freshness check in DBT

6. DBT Custom Macros

Implementing Table, View and Ephemeral Model
Create custom macro
Understanding DBT package
Labs: Building incremental load

7. Working with DBT Snapshots

Snapshots overview
Creating a snapshot
Labs: create snapshot

8. DBT Hooks

What are DBT Hooks?
Understanding pre-hook, post-hook, on-run-start, on-run-end hooks
Implementing DBT Hook
Labs: implementing DBT hook

Training material provided: Yes (Digital format)

Hands-on Lab: Instructions will be provided to set up a free tier snowflake account. Also, students will be provided on how to install DBT tool on Windows/Mac

The post Mastering Data Build Tool (DBT) for Snowflake first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/mastering-data-build-tool-dbt-for-snowflake/feed/ 0 Kickstart Snowpark with Python https://project.bigdatatrunk.com/courses/kickstart-snowpark-with-python/ https://project.bigdatatrunk.com/courses/kickstart-snowpark-with-python/#respond Thu, 07 Sep 2023 05:42:52 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=43391 Snowpark is a new developer experience for Snowflake that allows developers to write code in their preferred language: Scala, Java or Python to supplement the original SQL interface.

The post Kickstart Snowpark with Python first appeared on Big Data Trunk.

]]>

Overview
Prerequisites
Audience
Curriculum

Description:

Unlock the full potential of Snowpark, the innovative developer experience for Snowflake. This course equips you with the expertise to leverage your preferred language—Scala, Java, or Python—alongside the SQL interface. Discover how to harness the Snowpark API to create a customized software development environment. Say goodbye to exporting data to external environments and tap into Snowflake's powerful computing capabilities. Dive into reading and writing operations, transformations, queries, and the creation of Python UDFs (user-defined functions) using Snowpark.

Duration: 1 day

Course Code: BDT300

Learning Objectives

By the end of this course, you will:

Get Started with Snowpark and Python Integration in Snowflake
Leverage Snowpark for -Efficient Structured Data Reading and Writing in Snowflake
Master the Art of Handling Semi-Structured Data Using Snowpark
Perform Real-Time Data -Transformations While --Loading with Snowpark
Seamlessly Integrate Third-Party Python Libraries to Create User-Defined Functions (UDFs) in Snowpark.

Basic knowledge Snowflake and Python.

This course is designed for anyone interested in using the Snowpark API using Python. It is geared towards data engineers, architects, QA engineers, BI professionals, and data analysts who want to use Python to handle data processing in Snowflake.

Course Outline:

1. Introduction to Snowpark and Python Integration

Overview of Snowpark and its importance in Snowflake data processing
Brief introduction to Python’s role in Snowpark development
Setting up Snowpark development environment with Python
Hands-on: Execute a basic Python script to use Snowpark API

2. Use Snowpark to read and write structured data in Snowflake

Create Snowpark Dataframe
Apply schema to Dataframe
Read from S3: CSV and JSON
Write from S3 to Snowflake table, CSV, JSON
Hands-on lab with these topics

3. Handling semi-structured data with Snowpark

Create dataframe from S3 JSON files
Copy data into snowflake dataframe
Create dataframe from parquet files
Copy data into S3 parquet files into Snowflake table
Handle error records
Hands-on labs with these topics

4. Perform transformations while loading

Using the Snowpark’ aggregation framework
Perform grouping of data
Using Window functions
Using Join and the “using” clause
Hands-on labs with these topics

5. Integration third party Python libraries to create UDF

Build generic usable components library in Python
Create Snowpark UDF
Using vectorized UDFs
Integrating external packages
Hands-on lab with these topics

Training material provided: Yes (Digital format)

Hands-on Lab: Instructions will be provided to students to create “trial” snowflake account. Instructions will be provided in class to install and use Snowpark

The post Kickstart Snowpark with Python first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/kickstart-snowpark-with-python/feed/ 0 Byte-Sized Deep Learning Series: Applied Deep Learning for Natural Language Understanding https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-applied-deep-learning-for-natural-language-understanding/ https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-applied-deep-learning-for-natural-language-understanding/#respond Mon, 21 Jun 2021 07:48:35 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=22548 This 90-minute session will explore the application of deep learning models known as transformers to solve common natural language understanding tasks.

The post Byte-Sized Deep Learning Series: Applied Deep Learning for Natural Language Understanding first appeared on Big Data Trunk.

]]>

Overview
Prerequisite
Audience
Curriculum

Description:

This 90-minute session will explore the application of deep learning models known as transformers to solve common natural language understanding tasks (e.g., question answering, sentiment classification, text summarization, text generation). Learners will use the popular Hugging Face library.

Course Code/Duration:

BDT148 / 90 minutes

Learners should have a basic knowledge of Python programming.

This course is for those who would like to understand how to apply deep learning to common natural language understanding tasks.

Course Outline:

During this course, you will have the opportunity to:

Understand the transformer model and why it is superior to previous approaches to natural language understanding.
Learn how to use the popular Hugging Face library to solve common use cases.
Apply pre-trained models to answer questions related to a corpus of text, summarize text, generate novel text, and several other use cases including paraphrasing, sentiment classification, and text completion.

Training material provided: Yes (Digital format)

The post Byte-Sized Deep Learning Series: Applied Deep Learning for Natural Language Understanding first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-applied-deep-learning-for-natural-language-understanding/feed/ 0 Byte-Sized Deep Learning Series: Understanding Language https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-understanding-language/ https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-understanding-language/#respond Mon, 21 Jun 2021 07:44:58 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=22546 This 90-minute session will explore the use of deep learning and the role of recurrent neural networks in language understanding.

The post Byte-Sized Deep Learning Series: Understanding Language first appeared on Big Data Trunk.

]]>

Overview
Prerequisite
Audience
Curriculum

Session Description:

Course Code/Duration:

BDT151 / 90 minutes

Learning Objectives:

During this course, you will have the opportunity to:

Understand the transformer model and why it is superior to previous approaches to natural language understanding.
Learn how to use the popular Hugging Face library to solve common use cases.
Apply pre-trained models to answer questions related to a corpus of text, summarize text, generate novel text, and several other use cases including paraphrasing, sentiment classification, and text completion.

Training material provided: Yes (Digital format)

Learners should have a basic knowledge of Python programming.

This course is for those who would like to understand how to apply deep learning to common natural language understanding tasks.

The post Byte-Sized Deep Learning Series: Understanding Language first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-understanding-language/feed/ 0 Byte-Sized Deep Learning Series: Image Recognition https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-image-recognition/ https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-image-recognition/#respond Mon, 21 Jun 2021 07:41:28 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=22544 This course is for those who are interested in gaining an understanding of how deep learning is used to recognize and classify images

The post Byte-Sized Deep Learning Series: Image Recognition first appeared on Big Data Trunk.

]]>

Overview
Prerequisite
Audience
Curriculum

Description:

This 90-minute session will explore the use of deep learning and the role of convolution neural networks in computer vision and image classification. Learners will use TensorFlow to build and train deep learning models to classify images.

Course Code/Duration:

BDT149 / 90 minutes

Learners should have a basic knowledge of Python programming.

This course is for those who are interested in gaining an understanding of how deep learning is used to recognize and classify images (e.g., medical imaging, face recognition, image classification).

Course Outline:

During this course, you will have the opportunity to:

Explore deep learning and the advantages of deep learning models.
Understand convolutional neural networks and how they learn to recognize images.
Use TensorFlow to build and train a deep neural network to classify images.

Training Material Provided: Yes (Digital Format)

The post Byte-Sized Deep Learning Series: Image Recognition first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/byte-sized-deep-learning-series-image-recognition/feed/ 0 KickStart on The Internet of Things (IoT) https://project.bigdatatrunk.com/courses/kickstart-on-the-internet-of-things-iot/ https://project.bigdatatrunk.com/courses/kickstart-on-the-internet-of-things-iot/#respond Mon, 14 Jun 2021 09:31:07 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=22528 A 1-day session intended to provide an introduction to the Internet of Things (IOT), and to learn IoT Core on AWS.

The post KickStart on The Internet of Things (IoT) first appeared on Big Data Trunk.

]]>

Overview
Prerequisite
Audience
Curriculum

Description:

A 1-day session intended to provide an introduction to the Internet of Things (IOT), and to learn IoT Core on AWS.

Course Code/Duration:

BDT146 / 1 Day (8 hours)

Learning Objectives:

Learn how to securely onboard IoT devices on AWS.
Learn how to securely push telemetry from IoT devices to IoT Core on AWS.
Learn how to securely push command and control messages to connected IoT devices via the Device Shadow.
Learn how to Use IoT Rules to route messages to other AWS Services.

Having some basic idea of what it takes to create cloud infrastructure in AWS, helps you learn how to work within the IoT ecosystem of connected devices, and IoT cloud infrastructure.

This session is designed for developers, architects, security specialist and product owners who

Have some basic understanding of cloud computing.
Are interested to learn how to develop IoT Solutions on AWS.

Course Outline:

Sure, based on the learning objectives and lab descriptions, here’s a concise curriculum layout for the course:

Secure Onboarding of IoT Devices on AWS
Pushing Telemetry from IoT Devices to AWS IoT Core
Command and Control Messages via Device Shadow
Using IoT Rules for Message Routing

Hands-on Lab 1: Node-Red

Configuring MQTT Client for Secure Connection
Subscribing to MQTT Topic
Republishing Telemetry Messages to IoT Rule
Sending Command and Control Messages
Configuring AWS IoT Analytics
Creating IoT Telemetry Dashboard on Grafana

Hands-on Lab 2: Grafana

Configuring Data Source with Amazon Timeseries Plugin
Creating IoT Dashboard for Telemetry
Registering an IoT Thing using AWS Management Console
Running an IoT Simulator on AWS EC2

Optional Lab: Qwiklabs Lab Introduction to IoT

Registering an IoT Thing using AWS Management Console
Running an IoT Simulator on AWS EC2

Training material provided: Yes (Digital format)

The post KickStart on The Internet of Things (IoT) first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/kickstart-on-the-internet-of-things-iot/feed/ 0 Byte-Sized IoT Series: Introduction to Internet of Things and IoT Device Onboarding https://project.bigdatatrunk.com/courses/introduction-to-the-internet-of-things-and-iot-and-device-onboarding/ https://project.bigdatatrunk.com/courses/introduction-to-the-internet-of-things-and-iot-and-device-onboarding/#respond Tue, 08 Jun 2021 08:32:28 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=22488 A short session intended to provide an introduction to the Internet of Things (IOT).

The post Byte-Sized IoT Series: Introduction to Internet of Things and IoT Device Onboarding first appeared on Big Data Trunk.

]]>

Overview
Prerequisite
Audience
Curriculum

Description:

A short session intended to provide an introduction to the Internet of Things (IOT), and to learn to securely connect devices to IoT Core on AWS.

Course Code/Duration:

BDT140 / 90 Minutes

Learning Objectives:

Learn about IoT and how to securely onboard IoT devices, and we will then perform a hands-on lab to do the following:

Register a Thing on AWS IoT Core.
Configure a device with a certificate, private key and RootCA certificate to securely connect to IoT Core using the MQTT protocol.
Scope down permissions using IoT policies using least privilege.

Having some basic idea of what it takes to create cloud infrastructure in AWS, helps you learn how to work within the IoT ecosystem of connected devices, edge computing, and IoT cloud infrastructure.

This session is designed for developers, architects, security specialist and product owners who

Have some basic understanding of cloud computing.
Are interested to learn how to securely onboard IoT devices on the AWS Cloud.

Course Outline:

Introduction to IoT

Secure Onboarding of IoT Devices

Hands-on Lab: AWS IoT Core

Registering a Thing
Configuring Device for Secure Connection
Implementing Least Privilege with IoT Policies

Training Material Provided: Yes (Digital Format)

The post Byte-Sized IoT Series: Introduction to Internet of Things and IoT Device Onboarding first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/introduction-to-the-internet-of-things-and-iot-and-device-onboarding/feed/ 0 Byte-Sized IoT Series: Internet of Things (IoT) Device Telemetry https://project.bigdatatrunk.com/courses/internet-of-thingsiotdevice-telemetry/ https://project.bigdatatrunk.com/courses/internet-of-thingsiotdevice-telemetry/#respond Tue, 08 Jun 2021 07:57:19 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=22466 Get a concise overview of essential device sensors, including Accelerometer, Gyroscope, Magnetometer, Temperature, Humidity, Pressure, Light, and Sound. Learn the secure process of pushing telemetry data from connected devices to AWS IoT Core. Enhance your knowledge of IoT and sensor technologies in this short yet informative session.

The post Byte-Sized IoT Series: Internet of Things (IoT) Device Telemetry first appeared on Big Data Trunk.

]]>

Overview
Prerequisites
Audience
Curriculum

Description:

Get a concise overview of essential device sensors, including Accelerometer, Gyroscope, Magnetometer, Temperature, Humidity, Pressure, Light, and Sound. Learn the secure process of pushing telemetry data from connected devices to AWS IoT Core. Enhance your knowledge of IoT and sensor technologies in this short yet informative session.

Course Code/Duration:

BDT141 / 90 Minutes

Learning Objectives:

Learn about IoT and how to securely push telemetry from IoT devices to IoT Core on AWS, and we will then perform a hands-on lab to do the following:

Publish MQTT messages from an IoT Device to IoT Core on AWS
Configure an IoT rule to select messagesfrom a topic, transform them and route them to other AWS Services such as Lambda, SQS, Amazon Timestream etc.

Having some basic idea of what it takes to create cloud infrastructure in AWS, helps you learn how to work within the IoT ecosystem of connected devices, edge computing, and IoT cloud infrastructure.

This session is designed for developers, architects, security specialist and product owners who

Have some basic understanding of cloud computing.
Are interested to learn how to securely onboard IoT devices on the AWS Cloud.

Course Outline:

Introduction to IoT and AWS IoT Core

Secure Telemetry Push from IoT Devices

Hands-on Lab: AWS IoT Core

Publishing MQTT Messages from an IoT Device
Configuring IoT Rules for Message Transformation and Routing

Training material provided: Yes (Digital format)

The post Byte-Sized IoT Series: Internet of Things (IoT) Device Telemetry first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/internet-of-thingsiotdevice-telemetry/feed/ 0 Byte-Sized IoT Series: Internet of Things (IoT) Analytics https://project.bigdatatrunk.com/courses/byte-sized-iot-series-internet-of-things-iot-analytics/ https://project.bigdatatrunk.com/courses/byte-sized-iot-series-internet-of-things-iot-analytics/#respond Mon, 07 Jun 2021 11:01:02 +0000 https://www.bigdatatrunk.com/?post_type=lp_course&p=22474 A short session intended to provide you an overview of IoT Analytics services on AWS.

The post Byte-Sized IoT Series: Internet of Things (IoT) Analytics first appeared on Big Data Trunk.

]]>

Overview
Prerequisites
Audience
Curriculum

Description:

A short session intended to provide you an overview of IoT Analytics services on AWS.

Course Code/Duration:

BDT144 / 90 Minutes

Learning Objectives:

Gain a comprehensive understanding of IoT Analytics services on AWS.
Learn how to leverage these services for effective IoT data analysis.
Get hands-on experience with AWS IoT Analytics through practical examples.

Having some basic idea of what it takes to create cloud infrastructure in AWS, helps you learn how to work within the IoT ecosystem of connected devices, edge computing, and IoT cloud infrastructure.

This session is designed for developers, architects, security specialist and product owners who

Have some basic understanding of cloud computing.
Are interested to learn how to configure a Telemetry Pipeline on the AWS Cloud.

Course Outline:

Introduction to IoT Analytics on AWS
Working with AWS IoT Analytics Services
Hands-on Lab: Practical Application of AWS IoT Analytics

Training material provided: Yes (Digital format)

The post Byte-Sized IoT Series: Internet of Things (IoT) Analytics first appeared on Big Data Trunk.

]]> https://project.bigdatatrunk.com/courses/byte-sized-iot-series-internet-of-things-iot-analytics/feed/ 0