Introduction to the Fundamentals of Data Engineering
This training offers a clear and accessible introduction to the fundamental principles of data engineering. It helps you understand how data pipelines work, the stages of data collection and transformation, as well as the roles of p
Create your free account
How the course works
Each lesson is a structured text, illustrated with examples and diagrams, followed by a quiz that is automatically graded. You progress at your own pace, and your notes stay attached to the lesson in which you took them.
-
Gain a solid understanding of the core concepts of data engineering
-
Understand the different stages of data collection, transformation and storage
-
Discover how data pipelines work and their main applications
-
Identify the tools and technologies used in data processing environments
-
Develop the knowledge needed to understand the challenges of data quality, reliability and security
-
Junior data engineer: participation in data collection, transformation and storage.
-
Data pipeline developer: design and automation of data flows.
-
Assistant data analyst: preparation and organization of data for analysis.
-
Junior database administrator: management, organization and maintenance of databases.
-
Data management technician: checking the quality and consistency of information.
-
Data analyst: exploiting data to identify trends and produce analyses.
-
Junior data engineering consultant: supporting organizations in implementing data management solutions.
-
Data integration specialist: collecting and centralizing information from different sources.
-
Assistant in data architecture: participation in the design and organization of data storage and processing systems.
Accessibility

You have a disability and wish to take a training course with us? This email address is dedicated to you: contact@datagoz.com . For any other situation, don't hesitate to contact us to assess how to ensure a learning experience.
How do I finance my training?
Certificate of completion: at the end of this course, you will receive a DATAGOZ certificate of completion, issued in your name and verifiable online.
The DATAGOZ team wishes you a great training!
6 chapters, 59 lessons
- 2. The fundamentals of data engineeringopen
- 2. The missions and responsibilities of a data engineeropen
- 2. Data engineer and data scientist: understanding their differencesopen
- 2. The main challenges of data engineeringopen
- 2. The technologies and tools of the data engineeropen
- 2. Overview of the different types of databasesopen
- 2. Managing and running data processingopen
- 2. Task Automation and Schedulingopen
- 2. Discovering Cloud Service Providersopen
- 2. The Benefits of Cloud Computingopen
- 2. Overview of the Main Cloud Providersopen
- 2. The Different Services Offered in the Cloudopen
- 3. The Different Database Management Systemssubscribers
- 3. Comparing SQL and NoSQL Databasessubscribers
- 3. Designing and Organizing Database Schemassubscribers
- 3. Combining Tables with Relational Joinssubscribers
- 3. Data Modeling with the Star Schemasubscribers
- 3. Understanding the Principles of Parallel Computingsubscribers
- 3. The advantages of parallel data processingsubscribers
- 3. Breaking a process down into several independent taskssubscribers
- 3. Manipulating data with DataFramessubscribers
- 3. The main distributed computing frameworkssubscribers
- 3. Discovering the Spark, Hadoop and Hive technologiessubscribers
- 3. Grouping and aggregating data with PySparksubscribers
- 3. Running and managing PySpark scriptssubscribers
- 3. Workflow scheduling and orchestration toolssubscribers
- 3. Automating data processing with Airflow, Prefect and cronsubscribers
- 3. Creating and organizing DAGs with Apache Airflowsubscribers
- 4. The different data extraction methodssubscribers
- 4. Identify and explore data sourcessubscribers
- 4. Extract information from an APIsubscribers
- 4. Import data from a databasesubscribers
- 4. The data transformation stepssubscribers
- 4. Extract and break down rental price informationsubscribers
- 4. Prepare and organize transformation operationssubscribers
- 4. Enrich data through joins with reviewssubscribers
- 4. Data loading methodssubscribers
- 4. Understand the differences between OLAP and OLTPsubscribers
- 4. Export data to filessubscribers
- 4. Insert and store data in PostgreSQLsubscribers
- 4. Assemble the different steps of an ETL processsubscribers
- 4. Build a directed acyclic graph (DAG)subscribers
- 4. Set up an Apache Airflow environmentsubscribers
- 4. Analyze and understand the structure of a DAGsubscribers
- 5. Get familiar with the course documentationsubscribers
- 5. Discover the structure of the databasesubscribers
- 5. Explore the data using SQL queriessubscribers
- 5. Calculate the average rating given to each coursesubscribers
- 5. Use the reviews to formulate recommendationssubscribers
- 5. Detect and exclude erroneous datasubscribers
- 5. Apply the transformations needed to generate recommendationssubscribers
- 5. Automate the scheduling of daily processing taskssubscribers
- 5. Determine the destination table for the datasubscribers
- 5. Build the DAG to orchestrate the processsubscribers
- 5. Activate and run the DAGsubscribers
- 5. Query the data to get the generated recommendationssubscribers
- 5. Project wrap-up: data engineering projectsubscribers
Everything that comes with the lessons
How long, depending on your availability
What you get at the end
Once the course is completed, the certificate is issued in your name. Its code allows a recruiter to verify it in just a few seconds.
This is not a diploma or a professional certification; it is verifiable proof that you have followed and successfully completed this course.
Who writes this course
A clear, hands-on teaching approach, with concrete examples to help you learn effectively.
Before you start
Start this course today
13 lessons open, no credit card required. You'll see the level and tone before deciding.