DATA-ENG-PRO Sample Questions

DATA-ENG-PRO Sample Questions & Answers

Advanced data transformations and quality tie with writing and testing Python or SQL ETL pipelines for the top weight, alongside multi-format ingestion, cost and performance tuning, Lakehouse Federation paired with Delta Sharing, security, debugging, and data modeling.

Launch the full DATA-ENG-PRO simulator →

Showing 10 of 20 free samples.

  1. Question 1Beginner

    Data Ingestion & Acquisition · Auto Loader

    A data engineer needs to ingest a large volume of continuously arriving JSON files from a cloud storage location. The schema of the JSON files is known to evolve over time, with new columns being added frequently. The pipeline must be resilient to malformed JSON records and should infer and handle schema changes automatically. Which Databricks feature is best suited for this ingestion task?

    Show answer & explanation

    Correct answer: B

    Auto Loader (cloudFiles) is specifically designed for this use case. It provides scalable file discovery, robust schema inference, and automatic schema evolution, making it the ideal choice for ingesting evolving files from cloud storage.

  2. Question 2Intermediate

    Data Transformation, Cleansing and Quality · Delta Live Tables Expectations

    A data pipeline needs to transform a bronze Delta table containing raw customer orders into a silver table. During this transformation, any order record that does not have a valid order_id or has a negative total_amount should be moved to a quarantine table for later analysis, instead of being loaded into the silver table. The main pipeline should continue processing valid records without interruption. Which Delta Live Tables (DLT) feature should be used to implement this data quality and quarantining process?

    Show answer & explanation

    Correct answer: D

    The correct approach is a multi-hop architecture. First, define expectations on the bronze-to-silver transformation. DLT automatically records records that violate these expectations. Then, define a separate quarantine table that reads from the violation records of the silver table (e.g., dlt.read_stream("LIVE.silver_orders_violation")). This isolates bad data without stopping the pipeline.

  3. Question 3Intermediate

    Cost & Performance Optimisation · Delta Lake Optimization Techniques

    An analytics query on a large, date-partitioned Delta table is running slowly. The query frequently filters on a high-cardinality user_id column within specific date ranges. The Spark UI shows that a large number of files are being read even for queries that target a small number of users. The table's files are already compacted to an optimal size (around 1GB). Which optimization technique should be applied to improve data skipping and reduce the number of files scanned?

    Show answer & explanation

    Correct answer: C

    Z-Ordering is the ideal solution for this scenario. It co-locates related data within files based on the specified column (user_id). This allows the query engine to use the min/max statistics on user_id to skip a significant number of files that do not contain the relevant user data, drastically improving query performance.

  4. Question 4Beginner

    Data Governance · Unity Catalog Metastore

    What is the primary function of the metastore in the context of the Databricks Lakehouse Platform?

    Show answer & explanation

    Correct answer: C

    This is the core function of the metastore. It acts as a catalog, storing all the metadata (schema definitions, data locations, table properties, permissions) for the data assets within the lakehouse, enabling data discovery and governance.

  5. Question 5AdvancedSelect 3

    Data Processing · Delta Live Tables APPLY CHANGES INTO

    An engineering team is building a near-real-time ETL pipeline using Delta Live Tables (DLT) to process a Change Data Capture (CDC) feed from a relational database. The source system provides records with operation_type ('INSERT', 'UPDATE', 'DELETE'), a primary key, and a timestamp for ordering. The goal is to efficiently replicate these changes into a target Delta table.

    Which THREE DLT features or commands are essential for correctly implementing this CDC pipeline? (Select THREE)

    Show answer & explanation

    Correct answers: A, B, C

    dlt.apply_changes() (or APPLY CHANGES INTO in SQL) is the primary DLT command designed specifically to simplify CDC and MERGE operations, making it essential for this use case.

    The SEQUENCE BY clause is critical for ensuring that out-of-order CDC events are processed correctly. It uses the specified column (like a timestamp or version number) to handle late-arriving records and maintain data integrity.

    These clauses are used within the apply_changes function to map the source operation type (e.g., operation_type = 'DELETE') to the corresponding action on the target table. APPLY AS TRUNCATE is also supported for full reloads.

  6. Question 6Intermediate

    Monitoring and Alerting · Spark UI

    A data engineer is investigating a slow-running Databricks job. They need to analyze the execution plan, identify data shuffling, and view detailed metrics for each stage of the job. Which component of the Databricks UI provides the most comprehensive information for this type of performance analysis?

    Show answer & explanation

    Correct answer: B

    The Spark UI is the primary tool for deep-diving into the performance of a Spark application. It provides the Directed Acyclic Graph (DAG) of jobs, detailed stage and task metrics, information on data shuffling, execution plans (for SQL queries), and environmental details.

  7. Question 7Beginner

    Data Sharing and Federation · Delta Sharing

    A data provider wants to securely share a specific Delta table named live_sales_data with an external partner who is not on Databricks. The provider wants to ensure the partner always has access to the most recent version of the data without creating copies or managing complex data export pipelines. Which Databricks technology is designed for this purpose?

    Show answer & explanation

    Correct answer: C

    Delta Sharing is the correct technology. Specifically, its open sharing protocol allows secure, live sharing of Delta tables with any recipient on any platform using open-source connectors (e.g., for Pandas, Power BI, Tableau), perfectly matching the requirement.

  8. Question 8Beginner

    Data Modeling · Dimensional Modeling

    When designing a dimensional model for an analytical workload in a data warehouse, fact tables are primarily used to store quantitative transactional data or measurements.

    Show answer & explanation

    Correct answer: A

    True. This is the fundamental definition of a fact table in dimensional modeling. It contains the numeric measures (the 'facts') of a business process and foreign keys that link to descriptive dimension tables.

  9. Question 9Intermediate

    Ensuring Data Security and Compliance · Unity Catalog Governance Implementation

    Company Background:
    A multinational corporation uses Databricks for its global data operations. They have a central Unity Catalog metastore managing data from various business units. A new regulation requires that Personally Identifiable Information (PII) must be handled with extreme care.

    Data and Users:
    A critical table, employees.personal_details, contains columns such as full_name, national_id, salary, and country. The data governance team has defined three primary roles for accessing this table:

    • HR Analysts: Should see all employee data but only for employees within their specific country. For example, an analyst in the 'HR_Germany' group should only see records where country = 'Germany'.
    • Finance Analysts: Should see salary information for all employees but must not see full_name or national_id. These PII columns should appear as masked values.
    • Data Stewards: Require full, unrestricted access to all rows and columns for data quality and auditing purposes.

    Technical Challenge:
    The solution must be implemented using native Unity Catalog features to provide robust, centralized governance. The company wants to avoid creating physical copies of the data or managing a complex web of views for each role.

    Show answer & explanation

    Correct answer: B

    This is the ideal solution using native Unity Catalog features. A row filter function can dynamically restrict rows based on the HR Analyst's group membership. Column mask functions can conditionally show or mask PII based on whether a user is a Finance Analyst or a Data Steward. This provides a single, centrally governed policy on the base table.

  10. Question 10Advanced

    Cost & Performance Optimisation · Spark Join Optimization

    A batch job processing a 10TB Delta table is taking an unexpectedly long time to complete. The job involves a self-join to find related records and a subsequent aggregation. Analysis of the Spark UI reveals a massive shuffle operation during the join, with terabytes of data being written to and read from disk. The cluster is already using memory-optimized instances and is not showing signs of memory pressure. What is the most likely cause of this performance bottleneck and the most effective way to mitigate it?

    Show answer & explanation

    Correct answer: C

    A massive shuffle in a self-join on a large table is a classic symptom of data skew, where a few join keys have a disproportionately large number of records. AQE is specifically designed to detect and handle this by splitting skewed tasks into smaller, more manageable ones, thereby balancing the workload across executors and mitigating the shuffle bottleneck.

Ready for the real thing?

The full DATA-ENG-PRO simulator has every exam-style question, timed mode, and instant scoring.

Go to the DATA-ENG-PRO simulator →