Databricks-Certified-Professional-Data-Engineer Practice Exam Questions and Answers

A.  

Task queueing resulting from improper thread pool assignment.

B.  

Spill resulting from attached volume storage being too small.

C.  

Network latency due to some cluster nodes being in different regions from the source data

D.  

Skew caused by more data being assigned to a subset of spark-partitions.

E.  

Credential validation errors while pulling data from an external system.

A.  

Because another team uses this table to support a frequently running application, two-phase locking is preventing the operation from committing.

B.  

The activity details table already exists; CHECK constraints can only be added during initial table creation.

C.  

The activity details table already contains records that violate the constraints; all existing data must pass CHECK constraints in order to add them to an existing table.

D.  

The activity details table already contains records; CHECK constraints can only be added prior to inserting values into a table.

E.  

The current table schema does not contain the field valid coordinates; schema evolution will need to be enabled before altering the table to add a constraint.

A.  

Validates interactions between subsystems of your application

B.  

Requires an automated testing framework

C.  

Requires manual intervention

D.  

Validates an application use case

E.  

Validates behavior of individual elements of your application

A.  

An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 G

B.  

B.  

Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job.

C.  

Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.

D.  

Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.

E.  

An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 M

B.  

A.  

Improves the quality of your data

B.  

Validates a complete use case of your application

C.  

Troubleshooting is easier since all steps are isolated and tested individually

D.  

Yields faster deployment and execution times

E.  

Ensures that all steps interact correctly to achieve the desired end result

A.  

Cluster: New Job Cluster;

Retries: Unlimited;

Maximum Concurrent Runs: Unlimited

B.  

Cluster: New Job Cluster;

Retries: None;

Maximum Concurrent Runs: 1

C.  

Cluster: Existing All-Purpose Cluster;

Retries: Unlimited;

Maximum Concurrent Runs: 1

D.  

Cluster: Existing All-Purpose Cluster;

Retries: Unlimited;

Maximum Concurrent Runs: 1

E.  

Cluster: Existing All-Purpose Cluster;

Retries: None;

Maximum Concurrent Runs: 1

A.  

Scala is the only language that can be accurately tested using interactive notebooks; because the best performance is achieved by using Scala code compiled to JARs. all PySpark and Spark SQL logic should be refactored.

B.  

The only way to meaningfully troubleshoot code execution times in development notebooks Is to use production-sized data and production-sized clusters with Run All execution.

C.  

Production code development should only be done using an IDE; executing code against a local build of open source Spark and Delta Lake will provide the most accurate benchmarks for how code will perform in production.

D.  

Calling display () forces a job to trigger, while many transformations will only add to the logical query plan; because of caching, repeated execution of the same logic does not provide meaningful results.

E.  

The Jobs Ul should be leveraged to occasionally run the notebook as a job and track execution time during incremental code development because Photon can only be enabled on clusters launched for scheduled jobs.

A.  

• Total VMs; 1

• 400 GB per Executor

• 160 Cores / Executor

B.  

• Total VMs: 8

• 50 GB per Executor

• 20 Cores / Executor

C.  

• Total VMs: 4

• 100 GB per Executor

• 40 Cores/Executor

D.  

• Total VMs:2

• 200 GB per Executor

• 80 Cores / Executor

A.  

spark.sql.files.maxPartitionBytes

B.  

spark.sql.autoBroadcastJoinThreshold

C.  

spark.sql.files.openCostInBytes

D.  

spark.sql.adaptive.coalescePartitions.minPartitionNum

E.  

spark.sql.adaptive.advisoryPartitionSizeInBytes

A.  

Three columns will be returned, but one column will be named "redacted" and contain only null values.

B.  

Only the email and itv columns will be returned; the email column will contain all null values.

C.  

The email and ltv columns will be returned with the values in user itv.

D.  

The email, age. and ltv columns will be returned with the values in user ltv.

E.  

Only the email and ltv columns will be returned; the email column will contain the string "REDACTED" in each row.

A.  

Because Parquet compresses data row by row. strings will only be compressed when a character is repeated multiple times.

B.  

Delta Lake automatically collects statistics on the first 32 columns of each table which are leveraged in data skipping based on query filters.

C.  

Views in the Lakehouse maintain a valid cache of the most recent versions of source tables at all times.

D.  

Primary and foreign key constraints can be leveraged to ensure duplicate values are never entered into a dimension table.

E.  

Z-order can only be applied to numeric values stored in Delta Lake tables

A.  

/jobs/runs/list

B.  

/jobs/runs/get-output

C.  

/jobs/runs/get

D.  

/jobs/get

E.  

/jobs/list

A.  

sbt

B.  

CRAN

C.  

CRAM

D.  

nom

E.  

Wheels

F.  

jars

A.  

Because the SHA256 hash is used to obfuscate stored secrets, reversing this hash will display the value in plain text.

B.  

Account administrators can see all secrets in plain text by loggingon to the Databricks Accounts console.

C.  

Secrets are stored in an administrators-only table within the Hive Metastore; database administrators have permission to query this table by default.

D.  

Iterating through a stored secret and printing each character will display secret contents in plain text.

E.  

The Databricks REST API can be used to list secrets in plain text if the personal access token has proper credentials.

A.  

All records are cached to an operational database and then the filter is applied

B.  

The Parquet file footers are scanned for min and max statistics for the latitude column

C.  

All records are cached to attached storage and then the filter is applied

D.  

The Delta log is scanned for min and max statistics for the latitude column

E.  

The Hive metastore is scanned for min and max statistics for the latitude column

A.  

date = spark.conf.get("date")

B.  

input_dict = input()

date= input_dict["date"]

C.  

import sys

date = sys.argv[1]

D.  

date = dbutils.notebooks.getParam("date")

E.  

dbutils.widgets.text("date", "null")

date = dbutils.widgets.get("date")

A.  

Use Auto Loader to subscribe to new files in the account history directory; configure a Structured Streaminq trigger once job to batch update newly detected files into the account current table.

B.  

Overwrite the account current table with each batch using the results of a query against the account history table grouping by user id and filtering for the max value of last updated.

C.  

Filter records in account history using the last updated field and the most recent hour processed, as well as the max last iogin by user id write a merge statement to update or insert the most recent value for each user id.

D.  

Use Delta Lake version history to get the difference between the latest version of account history and one version prior, then write these records to account current.

E.  

Filter records in account history using the last updated field and the most recent hour processed, making sure to deduplicate on username; write a merge statement to update or insert the

most recent value for each username.

A.  

Decrease the trigger interval to 5 seconds; triggering batches more frequently allows idle executors to begin processing the next batch while longer running tasks from previous batches finish.

B.  

Increase the trigger interval to 30 seconds; setting the trigger interval near the maximum execution time observed for each batch is always best practice to ensure no records are dropped.

C.  

The trigger interval cannot be modified without modifying the checkpoint directory; to maintain the current stream state, increase the number of shuffle partitions to maximize parallelism.

D.  

Use the trigger once option and configure a Databricks job to execute the query every 10 seconds; this ensures all backlogged records are processed with each batch.

E.  

Decrease the trigger interval to 5 seconds; triggering batches more frequently may prevent records from backing up and large batches from causing spill.