Free Databricks Associate-Developer-Apache-Spark-3.5 Exam 2026 Practice Materials Collection [Q47-Q62]

Associate-Developer-Apache-Spark-3.5 Databricks

admin • February 19, 2026

Free Databricks Associate-Developer-Apache-Spark-3.5 Exam 2026 Practice Materials Collection [Q47-Q62]

Rate this post

Free Databricks Associate-Developer-Apache-Spark-3.5 Exam 2026 Practice Materials Collection

Associate-Developer-Apache-Spark-3.5 Exam Info and Free Practice Test All-in-One Exam Guide Feb-2026

NEW QUESTION 47
A developer notices that all the post-shuffle partitions in a dataset are smaller than the value set for spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold.
Which type of join will Adaptive Query Execution (AQE) choose in this case?

A Cartesian join

A shuffled hash join

A broadcast nested loop join

A sort-merge join

Adaptive Query Execution (AQE) dynamically selects join strategies based on actual data sizes at runtime. If the size of post-shuffle partitions is below the threshold set by:
spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold
then Spark prefers to use a shuffled hash join.
From the Spark documentation:
“AQE selects a shuffled hash join when the size of post-shuffle data is small enough to fit within the configured threshold, avoiding more expensive sort-merge joins.” Therefore:
A is wrong – Cartesian joins are only used with no join condition.
B is correct – this is the optimized join for small partitioned shuffle data under AQE.
C and D are used under other scenarios but not for this case.
Final answer: B

NEW QUESTION 48
A data scientist is working with a Spark DataFrame called customerDF that contains customer information. The DataFrame has a column named email with customer email addresses. The data scientist needs to split this column into username and domain parts.
Which code snippet splits the email column into username and domain columns?

customerDF.select(
col(“email”).substr(0, 5).alias(“username”),
col(“email”).substr(-5).alias(“domain”)
)

customerDF.withColumn(“username”, split(col(“email”), “@”).getItem(0))
.withColumn(“domain”, split(col(“email”), “@”).getItem(1))

customerDF.withColumn(“username”, substring_index(col(“email”), “@”, 1))
.withColumn(“domain”, substring_index(col(“email”), “@”, -1))

customerDF.select(
regexp_replace(col(“email”), “@”, “”).alias(“username”),
regexp_replace(col(“email”), “@”, “”).alias(“domain”)
)

Option B is the correct and idiomatic approach in PySpark to split a string column (like email) based on a delimiter such as “@”.
The split(col(“email”), “@”) function returns an array with two elements: username and domain.
getItem(0) retrieves the first part (username).
getItem(1) retrieves the second part (domain).
withColumn() is used to create new columns from the extracted values.
Example from official Databricks Spark documentation on splitting columns:
from pyspark.sql.functions import split, col
df.withColumn(“username”, split(col(“email”), “@”).getItem(0)) \
.withColumn(“domain”, split(col(“email”), “@”).getItem(1))
Why other options are incorrect:
A uses fixed substring indices (substr(0, 5)), which won’t correctly extract usernames and domains of varying lengths.
C uses substring_index, which is available but less idiomatic for splitting emails and is slightly less readable.
D removes “@” from the email entirely, losing the separation between username and domain, and ends up duplicating values in both fields.
Therefore, Option B is the most accurate and reliable solution according to Apache Spark 3.5 best practices.

NEW QUESTION 49
An application architect has been investigating Spark Connect as a way to modernize existing Spark applications running in their organization.
Which requirement blocks the adoption of Spark Connect in this organization?

Debuggability: the ability to perform interactive debugging directly from the application code

Upgradability: the ability to upgrade the Spark applications independently from the Spark driver itself

Complete Spark API support: the ability to migrate all existing code to Spark Connect without modification, including the RDD APIs

Stability: isolation of application code and dependencies from each other and the Spark driver

Spark Connect enables a decoupled client-server architecture, allowing remote clients to run Spark code via gRPC.
However, as of Spark 3.5, Spark Connect supports DataFrame and SQL APIs, but not RDD APIs.
Limitation:
Applications that rely heavily on RDD-based transformations or actions cannot be migrated directly to Spark Connect.
These APIs require tight driver integration, which Spark Connect intentionally decouples.
Thus, complete Spark API compatibility is not yet achieved – this is the key adoption blocker.
Why the other options are incorrect:
A: Debugging is possible through IDE integration and logs on the client side.
B: Spark Connect actually supports upgradable clients independent of the driver – this is an advantage, not a limitation.
D: Spark Connect provides strong isolation between the client and driver processes.
Reference:
Spark 3.5 Documentation – Spark Connect architecture and supported APIs.
Databricks Exam Guide (June 2025): Section “Using Spark Connect to Deploy Applications” – Spark Connect limitations (no RDD API support).

NEW QUESTION 50
A data engineer uses a broadcast variable to share a DataFrame containing millions of rows across executors for lookup purposes. What will be the outcome?

The job may fail if the memory on each executor is not large enough to accommodate the DataFrame being broadcasted

The job may fail if the executors do not have enough CPU cores to process the broadcasted dataset

The job will hang indefinitely as Spark will struggle to distribute and serialize such a large broadcast variable to all executors

The job may fail because the driver does not have enough CPU cores to serialize the large DataFrame

Comprehensive and Detailed Explanation From Exact Extract:
In Apache Spark, broadcast variables are used to efficiently distribute large, read-only data to all worker nodes. However, broadcasting very large datasets can lead to memory issues on executors if the data does not fit into the available memory.
According to the Spark documentation:
“Broadcast variables allow the programmer to keep a read-only variable cached on each machine rather than shipping a copy of it with tasks. This can greatly reduce the amount of data sent over the network.” However, it also notes:
“Using the broadcast functionality available in SparkContext can greatly reduce the size of each serialized task, and the cost of launching a job over a cluster. If your tasks use any large object from the driver program inside of them (e.g., a static lookup table), consider turning it into a broadcast variable.” But caution is advised when broadcasting large datasets:
“Broadcasting large variables can cause out-of-memory errors if the data does not fit in the memory of each executor.” Therefore, if the broadcasted DataFrame containing millions of rows exceeds the memory capacity of the executors, the job may fail due to memory constraints.
Reference:Spark 3.5.5 Documentation – Tuning

NEW QUESTION 51
Given a CSV file with the content:

And the following code:
from pyspark.sql.types import *
schema = StructType([
StructField(“name”, StringType()),
StructField(“age”, IntegerType())
])
spark.read.schema(schema).csv(path).collect()
What is the resulting output?

[Row(name=’bambi’), Row(name=’alladin’, age=20)]

[Row(name=’alladin’, age=20)]

[Row(name=’bambi’, age=None), Row(name=’alladin’, age=20)]

The code throws an error due to a schema mismatch.

In Spark, when a CSV row does not match the provided schema, Spark does not raise an error by default. Instead, it returns null for fields that cannot be parsed correctly.
In the first row, “hello” cannot be cast to Integer for the age field → Spark sets age=None In the second row, “20” is a valid integer → age=20 So the output will be:
[Row(name=’bambi’, age=None), Row(name=’alladin’, age=20)]
Final answer: C

NEW QUESTION 52
A data engineer is building an Apache Spark™ Structured Streaming application to process a stream of JSON events in real time. The engineer wants the application to be fault-tolerant and resume processing from the last successfully processed record in case of a failure. To achieve this, the data engineer decides to implement checkpoints.
Which code snippet should the data engineer use?

query = streaming_df.writeStream
.format(“console”)
.option(“checkpoint”, “/path/to/checkpoint”)
.outputMode(“append”)
.start()

query = streaming_df.writeStream
.format(“console”)
.outputMode(“append”)
.option(“checkpointLocation”, “/path/to/checkpoint”)
.start()

query = streaming_df.writeStream
.format(“console”)
.outputMode(“complete”)
.start()

query = streaming_df.writeStream
.format(“console”)
.outputMode(“append”)
.start()

To enable fault tolerance and ensure that Spark can resume from the last committed offset after failure, you must configure a checkpoint location using the correct option key: “checkpointLocation”.
From the official Spark Structured Streaming guide:
“To make a streaming query fault-tolerant and recoverable, a checkpoint directory must be specified using .option(“checkpointLocation”, “/path/to/dir”).” Explanation of options:
Option A uses an invalid option name: “checkpoint” (should be “checkpointLocation”) Option B is correct: it sets checkpointLocation properly Option C lacks checkpointing and won’t resume after failure Option D also lacks checkpointing configuration

NEW QUESTION 53
A DataFramedfhas columnsname,age, andsalary. The developer needs to sort the DataFrame byagein ascending order andsalaryin descending order.
Which code snippet meets the requirement of the developer?

df.orderBy(col(“age”).asc(), col(“salary”).asc()).show()

df.sort(“age”, “salary”, ascending=[True, True]).show()

df.sort(“age”, “salary”, ascending=[False, True]).show()

df.orderBy(“age”, “salary”, ascending=[True, False]).show()

Comprehensive and Detailed Explanation From Exact Extract:
To sort a PySpark DataFrame by multiple columns with mixed sort directions, the correct usage is:
python
CopyEdit
df.orderBy(“age”,”salary”, ascending=[True,False])
agewill be sorted in ascending order
salarywill be sorted in descending order
TheorderBy()andsort()methods in PySpark accept a list of booleans to specify the sort direction for each column.
Documentation Reference:PySpark API – DataFrame.orderBy

NEW QUESTION 54
A developer initializes a SparkSession:

spark = SparkSession.builder
.appName(“Analytics Application”)
.getOrCreate()
Which statement describes thesparkSparkSession?

ThegetOrCreate()method explicitly destroys any existing SparkSession and creates a new one.

A SparkSession is unique for eachappName, and callinggetOrCreate()with the same name will return an existing SparkSession once it has been created.

If a SparkSession already exists, this code will return the existing session instead of creating a new one.

A new SparkSession is created every time thegetOrCreate()method is invoked.

Comprehensive and Detailed Explanation From Exact Extract:
According to the PySpark API documentation:
“getOrCreate(): Gets an existing SparkSession or, if there is no existing one, creates a new one based on the options set in this builder.” This means Spark maintains a global singleton session within a JVM process. Repeated calls togetOrCreate() return the same session, unless explicitly stopped.
Option A is incorrect: the method does not destroy any session.
Option B incorrectly ties uniqueness toappName, which does not influence session reusability.
Option D is incorrect: it contradicts the fundamental behavior ofgetOrCreate().
(Source:PySpark SparkSession API Docs)

NEW QUESTION 55
An MLOps engineer is building a Pandas UDF that applies a language model that translates English strings into Spanish. The initial code is loading the model on every call to the UDF, which is hurting the performance of the data pipeline.
The initial code is:

def in_spanish_inner(df: pd.Series) -> pd.Series:
model = get_translation_model(target_lang=’es’)
return df.apply(model)
in_spanish = sf.pandas_udf(in_spanish_inner, StringType())
How can the MLOps engineer change this code to reduce how many times the language model is loaded?

Convert the Pandas UDF to a PySpark UDF

Convert the Pandas UDF from a Series # Series UDF to a Series # Scalar UDF

Run thein_spanish_inner()function in amapInPandas()function call

Convert the Pandas UDF from a Series # Series UDF to an Iterator[Series] # Iterator[Series] UDF

Comprehensive and Detailed Explanation From Exact Extract:
The provided code defines a Pandas UDF of type Series-to-Series, where a new instance of the language modelis created on each call, which happens per batch. This is inefficient and results in significant overhead due to repeated model initialization.
To reduce the frequency of model loading, the engineer should convert the UDF to an iterator-based Pandas UDF (Iterator[pd.Series] -> Iterator[pd.Series]). This allows the model to be loaded once per executor and reused across multiple batches, rather than once per call.
From the official Databricks documentation:
“Iterator of Series to Iterator of Series UDFs are useful when the UDF initialization is expensive… For example, loading a ML model once per executor rather than once per row/batch.”
– Databricks Official Docs: Pandas UDFs
Correct implementation looks like:
python
CopyEdit
@pandas_udf(“string”)
def translate_udf(batch_iter: Iterator[pd.Series]) -> Iterator[pd.Series]:
model = get_translation_model(target_lang=’es’)
for batch in batch_iter:
yield batch.apply(model)
This refactor ensures theget_translation_model()is invoked once per executor process, not per batch, significantly improving pipeline performance.

NEW QUESTION 56
An engineer notices a significant increase in the job execution time during the execution of a Spark job. After some investigation, the engineer decides to check the logs produced by the Executors.
How should the engineer retrieve the Executor logs to diagnose performance issues in the Spark application?

Locate the executor logs on the Spark master node, typically under the /tmp directory.

Use the command spark-submit with the -verbose flag to print the logs to the console.

Use the Spark UI to select the stage and view the executor logs directly from the stages tab.

Fetch the logs by running a Spark job with the spark-sql CLI tool.

The Spark UI is the standard and most effective way to inspect executor logs, task time, input size, and shuffles.
From the Databricks documentation:
“You can monitor job execution via the Spark Web UI. It includes detailed logs and metrics, including task-level execution time, shuffle reads/writes, and executor memory usage.” (Source: Databricks Spark Monitoring Guide) Option A is incorrect: logs are not guaranteed to be in /tmp, especially in cloud environments.
B . -verbose helps during job submission but doesn’t give detailed executor logs.
D . spark-sql is a CLI tool for running queries, not for inspecting logs.
Hence, the correct method is using the Spark UI → Stages tab → Executor logs.

NEW QUESTION 57
12 of 55.
A data scientist has been investigating user profile data to build features for their model. After some exploratory data analysis, the data scientist identified that some records in the user profiles contain NULL values in too many fields to be useful.
The schema of the user profile table looks like this:
user_id STRING,
username STRING,
date_of_birth DATE,
country STRING,
created_at TIMESTAMP
The data scientist decided that if any record contains a NULL value in any field, they want to remove that record from the output before further processing.
Which block of Spark code can be used to achieve these requirements?

filtered_users = raw_users.na.drop(“any”)

filtered_users = raw_users.na.drop(“all”)

filtered_users = raw_users.dropna(how=”any”)

filtered_users = raw_users.dropna(how=”all”)

In Spark’s DataFrame API, the dropna() (or equivalently, DataFrameNaFunctions.drop()) method removes rows containing null values.
Behavior:
how=”any” → drops rows where any column has a null value.
how=”all” → drops rows where all columns are null.
Since the data scientist wants to drop records with any null field, the correct parameter is how=”any”.
Correct syntax:
filtered_users = raw_users.dropna(how=”any”)
This will remove all records that have at least one null value in any column.
Why the other options are incorrect:
A: Uses na.drop(“any”) but missing parentheses context (works only as raw_users.na.drop(“any”), which is equivalent to option C).
B/D: how=”all” only removes rows where all values are null – too strict for this use case.
Reference:
PySpark DataFrame API – DataFrameNaFunctions.drop() and DataFrame.dropna().
Databricks Exam Guide (June 2025): Section “Developing Apache Spark DataFrame/DataSet API Applications” – covers handling missing data and DataFrame cleaning operations.

NEW QUESTION 58
A Data Analyst needs to retrieve employees with 5 or more years of tenure.
Which code snippet filters and shows the list?

employees_df.filter(employees_df.tenure >= 5).show()

employees_df.where(employees_df.tenure >= 5)

filter(employees_df.tenure >= 5)

employees_df.filter(employees_df.tenure >= 5).collect()

Comprehensive and Detailed Explanation From Exact Extract:
To filter rows based on a condition and display them in Spark, usefilter(…).show():
employees_df.filter(employees_df.tenure >= 5).show()
Option A is correct and shows the results.
Option B filters but doesn’t display them.
Option C uses Python’s built-infilter, not Spark.
Option D collects the results to the driver, which is unnecessary if.show()is sufficient.
Final Answer: A

NEW QUESTION 59
What is the risk associated with this operation when converting a large Pandas API on Spark DataFrame back to a Pandas DataFrame?

The conversion will automatically distribute the data across worker nodes

The operation will fail if the Pandas DataFrame exceeds 1000 rows

Data will be lost during conversion

The operation will load all data into the driver’s memory, potentially causing memory overflow

When you convert a large pyspark.pandas (aka Pandas API on Spark) DataFrame to a local Pandas DataFrame using .toPandas(), Spark collects all partitions to the driver.
From the Spark documentation:
“Be careful when converting large datasets to Pandas. The entire dataset will be pulled into the driver’s memory.” Thus, for large datasets, this can cause memory overflow or out-of-memory errors on the driver.
Final answer: D

NEW QUESTION 60
A data engineer wants to write a Spark job that creates a new managed table. If the table already exists, the job should fail and not modify anything.
Which save mode and method should be used?

saveAsTable with mode ErrorIfExists

saveAsTable with mode Overwrite

save with mode Ignore

save with mode ErrorIfExists

The method saveAsTable() creates a new table and optionally fails if the table exists.
From Spark documentation:
“The mode ‘ErrorIfExists’ (default) will throw an error if the table already exists.” Thus:
Option A is correct.
Option B (Overwrite) would overwrite existing data – not acceptable here.
Option C and D use save(), which doesn’t create a managed table with metadata in the metastore.
Final answer: A

NEW QUESTION 61
You have:
DataFrame A: 128 GB of transactions
DataFrame B: 1 GB user lookup table
Which strategy is correct for broadcasting?

DataFrame B should be broadcasted because it is smaller and will eliminate the need for shuffling itself

DataFrame B should be broadcasted because it is smaller and will eliminate the need for shuffling DataFrame A

DataFrame A should be broadcasted because it is larger and will eliminate the need for shuffling DataFrame B

DataFrame A should be broadcasted because it is smaller and will eliminate the need for shuffling itself

Broadcast joins work by sending the smaller DataFrame to all executors, eliminating the shuffle of the larger DataFrame.
From Spark documentation:
“Broadcast joins are efficient when one DataFrame is small enough to fit in memory. Spark avoids shuffling the larger table.” DataFrame B (1 GB) fits within the default threshold and should be broadcasted.
It eliminates the need to shuffle the large DataFrame A.
Final answer: B

NEW QUESTION 62
7 of 55.
A developer has been asked to debug an issue with a Spark application. The developer identified that the data being loaded from a CSV file is being read incorrectly into a DataFrame.
The CSV file has been read using the following Spark SQL statement:
CREATE TABLE locations
USING csv
OPTIONS (path ‘/data/locations.csv’)
The first lines of the command SELECT * FROM locations look like this:
| city | lat | long |
| ALTI Sydney | -33… | … |
Which parameter can the developer add to the OPTIONS clause in the CREATE TABLE statement to read the CSV data correctly again?

‘header’ ‘true’

‘header’ ‘false’

‘sep’ ‘,’

‘sep’ ‘|’

When reading CSV files using Spark SQL or the DataFrame API, Spark by default assumes that the first line of the file is data, not headers. To interpret the first line as column names, the header option must be set to true.
Correct syntax:
CREATE TABLE locations
USING csv
OPTIONS (
path ‘/data/locations.csv’,
header ‘true’
);
This tells Spark to read the first row as column headers and correctly map columns like city, lat, and long.
Why the other options are incorrect:
B (header ‘false’): Default behavior; would keep reading header as data.
C / D (sep): Used to specify the delimiter; not relevant unless the file uses a different separator (e.g., |).
Reference (Databricks Apache Spark 3.5 – Python / Study Guide):
PySpark SQL Data Sources – CSV options (header, inferSchema, sep).
Databricks Exam Guide (June 2025): Section “Using Spark SQL” – Reading data from files with different formats using Spark SQL and DataFrame APIs.

Loading …