| Stage Id ▾ | Description | Submitted | Duration | Tasks: Succeeded/Total | Input | Output | Shuffle Read | Shuffle Write | Failure Reason |
|---|---|---|---|---|---|---|---|---|---|
| 79 | Spark Connect - session_id: "a0ccde36-08dc-48be-99dc-4dc1fd142e78"
plan {
command {
create_dataframe_view {
input {
commo... Spark Connect - session_id: "a0ccde36-08dc-48be-99dc-4dc1fd142e78"
plan {
command {
create_dataframe_view {
input {
commo... session_id: "a0ccde36-08dc-48be-99dc-4dc1fd142e78"
plan {
command {
create_dataframe_view {
input {
common {
plan_id: 42
}
read {
data_source {
format: "parquet"
schema: ""
options {
key: "mergeSchema"
value: "true"
}
paths: "/home/jovyan/data/yellow-taxi"
}
}
}
name: "trips"
replace: true
}
}
}
client_type: "_SPARK_CONNECT_PYTHON spark/4.1.0 os/linux python/3.13.11"
request_options {
result_chunking_options {
allow_arrow_batch_chunking: true
}
}
request_options {
reattach_options {
reattachable: true
}
}
operation_id: "57a2fd73-b74c-4586-8649-69e177862a95"
client_observed_server_side_session_id: "1e5ca42a-df22-49f9-a9d3-84f92f7f947a"
| 2026/09/02 21:11:39 | 0.5 s |
0/2
(6 failed)
(1 killed: Stage cancelled: [CANNOT_MERGE_SCHEMAS] Failed merging schemas:
Initial schema:
"STRUCT<VendorID: BIGINT, tpep_pickup_datetime: TIMESTAMP_NTZ, tpep_dropoff_datetime: TIMESTAMP_NTZ, passenger_count: BIGINT, trip_distance: DOUBLE, RatecodeID: BIGINT, store_and_fwd_flag: STRING, PULocationID: BIGINT, DOLocationID: BIGINT, payment_type: BIGINT, fare_amount: DOUBLE, extra: DOUBLE, mta_tax: DOUBLE, tip_amount: DOUBLE, tolls_amount: DOUBLE, improvement_surcharge: DOUBLE, total_amount: DOUBLE, congestion_surcharge: DOUBLE, airport_fee: DOUBLE>"
Schema that cannot be merged with the initial schema:
"STRUCT<VendorID: BIGINT, tpep_pickup_datetime: TIMESTAMP_NTZ, tpep_dropoff_datetime: TIMESTAMP_NTZ, passenger_count: BIGINT, trip_distance: DOUBLE, RatecodeID: BIGINT, store_and_fwd_flag: STRING, PULocationID: BIGINT, DOLocationID: BIGINT, payment_type: BIGINT, fare_amount: DOUBLE, extra: DOUBLE, mta_tax: DOUBLE, tip_amount: DOUBLE, tolls_amount: DOUBLE, improvement_surcharge: DOUBLE, total_amount: DOUBLE, congestion_surcharge: VOID, airport_fee: VOID>". SQLSTATE: 42KD9)
| CANNOT_MERGE_SCHEMAS[CANNOT_MERGE_SCHEMAS] Failed merging schemas: Initial schema: "STRUCT<VendorID: BIGINT, tpep_pickup_datetime: TIMESTAMP_NTZ, tpep_dropoff_datetime: TIMESTAMP_NTZ, passenger_count: BIGINT, trip_distance: DOUBLE, RatecodeID: BIGINT, store_and_fwd_flag: STRING, PULocationID: BIGINT, DOLocationID: BIGINT, payment_type: BIGINT, fare_amount: DOUBLE, extra: DOUBLE, mta_tax: DOUBLE, tip_amount: DOUBLE, tolls_amount: DOUBLE, improvement_surcharge: DOUBLE, total_amount: DOUBLE, congestion_surcharge: DOUBLE, airport_fee: DOUBLE>" Schema that cannot be merged with the initial schema: "STRUCT<VendorID: BIGINT, tpep_pickup_datetime: TIMESTAMP_NTZ, tpep_dropoff_datetime: TIMESTAMP_NTZ, passenger_count: BIGINT, trip_distance: DOUBLE, RatecodeID: BIGINT, store_and_fwd_flag: STRING, PULocationID: BIGINT, DOLocationID: BIGINT, payment_type: BIGINT, fare_amount: DOUBLE, extra: DOUBLE, mta_tax: DOUBLE, tip_amount: DOUBLE, tolls_amount: DOUBLE, improvement_surcharge: DOUBLE, total_amount: DOUBLE, congestion_surcharge: VOID, airport_fee: VOID>". SQLSTATE: 42KD9 |