[ Web Proxy ]
URL:
Viewing:
https://spark.apache.org/docs/latest/api/python/reference/../development/index.html
[Back]
[Original]
Development — PySpark 4.2.0 documentation
Skip to main content
Search the docs ...:
[Logo image]
Site Navigation
Overview
Getting Started
Tutorials
User Guide
API Reference
Development
More
Migration Guides
GitHub
PyPI
Site Navigation
Overview
Getting Started
Tutorials
User Guide
API Reference
Development
More
Migration Guides
GitHub
PyPI
Section Navigation
Contributing to PySpark
Testing PySpark
Debugging PySpark
Setting up IDEs
Logging in PySpark
Error classes in PySpark
Development
Development
#
Contributing to PySpark
Contributing by Testing Releases
Contributing Documentation Changes
Preparing to Contribute Code Changes
Environment Setup
Contributing and Maintaining Type Hints
Code and Docstring Guide
Contributing Error and Exception
Testing PySpark
Running Individual PySpark Tests
breakpoint()
Support in PySpark Tests
Running Tests using GitHub Actions
Running Tests for Spark Connect
Debugging PySpark
Remote Debugging (PyCharm Professional)
Checking Resource Usage (
top
and
ps
)
Profiling Memory Usage (Memory Profiler)
Identifying Hot Loops (Python Profilers)
Common Exceptions / Errors
Stack Traces
Setting up IDEs
PyCharm
Logging in PySpark
Introduction
Customizing Log Format
Setting Up
Usage
Error classes in PySpark
APPLICATION_NAME_NOT_SET
ARGUMENT_REQUIRED
ARROW_LEGACY_IPC_FORMAT
ATTRIBUTE_NOT_CALLABLE
ATTRIBUTE_NOT_SUPPORTED
AXIS_LENGTH_MISMATCH
BROADCAST_VARIABLE_NOT_LOADED
CALL_BEFORE_INITIALIZE
CANNOT_ACCESS_TO_DUNDER
CANNOT_APPLY_IN_FOR_COLUMN
CANNOT_BE_EMPTY
CANNOT_BE_NONE
CANNOT_CONFIGURE_SPARK_CONNECT
CANNOT_CONFIGURE_SPARK_CONNECT_MASTER
CANNOT_CONVERT_COLUMN_INTO_BOOL
CANNOT_CONVERT_TYPE
CANNOT_DETERMINE_TYPE
CANNOT_GET_BATCH_ID
CANNOT_INFER_ARRAY_ELEMENT_TYPE
CANNOT_INFER_EMPTY_SCHEMA
CANNOT_INFER_SCHEMA_FOR_TYPE
CANNOT_INFER_TYPE_FOR_FIELD
CANNOT_MERGE_TYPE
CANNOT_OPEN_SOCKET
CANNOT_PARSE_DATATYPE
CANNOT_PROVIDE_METADATA
CANNOT_REGISTER_UDTF
CANNOT_SET_TOGETHER
CANNOT_SPECIFY_RETURN_TYPE_FOR_UDF
CANNOT_WITHOUT
CLASSIC_OPERATION_NOT_SUPPORTED_ON_DF
COLLATION_INVALID_PROVIDER
COLUMN_IN_LIST
CONFLICTING_PIPELINE_REFRESH_OPTIONS
CONNECT_URL_NOT_SET
CONTEXT_ONLY_VALID_ON_DRIVER
CONTEXT_UNAVAILABLE_FOR_REMOTE_CLIENT
DATA_SOURCE_EXTRANEOUS_FILTERS
DATA_SOURCE_INVALID_RETURN_TYPE
DATA_SOURCE_PUSHDOWN_DISABLED
DATA_SOURCE_RETURN_SCHEMA_MISMATCH
DATA_SOURCE_TYPE_MISMATCH
DECORATOR_ARGUMENT_NOT_CALLABLE
DIFFERENT_PANDAS_DATAFRAME
DIFFERENT_PANDAS_INDEX
DIFFERENT_PANDAS_MULTIINDEX
DIFFERENT_PANDAS_SERIES
DIFFERENT_ROWS
DIFFERENT_SCHEMA
DISALLOWED_TYPE_FOR_CONTAINER
DUPLICATED_ARTIFACT
DUPLICATED_FIELD_NAME_IN_ARROW_STRUCT
ERROR_OCCURRED_WHILE_CALLING
FIELD_DATA_TYPE_UNACCEPTABLE
FIELD_DATA_TYPE_UNACCEPTABLE_WITH_NAME
FIELD_NOT_NULLABLE
FIELD_NOT_NULLABLE_WITH_NAME
FIELD_STRUCT_LENGTH_MISMATCH
FIELD_STRUCT_LENGTH_MISMATCH_WITH_NAME
FIELD_TYPE_MISMATCH
FIELD_TYPE_MISMATCH_WITH_NAME
GRAPH_ELEMENT_DEFINED_OUTSIDE_OF_DECLARATIVE_PIPELINE
HIGHER_ORDER_FUNCTION_SHOULD_RETURN_COLUMN
INCORRECT_CONF_FOR_PROFILE
INDEX_NOT_POSITIVE
INDEX_OUT_OF_RANGE
INPUT_NOT_FULLY_CONSUMED
INVALID_ARROW_UDTF_RETURN_TYPE
INVALID_ARROW_UDTF_TABLE_ARGUMENT
INVALID_ARROW_UDTF_WITH_ANALYZE
INVALID_BROADCAST_OPERATION
INVALID_CALL_ON_UNRESOLVED_OBJECT
INVALID_CONNECT_URL
INVALID_INTERVAL_CASTING
INVALID_ITEM_FOR_CONTAINER
INVALID_JSON_DATA_TYPE_FOR_COLLATIONS
INVALID_MULTIPLE_ARGUMENT_CONDITIONS
INVALID_NDARRAY_DIMENSION
INVALID_NUMBER_OF_DATAFRAMES_IN_GROUP
INVALID_PANDAS_UDF
INVALID_PANDAS_UDF_TYPE
INVALID_RETURN_TYPE_FOR_ARROW_UDF
INVALID_RETURN_TYPE_FOR_PANDAS_UDF
INVALID_SESSION_UUID_ID
INVALID_STREAMING_SOURCE_NAME
INVALID_TIMEOUT_TIMESTAMP
INVALID_TYPE
INVALID_TYPENAME_CALL
INVALID_TYPE_DF_EQUALITY_ARG
INVALID_UDF_EVAL_TYPE
INVALID_UDTF_BOTH_RETURN_TYPE_AND_ANALYZE
INVALID_UDTF_EVAL_TYPE
INVALID_UDTF_HANDLER_TYPE
INVALID_UDTF_NO_EVAL
INVALID_UDTF_RETURN_TYPE
INVALID_WHEN_USAGE
INVALID_WINDOW_BOUND_TYPE
JAVA_GATEWAY_EXITED
JVM_ATTRIBUTE_NOT_SUPPORTED
KEY_NOT_EXISTS
KEY_VALUE_PAIR_REQUIRED
LENGTH_SHOULD_BE_THE_SAME
LOCAL_RELATION_SIZE_LIMIT_EXCEEDED
MALFORMED_GEOGRAPHY
MALFORMED_GEOMETRY
MALFORMED_VARIANT
MASTER_URL_INVALID
MASTER_URL_NOT_SET
MEMORY_PROFILE_INVALID_SOURCE
MISSING_LIBRARY_FOR_PROFILER
MISSING_VALID_PLAN
MIXED_TYPE_REPLACEMENT
MULTIPLE_PIPELINE_SPEC_FILES_FOUND
NEAREST_BY_JOIN
NEGATIVE_VALUE
NOT_EXPECTED_TYPE
NOT_IMPLEMENTED
NOT_IN_BARRIER_STAGE
NOT_ITERABLE
NOT_NUMERIC_COLUMNS
NOT_SAME_TYPE
NO_ACTIVE_EXCEPTION
NO_ACTIVE_OR_DEFAULT_SESSION
NO_ACTIVE_SESSION
NO_OBSERVE_BEFORE_GET
NO_SCHEMA_AND_DRIVER_DEFAULT_SCHEME
ONLY_ALLOWED_FOR_SINGLE_COLUMN
ONLY_ALLOW_SINGLE_TRIGGER
ONLY_SUPPORTED_WITH_SPARK_CONNECT
OUTPUT_EXCEEDS_INPUT_ROWS
PACKAGE_NOT_INSTALLED
PANDAS_API_ON_SPARK_FAIL_ON_ANSI_MODE
PIPELINE_SPEC_DICT_KEY_NOT_STRING
PIPELINE_SPEC_DICT_VALUE_NOT_STRING
PIPELINE_SPEC_FIELD_NOT_DICT
PIPELINE_SPEC_FILE_DOES_NOT_EXIST
PIPELINE_SPEC_FILE_NOT_FOUND
PIPELINE_SPEC_INVALID_GLOB_PATTERN
PIPELINE_SPEC_MISSING_REQUIRED_FIELD
PIPELINE_SPEC_UNEXPECTED_FIELD
PIPELINE_UNSUPPORTED_DEFINITIONS_FILE_EXTENSION
PIPE_FUNCTION_EXITED
PLOT_INVALID_TYPE_COLUMN
PROTOCOL_ERROR
PYTHON_HASH_SEED_NOT_SET
PYTHON_STREAMING_DATA_SOURCE_RUNTIME_ERROR
PYTHON_VERSION_MISMATCH
RDD_TRANSFORM_ONLY_VALID_ON_DRIVER
READ_ONLY
RESPONSE_ALREADY_RECEIVED
RESULT_COLUMN_NAMES_MISMATCH
RESULT_COLUMN_SCHEMA_MISMATCH
RESULT_COLUMN_TYPES_MISMATCH
RESULT_ROWS_MISMATCH
REUSE_OBSERVATION
SCHEMA_MISMATCH_FOR_PANDAS_UDF
SESSION_ALREADY_EXIST
SESSION_MUTATION_IN_DECLARATIVE_PIPELINE
SESSION_NEED_CONN_STR_OR_BUILDER
SESSION_NOT_SAME
SESSION_OR_CONTEXT_EXISTS
SESSION_OR_CONTEXT_NOT_EXISTS
SIMPLE_STREAM_READER_OFFSET_DID_NOT_ADVANCE
SLICE_WITH_STEP
STATE_NOT_EXISTS
STOP_ITERATION_OCCURRED
STREAMING_CONNECT_SERIALIZATION_ERROR
ST_INVALID_ALGORITHM_VALUE
ST_INVALID_CRS_VALUE
ST_INVALID_ENDIANNESS_VALUE
ST_INVALID_SRID_VALUE
TEST_CLASS_NOT_COMPILED
TOO_MANY_VALUES
TYPE_HINT_SHOULD_BE_SPECIFIED
UDF_RETURN_TYPE
UDTF_ARROW_DATA_CONVERSION_ERROR
UDTF_ARROW_TYPE_CONVERSION_ERROR
UDTF_CONSTRUCTOR_INVALID_IMPLEMENTS_ANALYZE_METHOD
UDTF_CONSTRUCTOR_INVALID_NO_ANALYZE_METHOD
UDTF_EVAL_METHOD_ARGUMENTS_DO_NOT_MATCH_SIGNATURE
UDTF_EXEC_ERROR
UDTF_INVALID_OUTPUT_ROW_TYPE
UDTF_RETURN_NOT_ITERABLE
UDTF_RETURN_SCHEMA_MISMATCH
UDTF_RETURN_TYPE_MISMATCH
UDTF_SERIALIZATION_ERROR
UNEXPECTED_RESPONSE_FROM_SERVER
UNEXPECTED_TUPLE_WITH_STRUCT
UNKNOWN_EXPLAIN_MODE
UNKNOWN_INTERRUPT_TYPE
UNKNOWN_RESPONSE
UNKNOWN_VALUE_FOR
UNSUPPORTED_DATA_TYPE
UNSUPPORTED_DATA_TYPE_FOR_ARROW
UNSUPPORTED_DATA_TYPE_FOR_ARROW_CONVERSION
UNSUPPORTED_JOIN_TYPE
UNSUPPORTED_LITERAL
UNSUPPORTED_LOCAL_CONNECTION_STRING
UNSUPPORTED_NUMPY_ARRAY_SCALAR
UNSUPPORTED_OPERATION
UNSUPPORTED_PACKAGE_VERSION
UNSUPPORTED_PARAM_TYPE_FOR_HIGHER_ORDER_FUNCTION
UNSUPPORTED_PIE_PLOT_PARAM
UNSUPPORTED_PIPELINES_DATASET_TYPE
UNSUPPORTED_PLOT_BACKEND
UNSUPPORTED_PLOT_BACKEND_PARAM
UNSUPPORTED_PLOT_KIND
UNSUPPORTED_SIGNATURE
VALUE_ALLOWED
VALUE_NOT_ACCESSIBLE
VALUE_NOT_ALLOWED
VALUE_NOT_ANY_OR_ALL
VALUE_NOT_BETWEEN
VALUE_NOT_NON_EMPTY_STR
VALUE_NOT_PEARSON
VALUE_NOT_PLAIN_COLUMN_REFERENCE
VALUE_NOT_POSITIVE
VALUE_NOT_TRUE
VALUE_OUT_OF_BOUNDS
WKB_PARSE_ERROR
WRONG_NUM_ARGS_FOR_HIGHER_ORDER_FUNCTION
WRONG_NUM_COLUMNS
ZERO_INDEX
Show Source
Web Proxy Viewer |
New URL
|
Original Page
![[Logo image]](https://spark.apache.org/images/spark-logo.png){kind=link}