Extension#

Extension Type classes#

class Bool8Type : public arrow::ExtensionType#

Bool8 is an alternate representation for boolean arrays using 8 bits instead of 1 bit per value.

The underlying storage type is int8.

Public Functions

inline Bool8Type()#

Construct a Bool8Type.

inline virtual std::string extension_name() const override#

Unique name of extension type used to identify type for serialization.

Returns:

the string name of the extension

virtual std::string ToString(bool show_metadata = false) const override#

A string representation of the type, including any children.

virtual bool ExtensionEquals(const ExtensionType &other) const override#

Determine if two instances of the same extension types are equal.

Invoked from ExtensionType::Equals

Parameters:

other [in] the type to compare this type with

Returns:

bool true if type instances are equal

virtual std::string Serialize() const override#

Create a serialized representation of the extension types metadata.

The storage type will be handled automatically in IPC code paths

Returns:

the serialized representation

virtual Result<std::shared_ptr<DataType>> Deserialize(std::shared_ptr<DataType> storage_type, const std::string &serialized_data) const override#

Create an instance of the ExtensionType given the actual storage type and the serialized representation.

Parameters:

• storage_type [in] the physical storage type of the extension

• serialized_data [in] the serialized representation produced by Serialize

virtual std::shared_ptr<Array> MakeArray(std::shared_ptr<ArrayData> data) const override#

Create a Bool8Array from ArrayData.

class FixedShapeTensorType : public arrow::ExtensionType#

Concrete type class for constant-size Tensor data.

This is a canonical arrow extension type. See: https://arrow.apache.org/docs/format/CanonicalExtensions.html

Public Functions

inline virtual std::string extension_name() const override#

Unique name of extension type used to identify type for serialization.

Returns:

the string name of the extension

virtual std::string ToString(bool show_metadata = false) const override#

A string representation of the type, including any children.

inline size_t ndim() const#

Number of dimensions of tensor elements.

inline const std::vector<int64_t> &shape() const#

Shape of tensor elements.

inline const std::shared_ptr<DataType> &value_type() const#

Value type of tensor elements.

const std::vector<int64_t> &strides()#

Strides of tensor elements.

Strides state offset in bytes between adjacent elements along each dimension. In case permutation is non-empty strides are computed from permuted tensor elements shape.

inline const std::vector<int64_t> &permutation() const#

Permutation mapping from logical to physical memory layout of tensor elements.

inline const std::vector<std::string> &dim_names() const#

Dimension names of tensor elements. Dimensions are ordered physically.

virtual bool ExtensionEquals(const ExtensionType &other) const override#

Determine if two instances of the same extension types are equal.

Invoked from ExtensionType::Equals

Parameters:

other [in] the type to compare this type with

Returns:

bool true if type instances are equal

virtual std::string Serialize() const override#

Create a serialized representation of the extension types metadata.

The storage type will be handled automatically in IPC code paths

Returns:

the serialized representation

virtual Result<std::shared_ptr<DataType>> Deserialize(std::shared_ptr<DataType> storage_type, const std::string &serialized_data) const override#

Create an instance of the ExtensionType given the actual storage type and the serialized representation.

Parameters:

• storage_type [in] the physical storage type of the extension

• serialized_data [in] the serialized representation produced by Serialize

virtual std::shared_ptr<Array> MakeArray(std::shared_ptr<ArrayData> data) const override#

Create a FixedShapeTensorArray from ArrayData.

Public Static Functions

static Result<std::shared_ptr<Tensor>> MakeTensor(const std::shared_ptr<ExtensionScalar> &scalar)#

Create a Tensor from an ExtensionScalar from a FixedShapeTensorArray.

This method will return a Tensor from ExtensionScalar with strides derived from shape and permutation of FixedShapeTensorType. Shape and dim_names will be permuted according to permutation stored in the FixedShapeTensorType metadata.

static Result<std::shared_ptr<DataType>> Make(const std::shared_ptr<DataType> &value_type, const std::vector<int64_t> &shape, const std::vector<int64_t> &permutation = {}, const std::vector<std::string> &dim_names = {})#

Create a FixedShapeTensorType instance.

class OpaqueType : public arrow::ExtensionType#

Opaque is a placeholder for a type from an external (usually non-Arrow) system that could not be interpreted.

Public Functions

inline explicit OpaqueType(std::shared_ptr<DataType> storage_type, std::string type_name, std::string vendor_name)#

Construct an OpaqueType.

Parameters:

• storage_type [in] The underlying storage type. Should be arrow::null if there is no data.

• type_name [in] The name of the type in the external system.

• vendor_name [in] The name of the external system.

inline virtual std::string extension_name() const override#

Unique name of extension type used to identify type for serialization.

Returns:

the string name of the extension

virtual std::string ToString(bool show_metadata) const override#

A string representation of the type, including any children.

virtual bool ExtensionEquals(const ExtensionType &other) const override#

Determine if two instances of the same extension types are equal.

Invoked from ExtensionType::Equals

Parameters:

other [in] the type to compare this type with

Returns:

bool true if type instances are equal

virtual std::string Serialize() const override#

Create a serialized representation of the extension types metadata.

The storage type will be handled automatically in IPC code paths

Returns:

the serialized representation

virtual Result<std::shared_ptr<DataType>> Deserialize(std::shared_ptr<DataType> storage_type, const std::string &serialized_data) const override#

Create an instance of the ExtensionType given the actual storage type and the serialized representation.

Parameters:

• storage_type [in] the physical storage type of the extension

• serialized_data [in] the serialized representation produced by Serialize

virtual std::shared_ptr<Array> MakeArray(std::shared_ptr<ArrayData> data) const override#

Create an OpaqueArray from ArrayData.

class JsonExtensionType : public arrow::ExtensionType#

Concrete type class for variable-size JSON data, utf8-encoded.

Public Functions

inline virtual std::string extension_name() const override#

Unique name of extension type used to identify type for serialization.

Returns:

the string name of the extension

virtual bool ExtensionEquals(const ExtensionType &other) const override#

Determine if two instances of the same extension types are equal.

Invoked from ExtensionType::Equals

Parameters:

other [in] the type to compare this type with

Returns:

bool true if type instances are equal

virtual Result<std::shared_ptr<DataType>> Deserialize(std::shared_ptr<DataType> storage_type, const std::string &serialized_data) const override#

Create an instance of the ExtensionType given the actual storage type and the serialized representation.

Parameters:

• storage_type [in] the physical storage type of the extension

• serialized_data [in] the serialized representation produced by Serialize

virtual std::string Serialize() const override#

Create a serialized representation of the extension types metadata.

The storage type will be handled automatically in IPC code paths

Returns:

the serialized representation

virtual std::shared_ptr<Array> MakeArray(std::shared_ptr<ArrayData> data) const override#

Wrap built-in Array type in a user-defined ExtensionArray instance.

Parameters:

data [in] the physical storage for the extension type

class UuidType : public arrow::ExtensionType#

UuidType is a canonical arrow extension type for UUIDs.

UUIDs are stored as FixedSizeBinary(16) with big-endian notation and this does not interpret the bytes in any way. Specific UUID version is not required or guaranteed.

Public Functions

inline UuidType()#

Construct a UuidType.

inline virtual std::string extension_name() const override#

Unique name of extension type used to identify type for serialization.

Returns:

the string name of the extension

virtual std::string ToString(bool show_metadata = false) const override#

A string representation of the type, including any children.

virtual bool ExtensionEquals(const ExtensionType &other) const override#

Determine if two instances of the same extension types are equal.

Invoked from ExtensionType::Equals

Parameters:

other [in] the type to compare this type with

Returns:

bool true if type instances are equal

virtual std::shared_ptr<Array> MakeArray(std::shared_ptr<ArrayData> data) const override#

Create a UuidArray from ArrayData.

virtual Result<std::shared_ptr<DataType>> Deserialize(std::shared_ptr<DataType> storage_type, const std::string &serialized) const override#

Create an instance of the ExtensionType given the actual storage type and the serialized representation.

Parameters:

• storage_type [in] the physical storage type of the extension

• serialized_data [in] the serialized representation produced by Serialize

inline virtual std::string Serialize() const override#

Create a serialized representation of the extension types metadata.

The storage type will be handled automatically in IPC code paths

Returns:

the serialized representation

Public Static Functions

static inline Result<std::shared_ptr<DataType>> Make()#

Create a UuidType instance.

Extension Array classes#

class Bool8Array : public arrow::ExtensionArray#

Bool8 is an alternate representation for boolean arrays using 8 bits instead of 1 bit per value.

The underlying storage type is int8.

Public Functions

explicit ExtensionArray(const std::shared_ptr<ArrayData> &data)#

Construct an ExtensionArray from an ArrayData.

The ArrayData must have the right ExtensionType.

ExtensionArray(const std::shared_ptr<DataType> &type, const std::shared_ptr<Array> &storage)#

Construct an ExtensionArray from a type and the underlying storage.

class FixedShapeTensorArray : public arrow::ExtensionArray#

Public Functions

const Result<std::shared_ptr<Tensor>> ToTensor() const#

Create a Tensor from FixedShapeTensorArray.

This method will create a Tensor from a FixedShapeTensorArray, setting its first dimension as length equal to the FixedShapeTensorArrays length and the remaining dimensions as the FixedShapeTensorTypes shape. Shape and dim_names will be permuted according to permutation stored in the FixedShapeTensorType metadata.

explicit ExtensionArray(const std::shared_ptr<ArrayData> &data)#

Construct an ExtensionArray from an ArrayData.

The ArrayData must have the right ExtensionType.

ExtensionArray(const std::shared_ptr<DataType> &type, const std::shared_ptr<Array> &storage)#

Construct an ExtensionArray from a type and the underlying storage.

Public Static Functions

static Result<std::shared_ptr<FixedShapeTensorArray>> FromTensor(const std::shared_ptr<Tensor> &tensor)#

Create a FixedShapeTensorArray from a Tensor.

This method will create a FixedShapeTensorArray from a Tensor, taking its first dimension as the number of elements in the resulting array and the remaining dimensions as the shape of the individual tensors. If Tensor provides strides, they will be used to determine dimension permutation. Otherwise, row-major layout (i.e. no permutation) will be assumed.

Parameters:

tensor [in] The Tensor to convert to a FixedShapeTensorArray

class OpaqueArray : public arrow::ExtensionArray#

Opaque is a wrapper for (usually binary) data from an external (often non-Arrow) system that could not be interpreted.

Public Functions

explicit ExtensionArray(const std::shared_ptr<ArrayData> &data)#

Construct an ExtensionArray from an ArrayData.

The ArrayData must have the right ExtensionType.

ExtensionArray(const std::shared_ptr<DataType> &type, const std::shared_ptr<Array> &storage)#

Construct an ExtensionArray from a type and the underlying storage.

class UuidArray : public arrow::ExtensionArray#

UuidArray stores array of UUIDs.

Underlying storage type is FixedSizeBinary(16).

Public Functions

explicit ExtensionArray(const std::shared_ptr<ArrayData> &data)#

Construct an ExtensionArray from an ArrayData.

The ArrayData must have the right ExtensionType.

ExtensionArray(const std::shared_ptr<DataType> &type, const std::shared_ptr<Array> &storage)#

Construct an ExtensionArray from a type and the underlying storage.