Data Types¶

enum arrow::Type::type¶

Main data type enumeration.

This enumeration provides a quick way to interrogate the category of a DataType instance.

Values:

NA = 0¶: A NULL type having no physical storage.

BOOL¶: Boolean as 1 bit, LSB bit-packed ordering.

UINT8¶: Unsigned 8-bit little-endian integer.

INT8¶: Signed 8-bit little-endian integer.

UINT16¶: Unsigned 16-bit little-endian integer.

INT16¶: Signed 16-bit little-endian integer.

UINT32¶: Unsigned 32-bit little-endian integer.

INT32¶: Signed 32-bit little-endian integer.

UINT64¶: Unsigned 64-bit little-endian integer.

INT64¶: Signed 64-bit little-endian integer.

HALF_FLOAT¶: 2-byte floating point value

FLOAT¶: 4-byte floating point value

DOUBLE¶: 8-byte floating point value

STRING¶: UTF8 variable-length string as List<Char>

BINARY¶: Variable-length bytes (no guarantee of UTF8-ness)

FIXED_SIZE_BINARY¶: Fixed-size binary. Each value occupies the same number of bytes.

DATE32¶: int32_t days since the UNIX epoch

DATE64¶: int64_t milliseconds since the UNIX epoch

TIMESTAMP¶: Exact timestamp encoded with int64 since UNIX epoch Default unit millisecond.

TIME32¶: Time as signed 32-bit integer, representing either seconds or milliseconds since midnight.

TIME64¶: Time as signed 64-bit integer, representing either microseconds or nanoseconds since midnight.

INTERVAL_MONTHS¶: YEAR_MONTH interval in SQL style.

INTERVAL_DAY_TIME¶: DAY_TIME interval in SQL style.

DECIMAL128¶: Precision- and scale-based decimal type with 128 bits.

DECIMAL = DECIMAL128 ¶: Defined for backward-compatibility.

DECIMAL256¶: Precision- and scale-based decimal type with 256 bits.

LIST¶: A list of some logical data type.

STRUCT¶: Struct of logical types.

SPARSE_UNION¶: Sparse unions of logical types.

DENSE_UNION¶: Dense unions of logical types.

DICTIONARY¶

Dictionary-encoded type, also called “categorical” or “factor” in other programming languages.

Holds the dictionary value type but not the dictionary itself, which is part of the ArrayData struct

MAP¶: Map, a repeated struct logical type.

EXTENSION¶: Custom data type, implemented by user.

FIXED_SIZE_LIST¶: Fixed size list of some logical type.

DURATION¶: Measure of elapsed time in either seconds, milliseconds, microseconds or nanoseconds.

LARGE_STRING¶: Like STRING, but with 64-bit offsets.

LARGE_BINARY¶: Like BINARY, but with 64-bit offsets.

LARGE_LIST¶: Like LIST, but with 64-bit offsets.

MAX_ID¶

class DataType : public arrow::detail::Fingerprintable¶

Base class for all data types.

Data types in this library are all logical. They can be expressed as either a primitive physical type (bytes or bits of some fixed size), a nested type consisting of other data types, or another data type (e.g. a timestamp encoded as an int64).

Simple datatypes may be entirely described by their Type::type id, but complex datatypes are usually parametric.

Subclassed by arrow::BaseBinaryType, arrow::ExtensionType, arrow::FixedWidthType, arrow::NestedType, arrow::NullType

Public Functions

bool Equals(const DataType &other, bool check_metadata = false) const¶

Return whether the types are equal.

Types that are logically convertible from one to another (e.g. List<UInt8> and Binary) are NOT equal.

bool Equals(const std::shared_ptr<DataType> &other) const¶: Return whether the types are equal.

const std::shared_ptr<Field> &field(int i) const¶: Returns the child-field at index i.

const std::vector<std::shared_ptr<Field>> &fields() const¶: Returns the children fields associated with this type.

int num_fields() const¶: Returns the number of children fields associated with this type.

virtual std::string ToString() const = 0¶: A string representation of the type, including any children.

size_t Hash() const¶: Return hash value (excluding metadata in child fields)

virtual std::string name() const = 0¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

virtual DataTypeLayout layout() const = 0¶

Return the data type layout.

Children are not included.

Note: Experimental API

Type::type id() const¶: Return the type category.

Factory functions¶

These functions are recommended for creating data types. They may return new objects or existing singletons, depending on the type requested.

std::shared_ptr<DataType> arrow::null()¶: Return a NullType instance.

std::shared_ptr<DataType> arrow::boolean()¶: Return a BooleanType instance.

std::shared_ptr<DataType> arrow::int8()¶: Return a Int8Type instance.

std::shared_ptr<DataType> arrow::int16()¶: Return a Int16Type instance.

std::shared_ptr<DataType> arrow::int32()¶: Return a Int32Type instance.

std::shared_ptr<DataType> arrow::int64()¶: Return a Int64Type instance.

std::shared_ptr<DataType> arrow::uint8()¶: Return a UInt8Type instance.

std::shared_ptr<DataType> arrow::uint16()¶: Return a UInt16Type instance.

std::shared_ptr<DataType> arrow::uint32()¶: Return a UInt32Type instance.

std::shared_ptr<DataType> arrow::uint64()¶: Return a UInt64Type instance.

std::shared_ptr<DataType> arrow::float16()¶: Return a HalfFloatType instance.

std::shared_ptr<DataType> arrow::float32()¶: Return a FloatType instance.

std::shared_ptr<DataType> arrow::float64()¶: Return a DoubleType instance.

std::shared_ptr<DataType> arrow::utf8()¶: Return a StringType instance.

std::shared_ptr<DataType> arrow::large_utf8()¶: Return a LargeStringType instance.

std::shared_ptr<DataType> arrow::binary()¶: Return a BinaryType instance.

std::shared_ptr<DataType> arrow::large_binary()¶: Return a LargeBinaryType instance.

std::shared_ptr<DataType> arrow::date32()¶: Return a Date32Type instance.

std::shared_ptr<DataType> arrow::date64()¶: Return a Date64Type instance.

std::shared_ptr<DataType> arrow::fixed_size_binary(int32_t byte_width)¶: Create a FixedSizeBinaryType instance.

std::shared_ptr<DataType> arrow::decimal(int32_t precision, int32_t scale)¶

Create a DecimalType instance depending on the precision.

If the precision is greater than 38, a Decimal128Type is returned, otherwise a Decimal256Type.

std::shared_ptr<DataType> arrow::decimal128(int32_t precision, int32_t scale)¶: Create a Decimal128Type instance.

std::shared_ptr<DataType> arrow::decimal256(int32_t precision, int32_t scale)¶: Create a Decimal256Type instance.

std::shared_ptr<DataType> arrow::list(const std::shared_ptr<Field> &value_type)¶: Create a ListType instance from its child Field type.

std::shared_ptr<DataType> arrow::list(const std::shared_ptr<DataType> &value_type)¶: Create a ListType instance from its child DataType.

std::shared_ptr<DataType> arrow::large_list(const std::shared_ptr<Field> &value_type)¶: Create a LargeListType instance from its child Field type.

std::shared_ptr<DataType> arrow::large_list(const std::shared_ptr<DataType> &value_type)¶: Create a LargeListType instance from its child DataType.

std::shared_ptr<DataType> arrow::map(std::shared_ptr<DataType> key_type, std::shared_ptr<DataType> item_type, bool keys_sorted = false)¶: Create a MapType instance from its key and value DataTypes.

std::shared_ptr<DataType> arrow::map(std::shared_ptr<DataType> key_type, std::shared_ptr<Field> item_field, bool keys_sorted = false)¶

Create a MapType instance from its key DataType and value field.

The field override is provided to communicate nullability of the value.

std::shared_ptr<DataType> arrow::fixed_size_list(const std::shared_ptr<Field> &value_type, int32_t list_size)¶: Create a FixedSizeListType instance from its child Field type.

std::shared_ptr<DataType> arrow::fixed_size_list(const std::shared_ptr<DataType> &value_type, int32_t list_size)¶: Create a FixedSizeListType instance from its child DataType.

std::shared_ptr<DataType> arrow::duration(TimeUnit::type unit)¶: Return a Duration instance (naming use _type to avoid namespace conflict with built in time classes).

std::shared_ptr<DataType> arrow::day_time_interval()¶: Return a DayTimeIntervalType instance.

std::shared_ptr<DataType> arrow::month_interval()¶: Return a MonthIntervalType instance.

std::shared_ptr<DataType> arrow::timestamp(TimeUnit::type unit)¶: Create a TimestampType instance from its unit.

std::shared_ptr<DataType> arrow::timestamp(TimeUnit::type unit, const std::string &timezone)¶: Create a TimestampType instance from its unit and timezone.

std::shared_ptr<DataType> arrow::time32(TimeUnit::type unit)¶

Create a 32-bit time type instance.

Unit can be either SECOND or MILLI

std::shared_ptr<DataType> arrow::time64(TimeUnit::type unit)¶

Create a 64-bit time type instance.

Unit can be either MICRO or NANO

std::shared_ptr<DataType> arrow::struct_(const std::vector<std::shared_ptr<Field>> &fields)¶: Create a StructType instance.

std::shared_ptr<DataType> arrow::sparse_union(FieldVector child_fields, std::vector<int8_t> type_codes = {})¶: Create a SparseUnionType instance.

std::shared_ptr<DataType> arrow::dense_union(FieldVector child_fields, std::vector<int8_t> type_codes = {})¶: Create a DenseUnionType instance.

std::shared_ptr<DataType> arrow::sparse_union(const ArrayVector &children, std::vector<std::string> field_names = {}, std::vector<int8_t> type_codes = {})¶: Create a SparseUnionType instance.

std::shared_ptr<DataType> arrow::dense_union(const ArrayVector &children, std::vector<std::string> field_names = {}, std::vector<int8_t> type_codes = {})¶: Create a DenseUnionType instance.

std::shared_ptr<DataType> arrow::union_(const std::vector<std::shared_ptr<Field>> &child_fields, const std::vector<int8_t> &type_codes, UnionMode::type mode = UnionMode::SPARSE)¶: Create a UnionType instance.

std::shared_ptr<DataType> arrow::union_(const std::vector<std::shared_ptr<Field>> &child_fields, UnionMode::type mode = UnionMode::SPARSE)¶: Create a UnionType instance.

std::shared_ptr<DataType> arrow::union_(const std::vector<std::shared_ptr<Array>> &children, const std::vector<std::string> &field_names, const std::vector<int8_t> &type_codes, UnionMode::type mode = UnionMode::SPARSE)¶: Create a UnionType instance.

std::shared_ptr<DataType> arrow::union_(const std::vector<std::shared_ptr<Array>> &children, const std::vector<std::string> &field_names, UnionMode::type mode = UnionMode::SPARSE)¶: Create a UnionType instance.

std::shared_ptr<DataType> arrow::union_(const std::vector<std::shared_ptr<Array>> &children, UnionMode::type mode = UnionMode::SPARSE)¶: Create a UnionType instance.

std::shared_ptr<DataType> arrow::dictionary(const std::shared_ptr<DataType> &index_type, const std::shared_ptr<DataType> &dict_type, bool ordered = false)¶

Create a DictionaryType instance.

Parameters

[in] index_type: the type of the dictionary indices (must be a signed integer)
[in] dict_type: the type of the values in the variable dictionary
[in] ordered: true if the order of the dictionary values has semantic meaning and should be preserved where possible

Concrete type subclasses¶

Primitive¶

class NullType : public arrow::DataType ¶

Concrete type class for always-null data.

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class BooleanType : public arrow::detail::CTypeImpl<BooleanType, PrimitiveCType, Type::BOOL, bool>¶

Concrete type class for boolean data.

Public Functions

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

class Int8Type : public arrow::detail::IntegerTypeImpl<Int8Type, Type::INT8, int8_t>¶: Concrete type class for signed 8-bit integer data.

class Int16Type : public arrow::detail::IntegerTypeImpl<Int16Type, Type::INT16, int16_t>¶: Concrete type class for signed 16-bit integer data.

class Int32Type : public arrow::detail::IntegerTypeImpl<Int32Type, Type::INT32, int32_t>¶: Concrete type class for signed 32-bit integer data.

class Int64Type : public arrow::detail::IntegerTypeImpl<Int64Type, Type::INT64, int64_t>¶: Concrete type class for signed 64-bit integer data.

class UInt8Type : public arrow::detail::IntegerTypeImpl<UInt8Type, Type::UINT8, uint8_t>¶: Concrete type class for unsigned 8-bit integer data.

class UInt16Type : public arrow::detail::IntegerTypeImpl<UInt16Type, Type::UINT16, uint16_t>¶: Concrete type class for unsigned 16-bit integer data.

class UInt32Type : public arrow::detail::IntegerTypeImpl<UInt32Type, Type::UINT32, uint32_t>¶: Concrete type class for unsigned 32-bit integer data.

class UInt64Type : public arrow::detail::IntegerTypeImpl<UInt64Type, Type::UINT64, uint64_t>¶: Concrete type class for unsigned 64-bit integer data.

class HalfFloatType : public arrow::detail::CTypeImpl<HalfFloatType, FloatingPointType, Type::HALF_FLOAT, uint16_t>¶: Concrete type class for 16-bit floating-point data.

class FloatType : public arrow::detail::CTypeImpl<FloatType, FloatingPointType, Type::FLOAT, float>¶: Concrete type class for 32-bit floating-point data (C “float”)

class DoubleType : public arrow::detail::CTypeImpl<DoubleType, FloatingPointType, Type::DOUBLE, double>¶: Concrete type class for 64-bit floating-point data (C “double”)

Time-related¶

enum arrow::TimeUnit::type¶

The unit for a time or timestamp DataType.

Values:

SECOND = 0¶

MILLI = 1¶

MICRO = 2¶

NANO = 3¶

class Date32Type : public arrow::DateType¶

Concrete type class for 32-bit date data (as number of days since UNIX epoch)

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class Date64Type : public arrow::DateType¶

Concrete type class for 64-bit date data (as number of milliseconds since UNIX epoch)

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class Time32Type : public arrow::TimeType¶

Concrete type class for 32-bit time data (as number of seconds or milliseconds since midnight)

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class Time64Type : public arrow::TimeType¶

Concrete type class for 64-bit time data (as number of microseconds or nanoseconds since midnight)

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class TimestampType : public arrow::TemporalType, public arrow::ParametricType¶

Concrete type class for datetime data (as number of seconds, milliseconds, microseconds or nanoseconds since UNIX epoch)

If supplied, the timezone string should take either the form (i) “Area/Location”, with values drawn from the names in the IANA Time Zone Database (such as “Europe/Zurich”); or (ii) “(+|-)HH:MM” indicating an absolute offset from GMT (such as “-08:00”). To indicate a native UTC timestamp, one of the strings “UTC”, “Etc/UTC” or “+00:00” should be used.

If any non-empty string is supplied as the timezone for a TimestampType, then the Arrow field containing that timestamp type (and by extension the column associated with such a field) is considered “timezone-aware”. The integer arrays that comprise a timezone-aware column must contain UTC normalized datetime values, regardless of the contents of their timezone string. More precisely, (i) the producer of a timezone-aware column must populate its constituent arrays with valid UTC values (performing offset conversions from non-UTC values if necessary); and (ii) the consumer of a timezone-aware column may assume that the column’s values are directly comparable (that is, with no offset adjustment required) to the values of any other timezone-aware column or to any other valid UTC datetime value (provided all values are expressed in the same units).

If a TimestampType is constructed without a timezone (or, equivalently, if the timezone supplied is an empty string) then the resulting Arrow field (column) is considered “timezone-naive”. The producer of a timezone-naive column may populate its constituent integer arrays with datetime values from any timezone; the consumer of a timezone-naive column should make no assumptions about the interoperability or comparability of the values of such a column with those of any other timestamp column or datetime value.

If a timezone-aware field contains a recognized timezone, its values may be localized to that locale upon display; the values of timezone-naive fields must always be displayed “as is”, with no localization performed on them.

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

Binary-like¶

class BinaryType : public arrow::BaseBinaryType¶

Concrete type class for variable-size binary data.

Subclassed by arrow::StringType

Public Functions

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class StringType : public arrow::BinaryType ¶

Concrete type class for variable-size string data, utf8-encoded.

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class FixedSizeBinaryType : public arrow::FixedWidthType, public arrow::ParametricType¶

Concrete type class for fixed-size binary data.

Subclassed by arrow::DecimalType

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

class Decimal128Type : public arrow::DecimalType¶

Concrete type class for 128-bit decimal data.

Arrow decimals are fixed-point decimal numbers encoded as a scaled integer. The precision is the number of significant digits that the decimal type can represent; the scale is the number of digits after the decimal point (note the scale can be negative).

As an example, Decimal128Type(7, 3) can exactly represent the numbers 1234.567 and -1234.567 (encoded internally as the 128-bit integers 1234567 and -1234567, respectively), but neither 12345.67 nor 123.4567.

Decimal128Type has a maximum precision of 38 significant digits (also available as Decimal128Type::kMaxPrecision). If higher precision is needed, consider using Decimal256Type.

Public Functions

Decimal128Type(int32_t precision, int32_t scale)¶: Decimal128Type constructor that aborts on invalid input.

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

Public Static Functions

static Result<std::shared_ptr<DataType>> Make(int32_t precision, int32_t scale)¶: Decimal128Type constructor that returns an error on invalid input.

Nested¶

class ListType : public arrow::BaseListType¶

Concrete type class for list data.

List data is nested data where each value is a variable number of child items. Lists can be recursively nested, for example list(list(int32)).

Subclassed by arrow::MapType

Public Functions

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class MapType : public arrow::ListType ¶

Concrete type class for map data.

Map data is nested data where each value is a variable number of key-item pairs. Its physical representation is the same as a list of {key, item} structs.

Maps can be recursively nested, for example map(utf8, map(utf8, int32)).

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

class StructType : public arrow::NestedType¶

Concrete type class for struct data.

Public Functions

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

std::shared_ptr<Field> GetFieldByName(const std::string &name) const¶: Returns null if name not found.

std::vector<std::shared_ptr<Field>> GetAllFieldsByName(const std::string &name) const¶: Return all fields having this name.

int GetFieldIndex(const std::string &name) const¶: Returns -1 if name not found or if there are multiple fields having the same name.

std::vector<int> GetAllFieldIndices(const std::string &name) const¶: Return the indices of all fields having this name in sorted order.

class UnionType : public arrow::NestedType¶

Base type class for union data.

Subclassed by arrow::DenseUnionType, arrow::SparseUnionType

Public Functions

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

std::string ToString() const¶: A string representation of the type, including any children.

const std::vector<int8_t> &type_codes() const¶

The array of logical type ids.

For example, the first type in the union might be denoted by the id 5 (instead of 0).

const std::vector<int> &child_ids() const¶: An array mapping logical type ids to physical child ids.

Dictionary-encoded¶

class DictionaryType : public arrow::FixedWidthType¶

Dictionary-encoded value type with data-dependent dictionary.

Indices are represented by any integer types.

Public Functions

std::string ToString() const¶: A string representation of the type, including any children.

std::string name() const¶

A string name of the type, omitting any child fields.

Note: Experimental API
Since: 0.7.0

DataTypeLayout layout() const¶

Return the data type layout.

Children are not included.

Note: Experimental API

Fields and Schemas¶

std::shared_ptr<Field> arrow::field(std::string name, std::shared_ptr<DataType> type, bool nullable = true, std::shared_ptr<const KeyValueMetadata> metadata = NULLPTR)¶

Create a Field instance.

Parameters

name: the field name
type: the field value type
nullable: whether the values are nullable, default true
metadata: any custom key-value metadata, default null

std::shared_ptr<Field> arrow::field(std::string name, std::shared_ptr<DataType> type, std::shared_ptr<const KeyValueMetadata> metadata)¶

Create a Field instance with metadata.

The field will be assumed to be nullable.

Parameters

name: the field name
type: the field value type
metadata: any custom key-value metadata

std::shared_ptr<Schema> arrow::schema(std::vector<std::shared_ptr<Field>> fields, std::shared_ptr<const KeyValueMetadata> metadata = NULLPTR)¶

Create a Schema instance.

Return

schema shared_ptr to Schema

Parameters

fields: the schema’s fields
metadata: any custom key-value metadata, default null

std::shared_ptr<Schema> arrow::schema(std::vector<std::shared_ptr<Field>> fields, Endianness endianness, std::shared_ptr<const KeyValueMetadata> metadata = NULLPTR)¶

Create a Schema instance.

Return

schema shared_ptr to Schema

Parameters

fields: the schema’s fields
endianness: the endianness of the data
metadata: any custom key-value metadata, default null

class Field : public arrow::detail::Fingerprintable¶

The combination of a field name and data type, with optional metadata.

Fields are used to describe the individual constituents of a nested DataType or a Schema.

A field’s metadata is represented by a KeyValueMetadata instance, which holds arbitrary key-value pairs.

Public Functions

std::shared_ptr<const KeyValueMetadata> metadata() const¶: Return the field’s attached metadata.

bool HasMetadata() const¶: Return whether the field has non-empty metadata.

std::shared_ptr<Field> WithMetadata(const std::shared_ptr<const KeyValueMetadata> &metadata) const¶: Return a copy of this field with the given metadata attached to it.

std::shared_ptr<Field> WithMergedMetadata(const std::shared_ptr<const KeyValueMetadata> &metadata) const¶: EXPERIMENTAL: Return a copy of this field with the given metadata merged with existing metadata (any colliding keys will be overridden by the passed metadata)

std::shared_ptr<Field> RemoveMetadata() const¶: Return a copy of this field without any metadata attached to it.

std::shared_ptr<Field> WithType(const std::shared_ptr<DataType> &type) const¶: Return a copy of this field with the replaced type.

std::shared_ptr<Field> WithName(const std::string &name) const¶: Return a copy of this field with the replaced name.

std::shared_ptr<Field> WithNullable(bool nullable) const¶: Return a copy of this field with the replaced nullability.

Result<std::shared_ptr<Field>> MergeWith(const Field &other, MergeOptions options = MergeOptions::Defaults()) const¶

Merge the current field with a field of the same name.

The two fields must be compatible, i.e:

have the same name
have the same type, or of compatible types according to options.

The metadata of the current field is preserved; the metadata of the other field is discarded.

bool Equals(const Field &other, bool check_metadata = false) const¶

Indicate if fields are equals.

Return

true if fields are equal, false otherwise.

Parameters

[in] other: field to check equality with.
[in] check_metadata: controls if it should check for metadata equality.

bool IsCompatibleWith(const Field &other) const¶

Indicate if fields are compatibles.

See the criteria of MergeWith.

Return: true if fields are compatible, false otherwise.

std::string ToString(bool show_metadata = false) const¶

Return a string representation ot the field.

Parameters

[in] show_metadata: when true, if KeyValueMetadata is non-empty, print keys and values in the output

const std::string &name() const¶: Return the field name.

const std::shared_ptr<DataType> &type() const¶: Return the field data type.

bool nullable() const¶: Return whether the field is nullable.

struct MergeOptions¶

Options that control the behavior of MergeWith.

Options are to be added to allow type conversions, including integer widening, promotion from integer to float, or conversion to or from boolean.

Public Members

bool promote_nullability = true¶

If true, a Field of NullType can be unified with a Field of another type.

The unified field will be of the other type and become nullable. Nullability will be promoted to the looser option (nullable if one is not nullable).

class Schema : public arrow::detail::Fingerprintable, public arrow::util::EqualityComparable<Schema>, public arrow::util::ToStringOstreamable<Schema>¶

Sequence of arrow::Field objects describing the columns of a record batch or table data structure.

Public Functions

bool Equals(const Schema &other, bool check_metadata = false) const¶: Returns true if all of the schema fields are equal.

std::shared_ptr<Schema> WithEndianness(Endianness endianness) const¶

Set endianness in the schema.

Return: new Schema

Endianness endianness() const¶: Return endianness in the schema.

bool is_native_endian() const¶: Indicate if endianness is equal to platform-native endianness.

int num_fields() const¶: Return the number of fields (columns) in the schema.

const std::shared_ptr<Field> &field(int i) const¶: Return the ith schema element. Does not boundscheck.

std::shared_ptr<Field> GetFieldByName(const std::string &name) const¶: Returns null if name not found.

FieldVector GetAllFieldsByName(const std::string &name) const¶: Return the indices of all fields having this name in sorted order.

int GetFieldIndex(const std::string &name) const¶: Returns -1 if name not found.

std::vector<int> GetAllFieldIndices(const std::string &name) const¶: Return the indices of all fields having this name.

Status CanReferenceFieldsByNames(const std::vector<std::string> &names) const¶: Indicate if fields named names can be found unambiguously in the schema.

const std::shared_ptr<const KeyValueMetadata> &metadata() const¶

The custom key-value metadata, if any.

Return: metadata may be null

std::string ToString(bool show_metadata = false) const¶

Render a string representation of the schema suitable for debugging.

Parameters

[in] show_metadata: when true, if KeyValueMetadata is non-empty, print keys and values in the output

std::shared_ptr<Schema> WithMetadata(const std::shared_ptr<const KeyValueMetadata> &metadata) const¶

Replace key-value metadata with new metadata.

Return

new Schema

Parameters

[in] metadata: new KeyValueMetadata

std::shared_ptr<Schema> RemoveMetadata() const¶: Return copy of Schema without the KeyValueMetadata.

bool HasMetadata() const¶: Indicate that the Schema has non-empty KevValueMetadata.

bool HasDistinctFieldNames() const¶: Indicate that the Schema has distinct field names.

Memory (management) Arrays