Conversion of range of `std::tuple`-like to `Table` instances#

While the above example shows a quite manual approach of a row to columnar conversion, Arrow also provides some template logic to convert ranges of std::tuple<..>-like objects to tables.

In the most simple case, you only need to provide the input data and the type conversion is then inferred at compile time.

std::vector<std::tuple<double, std::string>> rows = ..
std::shared_ptr<Table> table;

if (!arrow::stl::TableFromTupleRange(
      arrow::default_memory_pool(),
      rows, names, &table).ok()
) {
  // Error handling code should go here.
}

In reverse, you can use TupleRangeFromTable to fill an already pre-allocated range with the data from a Table instance.

// An important aspect here is that the table columns need to be in the
// same order as the columns will later appear in the tuple. As the tuple
// is unnamed, matching is done on positions.
std::shared_ptr<Table> table = ..

arrow::compute::ExecContext ctx;
arrow::compute::CastOptions cast_options;

// The range needs to be pre-allocated to the respective amount of rows.
// This allows us to pass in an arbitrary range object, not only
// `std::vector`.
std::vector<std::tuple<double, std::string>> rows(2);
if (!arrow::stl::TupleRangeFromTable(*table, cast_options, &ctx, &rows).ok()) {
  // Error handling code should go here.
}

Arrow itself already supports some C(++) data types for this conversion. If you want to support additional data types, you need to implement a specialization of arrow::stl::ConversionTraits<T> and the more general arrow::CTypeTraits<T>.

namespace arrow {

template<>
struct CTypeTraits<boost::posix_time::ptime> {
  using ArrowType = ::arrow::TimestampType;

  static std::shared_ptr<::arrow::DataType> type_singleton() {
    return ::arrow::timestamp(::arrow::TimeUnit::MICRO);
  }
};

}

namespace arrow { namespace stl {

template <>
struct ConversionTraits<boost::posix_time::ptime> : public CTypeTraits<boost::posix_time::ptime> {
  constexpr static bool nullable = false;

  // This is the specialization to load a scalar value into an Arrow builder.
  static Status AppendRow(
        typename TypeTraits<TimestampType>::BuilderType& builder,
        boost::posix_time::ptime cell) {
    boost::posix_time::ptime const epoch({1970, 1, 1}, {0, 0, 0, 0});
    return builder.Append((cell - epoch).total_microseconds());
  }

  // Specify how we can fill the tuple from the values stored in the Arrow
  // array.
  static boost::posix_time::ptime GetEntry(
        const TimestampArray& array, size_t j) {
    return psapp::arrow::internal::timestamp_epoch
        + boost::posix_time::time_duration(0, 0, 0, array.Value(j));
  }
};

}}

Row to columnar conversion

Conversion of RecordBatch to Tensor instances

Conversion of range of std::tuple-like to Table instances#

Conversion of range of `std::tuple`-like to `Table` instances#