pyarrow.concat_tables#

pyarrow.concat_tables(tables, MemoryPool memory_pool=None, str promote_options='none', **kwargs)#

Concatenate pyarrow.Table objects.

If promote_options=none, a zero-copy concatenation will be performed. The schemas of all the Tables must be the same (except the metadata), otherwise an exception will be raised. The result Table will share the metadata with the first table.

If promote_options=default, any null type arrays will be casted to the type of other arrays in the column of the same name. If a table is missing a particular field, null values of the appropriate type will be generated to take the place of the missing field. The new schema will share the metadata with the first table. Each field in the new schema will share the metadata with the first table which has the field defined. Note that type promotions may involve additional allocations on the given memory_pool.

If promote_options=permissive, the behavior of default plus types will be promoted to the common denominator that fits all the fields.

Parameters:

tablesiterable of pyarrow.Table objects

Pyarrow tables to concatenate into a single Table.

memory_poolMemoryPool, default None

For memory allocations, if required, otherwise use default pool.

promote_optionsstr, default none

Accepts strings none, default and permissive.

**kwargsdict, optional

Examples

>>> import pyarrow as pa
>>> t1 = pa.table([
...     pa.array([2, 4, 5, 100]),
...     pa.array(["Flamingo", "Horse", "Brittle stars", "Centipede"])
...     ], names=['n_legs', 'animals'])
>>> t2 = pa.table([
...     pa.array([2, 4]),
...     pa.array(["Parrot", "Dog"])
...     ], names=['n_legs', 'animals'])
>>> pa.concat_tables([t1,t2])
pyarrow.Table
n_legs: int64
animals: string
----
n_legs: [[2,4,5,100],[2,4]]
animals: [["Flamingo","Horse","Brittle stars","Centipede"],["Parrot","Dog"]]