pyspark.RDD.treeReduce#

RDD.treeReduce(f, depth=2)[source]#

Reduces the elements of this RDD in a multi-level tree pattern.

New in version 1.3.0.

Parameters

ffunction: the reduce function
depthint, optional, default 2: suggested depth of the tree (default: 2)

Returns

T: the aggregated result

See also

RDD.reduce()
RDD.aggregate()
RDD.treeAggregate()

Examples

>>> add = lambda x, y: x + y
>>> rdd = sc.parallelize([-5, -4, -3, -2, -1, 1, 2, 3, 4], 10)
>>> rdd.treeReduce(add)
-5
>>> rdd.treeReduce(add, 1)
-5
>>> rdd.treeReduce(add, 2)
-5
>>> rdd.treeReduce(add, 5)
-5
>>> rdd.treeReduce(add, 10)
-5

pyspark.RDD.treeAggregate

pyspark.RDD.union

Web Proxy Viewer | New URL | Original Page