Compute the number of records within each partition of a Spark DataFrame

sdf_partition_sizes

Description

Compute the number of records within each partition of a Spark DataFrame

Usage

sdf_partition_sizes(x)

Arguments

Arguments	Description
x	A `spark_connection`, `ml_pipeline`, or a `tbl_spark`.

Examples




library(sparklyr)
sc <- spark_connect(master = "spark://HOST:PORT")
example_sdf <- sdf_len(sc, 100L, repartition = 10L)
example_sdf %>%
  sdf_partition_sizes() %>%
  print()