library(sparklyr)
sc <- spark_connect(master = "spark://HOST:PORT")
example_sdf <- sdf_len(sc, 100L, repartition = 10L)
example_sdf %>%
sdf_partition_sizes() %>%
print() Compute the number of records within each partition of a Spark DataFrame
R/sdf_interface.R
sdf_partition_sizes
Description
Compute the number of records within each partition of a Spark DataFrame
Usage
sdf_partition_sizes(x) Arguments
| Arguments | Description |
|---|---|
| x | A spark_connection, ml_pipeline, or a tbl_spark. |