Packages
On CRAN
| Package | Description |
|---|---|
| pysparklyr | Integrate sparklyr with Spark Connect and Databricks Connect (v2) |
sparkxgb |
A sparklyr extension that provides an interface to XGBoost on Spark. |
graphframes |
Enables graph analysis in Spark via GraphFrames |
On Github
| Package | Description |
|---|---|
sparktf |
Allows writing of Spark DataFrame’s to TFRecord, the recommended format for persisting data to be used in training with TensorFlow |
mleap |
A sparklyr extension that provides an interface to MLeap, which allows us to take Spark pipelines to production. |