pysparklyr
Provides a ‘PySpark’ Back-End for the ‘sparklyr’ Package
| Function(s) | Description |
|---|---|
| connection_databricks_shinyapp() | A Shiny app that can be used to construct a spark_connect statement |
| deploy_databricks() | Deploys Databricks backed content to publishing server |
| install_pyspark() install_databricks() | Installs PySpark and Python dependencies |
| installed_components() | Lists installed Python libraries |
| ml_prepare_dataset() | Creates the ‘label’ and ‘features’ columns |
%>% |
Pipe operator |
| requirements_write() | Writes the ‘requirements.txt’ file, containing the needed Python libraries |
| spark_connect_service_start() spark_connect_service_stop() | Starts and stops Spark Connect locally |