Access the Spark API
spark-api
Description
Access the commonly-used Spark objects associated with a Spark instance. These objects provide access to different facets of the Spark API.
Usage
spark_context(sc)
java_context(sc)
hive_context(sc)
spark_session(sc)Arguments
| Arguments | Description |
|---|---|
| sc | A spark_connection. |
Details
The Scala API documentation is useful for discovering what methods are available for each of these objects. Use invoke to call methods on these objects.
Spark Context
The main entry point for Spark functionality. The Spark Context represents the connection to a Spark cluster, and can be used to create RDDs, accumulators and broadcast variables on that cluster.