Access DBUtils from Scala app running under databricks-connect
After a lot of trial and error this is what worked for me.
SBT manifest:
name := "test-dbutils"
version := "0.1.0-SNAPSHOT"
scalaVersion := "2.12.15"
val sparkVersion = "3.1.3"
libraryDependencies += "org.apache.spark" %% "spark-sql" % sparkVersion
// https://mvnrepository.com/artifact/com.databricks/dbutils-api
libraryDependencies += "com.databricks" %% "dbutils-api" % "0.0.5"
(note the com.databricks... dependency!)
Then in Scala code this works:
val dbutils = com.databricks.service.DBUtils
val files = dbutils.fs.ls("/FileStore")
println(files)
Em, excuse me! Have Android 📱 and use Databricks?
You might be interested in my totally free (and ad-free) Pocket Bricks . You can get it from Google Play too:
To contact me, send an email anytime or leave a comment below.