Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to build Apache Spark using Gradle?

I am trying to build an Apache Spark application in Java in Eclipse. I am using Gradle as my build management system. What should I write in my build.gradle and which gradle commands should I use to do the same thing which this Maven POM and terminal command does?

<project>
  <groupId>edu.berkeley</groupId>
  <artifactId>simple-project</artifactId>
  <modelVersion>4.0.0</modelVersion>
  <name>Simple Project</name>
  <packaging>jar</packaging>
  <version>1.0</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_2.10</artifactId>
      <version>1.6.1</version>
    </dependency>
  </dependencies>
</project>

and

mvn package

$ YOUR_SPARK_HOME/bin/spark-submit \
--class "SimpleApp" \
--master local[4] \
target/simple-project-1.0.jar
like image 557
khateeb Avatar asked Sep 02 '25 05:09

khateeb


1 Answers

Try this. If it is a Java project, you can remove the first line:

apply plugin: 'scala'

buildscript {
    repositories {
        mavenCentral()
    }
}

repositories {
    mavenLocal()
    mavenCentral()
}

dependencies{
    compile "org.apache.spark:spark-core_2.10:1.6.1"
}
like image 134
Daniel Zolnai Avatar answered Sep 04 '25 18:09

Daniel Zolnai