Maven
Maven holds its configuration in files named pom.xml. It supports multimodule projects easily, while for sbt, there needs to be some extra work done. In Maven, each module simply has its own child pom.xml file.
To download Maven, go to https://maven.apache.org/download.cgi.
The following screenshot shows the structure of a skeleton Maven project:
The main pom.xml file is much longer than the preceding SBT solution. Let's have a look at its parts separately.
There is usually some metadata about the project and different properties that can be used in the POM files in the beginning:
<modelVersion>4.0.0</modelVersion>
<groupId>com.ivan.nikolov</groupId>
<artifactId>skeleton-mvn</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<scala.version>2.12.4</scala.version>
<scalatest.version>3.0.4</scalatest.version>
<spark.version>2.2.0</spark.version>
</properties>
Then, there are the dependencies:
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.datastax.spark</groupId>
<artifactId>spark-cassandra-connector_2.11</artifactId>
<version>2.0.5</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.12</artifactId>
<version>${scalatest.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
Finally, there are the build definitions. Here, we can use various plugins to do different things with our project and give hints to the compiler. The build definitions are enclosed in the <build> tags.
First, we specify some resources:
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<resources>
<resource>
<directory>${basedir}/src/main/resources</directory>
</resource>
</resources>
The first plugin we have used is scala-maven-plugin, which is used when working with Scala and Maven:
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.3.1</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
Another plugin we use is maven-assembly-plugin, which is used for building the fat JAR of the application:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
The complete pom.xml file is equivalent to the preceding sbt files that we presented.
As before, the Spark and Datastax dependencies are here just for illustration purposes.
If you look into the dependencies in more depth, you will see that we have imported junit, which is a Java testing framework. At first glance, someone might think that we don't actually need it. However, there is a catch. A quick Google search about how to run Scalatest unit tests with Maven would point to resources recommending the use of scalatest-maven-plugin. If we followed those instructions and tried running some tests from the command line, we would get a strange error. This is due to the fact that we used Scala 2.12 and the scalatest-maven-plugin at its current version is not binary compatible with this version of the language.
Like many things in software engineering, we have to find workarounds. Here, we could do two things:
- Use an older version of Scala.
- Force Maven to run our tests.
Of course, the second option is the more desirable. This means that the only thing we need to do in each Scalatest we write is to add the following annotation to each test class: @RunWith(classOf[JUnitRunner]) and make sure our test classes contain the word Test in their name.
Similarly to SBT, you can use Maven from the command line. Some of the commands you might find most useful with the example projects in this book are shown in the next tip.
- mvn clean test: This runs the application unit tests
- mvn clean compile: This compiles the application
- mvn clean package: This creates an assembly of the application (a fat JAR) that can be used to run as any other Java JAR