Scala - How to avoid if/else condition for object factory

Question

I am trying to solve the following problem. Problem

• I have to write a copy method to copy from one file system to other. (i.e. local to hdfs, s3 to s3, and few more later).
• This file system (local, s3, hdfs) could increase in future and so does actions (copy, move, delete)
• some of the actions are cross FileSystem (i.e. Copy, Move) some aren't i.e. (Delete, List, Find)
• I have a property file which contains source and destination location, along with some other fields (i.e count) which help me understand where to copy file.

I tried to solve the problem using Factory in following way, however it's still not able to solve Cross Platform action problem. And code doesn't look elegant.

Implementation

abstract class FileSystem(propFileURI: String) {
  def moveFile(): Unit
}

object FileSystem {

  private class HDFSystem(propFileURI: String) extends FileSystem(propFileURI) {
    override def moveFile(): Unit = {
      println(" HDFS  move file")
    }
  }

  private class S3System(propFileURI: String) extends FileSystem(propFileURI) {
    override def moveFile(): Unit = {
      println("S3 Move File ")
    }
  }

  def apply(propFileURI: String): Option[FileSystem] = {
    val properties: Properties = new Properties()

    val source = Source.fromFile( System.getProperty("user.dir")+"\\src\\main\\resources\\"+propFileURI).reader
    properties.load(source)
    val srcPath = properties.getProperty("srcPath")
    val destPath = properties.getProperty("destPath")

    if (destPath.contains("hdfs")){
       Some(new HDFSystem(propFileURI))
    }
    if (srcPath.contains("s3") && destPath.contains("s3")){
      Some(new S3System(propFileURI))
    }else{
       None
    }

  }

  def main(args: Array[String]): Unit = {
    val obj = FileSystem("test.properties")
    obj match {
      case Some(test) => test.moveFile()
      case None => println("None returned")
    }
  }
}

Question:

1. the current implementation of moveFile only handles s3->s3 and hdfs->hdfs. how to implement same method for local->hdfs and local->s3

2. How to move HDFSystem and S3System to separate file?

3. How to avoid if/else in apply method?

Answer 1

You can replace if-else with pattern matching. But, t's not about just if-else statement, right?. So it can be written as the following:

sealed abstract class FileSystem(propFileURI: String) {
  def moveFile(): Unit
}

case class HDFSystem(propFileURI: String) extends FileSystem(propFileURI) {
  override def moveFile(): Unit =
    println(" HDFS  move file")
}

case class S3System(propFileURI: String) extends FileSystem(propFileURI) {
  override def moveFile(): Unit =
    println("S3 Move File ")
}
case class MoveFile(hdfs: Option[HDFSystem] = None, s3: Option[S3System] = None)

object FileSystem {

  def apply(propFileURI: String): MoveFile = {
    val properties: Properties = new Properties()
    val source = Source.fromFile(System.getProperty("user.dir") + "\\src\\main\\resources\\" + propFileURI).reader
    properties.load(source)

    val srcPath = Option(properties.getProperty("srcPath")).fold(false)(_.contains("hdfs"))
    val destPath = Option(properties.getProperty("destPath")).fold(false)(_.contains("s3"))

    (destPath, srcPath) match {
      case (true, true) =>
        MoveFile(
          hdfs = Option(HDFSystem(propFileURI)),
          s3 = Option(S3System(propFileURI))
        )
      case (false, true) =>
        MoveFile(s3 = Option(S3System(propFileURI)))
      case (true, false) =>
        MoveFile(hdfs = Option(HDFSystem(propFileURI)))
      case _ =>
        MoveFile()
    }
  }
}

object TestObj {

  def main(args: Array[String]): Unit = {
    val obj = FileSystem("test.properties")
    (obj.hdfs, obj.s3) match {
      case (Some(hdfs), _) => hdfs.moveFile()
      case (_, Some(s3)) => s3.moveFile()
      case (_, _) => println("None returned")
    }
  }
}

Honestly I don't like above implementation and slightly modified for below use case. You can use them as an ADT without MoveFile wrapper:

def testMethod(fs: FileSystem): Unit = {
  fs.moveFile()
}

def main(args: Array[String]): Unit = {
// You can have a logic here for which way to go
  val obj = S3System("test.properties")
  testMethod(obj)
  val obj1 = HDFSystem("test.properties")
  testMethod(obj1)
}

In this case you can remove FileSystem object entirely. If you want to have some path checker you can have them inside each sub-types. HdfsSystem and S3Sytem should implement moveFile method

Answer 2

I would separate the creation of FileSystem which can move files internally in the same file system, and one that can do so between file systems.

For the simple file system implementation, I'd create a sealed trait with the various systems:

sealed trait FileSystem {
  def moveFile(path: String)
}
object FileSystem {
  class HDFSSystem extends FileSystem {
    override def moveFile(path: String): Unit = ???
  }

  class S3FileSystem extends FileSystem {
    override def moveFile(path: String): Unit = ???
  }

  def apply(path: String): Either[Throwable, FileSystem] = {
    val properties: Properties = new Properties()
    properties.load(
      Source
        .fromFile(s"${System.getProperty("user.dir")}\\src\\main\\resources\\$path")
        .reader
    )

    val srcPath = properties.getProperty("srcPath")
    val destPath = properties.getProperty("destPath")

    if (!srcPath.equalsIgnoreCase(destPath))
      Left(new Exception("Source and dest paths should be equal"))
    else {
      path.toLowerCase() match {
        case s3 if s3.startsWith("s3")       => Right(new S3FileSystem)
        case hdfs if hdfs.startsWith("hdfs") => Right(new HDFSSystem)
        case _                               => Left(new Exception(s"Received unknown file system prefix: $path"))
      }
    }
  }
}

For the multiple file system transfer, I'd use a different abstraction that wraps the FileSystem. Here's a sketch:

abstract class MultiFileSystemTransfer[A <: FileSystem, B <: FileSystem](
  val srcSystem: A,
  val dstSystem: B
) {
  def moveFile(srcPath: String, dstPath: String): Unit
}

object MultiFileSystemTransfer {
  class S3ToS3FileSystemTransfer private
      extends MultiFileSystemTransfer[FileSystem.S3FileSystem, FileSystem.S3FileSystem] {
    override def moveFile(srcPath: String, dstPath: String): Unit = ???
  }
}

We can further improve the paths are actually originating from the underlying file systems supplied by using Type Members:

sealed trait FileSystem {
  type Path
  def moveFile(path: Path)
}
object FileSystem {
  class HDFSSystem extends FileSystem {
    type Path = String
    override def moveFile(path: Path): Unit = ???
  }
}

abstract class MultiFileSystemTransfer[A <: FileSystem, B <: FileSystem](
  val srcSystem: A,
  val dstSystem: B
) {
  def moveFile(srcPath: srcSystem.Path, dstPath: dstSystem.Path): Unit
}

object MultiFileSystemTransfer {
  class S3ToS3FileSystemTransfer(srcPath: FileSystem.S3FileSystem, dstPath: FileSystem.S3FileSystem)
      extends MultiFileSystemTransfer[FileSystem.S3FileSystem, FileSystem.S3FileSystem](
        srcPath,
        dstPath
      ) {
    override def moveFile(srcPath: srcSystem.Path, dstPath: dstSystem.Path): Unit = ???
  }
}