Gabbler, a reactive chat app – part 1

In my previous blog post I introduced the new Akka I/O and already kind of demonstrated that you can use it – in combination with spray – to build HTTP servers with Akka. This post is the first in a series which build on these core concepts and dive into the domain of modern and reactive web applications: We’ll create Gabbler, a simple push-enabled chat application using the latest and greatest tools like Scala, Akka, spray and AngularJS.

Setting up the project

As usual, we are going to use the amazing sbt build tool. I have heard rumors that some don’t like it, but anyway, here’s some interesting stuff from our build.sbt:

scalaVersion := "2.10.2"

// TODO Remove as soon as spray-json is on Maven Cental
resolvers += "spray repo" at "http://repo.spray.io"

// TODO Remove as soon as spray is final and on Maven Cental
resolvers += "spray nightlies repo" at "http://nightlies.spray.io"

libraryDependencies ++= Dependencies.gabbler

And as I try to stick to the rules Josh laid down in his Effective sbt talk at Scala Days 2013 here is our project/Dependencies.scala (rule #2: track dependencies in one place):

object Library {

  // Versions
  val akkaVersion = "2.2.0-RC2"
  val logbackVersion = "1.0.13"
  val sprayVersion = "1.2-20130628" // Compatible with Akka 2.2.0-RC2
  val sprayJsonVersion = "1.2.5"

  // Libraries
  val akkaActor = "com.typesafe.akka" %% "akka-actor" % akkaVersion
  val akkaSlf4j = "com.typesafe.akka" %% "akka-slf4j" % akkaVersion
  val logbackClassic = "ch.qos.logback" % "logback-classic" % logbackVersion
  val sprayCan = "io.spray" % "spray-can" % sprayVersion
  val sprayRouting = "io.spray" % "spray-routing" % sprayVersion
  val sprayJson = "io.spray" %% "spray-json" % sprayJsonVersion
}

object Dependencies {

  import Library._

  val gabbler = List(
    akkaActor,
    akkaSlf4j,
    logbackClassic,
    sprayCan,
    sprayRouting,
    sprayJson
  )
}

As you can see, we are still on the latest released version 2.10.2 of Scala, although I’d love to go for Scala 2.11 milestone whatsoever. But we all know that it can be a little tricky to find our library dependencies published against these latest development versions. As in my previous post we are using Akka 2.2 and spray 1.2.

The user interface

Don’t expect anything fancy or stylish here! I’m not a web designer and never will be. Luckily there’s Twitter Bootstrap which prevents me from creating a complete mess. Enough talk, here’s the Gabbler UI:

Gabbler

As you can see, we are using the Bootstrap starter template. We add two columns: the left contains a textarea for new messages and a button to send them off, the right hosts the list of received messages.

We have to logon first. For sake of simplicity we can use any username-password combination with equal username and password. Then we can send messages which are pushed to all connected clients.

The AngularJS client

To build the client, we are using AngularJS, which seems to be one of the most promising JavaScript frameworks for reactive web applications out there. We won’t cover the details here, but just point out some of the most interesting aspects.

Let’s first take a look at the view portion of the client, which is a single HTML 5 page:

<!doctype html>
<html data-ng-app="gabbler">

…

<body data-ng-controller="GabblerCtrl">

…

<div class="container">
  <div class="row">

    <!--### The form for a new message ###-->
    <div class="span6">
      <h3>Gabble away</h3>
      <form name="form" novalidate>
        <fieldset>
          <div>
            <textarea id="text-input" rows="3" placeholder="Enter your message ..."
                      data-ng-model="message.text" required></textarea>
          </div>
          <div>
            <button type="submit" class="btn btn-primary" data-ng-click="sendMessage()"
                    data-ng-disabled="form.$invalid">Gabble away</button>
          </div>
        </fieldset>
      </form>
    </div>

    <!--### The list of messages ###-->
    <div class="span6 messages">
      <h3 data-ng-init="getMessages()">Gibble gabble</h3>
      <div data-ng-repeat="msg in messages">
        <div class="username">{{ msg.username }}</div>
        <div class="text">{{ msg.text }}</div>
      </div>
    </div>

  </div>
</div>

…

</body>
</html>

As you can see, this is a completely logic-free HTML 5 page which can be opened in any browser. All interactivity comes through AngularJS data-ng- directives, which usually show up as attributes of HTML elements, and AngularJS expressions, embedded in {{ }}, which are evaluated kind of like JavaScript expressions.

The controller part of the client, which gets connected to the view through the data-ng-controller directive of the body element, is a JavaScript object:

function GabblerCtrl($scope, Message) {

  $scope.messages = [];

  $scope.getMessages = function() {
    var messages = Message.query(function() {
      $scope.messages = messages.concat($scope.messages);
      $scope.getMessages();
    });
  }

  $scope.message = new Message({ "username": "" });

  $scope.sendMessage = function() {
    $scope.message.$save();
    $scope.message = new Message({ "username": "" });
  };
}

Our GabblerCtrl defines the $scope.getMessages function that queries the server for messages asynchronously and adds any new messages sent from the server to the messages array. Notice that the callback for the query calls $scope.getMessages again, hence the client is polling the server. We don’t use WebSocket here by purpose, because it will be illustrative to use Akka actors to implement long polling in a very elegant fashion on the server side. The controller also defines the $scope.sendMessage function which sends a new message to the server and reinitializes – i.e. empties – the local message which is stored in the $scope.message variable.

The model part of the client is made up from the $scope.messages array and the $scope.message object, which are synchronized with the view through AngularJS’s two-way data binding. Therefore the view gets updated anytime new messages are sent from the server, resulting in new entries in the list of messages in the right column. Also, when the user clicks the “Gabble away” button, reinitializing the $scope.message variable leads to blanking out the textarea for new messages in the left column.

The server skeleton

With the client in place, let’s start looking at the RESTful HTTP server. In this post we just cover a server skeleton with incomplete implementations for accetping GET and POST requests.

In my previous post I have shown how to build a HTTP server with Akka I/O and spray-can. This time we add another spray module – spray-routing – which provides a high-level, very flexible routing DSL for elegantly defining RESTful web services.

First we have to create an application:

object GabblerServiceApp extends App {

  val system = ActorSystem("gabbler-service-system")
  val interface = GabblerSettings(system).interface
  val port = GabblerSettings(system).port
  system.actorOf(GabblerService.props(interface, port), "gabbler-service")

  readLine(s"Hit ENTER to exit ...$newLine")
  system.shutdown()
}

As usual we create an ActorSystem and shut it down at the end, after hitting “ENTER”, which is a feature we probably don’t want to add to a mission critical high-available production system. We also get some configuration settings from the GabblerSettings Akka extension which we use to create the GabblerService top-level actor:

class GabblerService(interface: String, port: Int) extends HttpServiceActor with ActorLogging {

  import GabblerService._
  import SprayJsonSupport._
  import context.dispatcher

  IO(Http)(context.system) ! Http.Bind(self, interface, port)

  …

As you can see, GabblerService extends HttpServiceActor from spray-routing which makes the routing DSL available. When the GabblerService actor is created, we send a Http.Bind message to the I/O manager for HTTP to register itself as listener for HTTP connections.

Other than in my previous post we don’t handle the various HTTP messages natively, but instead use the routing DSL to define the actor’s behavior. The core concepts of the routing DSL are routes and directives.

Routes, which are simply functions of type RequestContext =&gt; Unit and can be nested or composed, are used to either complete a request or recejct or ignore it. Here is a very simple route that completes a request:

context => context.complete("Hello, world!")

Directives are small building blocks for arbitrarily complex routes which can have inner routes, transform or filter the incoming RequestContext or extract values form it. spray-routing already provides many useful directives, e.g. for completing a request, matching against the HTTP verb or extracting path elements:

path("order" / IntNumber)(id =>
  get(
    complete(
      "Received GET request for order " + id
    )
  ) ~
  put(
    complete(
      "Received PUT request for order " + id
    )
  )
)

Using this routing DSL we define the behavior of the GabblerService actor:

  …

  override def receive: Receive =
    runRoute(apiRoute ~ staticRoute)

  def apiRoute: Route =
  // format: OFF
    authenticate(BasicAuth(UsernameEqualsPasswordAuthenticator, "Gabbler"))(user =>
      path("api" / "messages")(
        get(context =>
          log.debug("User '{}' is asking for messages ...", user.username)
          // TODO Complete this user's request later, when messages are available!
        ) ~
        post(
          entity(as[Message]) { message =>
            complete {
              log.debug("User '{}' has posted '{}'", user.username, message.text)
              // TODO Dispatch message to all users!
              StatusCodes.NoContent
            }
          }
        )
      )
    )
  // format: ON

  def staticRoute: Route =
    path("")(getFromResource("web/index.html")) ~ getFromResourceDirectory("web")
}

As you can see we define the receive method in terms of the runRoute method which gets a route composed of apiRoute and staticRoute.

Let’s first look at the pretty straightforward staticRoute, which is a composite of two simple routes. The first is made up from the path direcitve matching the empty path and the getFromResource directive which completes a GET request with the given resource, which is web/index.html in our case. The second route is constructed with the getFromResourceDirectory direcitve and completes all GET requests with resources from the web directory. Essentially, staticRoute is used to serve Gabbler‘s one and only HTML file as well as all the CSS, JavaScript, image, etc. resources.

Now let’s take a look at apiRoute. It extracts the authenticated user and then matches requests with path api/messages. Within that, GET and POST requests are accepted.

We use basic authentication with a simple scheme: We accept any username-password combination with equal username and password:

object UsernameEqualsPasswordAuthenticator extends UserPassAuthenticator[BasicUserContext] {

  override def apply(userPass: Option[UserPass]): Future[Option[BasicUserContext]] = {
    val basicUserContext =
      userPass flatMap {
        case UserPass(username, password) if username == password => Some(BasicUserContext(username))
        case _ => None
      }
    Promise.successful(basicUserContext).future
  }
}

On receiving a GET request we simply log that the authenticated user is asking for messages. Notice that we dont’ complete the request here in order to support long polling, which will be introduced in the next post.

On receiving a POST request we extract the body into an instance of the Message case class by using the entity directive. This is supported by spray-json integration with the following Message companions:

object Message extends DefaultJsonProtocol {
  implicit val format = jsonFormat2(apply)
}

case class Message(username: String, text: String)

As you can see, marshalling and unmarshalling of case classes with spray is a breeze.

Within the entity directive we complete the request with status code 204 “No Content” after logging that the authenticated user has posted a message. Again, we save an essential aspect – sending the message to all connected users – for the next post.

Conclusion

We have introduced Gabbler, a modern and reactive push-enabled chat application. Actually we have only shown Gabbler‘s client portion, built with AngularJS and a server skeleton built with Scala, Akka I/O and spray. In the next posts we will complete the server step by step towards a fully functional solution.

The complete source code is available on GitHub, the current state under the tag step-01.

Advertisements
Gabbler, a reactive chat app – part 1

Introduction to Akka I/O

The recently released Akka 2.2.0-RC1 introduces the new akka.io package in the akka-actor module. According to the documentation, “the guiding design goal for this I/O implementation was to reach extreme scalability, make no compromises in providing an API correctly matching the underlying transport mechanism and to be fully event-driven, non-blocking and asynchronous.”

For some of you this completely actor-based I/O implementation might look familiar, because it has been developed as a joint effort by the spray and Akka teams, based on the “old” spray.io module.

In this blog post we will introduce the new Akka I/O by some very simple networking examples, namely a TCP and a HTTP server. In follow-up posts I will then show how Akka I/O and spray can be used to build a modern and reactive single-page web application.

Setting up the project

All right, to get stated we need an sbt project with the following dependencies:

  • Scala 2.10.x
  • akka-actor 2.2.0-RC2: provides Akka I/O
  • spray-can 1.2-20130628: only needed for HTTP

For convenience we also add some logging support, hence we need the following sbt settings in build.sbt:

scalaVersion := "2.10.2"

// TODO Remove as soon as spray is final and on Maven Cental
resolvers += "spray repo" at "http://nightlies.spray.io"

libraryDependencies ++= Dependencies.demoAkka

And this nice little class in project/Dependendies.scala:

object Library {

  // Versions
  val akkaVersion = "2.2.0-RC2"
  val logbackVersion = "1.0.13"
  val sprayVersion = "1.2-20130628" // Compatible with Akka 2.2.0-RC2

  // Libraries
  val akkaActor = "com.typesafe.akka" %% "akka-actor" % akkaVersion
  val akkaSlf4j = "com.typesafe.akka" %% "akka-slf4j" % akkaVersion
  val logbackClassic = "ch.qos.logback" % "logback-classic" % logbackVersion
  val sprayCan = "io.spray" % "spray-can" % sprayVersion
}

object Dependencies {

  import Library._

  val demoAkka = List(akkaActor, akkaSlf4j, logbackClassic, sprayCan)
}

TCP server

As we want to write a server, we need something we can run, i.e. an App:

object EchoServiceApp extends App {

  val system = ActorSystem("echo-service-system")
  val endpoint = new InetSocketAddress("localhost", 11111)
  system.actorOf(EchoService.props(endpoint), "echo-service")

  readLine(s"Hit ENTER to exit ...${System.getProperty("line.separator")}")
  system.shutdown()
}

We call it EchoServiceApp, because all this very simple server will be able to do is echoing back whatever we tell. As you can see, we create an actor system and an endpoint on localhost and port 11111.

Then we create the yet to be defined EchoService actor. Please notice, that ~~passing constructor arguments to an actor has changed in Akka 2.2; while the old style is still supported, it is now deprecated in order to avoid closing over non-serializable state~~ we are following a recommended implementation pattern and are using a props factory method for Props in the also to be defined companion object to avoid closing over non-serializable state: If you pass a closure to Props from within an actor, this gets included, which refers to an Actor, which is not serializable.

Finally we make sure we can shutdown our server via the ENTER key.

I/O manager and listener

The new Akka I/O implementation abstracts over transport protocols via so called “drivers” and introduces the concept of an “I/O manager” as the API for a particular driver. Let’s create the EchoService companion object defining the Props factory as well as the EchoService actor and create an I/O manager for TCP in its constructor:

object EchoService {
  def props(endpoint: InetSocketAddress): Props =
    Props(new EchoService(endpoint))
}

class EchoService(endpoint: InetSocketAddress) extends Actor with ActorLogging {
  IO(Tcp) ! Tcp.Bind(self, endpoint)
  …
}

We send a Tcp.Bind command to the I/O manager for TCP in order to register self – i.e. this very EchoService actor instance – as so called “listener” for TCP connections. Of course we need to listen for TCP connections now:

override def receive: Receive = {
  case Tcp.Connected(remote, _) =>
    log.debug("Remote address {} connected", remote)
    sender ! Tcp.Register(context.actorOf(EchoConnectionHandler.props(remote, sender)))
}

Listening to TCP connections means handling messages of type Tcp.Connected. All we do here is log a nice message at DEBUG level and then register a new instance of the yet to be defined EchoConnectionHandler actor as so called “handler” for the connection.

Hence we create a new handler actor for each connection. As actors are really cheap, this shouldn’t be an issue at all. But if you prefer, you can register the listener itself as handler.

As you can see, we pass the sender, i.e. the connection, to the EchoConnectionHandler. We’ll explain the reason soon …

Connection handler

Now let’s take a look at the final missing piece:

object EchoConnectionHandler {
  def props(remote: InetSocketAddress, connection: ActorRef): Props =
    Props(new EchoConnectionHandler(remote, connection))
}

class EchoConnectionHandler(remote: InetSocketAddress, connection: ActorRef) extends Actor with ActorLogging {

  // We need to know when the connection dies without sending a `Tcp.ConnectionClosed`
  context.watch(connection)

  def receive: Receive = {
    case Tcp.Received(data) =>
      val text = data.utf8String.trim
      log.debug("Received '{}' from remote address {}", text, remote)
      text match {
        case "close" => context.stop(self)
        case _       => sender ! Tcp.Write(data)
      }
    case _: Tcp.ConnectionClosed =>
      log.debug("Stopping, because connection for remote address {} closed", remote)
      context.stop(self)
    case Terminated(`connection`) =>
      log.debug("Stopping, because connection for remote address {} died", remote)
      context.stop(self)
  }
}

As you can see, the connection handler receives messages of type Tcp.Received which contain the payload as a ByteString which – according to the documentation – is an “efficient, immutable alternative to the traditional byte containers used for I/O on the JVM, such as Array[Byte] and ByteBuffer“.

Here we simply reply with a Tcp.Write containing the same payload or – in case the payload equals “close” – stop the connection handler.

In case a Tcp.ConnectionClosed message is received, we also stop the connection handler. And finally, we listen to Terminated messages for the underlying connection and stop ourselves, too, in this case. Now you know why we had to pass along the connection from the listener.

TCP server in action

Now that we have all the necessary pieces in place, let’s run the TCP server and connect via telnet. In one sbt session we run the EchoServerApp and in another terminal session we enter the following:

tmp$ telnet 127.0.0.1 11111
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello, TCP
Hello, TCP
Woot, it workz!
Woot, it workz!
close
Connection closed by foreign host.
tmp$

As you can see, our TCP echo server works as expected. We can either close the connection via the “close command” or even shut down the server via the ENTER key.

HTTP server

Now that we know how to create a TCP server, let’s look at HTTP which is probably ways more interesting for most of us. Actually there is not so much to say, because Akka I/O is designed in a way which pretty much abstracts away the transport protocol, which is pretty awesome.

The EchoServiceApp for HTTP looks exactly the same except for the constructor arguments of the HTTP listener. Hence let’s dive into the HTTP listener immediately:

class EchoService(host: String, port: Int) extends Actor with ActorLogging {

  import context.system

  IO(Http) ! Http.Bind(self, host, port)

  override def receive: Receive = {
    case Http.Connected(remote, _) =>
      log.debug("Remote address {} connected", remote)
      sender ! Http.Register(context.actorOf(EchoConnectionHandler.props(remote, sender)))
  }
}

As you can see, all we had to change here are the message types: Http.Bind and Http.Connected instead of the TCP likes.

And also the HTTP connection handler looks almost like the one for TCP:

class EchoConnectionHandler(remote: InetSocketAddress, connection: ActorRef) extends Actor with ActorLogging {

  // We need to know when the connection dies without sending a `Tcp.ConnectionClosed`
  context.watch(connection)

  def receive: Receive = {
    case HttpRequest(GET, uri, _, _, _) =>
      sender ! HttpResponse(entity = uri.path.toString)
    case _: Tcp.ConnectionClosed =>
      log.debug("Stopping, because connection for remote address {} closed", remote)
      context.stop(self)
    case Terminated(`connection`) =>
      log.debug("Stopping, because connection for remote address {} died", remote)
      context.stop(self)
  }
}

As you can see, the HTTP connection handler receives and GET messages and replies with the path component of the URI. The other received messages are the same like for the TCP connection handler.

HTTP server in action

All right, let’s finally take a look at the HTTP server. We run it in sbt and in another terminal session we enter the following:

tmp$ curl localhost:8080/Akka-IO-rocks
/Akka-IO-rockstmp$
tmp$

Conclusion

We have introduced the new Akka I/O by two very simple networking examples. Hopefully it has become obvious, that it is pretty easy to create an actor-based and hence completely asynchronous I/O server, most remarkably almost independent of the transport protocol.

The full source code is available on GitHub.

Introduction to Akka I/O

Implicits unchained – type-safe equality – part 3

In the last post we have developed a type-wise balanced type-safe equality operation:

scala> import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._
import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._

scala> Seq(1, 2, 3) === List(1, 2, 3)
res0: Boolean = true

scala> List(1, 2, 3) === Seq(1, 2, 3)
res1: Boolean = true

scala> 123 === "a"
<console>:11: error: TypeWiseBalancedEquality requires Int and String to be in a subtype relationship!

Before moving on, let’s discuss two minor issues related to the design and the implementation of the current state of the solution.

First, one can easily create new type class instances for Equality that “override” the default which is based on natural equality, i.e. delegates to ==:

scala> import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._
import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._

scala> implicit val weirdIntEquality = new Equality[Int, Int] {
     |   def areEqual(n: Int, m: Int) = n != m
     | }
weirdIntEquality: name.heikoseeberger.demoequality.TypeWiseBalancedEquality.Equality[Int,Int] = $anon$1@6a5f7445

scala> 1 === 1
res0: Boolean = false

Due to the rules of implicit resolution, all locally defined or imported type class instances override the ones defined in the implicit scope (in the companion object of the type class instance). Therefore it’s easy to override the default.

There are many situations where we want this behavior, but I think that in our use case intuition commands that === – which almost looks like == – behaves like ==. Therefore we need a way to prevent others from creating Equality type class instances. This can easily be achieved by sealing Equality:

sealed trait Equality[L, R] { … }

The second issue is related to performance. Currently the Equality type class instances are provided by the polymorphic rightSubtypeOfLeftEquality and leftSubtypeOfRightEquality methods, which create new instances every time they get called. Therefore we add the AnyEquality singleton object and simply use it as the return value for rightSubtypeOfLeftEquality and leftSubtypeOfRightEquality:

implicit def rightSubtypeOfLeftEquality[L, R <: L]: Equality[L, R] =
  AnyEquality.asInstanceOf[Equality[L, R]]

implicit def leftSubtypeOfRightEquality[R, L <: R]: Equality[L, R] =
  AnyEquality.asInstanceOf[Equality[L, R]]

private object AnyEquality extends Equality[Any, Any] {
  override def areEqual(left: Any, right: Any): Boolean = left == right
}

The type cast looks ugly, but thanks to type erasure it doesn’t cause any problems.

All right, that’s our final solution for type-wise balanced type-safe equality. Here is the complete code:

object TypeWiseBalancedEquality {
  implicit class Equal[L](val left: L) extends AnyVal {
    def ===[R](right: R)(implicit equality: Equality[L, R]): Boolean =
      equality.areEqual(left, right)
  }
  @implicitNotFound("TypeWiseBalancedEquality requires ${L} and ${R} to be in a subtype relationship!")
  sealed trait Equality[L, R] {
    def areEqual(left: L, right: R): Boolean
  }
  object Equality extends LowPriorityEqualityImplicits {
    implicit def rightSubtypeOfLeftEquality[L, R <: L]: Equality[L, R] =
      AnyEquality.asInstanceOf[Equality[L, R]]
  }
  trait LowPriorityEqualityImplicits {
    implicit def leftSubtypeOfRightEquality[R, L <: R]: Equality[L, R] =
      AnyEquality.asInstanceOf[Equality[L, R]]
  }
  private object AnyEquality extends Equality[Any, Any] {
    override def areEqual(left: Any, right: Any): Boolean =
      left == right
  }
}

View-wise balanced type-safe equality

While this type-wise balanced type-safe equality operation works in most cases, there are still some glitches:

scala> 1 === 1L
<console>:11: error: TypeWiseBalancedEquality requires Int and Long to be in a subtype relationship!
scala> 1L === 1
<console>:11: error: TypeWiseBalancedEquality requires Long and Int to be in a subtype relationship!

The types in this example – Int and Long – are not in a subtype relationship and therefore our type-wise === doesn’t work. Well, while this is how we designed it, this behavior is kind of odd, because there is a certain relationship between Int and Long: a view-based one. That is, there is an implicit conversion from Int to Long which is the reason why we can assign Int values to Long variables:

scala> val l: Long = 1
l: Long = 1

While it is not a “must”, it’s reasonable to argue that type-safe equality should also work for types which are in a view-based relationship. So let’s add a view-based === to our solution!

Most of TypeWiseBalancedEquality can be copied over to ViewWiseBalancedEquality. The only difference is the type class instances. These have to be provided the following way:

implicit def rightToLeftEquality[L, R](implicit view: R => L): Equality[L, R] =
  new RightToLeftViewEquality(view)
implicit def leftToRightEquality[L, R](implicit view: L => R): Equality[L, R] =
  new LeftToRightViewEquality(view)
private class LeftToRightViewEquality[L, R](view: L => R) extends Equality[L, R] {
  override def areEqual(left: L, right: R): Boolean =
    view(left) == right
}
private class RightToLeftViewEquality[L, R](view: R => L) extends Equality[L, R] {
  override def areEqual(left: L, right: R): Boolean =
    left == view(right)
}

As you can see, we need to reqire implicit conversions from the left type to the right or vice versa to be in scope for the type class instances. With these changes we make the above example work:

scala> import name.heikoseeberger.demoequality.ViewWiseBalancedEquality._
import name.heikoseeberger.demoequality.ViewWiseBalancedEquality._

scala> 1 === 1L
res0: Boolean = true

scala> 1L === 1
res1: Boolean = true

OK, we’re done, that’s our solution for view-wise balanced type-safe equality. Here is the complete code:

object ViewWiseBalancedEquality {
  implicit class Equal[L](val left: L) extends AnyVal {
    def ===[R](right: R)(implicit equality: Equality[L, R]): Boolean =
      equality.areEqual(left, right)
  }
  @implicitNotFound("ViewWiseBalancedEquality requires ${L} and ${R} to be in an implicit conversion relationship, i.e. one can be viewed as the other!")
  sealed trait Equality[L, R] {
    def areEqual(left: L, right: R): Boolean
  }
  object Equality extends LowPriorityEqualityImplicits {
    implicit def rightToLeftEquality[L, R](implicit view: R => L): Equality[L, R] =
      new RightToLeftViewEquality(view)
  }
  trait LowPriorityEqualityImplicits {
    implicit def leftToRightEquality[L, R](implicit view: L => R): Equality[L, R] =
      new LeftToRightViewEquality(view)
  }
  private class LeftToRightViewEquality[L, R](view: L => R) extends Equality[L, R] {
    override def areEqual(left: L, right: R): Boolean =
      view(left) == right
  }
  private class RightToLeftViewEquality[L, R](view: R => L) extends Equality[L, R] {
    override def areEqual(left: L, right: R): Boolean =
      left == view(right)
  }
}

Conclusion

In this post we have improved the type-wise balanced type-safe equality and also added a view-based one. I think that type-safe equality is very important and therefore wonder whether the sample code which is available on GitHub should be “lifted” to something like a “util-equality” project. What do you think?

Implicits unchained – type-safe equality – part 3

Implicits unchained – type-safe equality – part 2

In the last post we have discussed the problems with untyped equality and shown how to provide a simple type-safe equality operation using implicit classes:

object SimpleEquality {
  implicit class Equal[A](val left: A) extends AnyVal {
    def ===(right: A): Boolean = left == right
  }
}

So why is this solution (too) simple? Let’s look at an example:

scala> import name.heikoseeberger.demoequality.SimpleEquality._
import name.heikoseeberger.demoequality.SimpleEquality._

scala> Seq(1, 2, 3) === List(1, 2, 3)
res0: Boolean = true

scala> List(1, 2, 3) === Seq(1, 2, 3)
<console>:11: error: type mismatch;
 found   : Seq[Int]
 required: List[Int]

As you can see, this simple type-safe equality is not type-wise balanced, i.e. it only works if the object on the right is a subtype – including the same type – of the object on the left. The other way round is not possible. Bummer!

The reason is pretty straightforward: When the left object is wrapped into Equal, the type argument A is inferred to the type of this left object. === by its signature requires the right object to be of that type and of course only subtypes, but no supertypes fulfill that constraint.

Type-wise balanced type-safe equality

Like this reason, the solution for this problem is also pretty straightforward: We must additionally allow for the type on the right to be a supertype of the type on the left. In other words, instead of a single constraint RightType subtypeOf LeftType like in the simple approach, we need to enforce either of the two constraints RightType subtypeOf LeftType and LeftType subtypeOf RightType.

Let’s first relax the above constraint by making === itself polymorphic:

object TypeWiseBalancedEquality {
  implicit class Equal[L](val left: L) extends AnyVal {
    def ===[R](right: R): Boolean = …
  }
}

All right, now we can use supertypes, but also arbitrary types, because we are not enforcing any constraints at all. In order to do so, let’s employ type classes. In Scala a type class is a polymorphic trait and type class instances are implicit values implementing that trait for particular types.

First we define the Equality type class representing equality of two arguments with arbitrary respective types:

@implicitNotFound("TypeWiseBalancedEquality requires ${L} and ${R} to be in a subtype relationship!")
trait Equality[L, R] {
  def areEqual(left: L, right: R): Boolean
}

As you can see, we have added a friendly error message if the compiler can’t find a type class instance. Then we add an implicit parameter of type Equality to === and implement === simply by delegating to this parameter:

implicit class Equal[L](val left: L) extends AnyVal {
  def ===[R](right: R)(implicit equality: Equality[L, R]): Boolean =
    equality.areEqual(left, right)
}

If we try to call === now, we’ll be out of luck, because the compiler can’t yet find a type class instance:

scala> import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._
import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._

scala> 1 === 1
<console>:11: error: TypeWiseBalancedEquality requires Int and Int to be in a subtype relationship!

Next we define a type class instance for Equality[L, R] for all L (left type) and R (right type) which fulfill RightType subtypeOf LeftType in the Equality companion object:

object Equality {
  implicit def rightSubtypeOfLeftEquality[L, R <: L]: Equality[L, R] =
    new Equality[L, R] {
      override def areEqual(left: L, right: R): Boolean = left == right
    }
}

Finally we have to define the type class instance for Equality[L, R] for all L (left type) and R (right type) which fulfill LeftType subtypeOf RightType. We can’t define that directly in the Equality companion object, too, because we would run into ambiguity issues if the two types on the left and right are the same. Therefore we move it to a trait we mix into the Equality companion object which results in lower precedence and thus avoids any ambiguities:

trait LowPriorityEqualityImplicits {
  implicit def leftSubtypeOfRightEquality[R, L <: R]: Equality[L, R] = …
}
object Equality extends LowPriorityEqualityImplicits { … }

If we look at the above example again, we see that our new type-safe equality operation is balanced:

scala> import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._
import name.heikoseeberger.demoequality.TypeWiseBalancedEquality._

scala> Seq(1, 2, 3) === List(1, 2, 3)
res0: Boolean = true

scala> List(1, 2, 3) === Seq(1, 2, 3)
res1: Boolean = true

scala> 123 === "a"
<console>:11: error: TypeWiseBalancedEquality requires Int and String to be in a subtype relationship!

Woot!

Here is the current state of our type-wise balanced solution:

object TypeWiseBalancedEquality {
  implicit class Equal[L](val left: L) extends AnyVal {
    def ===[R](right: R)(implicit equality: Equality[L, R]): Boolean =
      equality.areEqual(left, right)
  }
  @implicitNotFound("TypeWiseBalancedEquality requires ${L} and ${R} to be in a subtype relationship!")
  trait Equality[L, R] {
    def areEqual(left: L, right: R): Boolean
  }
  object Equality extends LowPriorityEqualityImplicits {
    implicit def rightSubtypeOfLeftEquality[L, R <: L]: Equality[L, R] =
      new Equality[L, R] {
        override def areEqual(left: L, right: R): Boolean = left == right
      }
  }
  trait LowPriorityEqualityImplicits {
    implicit def leftSubtypeOfRightEquality[R, L <: R]: Equality[L, R] =
      new Equality[L, R] {
        override def areEqual(left: L, right: R): Boolean = left == right
      }
  }
}

Conclusion

In this post we have moved from the simple and unbalanced type-safe equality operation from the previous post to a type-wise balanced one. In the next post we’ll look at some minor issues with the current design and implementation of this type-wise balanced solution and also extend it to types which are “compatible” via implicit conversions.

The full source code is available on GitHub.

Implicits unchained – type-safe equality – part 2

Implicits unchained – type-safe equality – part 1

Beside the fusion of object orientation and functional programming, I think that implicits – implicit classes, conversions, parameters and values – are Scala’s most intrinsic feature. They are a very powerful tool and therefore you should follow the Spider-Man principle: “with great power comes great responsibility”. But hey, they also say “no risk no fun”, so let’s fasten our seatbelts and start a discovery tour of implicitlandia.

Use case: type-safe equality

Our discovery tour will be centered around the use case of type-safe equality. In Java and – because of Java compatibility – also in Scala, equality is untyped:

abstract class Any {
  def equals(other: Any): Boolean = …
  …
}

As you can see, equals takes an argument of type Any. Therefore we can compare apples with pies, a Person with an Option[Person], etc. In most cases, this is not what we want, and this can lead to errors which are very hard to analyze. Instead, we’d like to have a type-safe equality operation like:

123 === 123 // true
"a" === "b" // false
123 === "a" // Should not compile!

First approach

All right, what we need is a === method that can be called on objects of some type and takes another object of that particular type instead of Any. In other words, we need a polymorphic method:

def ===[A](a: A): Boolean = …

But there is no === on Any or any of the other standard classes. And Scala doesn’t support extension methods, right? Well, not directly, but we can use implicits to achieve the same effect.

The basic idea is quite simple: Wrap an object that needs to be extended with one that provides the “extension method” and define an implicit conversion from the type that is to be extended to the wrapper. In our case the type to be extended is any type, so we have to provide a polymorphic conversion. Since Scala 2.10 these two steps can be unified by providing an implicit class:

object SimpleEquality {
  implicit class Equal[A](left: A) {
    def ===(right: A): Boolean = left == right
  }
}

As implicit classes are desugared into the wrapper class and the implicit conversion by the Scala compiler, they can’t be defined at the top-level. Therefore we define the implicit Equal class inside of the SimpleEquality singleton object. By importing SimpleEquality._ we bring the implicit conversion into scope and can make use of our nice type-safe equality operation:

scala> import name.heikoseeberger.demoequality.SimpleEquality._
import name.heikoseeberger.demoequality.SimpleEquality._

scala> 123 === 123
res0: Boolean = true

scala> "a" === "b"
res1: Boolean = false

scala> 123 === "a"
<console>:11: error: type mismatch;
 found   : String("a")
 required: Int

Nice, eh? Well, some of you might be concerned about the potential performance impact of creating wrapper objects at runtime. Fortunately Scala 2.10 offers value classes, which we can use to avoid this creation of the wrappers. All we have to do is extend from AnyVal and make the wrapped value a val. Then the Scala compiler will inline the “extension methods”. Here is the final state of our first and simple approach:

object SimpleEquality {
  implicit class Equal[A](val left: A) extends AnyVal {
    def ===(right: A): Boolean = left == right
  }
}

Conclusion

In this post we have discussed the problems with untyped equality and shown how to provide a simple type-safe equality operation using implicit classes. In the next post we’ll look at the drawbacks of our simple solution and provide a more sophisticated one which makes use of more implicit goodness.

The full source code is available on GitHub.

Implicits unchained – type-safe equality – part 1

SBT-Eclipse Integration

Yay, we finally have got a really good Eclipse plugin for Scala. Thanks to all the contributors for their great work (as a former Eclipse committer I know that it was hard work). But as lots of Scala projects use SBT as their build tool, a new need is arising, the need for an integration of SBT and Eclipse. For the last one and a half years or so I have been using SBT and IntelliJ IDEA, because the Scala plugin for IDEA is awesome and, most important, there is a really good SBT-IDEA integration. It is provided by the sbt-idea project which I take as an inspiring example for what I would like to achieve for SBT-Eclipse integration.

Therefore I started the sbteclipse project which aims at providing the same features as sbt-idea, i.e. the ability to create Eclipse project files (.project and .classpath) from an SBT project. As SBT will soon be on version 0.9 which differs a great lot from version 0.7 from a plugin developer’s perspective, I decided to develop sbteclipse as a plugin for SBT 0.9.

While there is still a very long way to go, I just released the very first version 0.1 of sbteclipse to the Typesafe repository which means that you can use it easily. Right now it will only create a single project with its source- and class-directories. Libraries and multi-module projects are not yet supported. And of course you will have to build SBT 0.9 on your own, because there are no binaries yet.

If you are interested, please take a look at the “Usage” section of the sbteclipse README. I would be glad to get your feedback, of course. Have fun!

SBT-Eclipse Integration

Applicatives are generalized functors

In my previous blog post about categories and functors I already threatened you with the possibility of a follow-up. Well, here we go. Actually I won’t talk about category theory this time, but about an important abstraction from the world of advanced (at least in my understanding) functional programming which turns out to be a close relative to the now well understood functor.

Functors

Just in case you don’t remember exactly what a functor looks like, here comes the definition:

trait GenericFunctor[->>[_, _], ->>>[_, _], F[_]] {

  def fmap[A, B](f: A ->> B): F[A] ->>> F[B]
}

trait Functor[F[_]] extends GenericFunctor[Function, Function, F] {

  final def fmap[A, B](as: F[A])(f: A => B): F[B] =
    fmap(f)(as)
}

For the sake of simplicity we will stick to the more specific Functor definition throughout the rest of this post. Such a functor is an endofunctor, because its source and target are the same (the category of Scala types and Scala functions). Maybe you remember that such a functor can be regarded as a provider of a computational context: The function f: A => B you give to fmap is lifted into the functor’s context which means that it is executed (maybe once, maybe several times or maybe even not at all) under the control of the functor.

As an example let’s look at how the OptionFunctor defined in the previous blog post is working: If we give fmap a Some the given function will be invoked, if we give it a None it won’t be invoked.

scala> fmap(Option(1)) { x => println("I was invoked!"); x + 1 }
I was invoked!
res0: Option[Int] = Some(2)

scala> fmap(None: Option[Int]) { x => println("I was invoked!"); x + 1 }
res1: Option[Int] = None

So far, so good. Using functors we can lift functions of arity-1 into a computational context.

Applicatives

But what if we have a function of higher arity? Can we still use a functor to lift a function of, let’s say, arity-2?

Let’s look at what happens if we call fmap to partially apply an arity-2 function to its first argument within the computational context of an Option:

scala> val f = (x: Int) => (y: Int) => x + y + 10
f: (Int) => (Int) => Int = <function1>

scala> fmap(Option(1))(f)
res0: Option[(Int) => Int] = Some()

What we get back is an Option[Int => Int], i.e. the “rest” of the partially applied function wrapped in an Option. Now we have a problem, because we cannot give this lifted function to another call of fmap.

scala> fmap(Option(2))(fmap(Option(1))(f))
:13: error: type mismatch;
 found   : Option[(Int) => Int]
 required: (Int) => ?
       fmap(Option(2))(fmap(Option(1))(f))

Of course we cannot, because fmap expects a pure function, not a lifted one. And that’s the moment when applicatives enter the stage. The idea is simple and follows intutively from what we have just seen: Instead of fmap taking a pure function, an Applicative defines the method apply taking a lifted function. And it defines the method pure to lift pure functions. Using these it is perfectly possible to partially apply an arity-n function to all of its arguments within a computational context. Before we look at our example from this new perspective, let’s code up our new abstraction:

trait Applicative[F[_]] extends Functor[F] {

  def pure[A](a: A): F[A]

  def apply[A, B](f: F[A => B]): F[A] => F[B]

  final def apply[A, B](fa: F[A])(f: F[A => B]): F[B] =
    apply(f)(fa)

  override def fmap[A, B](f: A => B): F[A] => F[B] =
    apply(pure(f))
}

As you can see, there is a strong relation between functors and applicatives: Each applicative is a functor and by one of the laws for applicatives the following has to hold true: fmap = apply ο pure. Well, this law is pretty intuitive, because it makes sure we can use an applicative as a functor, i.e. for a pure arity-1 function, and it will behave as expected.

In order to be able to work with applicatives we need some scaffolding and for our Option example we need an implementation of Applicative:

object Applicative {

  def pure[A, F[_]](a: A)(implicit applicative: Applicative[F]): F[A] =
    applicative pure a

  def apply[A, B, F[_]](fa: F[A])(f: F[A => B])(implicit applicative: Applicative[F]): F[B] =
    applicative.apply(fa)(f)

  implicit object OptionApplicative extends Applicative[Option] {

    override def pure[A](a: A): Option[A] =
      Option(a)

    override def apply[A, B](f: Option[A => B]): Option[A] => Option[B] =
      o => for { a <- o; p <- f } yield p(a)
    }
}

Now let’s look at our example from above with an arity-2 function. Using a functor we got stuck, but using an applicative we can make it all the way through the two arguments:

scala> apply(Option(1))(apply(Option(2))(pure(f)))
res0: Option[Int] = Some(13)

Ain’t that nice? You might answer, that this code looks cumbersome and is not necessary at all. Regarding the first argument please read on. Regarding the second: Yes, of course we don’t need an applicative for Option, because it already offers flatMap which does the job. But with this type class approach we can deal with any class, e.g. with Either from the Scala standard library or with classes from our own projects.

Conclusion

You have seen the basic principle of applicatives: We can apply functions of arbitrary arity (well, greater or even one) to its arguments within a computational context. As functors provide exactly this for arity-1, applicatives are generalized functors.

Thanks to Scala’s flexibility we can of course do much better than above. Using a little pimp my library and some operators we can get something that’s elegant and useful. Luckily the scalaz folks have already done this, so I will just show two ways of expressing the above example using this awesome library:

scala> Option(1) <*> (Option(2) <*> Option(f))
res0: Option[Int] = Some(13)

scala> (Option(1) <**> Option(2)) { _ + _ + 10 }
res1: Option[Int] = Some(13)

Now this is really nice, isn’t it? Especially the second one looks very concise and intuitive. And as I stated above we can use it for types that don’t bring helpful methods like flatMap. Let’s conclude with another example using Either which is a perfect candidate to be used for results that might fail with well defined errors:

scala> (1.right[String] <**> 2.right[String]) { _ + _ + 10 }     
res0: Either[String,Int] = Right(13)

scala> (1.right[String] <**> "Error".left[Int]) { _ + _ + 10 }
res1: Either[String,Int] = Left(Error)

If you are interested in the code examples shown above, you can find these here: https://github.com/weiglewilczek/fun

Applicatives are generalized functors