Currently i’m using 3 parties to get the picture complete

1. Github - Online Git repositories with nice collaboration features, if you know it, you’ll be using it, if you don’t, have a look, you’ll never look back
2. Cloudbees – Free PaaS with Git Repository and Jenkins CI instance, you could also run your apps here, but no support for Scala, Lift at the time I looked at it
3. Cloudfoundry – Free for now  (Beta) PaaS from VMWare – Support Scala and Lift, which makes it easy for me.

Registering a app at cloudfoundry

Once you have an account and installed the command-line tools you can claim your little application space at the hosted cloudfoundry (or download and run a micro.cloudfoundry instance in your local Vmware system)

$ vmc target api.cloudfoundry.com
$ vmc login [email]
$ vmc push my-app --path target/war --url my-app.cloudfoundry.com --instances 2 --mem 512k --runtime java
$ vmc create-service mysql my-app-db my-app

What this means is, i’m pushing my war based maven build webapplication to the hosted cloudfoundry as two instances with each 512k memory and runtime java, I’m also binding it to a mysql service called my-app-db.
To see the result of my actions I can type vmc apps

$ vmc apps
+-------------+----+---------+---------------------------+-------------+
| Application | #  | Health  | URLS                      | Services    |
+-------------+----+---------+---------------------------+-------------+
| my-app      | 1  | RUNNING | my-app.cloudfoundry.com   | my-app-db   |
+-------------+----+---------+---------------------------+-------------+

The commandline tools also give me options to check logs and crashes, start and stop the application etc.

Database connectivity

As cloud applications/instances/services are usually not as easily identifiable with an ip-address, your application needs to find out about services (like a database) in a different manner.
I’m doing this by using the org.cloudfoundry.runtime library to create a mysql Service for me

1
2
3
4
5
6
7
8
9
10
11
12
13
14

  object CloudFoundryConnection extends ConnectionManager {
    def newConnection(name: ConnectionIdentifier): Box[Connection] = {
      try {
        import org.cloudfoundry.runtime.env._
        import org.cloudfoundry.runtime.service.relational._
        Full(new MysqlServiceCreator(new CloudEnvironment())
                        .createSingletonService().service.getConnection())
 
      } catch {
        case e : Exception => Empty
      }
    }
    def releaseConnection(conn: Connection) {conn.close}
  }

This object will return a connection if it is running in the cloud, otherwise I’ll fall back to the normal database connection of in my case an H2 database which I use for development
so I also changed the DB initialisation part of Lift in its Boot class.
The order of DB discovery is now 1) jndi connection, 2) cloudfoundry 3) connection in Properties file 4) H2 database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

if (!DB.jndiJdbcConnAvailable_?) {
      val connection = CloudFoundryConnection
      connection.newConnection(DefaultConnectionIdentifier) match {
        case (Full(connection) => {
          DB.defineConnectionManager(DefaultConnectionIdentifier, connection)
        }
        case Empty => {
          val vendor = new StandardDBVendor(
            Props.get("db.class") openOr "org.h2.Driver",
            Props.get("db.url") openOr "jdbc:h2:lift_proto.db;AUTO_SERVER=TRUE",
            Props.get("db.user"),
            Props.get("db.password")
          )
          LiftRules.unloadHooks.append(vendor.closeAllConnections_! _)
          DB.defineConnectionManager(DefaultConnectionIdentifier, vendor)
        }
      }
    }

Maven integration

Cloudfoundry also provides a maven plugin that allows most of the functionality to be run as part of a maven goal, which is ideal when you want to deploy from your Jenkins build.
Here are the changes to my pom.xml
Add the repository for the org.cloudfoundry.runtime dependency

<repository>
    <id>springsource-milestones</id>
    <name>SpringSource Milestones Proxy</name>
    <url>https://oss.sonatype.org/content/repositories/springsource-milestones</url>
</repository>

Add the dependency itself

<dependency>
    <groupId>org.cloudfoundry</groupId>
    <artifactId>cloudfoundry-runtime</artifactId>
    <version>0.6.1</version>
</dependency>

For the plugin add the plugin repository

<pluginRepository>
    <id>repository.springframework.maven.milestone</id>
    <name>Spring Framework Maven Milestone Repository</name>
    <url>http://maven.springframework.org/milestone</url>
</pluginRepository>

and the plugin itself

<plugin>
    <groupId>org.cloudfoundry</groupId>
    <artifactId>cf-maven-plugin</artifactId>
    <version>1.0.0.M1</version>
    <configuration>
        <server>mycloudfoundry-instance</server>
        <target>http://api.cloudfoundry.com</target>
        <url>medicate.cloudfoundry.com</url>
        <memory>512</memory>
        <instances>2</instances>
    </configuration>
</plugin>

Username and password information you can specify in the servers section of your .m2/settings.xml file

<server>
    <id>mycloudfoundry-instance</id>
    <username>email@address.com</username>
    <password>s3cr3t</password>
</server>

now you can update your deployed application by a simple

$ mvn cf:update

 Continous Integration on Cloudbees

First setup an account at cloudbees.com, create a repository and a Jenkins instance, and make sure cloudbees has your public key to allow git pushes.

Then setup a maven 2/3 build in Jenkins, setting the scm section to watch for changes from your cloudbees git repository.

Make sure your build name doesn’t have any spaces in it, cloudbees will choke on that, at least with the scala compiler.

As we need the login information from the .m2/settings.xml, you can upload this through webdav to repository-[username].forge.cloudbees.com/private directory.

And then in our jenkins build set /private/[username]/settings.xml in the alternative maven settings field (advanced button in the maven section)

I’ve set the maven goal to clean scala:doc cf:update meaning it will compile, test, create scala docs and deploy the application to cloudfoundry.

to finish it up you need to add a remote to cloudbees in your local git repository and push your changes. this then will trigger a build and deploy your app.

git remote add cloudbees ssh://git@git.cloudbees.com/[username]/my-app.git

To conclude, besides the 2 i’ve mentioned, there are a number of solutions out there, that all work in similar ways, but each still has it own requirements, If you want to run Java, Ruby, python, php, node.js applications, with Mysql, postgres, MongoDb, CouchDb, or RabbitMQ, you can get it to work on one or more of these providers.

Last but not least here is a picture from the actual application i’m building to visualize all I’ve talked about above:

But code with hardly any tests on them always makes me feel uncomfortable. so I decided to do some (almost integration) tests but running as part of the normal unit test suite
I’m doing this by placing the tags on a jsp, running this on a embedded Jetty Server and using Selenium Webdriver to test the resulting html.

First the base class, which takes care of starting/stopping the embedded Jetty Server and setting up the web driver.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

public abstract class BaseTest {
    private static int port = 9595;
    private static Server server = new Server(port);
 
    protected static String baseUrl = "http://localhost:" + port + "/";
    protected static WebDriver driver;
 
    @BeforeClass
    public static void setup() throws Exception {
 
        //configure jetty as an exploded war
        URL webAppUrl = BaseTest.class.getClassLoader().getResource("/");
        WebAppContext wac = new WebAppContext();
        wac.setContextPath("/");
        wac.setWar(webAppUrl.toExternalForm());
 
        server.setHandler(wac);
        server.setStopAtShutdown(true); //makes sure the sever is stopped even if the @AfterClass is never reached
        server.start();
 
        //HtmlUnit drive doesn't give a popup, the rest of the drivers do
        driver = new HtmlUnitDriver();
    }
 
    @AfterClass
    public static void teardown() throws Exception {
        driver.close();
        server.stop();
    }

In my project I have setup a resources directory that contains a WEB-INF/ with a web.xml containing my custom tag definitions and my tld file as you would with a normal web project
Webdriver comes with a set of Drivers for different browsers (Firefox, chrome, IE, iPhone, Android, etc) for testing browser compatibility, I don’t care about that, the tags I’m testing do not output any html/css that might require this.
The HtmlUnit driver is internal and doesn’t popup a browser window so ideal for my purpose.

Writing a test now becomes very easy:

In this case I have a custom tag that when invoked simply outputs “Hello World”

my test jsp:

1
2
3
4
5
6
7
8
9
10

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"  isELIgnored="false" %>
<%@ taglib uri="/customtaglib" prefix="custom" %>
<html>
  <head>
    <title>Hello World</title>
  </head>        
  <body>
    <p><custom:helloworld /></p>
  </body>
</html>

The Test class:

1
2
3
4
5
6
7
8
9
10

public class HelloWorldTest extends BaseTest {
 
    @Test
    public void testHelloWorldTag() {
        driver.get(baseUrl + "helloworld.jsp");
        Assert.assertEquals("Hello World", driver.getTitle());
 
        WebElement element = driver.findElement(By.cssSelector("body p"));
        Assert.assertEquals("Hello World", element.getText());
    }

Testing the resulting html is made very easy, by having the WebElement findElement() and List findElements() methods.
and then setting predicates with the By class. for instance By.cssSelector(), By.tagName() etc.
It is also possible to test click throughs and form submits, by calling click() or submit() on WebElements.

Added bonus is that although the tag code is running in the embedded jetty server and not in the unit tests itself it’s coverage is measured by our code coverage tools

One thing to look out for is for the embedded Jetty to work with JSP’s you need to add jetty’s version of the jsp spec to your project not the javax.servlet.* ones.
my maven dependencies look likes this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

    <dependencies>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.12.0</version>
        </dependency>
        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jetty</artifactId>
            <version>6.1.22</version>
        </dependency>
        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jsp-api-2.1</artifactId>
            <version>6.1.14</version>
        </dependency>
         <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jsp-2.1</artifactId>
            <version>6.1.14</version>
        </dependency>
        <dependency>
             <groupId>junit</groupId>
             <artifactId>junit</artifactId>
             <version>4.7</version>
             <scope>test</scope>
        </dependency>
    </dependencies>

According to Wikipedia:

In computer science, the Actor model is a mathematical model of concurrent computation that treats “actors” as the universal primitives of concurrent digital computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received.

The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.
An actor is a computational entity that, in response to a message it receives, can concurrently:

• send a finite number of messages to other actors;
• create a finite number of new actors;
• designate the behavior to be used for the next message it receives.

There is no assumed sequence to the above actions and they could be carried out in parallel.

A number of languages have the actor model implemented in their core (Io, Erlang, Scala), for most other languages it’s available as a library (Akka for Java, Retlang for .Net, Haskell-Actor for Haskell, Parely and Pykka for Python)

In this example I’ll be using the Akka framework for Java and Scala, which brings actors to the enterprise level by adding for instance fault tolerance (supervisors, let it crash semantics) and remote actors, If you want to play around with this yourself the best place to start is to use the typesafe stack which includes Akka and Scala

I’m using a typical map/reduce use case, namely counting words in a document
I do this by having a ‘master’ actor that divides the work in lines, have a number of ‘worker’ actors that count the words for a single line, replying the result to the ‘master’ to aggregate

Messages that actors send to each other are simply objects (case classes, the receive method typically uses pattern matching to determine which message has arrived), multiple messages to an actor will be queued, and handled sequentially

So let’s create the message objects

30
31
32
33
34

  // Messages
  sealed trait MapReduceMessage
  case class CountDocument(document: Iterator[String]) extends MapReduceMessage
  case class CountLine(line: String) extends MapReduceMessage
  case class Result(values: Map[String, Int]) extends MapReduceMessage

The CountDocument which will be send to the master, the CountLine to each worker, and each worker will return a Result when done

as each receive method returns a Partial Function[T] which allows for several classes to handle all the messages, the sealed trait super type will actually generate a warning when not all messages are handled. neat trick that.
It will also make it impossible to add messages outside of your class which could be something you want. or not.

Now on to the Master

57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

// Master Actor, creates Worker Actors, distributes work and gathers results
  class Master(nrOfWorkers: Int, latch:CountDownLatch) extends Actor {
 
    val workers = Vector.fill(nrOfWorkers)(actorOf[CountLineWorker].start());
    val router = Routing.loadBalancerActor(CyclicIterator(workers)).start();
 
    val resultMap = new HashMap[String, Int]();
 
    var start : Long = _
    var count : Long = 0
 
    def receive = {
      case CountDocument(lines : Iterator[String]) =>
 
        lines.foreach(line =>
              if (!line.isEmpty) {
                count = count+1;
                router ! CountLine(line)
              })
 
        //shutdown actors
        router ! Broadcast(PoisonPill)
        router ! PoisonPill
 
      case Result(values: Map[String, Int]) =>
 
        for ((key, value) <- values) {
          resultMap.put(key, resultMap.getOrElse(key, 0)+value)
        }
        count = count - 1;
        if (count <= 0) self.stop()
    }
 
    override def preStart() {
      start = System.currentTimeMillis
    }
 
    override def postStop() {
      val end = System.currentTimeMillis-start
      println("Result after %s ms :".format(end))
      for((key, value) <- resultMap.toList.sortBy(_._2).reverse) {
        println("%s: %s".format(value, key))
      }
      latch.countDown()
    }
  }

An actor class needs to implement the Actor interface which has one method called receive
The Routing.LoadbalancingActor is a built in actor that forwards messages send to it according to the scheme passed to it, in our case in a round robin style.
When receiving a CountDocument messages it will send each line to a worker, it will then tell the workers and the router to shutdown by sending them a PosionPill message.
When receiving a Result message it will aggregate the result and after receiving all results it will stop itself.
we override the preStart() and postStop() methods to allow us to calculate spend time and reporting the result

the ! in

router ! CountLine(line)

is a so called bang method, meaning fire and forget, there are also methods that will reply eventually (!!) and reply eventually with a Future (!!!)

The Worker looks as follows:

36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

//Actor that counts the words for a single line
  class CountLineWorker extends Actor {
 
    def receive = {
       case CountLine(line) =>
         self reply Result(countWords(line))
    }
 
    def countWords(line: String):Map[String, Int] = {
      val result = new HashMap[String, Int]
 
      "[^A-Za-z0-9\u0020]".r.replaceAllIn(line, "")
            .split(" ")
            .foreach(word => {
              result.put(word, result.getOrElse(word, 0)+1)
            })
 
      result
    }
  }

Should speak for itself, sanitizes and then count the words in a line, and replies to the sending actor with the result.

now wrap it in a main class that loads the document and sends it off to the master

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

object MapReduce extends App {
 
  countWordsInFile("src/main/resources/test.txt", 6);
 
  def countWordsInFile(fileName: String, nrOfWorkers: Int) {
 
    val source = scala.io.Source.fromFile(fileName)
    val document = source.getLines()
 
    val latch = new CountDownLatch(1)
 
    val master = actorOf(new Master(nrOfWorkers, latch)).start()
 
    master ! CountDocument(document)
 
    latch.await()
 
    source.close()
  }
}

This will load the document and sends it off to the master also instructing it to create 6 workers, as actors are called asynchronously, the Countdown latch is there to prevent the program from stopping before all the work is done.

The full source code is available as a gist on github

A couple of other interesting Akka features are

Become:
you can tell each actor to become another, this allows for hot code swapping, which is essential in fault-tolerant, always up systems.

  def receive = {
     case updateYourSelf(newMe:ActorRef) => {
        become newMe
     }
  }

Typed Actors (example blatantly stolen from Akka documentation):
extending any POJO with TypedActor will turn them into actors
your class need to have a separate interface/implementation for this to work

trait RegistrationService {
  def register(user: User, cred: Credentials): Unit
  def getUserFor(username: String): User
}
public class RegistrationServiceImpl extends TypedActor with RegistrationService {
  def register(user: User, cred: Credentials): Unit = {
    ... // register user
  }
 
  def getUserFor(username: String): User = {
    ... // fetch user by username
   user
  }
}
 
val service = TypedActor.newInstance(classOf[RegistrationService], classOf[RegistrationServiceImpl], 1000)
//last parameter is timeout for Futures

Remote Actors (example blatantly stolen from the Akka homepage):

// server code
class HelloWorldActor extends Actor {
  def receive = {
    case msg => self reply (msg + " World")
  }
}
remote.start("localhost", 9999).register(
  "hello-service", actorOf[HelloWorldActor])
 
// client code
val actor = remote.actorFor(
  "hello-service", "localhost", 9999)
val result = actor !! "Hello"

When starting the mobile application and entered the correct login information, a request is being made to get the list of authors.
Clicking on an author will get a list of books from that author that the user owns, clicking on a book shows the details.

So the first query that needs implementing is: Give me a list of Authors for which the current user owned books

Given the Following Mapper Data Model:
Author <- (1) -> Book <- (2) -> User
(1) An Author can have more books while books can be written by multiple authors (N-N)
(2) A user can own multiple books, while a book can have multiple owners (N-N)

note: When successfully authenticating in a user his API key is returned which is used in all other requests

Below we will see the code that executes this query
what happens line by line is this:

• Line4: we get a list of books for the current user
• Line5: we map the colllection to one that contains the authors for each book, the result is a list of lists of authors List[List[Author]]
• Line6: foldLeft applies the method to each element in the list, from left to right, as the method ++ means adding two list’s together, we’ll end up with one list containing all authors
• Line7: distinct removes any duplicates in there
• Line8: We sort the list on the author’s lastName property
• Line9: we map it to a List of Xml fragments

Scala’s inline XML support allows us to simple put the Authors tag around it.
The second case does the same but gives a JSON response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

serve {
  case "api" :: key :: "user" :: "authors" :: "all" :: _ XmlGet _ => {
    <Authors> {
      Book.findAll(In(Book.id ,BookUser.book, By(BookUser.user, getUserIdFromKey(key))))
        .map(book => { book.authors.get })
        .foldLeft(List[Author]())(_ ++ _)
        .distinct
        .sortWith((a1:Author, a2:Author) => a1.lastName.is < a2.lastName.is)
        .map(author => author.toXml)
      }
    </Authors>
  }
  case "api" :: key :: "user" :: "authors" :: "all" :: _ JsonGet _ => {
    JsonWrapper ("Authors", {
      Book.findAll(In(Book.id ,BookUser.book, By(BookUser.user, getUserIdFromKey(key))))
        .map(book => { book.authors.get })
        .foldLeft(List[Author]())(_ ++ _)
        .distinct
        .sortWith((a1:Author, a2:Author) => a1.lastName.is < a2.lastName.is)
        .map(author => author.toJson)
      })
  }
}
 
//helper method
def getUserIdFromKey(key:String):Long = {
  User.find(By(User.key, key) match {
    case Full(user) => user.id
    case (_) => 0 
  }
}

Lets say we want to get a list of all books in the system when we access http://(server)/api/books/all

First we need to tell Lift we have a class that responds to requests in Boot.scala:

LiftRules.dispatch.append(BookRestApi)

then the class itself: BookRestApi.scala:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

object BookRestApi extends RestHelper  {
  serve {
    // list of books
    case "api" :: "books" :: "all" :: _ XmlGet _=>
      {
         <Books>
          {
            Book.findAll.map(book => book.toXml)
          }
         </Books>
      }
    case "api" :: "books" :: "all" :: _ JsonGet _=>
      {
        JsonWrapper("books", Book.findAll.map(book => book.toJS))
      }
  }
}

The RestHelper trait parses the Url and makes the relevant parts available for Scala’s pattern matching capabilities.

for the first case statement:

• the api :: books : all List will match if the url points to /api/books/all
• the XmlGet method will match if the client has indicated he can accept xml by setting an accept header or by ending the url with a .xml suffix

Scala’s inline XML capabilities make it very easy, to envelop the result in a <Books> element.

the second statement does the same but then when the client requests a json response.

For the Json part I used a JsonWrapper helper method because the inline code looked totally unreadable, this still isn’t very readable but at least it has a descriptive method name now

1
2
3

  def JsonWrapper(name : String, content : JValue) : JValue = {
    (name -> content)
  }

If you would call this method with the name “Books” and the content {“foo” : “bar” } it will output :

{"books" : 
  {"foo" : "bar"}
}

This shows both a positive and negative aspect to the Scala language, it makes for a very flexible and fast development, but loses a lot in readability, which makes maintenance more difficult.

Now if would want to get a single book when I specify the id of that book, I’d add the following:

1
2
3
4
5
6
7

case "api" :: "books" :: AsLong(id) :: _ XmlGet _=>
        {
          Book.findByKey(id) match {
            case Full(book) => {book.toXml}
            case (_) => NotFoundResponse("Book with id " + id + " not found\r\n")
          }
        }

This pattern will only match on /api/books/{id} and only if that Id can be converted to a Long, in which case it will try to look up a book with that id
if the book isn’t found (the Box returned from findByKey isn’t Full) it will give the text back

In the next article I’ll talk about PUT requests and authentication so we can add books through the rest interface

As an admin I want to be able to add/edit news articles and choose whether or not they appear on the homepage

For this we need to store the articles in a database, A template and Snippet to show them and some way of managing them.
Lift has 2 built-in persistence frameworks, Mapper and Record, but also JPA is fully supported.
As Mapper has CRUD and User management I’ve chosen to go with that one.

Our News article will have a Title, an Author, the content and a boolean that allows it to be seen on the homepage

1
2
3
4
5
6
7
8
9

class News extends LongKeyedMapper[News]  {
  def getSingleton = News
 
  object title extends MappedString(this, 100)
  object author extends MappedString(this, 100)
  object content extends MappedTextarea(this, 4000)
  object showOnHomepage extends MappedBoolean(this)
}
object News extends News with LongKeyedMetaMapper[News]

* A class and a object with the same name are called companion’s in Scala and the relationship is that the object is a singleton instance of the class, and they have access to each others private methods.

by extending News with LongKeyedMapper[News] we are able to persist it to the database.

Now we will mix-in some traits to add some extra functionality, a Trait in Scala is like a Java interface but allows for partial implementation. Say we want to add a primary key field and two date fields to store created and updated values, all we need to do is add the triats: IdPk (for primary key) and CreatedUpdated to the class definition.

1

class News extends LongKeyedMapper[News] with IdPK with CreatedUpdated

this will add the MappedLong object id, and 2 MappedDate objects created and updated to the News class

To add CRUD functionality we just add the CRUDify trait, this one needs to be added to the object instead of the class so that one now looks like:

object News extends News with LongKeyedMetaMapper[News] with CRUDify[Long, News]

This will add the forms, validation and handling to the news object.

To hook this all up into Lift’s bootstrapping we have to add some lines to Boot.scala, which is the class that lift calls before anything else.
The following line will make sure Lift knows about our persistence class and creates the database tables if needed

Schemifier.schemify(true, Schemifier.infoF _, News)

Lift works with a Sitemap, any page you want Lift to serve needs to be added to the sitemap. so we need to add our CRUD pages there fortunately CRUDify has a convenient helper method for this.

val menu = Menu(Loc("HomePage", "index" :: Nil, "Home Page", Hidden)) ::: News.menus
LiftRules.setSiteMap(SiteMap(menu:_*))

This will create a Homepage menu item and adds the news crud pages to it.

If we want to create a news item through code we could just execute the following line:

News.create.title("A news Item").author("gertjan").content("Breaking news, an item has just been created").showOnHomePage(true).saveMe

Next the snippet that will display a number of news items on the homepage

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

class Homepage {
 def news(in: NodeSeq) : NodeSeq = {
    val articles = News.findAll(By(News.showOnHomepage, true), OrderBy(News.createdAt, Descending), MaxRows[News](5))
 
    def bindArticles(template: NodeSeq): NodeSeq = {
      articles.flatMap {
                case (news) => bind("article", template, 
							"title" -> news.title, 
							"author" -> news.author, 
							"content" -> StringHelper.unescapeXml(news.content),
							"created" -> news.createdAt,
							"updated" -> news.updatedAt)
      }
   }
 
   Helpers.bind("news", in, "articles" -> bindArticles _)
  }
}

The first line will lookup 5 news items that have the showOnHomepage boolean set to true and order them on date created.
The method bindArticles method will create bindings for each item found. as Lift escapes all xml/html by default, we need to unescape it especially when we want rich content.
The last line, will bind the collection to <news:articles/>

Then all is left is the html template:

<lift:surround with="default" at="content">
    <div class="news">
    <lift:homepage.news>
	<news:articles>
          <h3><article:title /></h3>
          <h4><i>by:</i><article:author /> <i>created:</i><article:created /> <i>updated:</i><article:updated/></h4>
          <p><article:content /></p>
        </news:articles>
    </lift:homepage.news>
    </div>
</lift:surround>

Because the <news:articles> contains a collection, lift will loop over it and process the inner html with each item in the collection.

A thing about authorization, we want the CRUD pages (and probably some other pages) to be accessible only to logged in users, this can be handled in several ways.
1) The Menu class has a parameter that defines the visibility of it, this can be easily used to check if a user is logged in:

val adminLoc = Loc("AdminPage", "admin" :: Nil, "Admin Page", Test((u) => u = User.loggedin_?))

or override the specific menu methods that are defined in the CRUDify trait

override def editMenuLocParams = If(User.loggedIn_? _, RedirectResponse("/user_mgt/login")) :: super.editMenuLocParams

And there it is, with surprisingly little code, we now have the beginnings of a content management system.

I’m building a webapplication that holds my collection of books, so the first use case I want to present to you and go through the implementation is the following:

As a user I want to be able to quickly import a list of isbn numbers

Lift moves away from your typical model view controller framework, but focuses on keeping your template’s designer friendly without any logic or code in there. Besides that the main focus areas are security and scalability, read the documentation on the lift website if you want to know more about this.

Taking the use case, we need to build a form with a textarea that allows us to copy/paste a list of isbn numbers in and add those to the database. Later use cases will then look up title/author information from for instance amazon, google books, or isbndb

Our html template which contains the form looks like this:

<lift:surround with="default" at="content">
  <h3>Import Isbnn rumbers</h3>
  <lift:ImportIsbn.bulk form="POST">
    <entry:text /><br/>
    <entry:submit />
  </lift:ImportIsbn.bulk>
 </lift:surround>

• The <lift:surround> tells lift to ‘surround’ this template with the default.html template which contains the header, footer and navigation for our website, inserting this template at the <lift:content /> tag in that template.
• The <lift:ImportIsbn.bulk> tells Lift to look for a class named ImportIsbn with a method bulk with the inner html as a parameter for us to manipulate
• The <entry:*>’s we will bind to form elements in the snippet code later on

Something more about templates:

• if a snippet takes long to complete, surround it with <lift:lazy-load> this will show an ajax loading gif while the snippet is executed.
• if you have a lot of snippets that take a while, surround them with <lift:parallel> to execute them in different threads

now to create our form:

1
2
3
4
5
6
7
8

class ImportIsbn
  var isbnnrs = ""
  def bulk(in : NodeSeq) : NodeSeq = {
        bind("entry", in,
          "text" -> SHtml.textarea(isbnnrs, isbnnrs = _),
          "submit" -> (SHtml.hidden(processIsbnNrs) ++ <input type="submit" value="Add" />))
  }
}

The bind method is a helper method that looks in the html (NodeSeq) passed on for tags marked with the “entry” prefix and binds them to html elements, in good functional programming the SHtml.* methods take a function as a parameter, for the textarea, we assign the value of the textarea to it, the hidden one calls the method processIsbnNrs
we could also have used

SHtml.submit("Add", processIsbnNrs)

but the above allows us to quickly make a ajax form out of it.
To do this we simple put a Shtml.ajaxForm around the bind.

 SHtml.ajaxForm(
        bind(.... etc )
      )

This will create the javascript methods to submit this form without doing a normal post

Due to Scala’s inline xml capabilities a snippet can also return xhtml directly, lift only works with xhtml (and if you use the 2.2-SNAPSHOT even html5)

  def mySnippet(in: NodeSeq) : NodeSeq = {
  <p>This is returned from my snippet</p>
  }

After submitting the method processIsbnNrs is called which looks like this:

1
2
3
4
5
6
7
8
9
10
11
12

      def processIsbnNrs() = {
        isbnnrs split " " map {addBook _}
      }
 
      def addBook(nr : String) {
        Book.find(By(Book.isbn, nr)) match {
          case (Full(book)) => alreadyInDb += " " + nr
          case (_) => Book.create.isbn(nr).saveMe
        }
        msg = "Skipped: " + alreadyInDb
        S.warning(Text(msg))
      }

As the isbnnrs var is bound to the textarea it will now contains the list of isbnnrs, the first line will split it on a space and then calling the addBook method for each entry

The addBook method will try to find the book in the Database, (the following post will look at Mapper the persistence layer in Lift). if it doesn’t it will add it to the database.
and that is all to it, I left out validation and initializing of the var’s here.  but the code is public at github if you want to take closer look.

A close observer will see the use of the method Full() in there, Scala has the Option class with which can be an instance of Some or None, which is designed to make continuous null pointer exception checking unnecessary.
Lift takes this one step further by having a Box class, which can be Full, Empty or Failure.  This in combination with Scala’s match/case functionality is very powerful and creates very readable and thus maintable code.
In our case if a Book is found (the box is full) do nothing, if not save a book with that isbn nr to the database.

See the Box article on Liftweb for more information about this.

• Unittests takes times to write
  I’m even inclined to say that writing your tests should take at least the time as writing your functionality takes.
• Start with the unit tests
  Start writing your unittests first, setting up just enough of your classes to get the test compiling, this will force you write your test based on the functionality required not the buggy implementation you’ve just written.
• Independent
  as the name says, a unit test will test a single unit of code, use integration tests to see how parts are working together.
• Small
  A unit test should test a small bit of functionality, one class, or even one method. I usually write extra unittests that test entire use-cases, but those work next to the normal ones.
• Test negatives as wel as positives
  Your first test will probably check whether the functionality works when specifying the correct parameters , but also test whether the right results are returned or the correct exceptions are thrown when specifying the wrong ones.
• Use code coverage tools to measure effectiveness
  Coverage tools, like for instance Clover will tell you how much of your code is covered, and also which classes are the most risk (high complexy, low coverage)
• Be careful when using mock objects
  Using mock objects keeps your unittests indepent but it has the danger that you are testing your mock objects instead of your own functionality, write integration tests
• Don’t change outputted results into asserts
  It can be very tempting to first see what your class outputs when running a test, and then writing asserts to test on those results, but you will just be validating that bug that is still in there.

Advantages of rest

• Works over HTTP, known protocol with known data formats (mime types), no weird ports open in your firewall
• Every resource is uniquely identified by a URI
• Can do CRUD on those resources (they are just called POST, GET, PUT, DELETE)
• used by all the major players for cloud applications (SUN’s cloudApi, Microsoft’s Azure services platform)
• we might finally get rid of webdav

I’ll be using the Jersey Framework for this.
As i’ve been working on a backend that will allow you to manage agile development processes (Scrum, Kanban etc) I thought I use that.

In this application every object is represented by a Card for instance a Story, Task, Bug, work Item, etc.
Cards have a hierarchical structure and can each have custom properties
The class looks something like this (removed methods for clarity):

public abstract class Card {
	private long id;
	private List children = new ArrayList();
	private List properties = new ArrayList()
}

and then for instance the Story class with some default values set in the constructor:

public class Story extends Card {
	public Story() {
		addProperty("Title", "A new story");
		addProperty("Description", "as a .. I want .. for the benefit of ..");
		addProperty("Storypoints", "0");
	}
}

To be able to return these objects through REST they need to be in a format that can be distributed over HTTP, in this case we will be using XML and JSON.
The Java XML Bindings are perfectly suited to create xml representations of our classes

@XmlRootElement(name="card")
public abstract class Card {
	private long id;
 
	@XmlAttribute(name="type")
	protected String instance = this.getClass().getSimpleName();
 
	@XmlElement(name="card")
	private List children = new ArrayList();
 
	@XmlElement(name="property")
	private List properties = new ArrayList();
 
	@XmlAttribute
	public long getId() {}
}

* the Entry class is a simple class with a name and value property, I choose this approach as JAXB has an issue in representing HashMaps() in XML
this will produce the following xml

for JSON we don’t need to do anything special then include the jersey-json.jar to the project.

Now to configure and hookup the rest servlet
I added the jersey servlet to my web.xml

    	Rest Web Service
    	com.sun.jersey.spi.container.servlet.ServletContainer
    	1
 
    	Rest Web Service
    	/services/*

And create the service provider class

@PerRequest
@Path("/agileassistant")
@Consumes({ MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML })
@Produces({ MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML })
public class AgileAssistantService {
 ...
}

This will tell Jersey that this is a provider that will respond to requests to /rest/agileassistant, accept JSON and XML and can output JSON or XML depending on the accept headers sent by the client

Now if we for instance want to return the first card (rootCard) and all its children we add this method

	public static Card rootCard = Card.createCard(Root.class);
 
	@GET
	public Card getRoot() {
		return rootCard;
	}

(in my data model the rootCard is always present and functions as a starting point to add the rest of the cards in a hierarchical structure)
So requesting this card will effectively return the entire structure for my test data:

Doing a parameterized query is almost as easy

	@GET
	@Path("/card/{id}")
	public Card getCard(@PathParam("id") long id) {
		return getCardById(rootCard, id);
	}

here the id that you specify will be injected into the parameter
giving the following output

To conclude enabling rest for your applications is pretty easy, writing this blog took longer than writing the code.
In a next post I will add POST/PUT/DELETE requests to store data in your application

var someObject : bind myJavaclass.getProperty();

JavaFX needs to be aware of any changes to the bound variables to be able to update the related objects. It does this automagically for all javafx objects.
It seems JavaFX uses the Observer/Observable pattern for all its internal objects, well we can do the same to bind with JavaObjects.
To get myself familiar with JavaFX i decided to write a photoviewer/editor. This application has a java backend with in particular a Photo class which contains all the information of a certain picture, filename, path, tags, exif information etcetera

To get changes in the Java class to appear in the gui I create a PhotoAdapter javafx class that extends Observer and implemented the properties I need in my GUI.

public class PhotoAdapter extends Observer {
    public-read var filename: String;
    public-read var tags: String[];
    public-read var exif: String[];
 
    public-init var photo: Photo
         on replace {
            photo.addObserver(this);
            filename=photo.getFilename();
            tags = photo.getTags();
            exif = photo.getExif();
         };
 
    override function update(observable: Observable, arg: Object) {
        filename = photo.getFilename();
        tags = photo.getTags();
        exif = photo.getExif();
    }

after that I had my existing Java Photo class extend Observable and in every method that made changes I added the methods setChanged() and NotifyObservers() to inform the Observers about the change

public class Photo extends Observable {
    private Map<String, String> exifInformation; 
    ....
    public void setExifInformation(Map exifInfo) {
        this.exifInformation = exifInfo;
        setChanged();
        notifyObservers();
    }
}

in my main javafx class I can then use it as follows

    var photoAdapter = PhotoAdapter {
        photo : new Photo(startImage);
}

and use to fill a ListView swing object with for instance the tags:

    ListView {
        items:     bind photoAdapter.tags;
        width:     200;
        height:    300;
    }

et viola, thats all that there is to it