Currently i’m using 3 parties to get the picture complete

1. Github - Online Git repositories with nice collaboration features, if you know it, you’ll be using it, if you don’t, have a look, you’ll never look back
2. Cloudbees – Free PaaS with Git Repository and Jenkins CI instance, you could also run your apps here, but no support for Scala, Lift at the time I looked at it
3. Cloudfoundry – Free for now  (Beta) PaaS from VMWare – Support Scala and Lift, which makes it easy for me.

Registering a app at cloudfoundry

Once you have an account and installed the command-line tools you can claim your little application space at the hosted cloudfoundry (or download and run a micro.cloudfoundry instance in your local Vmware system)

$ vmc target api.cloudfoundry.com
$ vmc login [email]
$ vmc push my-app --path target/war --url my-app.cloudfoundry.com --instances 2 --mem 512k --runtime java
$ vmc create-service mysql my-app-db my-app

What this means is, i’m pushing my war based maven build webapplication to the hosted cloudfoundry as two instances with each 512k memory and runtime java, I’m also binding it to a mysql service called my-app-db.
To see the result of my actions I can type vmc apps

$ vmc apps
+-------------+----+---------+---------------------------+-------------+
| Application | #  | Health  | URLS                      | Services    |
+-------------+----+---------+---------------------------+-------------+
| my-app      | 1  | RUNNING | my-app.cloudfoundry.com   | my-app-db   |
+-------------+----+---------+---------------------------+-------------+

The commandline tools also give me options to check logs and crashes, start and stop the application etc.

Database connectivity

As cloud applications/instances/services are usually not as easily identifiable with an ip-address, your application needs to find out about services (like a database) in a different manner.
I’m doing this by using the org.cloudfoundry.runtime library to create a mysql Service for me

1
2
3
4
5
6
7
8
9
10
11
12
13
14

  object CloudFoundryConnection extends ConnectionManager {
    def newConnection(name: ConnectionIdentifier): Box[Connection] = {
      try {
        import org.cloudfoundry.runtime.env._
        import org.cloudfoundry.runtime.service.relational._
        Full(new MysqlServiceCreator(new CloudEnvironment())
                        .createSingletonService().service.getConnection())
 
      } catch {
        case e : Exception => Empty
      }
    }
    def releaseConnection(conn: Connection) {conn.close}
  }

This object will return a connection if it is running in the cloud, otherwise I’ll fall back to the normal database connection of in my case an H2 database which I use for development
so I also changed the DB initialisation part of Lift in its Boot class.
The order of DB discovery is now 1) jndi connection, 2) cloudfoundry 3) connection in Properties file 4) H2 database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

if (!DB.jndiJdbcConnAvailable_?) {
      val connection = CloudFoundryConnection
      connection.newConnection(DefaultConnectionIdentifier) match {
        case (Full(connection) => {
          DB.defineConnectionManager(DefaultConnectionIdentifier, connection)
        }
        case Empty => {
          val vendor = new StandardDBVendor(
            Props.get("db.class") openOr "org.h2.Driver",
            Props.get("db.url") openOr "jdbc:h2:lift_proto.db;AUTO_SERVER=TRUE",
            Props.get("db.user"),
            Props.get("db.password")
          )
          LiftRules.unloadHooks.append(vendor.closeAllConnections_! _)
          DB.defineConnectionManager(DefaultConnectionIdentifier, vendor)
        }
      }
    }

Maven integration

Cloudfoundry also provides a maven plugin that allows most of the functionality to be run as part of a maven goal, which is ideal when you want to deploy from your Jenkins build.
Here are the changes to my pom.xml
Add the repository for the org.cloudfoundry.runtime dependency

<repository>
    <id>springsource-milestones</id>
    <name>SpringSource Milestones Proxy</name>
    <url>https://oss.sonatype.org/content/repositories/springsource-milestones</url>
</repository>

Add the dependency itself

<dependency>
    <groupId>org.cloudfoundry</groupId>
    <artifactId>cloudfoundry-runtime</artifactId>
    <version>0.6.1</version>
</dependency>

For the plugin add the plugin repository

<pluginRepository>
    <id>repository.springframework.maven.milestone</id>
    <name>Spring Framework Maven Milestone Repository</name>
    <url>http://maven.springframework.org/milestone</url>
</pluginRepository>

and the plugin itself

<plugin>
    <groupId>org.cloudfoundry</groupId>
    <artifactId>cf-maven-plugin</artifactId>
    <version>1.0.0.M1</version>
    <configuration>
        <server>mycloudfoundry-instance</server>
        <target>http://api.cloudfoundry.com</target>
        <url>medicate.cloudfoundry.com</url>
        <memory>512</memory>
        <instances>2</instances>
    </configuration>
</plugin>

Username and password information you can specify in the servers section of your .m2/settings.xml file

<server>
    <id>mycloudfoundry-instance</id>
    <username>email@address.com</username>
    <password>s3cr3t</password>
</server>

now you can update your deployed application by a simple

$ mvn cf:update

 Continous Integration on Cloudbees

First setup an account at cloudbees.com, create a repository and a Jenkins instance, and make sure cloudbees has your public key to allow git pushes.

Then setup a maven 2/3 build in Jenkins, setting the scm section to watch for changes from your cloudbees git repository.

Make sure your build name doesn’t have any spaces in it, cloudbees will choke on that, at least with the scala compiler.

As we need the login information from the .m2/settings.xml, you can upload this through webdav to repository-[username].forge.cloudbees.com/private directory.

And then in our jenkins build set /private/[username]/settings.xml in the alternative maven settings field (advanced button in the maven section)

I’ve set the maven goal to clean scala:doc cf:update meaning it will compile, test, create scala docs and deploy the application to cloudfoundry.

to finish it up you need to add a remote to cloudbees in your local git repository and push your changes. this then will trigger a build and deploy your app.

git remote add cloudbees ssh://git@git.cloudbees.com/[username]/my-app.git

To conclude, besides the 2 i’ve mentioned, there are a number of solutions out there, that all work in similar ways, but each still has it own requirements, If you want to run Java, Ruby, python, php, node.js applications, with Mysql, postgres, MongoDb, CouchDb, or RabbitMQ, you can get it to work on one or more of these providers.

Last but not least here is a picture from the actual application i’m building to visualize all I’ve talked about above:

But code with hardly any tests on them always makes me feel uncomfortable. so I decided to do some (almost integration) tests but running as part of the normal unit test suite
I’m doing this by placing the tags on a jsp, running this on a embedded Jetty Server and using Selenium Webdriver to test the resulting html.

First the base class, which takes care of starting/stopping the embedded Jetty Server and setting up the web driver.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

public abstract class BaseTest {
    private static int port = 9595;
    private static Server server = new Server(port);
 
    protected static String baseUrl = "http://localhost:" + port + "/";
    protected static WebDriver driver;
 
    @BeforeClass
    public static void setup() throws Exception {
 
        //configure jetty as an exploded war
        URL webAppUrl = BaseTest.class.getClassLoader().getResource("/");
        WebAppContext wac = new WebAppContext();
        wac.setContextPath("/");
        wac.setWar(webAppUrl.toExternalForm());
 
        server.setHandler(wac);
        server.setStopAtShutdown(true); //makes sure the sever is stopped even if the @AfterClass is never reached
        server.start();
 
        //HtmlUnit drive doesn't give a popup, the rest of the drivers do
        driver = new HtmlUnitDriver();
    }
 
    @AfterClass
    public static void teardown() throws Exception {
        driver.close();
        server.stop();
    }

In my project I have setup a resources directory that contains a WEB-INF/ with a web.xml containing my custom tag definitions and my tld file as you would with a normal web project
Webdriver comes with a set of Drivers for different browsers (Firefox, chrome, IE, iPhone, Android, etc) for testing browser compatibility, I don’t care about that, the tags I’m testing do not output any html/css that might require this.
The HtmlUnit driver is internal and doesn’t popup a browser window so ideal for my purpose.

Writing a test now becomes very easy:

In this case I have a custom tag that when invoked simply outputs “Hello World”

my test jsp:

1
2
3
4
5
6
7
8
9
10

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"  isELIgnored="false" %>
<%@ taglib uri="/customtaglib" prefix="custom" %>
<html>
  <head>
    <title>Hello World</title>
  </head>        
  <body>
    <p><custom:helloworld /></p>
  </body>
</html>

The Test class:

1
2
3
4
5
6
7
8
9
10

public class HelloWorldTest extends BaseTest {
 
    @Test
    public void testHelloWorldTag() {
        driver.get(baseUrl + "helloworld.jsp");
        Assert.assertEquals("Hello World", driver.getTitle());
 
        WebElement element = driver.findElement(By.cssSelector("body p"));
        Assert.assertEquals("Hello World", element.getText());
    }

Testing the resulting html is made very easy, by having the WebElement findElement() and List findElements() methods.
and then setting predicates with the By class. for instance By.cssSelector(), By.tagName() etc.
It is also possible to test click throughs and form submits, by calling click() or submit() on WebElements.

Added bonus is that although the tag code is running in the embedded jetty server and not in the unit tests itself it’s coverage is measured by our code coverage tools

One thing to look out for is for the embedded Jetty to work with JSP’s you need to add jetty’s version of the jsp spec to your project not the javax.servlet.* ones.
my maven dependencies look likes this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

    <dependencies>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.12.0</version>
        </dependency>
        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jetty</artifactId>
            <version>6.1.22</version>
        </dependency>
        <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jsp-api-2.1</artifactId>
            <version>6.1.14</version>
        </dependency>
         <dependency>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>jsp-2.1</artifactId>
            <version>6.1.14</version>
        </dependency>
        <dependency>
             <groupId>junit</groupId>
             <artifactId>junit</artifactId>
             <version>4.7</version>
             <scope>test</scope>
        </dependency>
    </dependencies>

Ok, lets get started by imagining we have a web store that sells books and we store information about our visitors, and our Marketeer would like to target certain offers to customers based on which categories they visit the most.
I’m not explaining the customers implementation but i’m assuming a method from the business layer can be called. (the source-code contains a dummy implementation that returns random categories)

Since SDL Tridion 2011 there is an Ambient Data Framework (ADF) on the Content Delivery site. This framework allows sharing of data between applications. You can create so-called cartridges that can retrieve and store data within a request or session scope by specifying input and/or output claims.
The ADF will figure out any dependencies (if cartridge 1 has an input claim, that is an output claim  of cartridge 2, the ADF will make sure cartridge 2 is run first)

SDL SmartTarget comes with a session cartridge, which will provide information like the client’s browser, os, ip address etc
SDL Audience Management comes with it’s own cartridge which will provide information about the logged in Audience Manager visitor

What we need to do is:

1. Get the most looked at category into the Ambient Data Framework
2. Configure SmartTarget to use this data when doing a query for promotions
3. Add it as a trigger type so we can use it as a trigger in a promotion

Writing a cartridge for the Ambient Data Framework
A cartridge consists of a configuration file and one or more ClaimProcessor classes.
The configuration file will contain the input and output claims and which classes will implement the functionality, in our case just one output claim and just one implementing class

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

<CartridgeDefinition Uri="com:tridion:smarttarget:samples:recommendedbooks" Description="Recommended books cartridge.">
	<ClaimDefinitions>
		<ClaimDefinition Uri="com:tridion:smarttarget:samples:recommendbooks:category"
                         Scope="REQUEST"
                         Subject="com:tridion:smarttarget:samples:recommendbooks"
                         Description="The Most lookedat category" />
	</ClaimDefinitions>
	<ClaimProcessorDefinitions>
		<ClaimProcessorDefinition Uri="com:tridion:smarttarget:samples:recommendbooks"
				ImplementationClass="com.tridion.smarttarget.samples.recommendbooks.RecommendBooksClaimProcessor"
				Description="This will put the most looked at category into the claimstore">
			<RequestStart>
				<InputClaims />
				<OutputClaims>
					<ClaimDefinition Uri="com:tridion:smarttarget:samples:recommendbooks:category" />
				</OutputClaims>
			</RequestStart>
		</ClaimProcessorDefinition>
	</ClaimProcessorDefinitions>
</CartridgeDefinition>

and the ClaimProcessor looks like

1
2
3
4
5
6
7
8
9
10

public class RecommendBooksClaimProcessor extends AbstractClaimProcessor {
 
    private final static URI RECOMMENDED_CATEGORY_URI = URI.create("com:tridion:smarttarget:samples:recommendbooks:category");
 
    @Override
    public void onRequestStart(ClaimStore claimStore) throws AmbientDataException {
        String category = YourBusinessModel.getMostLookedAtCategory();
        claimStore.put(RECOMMENDED_CATEGORY_URI, category, true);
    }
}

so very simply get your category from your business layer and putting it on the ADF, the boolean at the end, defines whether or not other cartridges can override this value
true meaning it cannot be overwritten

Then the cartridge needs to be added to cd_ambient_conf

1
2
3
4
5
6

<Configuration>
    <Cartridges>
        <Cartridge File="/recommendedbooks_cartridge.xml" />
            <!-- other cartridges removed for brevity -->
    </Cartridges>
</Configuration>

Step 2 configuring SmartTarget
Here we configure how the category will be added to the query with a prefix to avoid name conflicts

1
2
3
4
5
6
7
8
9
10
11
12
13

<Configuration Version="1.1.0">
    <!-- the rest of the configuration left out for brevity -->
    ..
    <SmartTarget>
        ..
        <AmbientData>
            ..
            <Prefixes>
                ..
                <com_tridion_smarttarget_samples_recommendedbooks>rb</com_tridion_smarttarget_samples_recommendedbooks>
            </Prefixes>
        </AmbientData>
    </SmartTarget>

The name of the element should reflect the base part of the URI for the data stored in the ADF, the last part of the URI together with the prefix will become the name
so in our case storing “Science Fiction” in

com:tridion:smarttarget:samples:recommendbooks:category

will end up as

rb_category=Science+Fiction

in the query

Step 3: configuring the trigger-type
in the config directory of your fredhopper installation there is a trigger-types.xml
in here all the custom trigger-types are configured.

1
2
3
4
5
6
7
8
9
10
11
12
13

<trigger-types xmlns="http://www.fredhopper.com/schema/knowledge-model/trigger/type/1.0">
    <!-- other trigger-types removed for brevity -->
    ...
    <trigger-type url-param="rb_category" name="Category (most looked at)" basetype="text">
        <list-of-values multiselect="true">
		<value>Fantasy</value>
		<value>Fiction</value>
		<value>Romance</value>
                <value>Science Fiction</value>
                <value>Thriller</value>
	</list-of-values>
    </trigger-type>
</trigger-types>

The multiselect=”true” will create a list of checkboxes in the GUI allowing a promotion to trigger on more categories.

if you’re list of categories (or any other enumeration) is changing a lot, there is a rest interface that allows you to modify them runtime.
(in Audience Management we do that for instance to propagate changes in Audience management segments to SmartTarget)

The source code for the cartridge is available on github, with the sample configurations.
To run it, you’ll need maven and the cd_ambient.jar from the CD Installation in your maven repository
run mvn package, and copy the resulting jar to your website lib where you have the ambient data framework configured.

I hope this has given you some ideas on how to better integrate Smart Target with your business

According to Wikipedia:

In computer science, the Actor model is a mathematical model of concurrent computation that treats “actors” as the universal primitives of concurrent digital computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received.

The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.
An actor is a computational entity that, in response to a message it receives, can concurrently:

• send a finite number of messages to other actors;
• create a finite number of new actors;
• designate the behavior to be used for the next message it receives.

There is no assumed sequence to the above actions and they could be carried out in parallel.

A number of languages have the actor model implemented in their core (Io, Erlang, Scala), for most other languages it’s available as a library (Akka for Java, Retlang for .Net, Haskell-Actor for Haskell, Parely and Pykka for Python)

In this example I’ll be using the Akka framework for Java and Scala, which brings actors to the enterprise level by adding for instance fault tolerance (supervisors, let it crash semantics) and remote actors, If you want to play around with this yourself the best place to start is to use the typesafe stack which includes Akka and Scala

I’m using a typical map/reduce use case, namely counting words in a document
I do this by having a ‘master’ actor that divides the work in lines, have a number of ‘worker’ actors that count the words for a single line, replying the result to the ‘master’ to aggregate

Messages that actors send to each other are simply objects (case classes, the receive method typically uses pattern matching to determine which message has arrived), multiple messages to an actor will be queued, and handled sequentially

So let’s create the message objects

30
31
32
33
34

  // Messages
  sealed trait MapReduceMessage
  case class CountDocument(document: Iterator[String]) extends MapReduceMessage
  case class CountLine(line: String) extends MapReduceMessage
  case class Result(values: Map[String, Int]) extends MapReduceMessage

The CountDocument which will be send to the master, the CountLine to each worker, and each worker will return a Result when done

as each receive method returns a Partial Function[T] which allows for several classes to handle all the messages, the sealed trait super type will actually generate a warning when not all messages are handled. neat trick that.
It will also make it impossible to add messages outside of your class which could be something you want. or not.

Now on to the Master

57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

// Master Actor, creates Worker Actors, distributes work and gathers results
  class Master(nrOfWorkers: Int, latch:CountDownLatch) extends Actor {
 
    val workers = Vector.fill(nrOfWorkers)(actorOf[CountLineWorker].start());
    val router = Routing.loadBalancerActor(CyclicIterator(workers)).start();
 
    val resultMap = new HashMap[String, Int]();
 
    var start : Long = _
    var count : Long = 0
 
    def receive = {
      case CountDocument(lines : Iterator[String]) =>
 
        lines.foreach(line =>
              if (!line.isEmpty) {
                count = count+1;
                router ! CountLine(line)
              })
 
        //shutdown actors
        router ! Broadcast(PoisonPill)
        router ! PoisonPill
 
      case Result(values: Map[String, Int]) =>
 
        for ((key, value) <- values) {
          resultMap.put(key, resultMap.getOrElse(key, 0)+value)
        }
        count = count - 1;
        if (count <= 0) self.stop()
    }
 
    override def preStart() {
      start = System.currentTimeMillis
    }
 
    override def postStop() {
      val end = System.currentTimeMillis-start
      println("Result after %s ms :".format(end))
      for((key, value) <- resultMap.toList.sortBy(_._2).reverse) {
        println("%s: %s".format(value, key))
      }
      latch.countDown()
    }
  }

An actor class needs to implement the Actor interface which has one method called receive
The Routing.LoadbalancingActor is a built in actor that forwards messages send to it according to the scheme passed to it, in our case in a round robin style.
When receiving a CountDocument messages it will send each line to a worker, it will then tell the workers and the router to shutdown by sending them a PosionPill message.
When receiving a Result message it will aggregate the result and after receiving all results it will stop itself.
we override the preStart() and postStop() methods to allow us to calculate spend time and reporting the result

the ! in

router ! CountLine(line)

is a so called bang method, meaning fire and forget, there are also methods that will reply eventually (!!) and reply eventually with a Future (!!!)

The Worker looks as follows:

36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

//Actor that counts the words for a single line
  class CountLineWorker extends Actor {
 
    def receive = {
       case CountLine(line) =>
         self reply Result(countWords(line))
    }
 
    def countWords(line: String):Map[String, Int] = {
      val result = new HashMap[String, Int]
 
      "[^A-Za-z0-9\u0020]".r.replaceAllIn(line, "")
            .split(" ")
            .foreach(word => {
              result.put(word, result.getOrElse(word, 0)+1)
            })
 
      result
    }
  }

Should speak for itself, sanitizes and then count the words in a line, and replies to the sending actor with the result.

now wrap it in a main class that loads the document and sends it off to the master

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

object MapReduce extends App {
 
  countWordsInFile("src/main/resources/test.txt", 6);
 
  def countWordsInFile(fileName: String, nrOfWorkers: Int) {
 
    val source = scala.io.Source.fromFile(fileName)
    val document = source.getLines()
 
    val latch = new CountDownLatch(1)
 
    val master = actorOf(new Master(nrOfWorkers, latch)).start()
 
    master ! CountDocument(document)
 
    latch.await()
 
    source.close()
  }
}

This will load the document and sends it off to the master also instructing it to create 6 workers, as actors are called asynchronously, the Countdown latch is there to prevent the program from stopping before all the work is done.

The full source code is available as a gist on github

A couple of other interesting Akka features are

Become:
you can tell each actor to become another, this allows for hot code swapping, which is essential in fault-tolerant, always up systems.

  def receive = {
     case updateYourSelf(newMe:ActorRef) => {
        become newMe
     }
  }

Typed Actors (example blatantly stolen from Akka documentation):
extending any POJO with TypedActor will turn them into actors
your class need to have a separate interface/implementation for this to work

trait RegistrationService {
  def register(user: User, cred: Credentials): Unit
  def getUserFor(username: String): User
}
public class RegistrationServiceImpl extends TypedActor with RegistrationService {
  def register(user: User, cred: Credentials): Unit = {
    ... // register user
  }
 
  def getUserFor(username: String): User = {
    ... // fetch user by username
   user
  }
}
 
val service = TypedActor.newInstance(classOf[RegistrationService], classOf[RegistrationServiceImpl], 1000)
//last parameter is timeout for Futures

Remote Actors (example blatantly stolen from the Akka homepage):

// server code
class HelloWorldActor extends Actor {
  def receive = {
    case msg => self reply (msg + " World")
  }
}
remote.start("localhost", 9999).register(
  "hello-service", actorOf[HelloWorldActor])
 
// client code
val actor = remote.actorFor(
  "hello-service", "localhost", 9999)
val result = actor !! "Hello"

When starting the mobile application and entered the correct login information, a request is being made to get the list of authors.
Clicking on an author will get a list of books from that author that the user owns, clicking on a book shows the details.

So the first query that needs implementing is: Give me a list of Authors for which the current user owned books

Given the Following Mapper Data Model:
Author <- (1) -> Book <- (2) -> User
(1) An Author can have more books while books can be written by multiple authors (N-N)
(2) A user can own multiple books, while a book can have multiple owners (N-N)

note: When successfully authenticating in a user his API key is returned which is used in all other requests

Below we will see the code that executes this query
what happens line by line is this:

• Line4: we get a list of books for the current user
• Line5: we map the colllection to one that contains the authors for each book, the result is a list of lists of authors List[List[Author]]
• Line6: foldLeft applies the method to each element in the list, from left to right, as the method ++ means adding two list’s together, we’ll end up with one list containing all authors
• Line7: distinct removes any duplicates in there
• Line8: We sort the list on the author’s lastName property
• Line9: we map it to a List of Xml fragments

Scala’s inline XML support allows us to simple put the Authors tag around it.
The second case does the same but gives a JSON response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

serve {
  case "api" :: key :: "user" :: "authors" :: "all" :: _ XmlGet _ => {
    <Authors> {
      Book.findAll(In(Book.id ,BookUser.book, By(BookUser.user, getUserIdFromKey(key))))
        .map(book => { book.authors.get })
        .foldLeft(List[Author]())(_ ++ _)
        .distinct
        .sortWith((a1:Author, a2:Author) => a1.lastName.is < a2.lastName.is)
        .map(author => author.toXml)
      }
    </Authors>
  }
  case "api" :: key :: "user" :: "authors" :: "all" :: _ JsonGet _ => {
    JsonWrapper ("Authors", {
      Book.findAll(In(Book.id ,BookUser.book, By(BookUser.user, getUserIdFromKey(key))))
        .map(book => { book.authors.get })
        .foldLeft(List[Author]())(_ ++ _)
        .distinct
        .sortWith((a1:Author, a2:Author) => a1.lastName.is < a2.lastName.is)
        .map(author => author.toJson)
      })
  }
}
 
//helper method
def getUserIdFromKey(key:String):Long = {
  User.find(By(User.key, key) match {
    case Full(user) => user.id
    case (_) => 0 
  }
}

Lets say we want to get a list of all books in the system when we access http://(server)/api/books/all

First we need to tell Lift we have a class that responds to requests in Boot.scala:

LiftRules.dispatch.append(BookRestApi)

then the class itself: BookRestApi.scala:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

object BookRestApi extends RestHelper  {
  serve {
    // list of books
    case "api" :: "books" :: "all" :: _ XmlGet _=>
      {
         <Books>
          {
            Book.findAll.map(book => book.toXml)
          }
         </Books>
      }
    case "api" :: "books" :: "all" :: _ JsonGet _=>
      {
        JsonWrapper("books", Book.findAll.map(book => book.toJS))
      }
  }
}

The RestHelper trait parses the Url and makes the relevant parts available for Scala’s pattern matching capabilities.

for the first case statement:

• the api :: books : all List will match if the url points to /api/books/all
• the XmlGet method will match if the client has indicated he can accept xml by setting an accept header or by ending the url with a .xml suffix

Scala’s inline XML capabilities make it very easy, to envelop the result in a <Books> element.

the second statement does the same but then when the client requests a json response.

For the Json part I used a JsonWrapper helper method because the inline code looked totally unreadable, this still isn’t very readable but at least it has a descriptive method name now

1
2
3

  def JsonWrapper(name : String, content : JValue) : JValue = {
    (name -> content)
  }

If you would call this method with the name “Books” and the content {“foo” : “bar” } it will output :

{"books" : 
  {"foo" : "bar"}
}

This shows both a positive and negative aspect to the Scala language, it makes for a very flexible and fast development, but loses a lot in readability, which makes maintenance more difficult.

Now if would want to get a single book when I specify the id of that book, I’d add the following:

1
2
3
4
5
6
7

case "api" :: "books" :: AsLong(id) :: _ XmlGet _=>
        {
          Book.findByKey(id) match {
            case Full(book) => {book.toXml}
            case (_) => NotFoundResponse("Book with id " + id + " not found\r\n")
          }
        }

This pattern will only match on /api/books/{id} and only if that Id can be converted to a Long, in which case it will try to look up a book with that id
if the book isn’t found (the Box returned from findByKey isn’t Full) it will give the text back

In the next article I’ll talk about PUT requests and authentication so we can add books through the rest interface

So what is Page speed?
It consists of a Firefox/Firebug plugin that allows web developer’s to analyze requests and optimize their websites and a Apache module that optimizes resources
more information here

As an admin I want to be able to add/edit news articles and choose whether or not they appear on the homepage

For this we need to store the articles in a database, A template and Snippet to show them and some way of managing them.
Lift has 2 built-in persistence frameworks, Mapper and Record, but also JPA is fully supported.
As Mapper has CRUD and User management I’ve chosen to go with that one.

Our News article will have a Title, an Author, the content and a boolean that allows it to be seen on the homepage

1
2
3
4
5
6
7
8
9

class News extends LongKeyedMapper[News]  {
  def getSingleton = News
 
  object title extends MappedString(this, 100)
  object author extends MappedString(this, 100)
  object content extends MappedTextarea(this, 4000)
  object showOnHomepage extends MappedBoolean(this)
}
object News extends News with LongKeyedMetaMapper[News]

* A class and a object with the same name are called companion’s in Scala and the relationship is that the object is a singleton instance of the class, and they have access to each others private methods.

by extending News with LongKeyedMapper[News] we are able to persist it to the database.

Now we will mix-in some traits to add some extra functionality, a Trait in Scala is like a Java interface but allows for partial implementation. Say we want to add a primary key field and two date fields to store created and updated values, all we need to do is add the triats: IdPk (for primary key) and CreatedUpdated to the class definition.

1

class News extends LongKeyedMapper[News] with IdPK with CreatedUpdated

this will add the MappedLong object id, and 2 MappedDate objects created and updated to the News class

To add CRUD functionality we just add the CRUDify trait, this one needs to be added to the object instead of the class so that one now looks like:

object News extends News with LongKeyedMetaMapper[News] with CRUDify[Long, News]

This will add the forms, validation and handling to the news object.

To hook this all up into Lift’s bootstrapping we have to add some lines to Boot.scala, which is the class that lift calls before anything else.
The following line will make sure Lift knows about our persistence class and creates the database tables if needed

Schemifier.schemify(true, Schemifier.infoF _, News)

Lift works with a Sitemap, any page you want Lift to serve needs to be added to the sitemap. so we need to add our CRUD pages there fortunately CRUDify has a convenient helper method for this.

val menu = Menu(Loc("HomePage", "index" :: Nil, "Home Page", Hidden)) ::: News.menus
LiftRules.setSiteMap(SiteMap(menu:_*))

This will create a Homepage menu item and adds the news crud pages to it.

If we want to create a news item through code we could just execute the following line:

News.create.title("A news Item").author("gertjan").content("Breaking news, an item has just been created").showOnHomePage(true).saveMe

Next the snippet that will display a number of news items on the homepage

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

class Homepage {
 def news(in: NodeSeq) : NodeSeq = {
    val articles = News.findAll(By(News.showOnHomepage, true), OrderBy(News.createdAt, Descending), MaxRows[News](5))
 
    def bindArticles(template: NodeSeq): NodeSeq = {
      articles.flatMap {
                case (news) => bind("article", template, 
							"title" -> news.title, 
							"author" -> news.author, 
							"content" -> StringHelper.unescapeXml(news.content),
							"created" -> news.createdAt,
							"updated" -> news.updatedAt)
      }
   }
 
   Helpers.bind("news", in, "articles" -> bindArticles _)
  }
}

The first line will lookup 5 news items that have the showOnHomepage boolean set to true and order them on date created.
The method bindArticles method will create bindings for each item found. as Lift escapes all xml/html by default, we need to unescape it especially when we want rich content.
The last line, will bind the collection to <news:articles/>

Then all is left is the html template:

<lift:surround with="default" at="content">
    <div class="news">
    <lift:homepage.news>
	<news:articles>
          <h3><article:title /></h3>
          <h4><i>by:</i><article:author /> <i>created:</i><article:created /> <i>updated:</i><article:updated/></h4>
          <p><article:content /></p>
        </news:articles>
    </lift:homepage.news>
    </div>
</lift:surround>

Because the <news:articles> contains a collection, lift will loop over it and process the inner html with each item in the collection.

A thing about authorization, we want the CRUD pages (and probably some other pages) to be accessible only to logged in users, this can be handled in several ways.
1) The Menu class has a parameter that defines the visibility of it, this can be easily used to check if a user is logged in:

val adminLoc = Loc("AdminPage", "admin" :: Nil, "Admin Page", Test((u) => u = User.loggedin_?))

or override the specific menu methods that are defined in the CRUDify trait

override def editMenuLocParams = If(User.loggedIn_? _, RedirectResponse("/user_mgt/login")) :: super.editMenuLocParams

And there it is, with surprisingly little code, we now have the beginnings of a content management system.

I’m building a webapplication that holds my collection of books, so the first use case I want to present to you and go through the implementation is the following:

As a user I want to be able to quickly import a list of isbn numbers

Lift moves away from your typical model view controller framework, but focuses on keeping your template’s designer friendly without any logic or code in there. Besides that the main focus areas are security and scalability, read the documentation on the lift website if you want to know more about this.

Taking the use case, we need to build a form with a textarea that allows us to copy/paste a list of isbn numbers in and add those to the database. Later use cases will then look up title/author information from for instance amazon, google books, or isbndb

Our html template which contains the form looks like this:

<lift:surround with="default" at="content">
  <h3>Import Isbnn rumbers</h3>
  <lift:ImportIsbn.bulk form="POST">
    <entry:text /><br/>
    <entry:submit />
  </lift:ImportIsbn.bulk>
 </lift:surround>

• The <lift:surround> tells lift to ‘surround’ this template with the default.html template which contains the header, footer and navigation for our website, inserting this template at the <lift:content /> tag in that template.
• The <lift:ImportIsbn.bulk> tells Lift to look for a class named ImportIsbn with a method bulk with the inner html as a parameter for us to manipulate
• The <entry:*>’s we will bind to form elements in the snippet code later on

Something more about templates:

• if a snippet takes long to complete, surround it with <lift:lazy-load> this will show an ajax loading gif while the snippet is executed.
• if you have a lot of snippets that take a while, surround them with <lift:parallel> to execute them in different threads

now to create our form:

1
2
3
4
5
6
7
8

class ImportIsbn
  var isbnnrs = ""
  def bulk(in : NodeSeq) : NodeSeq = {
        bind("entry", in,
          "text" -> SHtml.textarea(isbnnrs, isbnnrs = _),
          "submit" -> (SHtml.hidden(processIsbnNrs) ++ <input type="submit" value="Add" />))
  }
}

The bind method is a helper method that looks in the html (NodeSeq) passed on for tags marked with the “entry” prefix and binds them to html elements, in good functional programming the SHtml.* methods take a function as a parameter, for the textarea, we assign the value of the textarea to it, the hidden one calls the method processIsbnNrs
we could also have used

SHtml.submit("Add", processIsbnNrs)

but the above allows us to quickly make a ajax form out of it.
To do this we simple put a Shtml.ajaxForm around the bind.

 SHtml.ajaxForm(
        bind(.... etc )
      )

This will create the javascript methods to submit this form without doing a normal post

Due to Scala’s inline xml capabilities a snippet can also return xhtml directly, lift only works with xhtml (and if you use the 2.2-SNAPSHOT even html5)

  def mySnippet(in: NodeSeq) : NodeSeq = {
  <p>This is returned from my snippet</p>
  }

After submitting the method processIsbnNrs is called which looks like this:

1
2
3
4
5
6
7
8
9
10
11
12

      def processIsbnNrs() = {
        isbnnrs split " " map {addBook _}
      }
 
      def addBook(nr : String) {
        Book.find(By(Book.isbn, nr)) match {
          case (Full(book)) => alreadyInDb += " " + nr
          case (_) => Book.create.isbn(nr).saveMe
        }
        msg = "Skipped: " + alreadyInDb
        S.warning(Text(msg))
      }

As the isbnnrs var is bound to the textarea it will now contains the list of isbnnrs, the first line will split it on a space and then calling the addBook method for each entry

The addBook method will try to find the book in the Database, (the following post will look at Mapper the persistence layer in Lift). if it doesn’t it will add it to the database.
and that is all to it, I left out validation and initializing of the var’s here.  but the code is public at github if you want to take closer look.

A close observer will see the use of the method Full() in there, Scala has the Option class with which can be an instance of Some or None, which is designed to make continuous null pointer exception checking unnecessary.
Lift takes this one step further by having a Box class, which can be Full, Empty or Failure.  This in combination with Scala’s match/case functionality is very powerful and creates very readable and thus maintable code.
In our case if a Book is found (the box is full) do nothing, if not save a book with that isbn nr to the database.

See the Box article on Liftweb for more information about this.