A good way to stay flexible is to write less code. --Pragmatic Programmer

December 8, 2011

Cloud, Development

(No comments)

Lately I’m becoming more and more aware of the possibilities that are out there for developers, both in building, testing and running your applications in the cloud.
So I changed a hobby project i’m working on to be build, tested and deployed in the cloud. and I like to share my experiences here.

Currently i’m using 3 parties to get the picture complete

  1. Github - Online Git repositories with nice collaboration features, if you know it, you’ll be using it, if you don’t, have a look, you’ll never look back
  2. Cloudbees – Free PaaS with Git Repository and Jenkins CI instance, you could also run your apps here, but no support for Scala, Lift at the time I looked at it
  3. Cloudfoundry – Free for now  (Beta) PaaS from VMWare – Support Scala and Lift, which makes it easy for me.
For this to work I needed to make a couple of changes to my application that I will explain below.

Registering a app at cloudfoundry

Once you have an account and installed the command-line tools you can claim your little application space at the hosted cloudfoundry (or download and run a micro.cloudfoundry instance in your local Vmware system)

$ vmc target api.cloudfoundry.com
$ vmc login [email]
$ vmc push my-app --path target/war --url my-app.cloudfoundry.com --instances 2 --mem 512k --runtime java
$ vmc create-service mysql my-app-db my-app

What this means is, i’m pushing my war based maven build webapplication to the hosted cloudfoundry as two instances with each 512k memory and runtime java, I’m also binding it to a mysql service called my-app-db.
To see the result of my actions I can type vmc apps

$ vmc apps
+-------------+----+---------+---------------------------+-------------+
| Application | #  | Health  | URLS                      | Services    |
+-------------+----+---------+---------------------------+-------------+
| my-app      | 1  | RUNNING | my-app.cloudfoundry.com   | my-app-db   |
+-------------+----+---------+---------------------------+-------------+

The commandline tools also give me options to check logs and crashes, start and stop the application etc.

Database connectivity

As cloud applications/instances/services are usually not as easily identifiable with an ip-address, your application needs to find out about services (like a database) in a different manner.
I’m doing this by using the org.cloudfoundry.runtime library to create a mysql Service for me

1
2
3
4
5
6
7
8
9
10
11
12
13
14
  object CloudFoundryConnection extends ConnectionManager {
    def newConnection(name: ConnectionIdentifier): Box[Connection] = {
      try {
        import org.cloudfoundry.runtime.env._
        import org.cloudfoundry.runtime.service.relational._
        Full(new MysqlServiceCreator(new CloudEnvironment())
                        .createSingletonService().service.getConnection())
 
      } catch {
        case e : Exception => Empty
      }
    }
    def releaseConnection(conn: Connection) {conn.close}
  }

This object will return a connection if it is running in the cloud, otherwise I’ll fall back to the normal database connection of in my case an H2 database which I use for development
so I also changed the DB initialisation part of Lift in its Boot class.
The order of DB discovery is now 1) jndi connection, 2) cloudfoundry 3) connection in Properties file 4) H2 database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
if (!DB.jndiJdbcConnAvailable_?) {
      val connection = CloudFoundryConnection
      connection.newConnection(DefaultConnectionIdentifier) match {
        case (Full(connection) => {
          DB.defineConnectionManager(DefaultConnectionIdentifier, connection)
        }
        case Empty => {
          val vendor = new StandardDBVendor(
            Props.get("db.class") openOr "org.h2.Driver",
            Props.get("db.url") openOr "jdbc:h2:lift_proto.db;AUTO_SERVER=TRUE",
            Props.get("db.user"),
            Props.get("db.password")
          )
          LiftRules.unloadHooks.append(vendor.closeAllConnections_! _)
          DB.defineConnectionManager(DefaultConnectionIdentifier, vendor)
        }
      }
    }

Maven integration

Cloudfoundry also provides a maven plugin that allows most of the functionality to be run as part of a maven goal, which is ideal when you want to deploy from your Jenkins build.
Here are the changes to my pom.xml
Add the repository for the org.cloudfoundry.runtime dependency

<repository>
    <id>springsource-milestones</id>
    <name>SpringSource Milestones Proxy</name>
    <url>https://oss.sonatype.org/content/repositories/springsource-milestones</url>
</repository>

Add the dependency itself

<dependency>
    <groupId>org.cloudfoundry</groupId>
    <artifactId>cloudfoundry-runtime</artifactId>
    <version>0.6.1</version>
</dependency>

For the plugin add the plugin repository

<pluginRepository>
    <id>repository.springframework.maven.milestone</id>
    <name>Spring Framework Maven Milestone Repository</name>
    <url>http://maven.springframework.org/milestone</url>
</pluginRepository>

and the plugin itself

<plugin>
    <groupId>org.cloudfoundry</groupId>
    <artifactId>cf-maven-plugin</artifactId>
    <version>1.0.0.M1</version>
    <configuration>
        <server>mycloudfoundry-instance</server>
        <target>http://api.cloudfoundry.com</target>
        <url>medicate.cloudfoundry.com</url>
        <memory>512</memory>
        <instances>2</instances>
    </configuration>
</plugin>

Username and password information you can specify in the servers section of your .m2/settings.xml file

<server>
    <id>mycloudfoundry-instance</id>
    <username>email@address.com</username>
    <password>s3cr3t</password>
</server>

now you can update your deployed application by a simple

$ mvn cf:update

 Continous Integration on Cloudbees

First setup an account at cloudbees.com, create a repository and a Jenkins instance, and make sure cloudbees has your public key to allow git pushes.

Then setup a maven 2/3 build in Jenkins, setting the scm section to watch for changes from your cloudbees git repository.

Make sure your build name doesn’t have any spaces in it, cloudbees will choke on that, at least with the scala compiler.

As we need the login information from the .m2/settings.xml, you can upload this through webdav to repository-[username].forge.cloudbees.com/private directory.

And then in our jenkins build set /private/[username]/settings.xml in the alternative maven settings field (advanced button in the maven section)

I’ve set the maven goal to clean scala:doc cf:update meaning it will compile, test, create scala docs and deploy the application to cloudfoundry.

to finish it up you need to add a remote to cloudbees in your local git repository and push your changes. this then will trigger a build and deploy your app.

git remote add cloudbees ssh://git@git.cloudbees.com/[username]/my-app.git

To conclude, besides the 2 i’ve mentioned, there are a number of solutions out there, that all work in similar ways, but each still has it own requirements, If you want to run Java, Ruby, python, php, node.js applications, with Mysql, postgres, MongoDb, CouchDb, or RabbitMQ, you can get it to work on one or more of these providers.

Last but not least here is a picture from the actual application i’m building to visualize all I’ve talked about above:
Develop for the Cloud