Grails

Grails + CouchDB #s2gx

Posted on by Matt Woodward

Scott Davis – thirstyhead.com

NoSQL Databases in General

  • given the number of big companies using them, clearly they're ready to use today
  • time to re-examine our unnatural obsession for relational databases
  • rdbms has been around for 50 years now–well understood, great tooling, lots of information
  • rdbmses are silos
    • still good at what they do, but aren't necessarily well-suited to all data
  • as developers we're being forced to use sql to express something that's crucial to the success of your application
    • not our native language, kind of foreign when it comes down to it
  • we use orm to insulate ourselves from sql
    • express yourself in the native language of your choice instead of in sql

Is ORM State of the Art?

  • really just a bridge
  • why aren't there pure java or groovy datastores?
  • persistence is pretty uninteresting to developers
  • orm is a reasonable bridge, but a rather leaky abstraction as well
  • ted neward: orm is the vietnam of computer science
    • "[ORM] represents a quagmire which starts well, gets more complicated as time passes, and before long entraps its users in a commitment that has no clear demarcation point, no clear win conditions, and no clear exit strategy."

What Drew Me to CouchDB

  • what if i didn't have to bridge technologies anymore?
  • what if i could save my objects in their native format?
    • couchdb is actually a json datastore, but grails makes it trivial to transfer pogo <-> json
  • just need a thin translation layer

NoSQL Solutions

  • Google BigTable
  • mongoDB
  • CouchDB
  • Cassandra
    • "this is the future, but no one believes us"
  • each one of these are a bit different and each has their strengths and weaknesses
  • NoSQL = "not only SQL"
  • don't think of nosql solutions as just another database; truly different way to think about persistence
  • if you think of it as just another database, it'll be the worst database you've ever used
  • need to get out of the mindset of "spreadsheet" type format for data
  • start thinking more about the right tool for the job

CouchDB History

  • starting point was Lotus Notes
    • largely ahead of its time
    • document database
    • not brand-new stuff–ideas and foundation has been around for a very long time
  • Apache project

RDBMS vs. CouchDB

  • rdbms
    • row/column oriented
    • language: sql
    • insert, select, update, delete
  • CouchDB
    • if your data has a more vertical orientation as opposed to horizontal, starts to look more like attachments
    • email is a good example: to, from, body, attachment
    • language: javascript (map/reduce functions)
    • put, get, post, delete (REST)
    • "Django may be built for the Web, but CouchDB is built of the Web." — Jacob Kaplan-Moss, Django Developer
    • can build entire apps in CouchDB
  • Couch = acronym for "cluster of unreliable commodity hardware"
  • clustering is much more difficult to do clustering–couch was built from the ground up to be massively distributed, clusters out of the box
  • O'Reilly book available — free online

Using CouchDB With Grails

  • grails has native json support out of the box

import grails.converters.* class AlbumController { def scaffold = true def listAsJson = { render Album.list() as JSON } def listAsXml = { render Album.list() as XML } } CouchDB 101

  • json up and down
  • restful interface
  • no drivers since it's just http
  • written in erlang
    • incredibly fast
    • designed for scalability and parallel processing

Installing CouchDB

  • sudo apt-get install couchdb
  • windows installer available

Kicking the Tires

  • ping
    • curl http://localhost:5984
      {"couchdb":"Welcome","version":"1.0.1"}
    • can also hit this in a browser, but of course can't do a POST from a URL in a browser
  • get databases
  • create a database
  • delete a database
  • uses standard HTTP response codes, e.g. a 201 response code for a database create
  • web UI available – "Futon"
  • create a document
  • create a document from a file
  • URIs for documents are essentially your primary key–unique way of representing the document
  • don't have to create schemas — just start throwing documents at the database
  • documents get etags so they're very cache friendly
  • documents also get revisions–keeps tracks of multiple versions of the document
    • have to provide version number when updating
    • versioning numbers are revision number (integer), then -, then md5 hash of the document itself
    • can explicitly compress the database to get rid of old versions to reduce size of database
  • couch prefers uuids for the ids, but you can use anything you want
  • get UUID(s) from couch
  • to update a document, you'll get the latest version of the document, then do the update, then pass your changes back to couchdb which includes the revision number
  • one of the major things couchdb gives you since it's document based is that the data is accurate at that point in time
    • if the data changes in the future, in an rdbms the old document would get the new data

CouchDB With Grails

  • domain class–id and _rev as properties
  • can add couchdb stuff to Config.groovy to do stuff like create-drop for couchdb databases
  • add stuff to BootStrap.groovy
  • showing CouchDBService that has convenience methods around a lot of the URL calls to couch

Map/Reduce

  • in sql you say select firstname, lastname from foo (this is map) where state = 'NE' (this is reduce)
  • map and reduce are stored in 2 separate javascript functions

Polyglot Web Development With the Grails Framework #s2gx

Posted on by Matt Woodward

Jeff Brown – SpringSource

Polyglot?

  • "many languages"
  • writing software in multiple languages
  • some people would say if you do any web development, you're doing polyglot
    • javascript, css, html, java, etc.
  • in the context of this talk, we'll be talking about implementing the actual business logic with multiple languages

Languages on the JVM

  • 200+ languages available on the JVM
  • many of these aren't exactly practical, but many are
  • at least 10-12 programming languages available on the JVM that could be used for serious development
  • big players are java, groovy, clojure, scala, jruby, jython
  • which of these is the best? no answer of course
    • personal preference, best tool for the job, etc.
  • many of these languages solve specific problems really well
  • all these languages are turing complete, so anything you can do in one you can do in another
  • but depending on the problem you're trying to solve, you may find one language or another is ideally suited to the task at hand
  • reached a point with CPU development where the speed of light is a factor in terms of increasing the speed
    • can't really make processors any faster with the current method of developing processors
    • instead of making faster processors, we're using more processors and multiple cores
  • concurrency is becoming more and more important
  • OO languages don't lend themselves to managing concurrency very well
    • allocate objects on the heap, objects in a shared mutable state
    • best we can do in OO languages is use locks so multiple threads can't access things at the same time
    • problem with locking is it's opt-in
  • functional languages make the concurrency problem almost disappear
    • no such thing as destructive assignment in a pure functional language
    • clojure and scala *do* allow destructive assignment
    • but in clojure, for example, you have to do this in a transaction
      • get a snapshot of the heap, and nothing can change on the heap while you're making changes
  • because of the advantages in terms of concurrency, it will be more common to use polyglot programming moving forward
    • e.g. write much of the application in groovy, but build parts of the application using a functional language
  • ultimately all this will run on the jvm, but we can take advantage of the best things each language has to offer
  • pretty different from the past–would have been rather unusual to write a C++ program that had other languages mixed in

Grails?

  • full stack MVC platform for the JVM
    • build system down to ORM, etc.
  • leverages proven staples
    • spring, hibernate, groovy, quartz, java, sitemesh
  • extensible plugin system
    • e.g. can pull out hibernate and use a different persistence mechanism, or can write your own plugins
  • since grails is built on the jvm you can take advantage of any language that will run on the jvm

Demo

  • showing how to write a grails app that uses a groovy controller, but a "math helper" class can be written in groovy, or java, or clojure
  • as long as there's a bean in the spring context, regardless of language, it can be injected into grails controllers
  • the grails controller doesn't care what language the classes it uses are written in

Clojure

  • core grails doesn't have clojure support, but there's a clojure plugin
  • plugin creates an src/clj file for clojure source
  • code in the grails controller doesn't have to change to take advantage of the math helper written in the different languages


(ns grails)
(defn addNumbers [x y]
  (+ x y ))

Using the Clojure Plugin

  • call via clj.mathHelper.addNumbers(x, y)
  • clj is the same as calling getClj()
  • the cloure plugin adds the getCjl method to all of the grails classes
    • classes*.metaClass*."getClj" = { return proxy } — this is done in the withDynamicProperties method in the plugin config
    • the proxy is an instance of the clojure proxy class (grails.clojure.ClojureProxy)
  • no addNumbers method in the proxy class — uses methodMissing
  • plugin looks for clojure methods in the grails namespace by default
    • if you need another namespace, clj['mynamespace'].methodName() will handle this
  • clojure plugin declares that it wants to watch all the files in src/clj
    • gets notified when files change and compiles the files if they change
  • if your plugin is adding things to the metaclasses on application startup, then you need to make sure your plugin also modifies controllers, services, etc. as they're changed while the application is running
  • can also observe only specific plugins for changes, e.g. only notify me when something involved with hibernate changes
  • can swap out other view technologies in grails
  • for taglibs, they have to be written in groovy, but inside the taglib you could be making calls to code in other languages

Who gets the credit?

  • grails? groovy? clojure? java? the jvm?
  • really it's the combination of all of them
  • don't have to walk away from grails to take advantage of any of the languages that run on the jvm

Advanced GORM: Performance, Customization, and Monitoring #s2gx

Posted on by Matt Woodward

Speaker: Burt Beckwith, SpringSource

Overview

  • demo of potential performance issues with mapped collections in GORM
  • using the hibernate 2nd-level cache
  • monitoring and managing 2nd-level caches
  • app info plugin

Standard Grails One-to-Many

  • library has many visits
  • visit class has person name and date, with backreference to library

What's the problem?

  • hasMany = [visits:Visit] creates a set
  • sets guarantee uniqueness
  • adding to the set required loading all instances from the database to guarantee uniqueness, even if you know the item is unique
  • likewise for a mapped list–lists don't guarantee uniqueness, but they do guarantee order, so they still have to pull all records from the db to get the order right
  • you get a false sense of security since it's lazy-loaded; only partially helpful
  • works fine in development when you only have a few visits, but imagine when you deploy to production and you have 1,000,000 visits and want to add one more
  • risk of artificial optimistic locking exceptions; altering a mapped collection bumps the version, so simultaneous visit creations can break but shouldn't

What's the Solution?

  • don't use collections
  • instead of visit belonging to a library, visit HAS a library
  • different syntax for persisting a visit
  • no cascading; to delete a library you need to delete its visits first in a transactional service method
  • you also lose your collection so you can't do library.visits.size(), etc. but you can still use dynamic finders,which is better anyway since you're only pulling what you need

Standard Grails Many-to-Many

  • user has many roles, role has many users
  • problem is that if all new users are granted a particular role, you get into scaling issues quickly
  • with many to many you have an intermediate table with pointers to both tables and can map the join table
  • the belongsTo in a many to many can go in either class since it's bidirectional
    • but this is the problem since the same amount of data will get loaded either way
  • more efficient to treat kind of like one-to-many and create the user, then grant the role
    • this way you're adding/deleting single records in a single table due to existence of a domain class describing the relationship
  • important to have well-defined equals() and hashCode() methods in your domain classes, as well as implement serializable so you can use second level caching
  • wind up with user.addToRole() or role.addToUsers()
  • no cascading like before–have to manage this yourself

So never used mapped collections?

  • no, you need to examine each case
  • standard approach is fine if the collections are reasonably small–for both sides in the case of many to many
  • the collections will contain proxies, so they're smaller than real isntances until initialized, but still a memory concern
  • great example of something that's convenient and easy out of the box, but when it becomes a problem, you just do it a different way

Using Hibernate 2nd-level Caching

  • great, but have to  be careful because it can burn you in the same way that a query cache in a database can bite you
  • great candidates for caching–anything that's read only and doesn't change often
  • can overuse cache to the point where you're spending more cycles flushing and aren't saving yourself any db traffic–can actually make things worse than just hitting the db

Caching Usage Notes

  • 1st level cache is the hibernate session itself
  • get is always cached
  • can significantly reduce db load by keeping instances in memory
  • can be distributed between multiple servers to let one instance load from the db and share updated instances, avoiding extra db trips
  • "cache true" creates a read/write cache, best for read-mostly object since frequently updated objects will result in excessive cache invalidation (and network traffic when distributed)
  • "cache usage: 'read only'" creates a read-only cache, best for lookup data (countries, zip codes, etc.) that never change

Query Cache

  • can set cacheable true on all the query options; this caches the query and you can grab class instances from this

Hibernate query cache considered harmful?

  • most queries are not good candidates for caching; must be same query with same parameters
  • updates to domain classes will pessimistically flush all potentially affected cache results
  • DomainClass.list() is a decent candidate if there aren't any (or many) updates and the total number isn't huge
  • great blog post by alex miller at http://tech.puredanger.com/2009/07/10/hibernate-query-cache

2nd Level Cache API

  • evict one instance: sessionFactory.evict(DomainClass, id)
  • can get stats (hits/misses, etc.)
  • can look at the stats and get a good sense of whether or not you're caching effectively
  • e.g. if miss count is high then your cache strategy isn't effective
  • appinfo plugin gives you tons of information about what's going on in your app, what's going on in hibernate, etc.

Location of Grails Webapp When Launched from SpringSource Tool Suite

Posted on by Matt Woodward

I'm having trouble getting SpringSource Tool Suite to compile and deploy a Java class located in a Grails project's src/java directory when I run the application from within STS. This is particularly odd given that I have a ServletFilter in this same location (albeit a different package) that gets compiled just fine, so I decided to dig into things a bit and figure out where STS deploys projects when you do grails run-app. This way I could at least see whether or not the class was getting deployed and if my class not found exceptions were related to some other issue.

Turns out if you watch the console as the app launches you get a big clue. Well, the answer actually. I'm on Ubuntu so in my case, my project was deployed to /home/mwoodward/.grails/1.2.1/projects/PROJECT_NAME/resources, so I poked around in there to discover that it even creates the package for my Java class but alas, there's no compiled class in the package.

At least I confirmed I'm not losing my mind. Now to solve the mystery of why STS isn't compiling and deploying that class.

March GroovyMag Available — Includes My “Grails For Switchers” Article

Posted on by Matt Woodward

Media_httpwwwgroovyma_qcrpj

The March edition of GroovyMag is available and includes an article I wrote entitled “Grails for Switchers,” which is my account of what I learned while first learning Grails.

If you’re at all interested in Groovy or Grails you need to subscribe to GroovyMag. It’s a great way to learn more about Groovy and Grails and keep up with what’s going on in the Groovy/Grails community.

Resolving CSS Issues With Grails UI Plugin

Posted on by Matt Woodward

I'm working on another Grails application and am using the fantastic Grails UI plugin for a lot of the UI controls. Grails UI is a really nice Grails-friendly wrapper around the YUI components and includes things like a dialog box, calendar controls, a rich text editor, and a whole lot more. This was my first real foray into using this plugin, so I started with a simple modal dialog box that would show the contact information details for people in a simple list.

The main point of this post is to outline the simple resolution to the CSS issues I was seeing because it took me a while to figure out what was going on, but I thought I'd outline some Grails and Grails UI magic along the way.

First, in order to use the Grails UI plugin you of course have to install it, which is as simple as:

grails install-plugin grails-ui

Next, on any view page on which you wish to use any Grails UI resources, you have to indicate which resources you're going to use on the page. The nice thing about this is it will only load the JavaScript for the specific UI resources you need on each page. In the case of this example I'm only using a dialog box, so I have this line in the head section of my view page:

<gui:resources components="dialog" />

Also in my head section I need to tell Grails I'll be using some AJAX on this page, so I use the javascript tag to load the Prototype library:

<g:javascript library="prototype" />

With Protoype loaded, pulling the contact details to be shown in the dialog box is dead simple using the Grails remoteLink tag:

<g:remoteLink controller="person"
               action="showDetail"
               id="${person.id}"
               update="personDiv"
               onComplete="showPersonDialog();">${person}</g:remoteLink>

If you're not familiar with Grails, what this does is tells Grails to make an AJAX call to the person controller, call the action showDetail, and pass the ID of the person object. We'll see what's returned by the AJAX call in a moment. The update attribute of the remoteLink tag tells Grails what DOM object to update with the results of the AJAX call, and the onComplete attribute indicates a JavaScript function to call when the AJAX call is complete.

If all I was doing was updating a DIV on the page I wouldn't need this, but since I need to show the Grails UI dialog box after I pull the contact details, I need a JavaScript function to handle that, so I added this to the head section of the view page:

<script type="text/javascript">
    function showPersonDialog() {
        GRAILSUI.personDialog.show();
    }
</script>

Next, let's check out the showDetail action in my Person controller to see what it's doing when the AJAX call is made:

def showDetail = {
    def personInstance = Person.get(params.id)
    if (!personInstance) {
        def message = "No person found with ID ${params.id}"
        render(view:"personDetail", model: [message : message])
    } else {
        render(view:"personDetail", model: [personInstance : personInstance])
    }
}

Here's the simple view that's rendered in the controller action above:

<g:if test="${message}">
    <p>${message}</p>
</g:if>

<g:if test="${personInstance}">
    <p>
        <strong>${personInstance.firstName} ${personInstance.lastName}</strong><br />
        ${personInstance.email}<br />
        Phone: ${personInstance.phone}<br />
        Cell: ${personInstance.cell}
    </p>
</g:if>

And finally, here's the code for the Grails UI dialog box and the DIV that is populated with the view above:

<div class="yui-skin-sam">
    <gui:dialog id="personDialog"
                width="400px"
                title="Contact Details"
                draggable="true"
                update="personDiv"
                modal="true">
        <div id="personDiv"></div>
    </gui:dialog>
</div>

This is all pretty straight-forward. What was happening, however, is when the dialog box was shown, there was no CSS being applied to it. The YUI components that are used by the Grails UI plugin have stylesheets associated with them, but when I checked the source code of the rendered page they seemed to be getting included just fine, and as you can see above I wrapped the dialog box in a div with the correct CSS class, which is yui-skin-sam.

At this point it's important to remember that when a Grails page is rendered it uses SiteMesh, which is basically a templating/page decoration framework. As many Grails applications do, I was using a main.gsp layout page, and each individual view page gets woven into this main template.

Therein lies the problem. As I said above this was simple enough in the end but since it took me a while to figure out I thought I'd share. Even though the YUI CSS was being included in the individual view page with the dialog box code on it, for some reason the CSS wasn't getting applied. I decided to experiment and put the yui-skin-sam class in the body tag in my main.gsp layout page, and this solved the problem.

In my case I didn't have any conflicting CSS involved so this solution didn't cause any issues, but if you have other CSS involved and applying a class to the body tag in the main layout page will cause issues, you can add the additional CSS references after the <g:layoutHead /> tag in the main layout page, and this will allow you to override any CSS that came earlier.

With all this in place the Grails UI components are being styled correctly and they're extremely nice additions to any Grails app.

ThirstyHead: Free Webinar: Getting Started with Groovy, Grails, and MySQL (February 18, 2010)

Posted on by Matt Woodward

Wednesday, Feb 10, 2010

Free Webinar: Getting Started with Groovy, Grails, and MySQL (February 18, 2010)

On February 18, 2010, join Scott Davis on a Sun/Oracle sponsored webinar: Getting Started with Groovy, Grails, and MySQL. We’ll spend some time working with MySQL from Groovy 1.7 scripts (Sql.eachRow(), Sql.withBatch()). Then we’ll switch gears and show you how easy it is to skin a MySQL database with Grails 1.2. Hope to see you there!

If you’re interested in Grails this would be a great one to attend. Scott’s a great presenter!

InfoQ: Getting Started with Grails, Second Edition – FREE!

Posted on by Matt Woodward

Grails is a Java- and Groovy-based web framework that is built for speed. First-time developers are amazed at how quickly you can get a page-centric MVC web site up and running thanks to the scaffolding and convention over configuration that Grails provides. Advanced web developers are often pleasantly surprised at how easy it is to leverage their existing Spring and Hibernate experience.

“Getting Started with Grails” brings you up to speed on this modern web framework. Companies as varied as LinkedIn, Wired, Tropicana, and Taco Bell are all using Grails. Are you ready to get started as well?

The second edition of “Getting Started with Grails” in now available for free on InfoQ, or you can buy the print version for only $22.95. The first edition was great so I’m really looking forward to reading the update.

In Seattle? Into Groovy and Grails? Join the Seattle Groovy/Grails Meetup!

Posted on by Matt Woodward

If you're in the Seattle area and interested in Groovy and Grails, make sure and join us at next month's Seattle Groovy/Grails Meetup on February 11. We get together every month at the Elephant & Castle, and it's a casual mix of Groovy, Grails, other technology, and whatever else comes up.

All are welcome so even if you're not using Groovy and Grails yet, come join us and we can all learn from each other. See you there!

Componentix blog – Run long batch processing jobs in Grails without memory leaks

Posted on by Matt Woodward

In one of our Grails applications we had to run a number of batch jobs. Nothing unusual and Grails supports it quite well with the excellent Quartz plugin.

But when we deployed application in production, we noticed that after running for some time, it consumed a lot of memory and JVM was spending all the time running garbage collection. The reason for it was that our jobs were quite long-running, taking several hours to complete, and Grails wasn’t really designed for such kind of use case.

But “not designed for” doesn’t mean it’s not possible, of course. Great info about how to clear out some of the things that will eat up memory over time in long-running Grails processes.