There have been some rumblings the last few days about the use of design patterns in the ColdFusion world, all of which seem to have been prompted by Brian Rinaldi’s post on the Gateway pattern and its use (some say misuse) in ColdFusion. As usual with things of this nature the discussion got rather heated so I thought I’d add some fuel to the fire. The building’s burning down anyway, so what the heck. Actually I love discussions like this because in my opinion anything that gets people this riled up is a good thing. When you question aspects of development at a very fundamental level you either come out of it with a completely different view, or it strengthens your beliefs in the way you’re already doing things. Either way much is learned through the process.

In any discussion of design patterns, we must first turn to The Holy Book of Design Patterns, namely Design Patterns: Elements of Reusable Object-Oriented Architecture by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, a.k.a The Gang of Four, The Four Wisemen, or the Four Horsemen depending on your perspective. I don’t think the Biblical comparisons are necessarily misplaced because, like the Bible, this book has many companion books that attempt to explain what the heck the contents of The Book actually mean, the most popular of which is Design Patterns Explained: A New Perspective on Object-Oriented Design by Alan Shalloway and James R. Trott. My copy even has one of those fancy attached ribbon bookmarks in it just like many Bibles. Also, as with religion people get extremely passionate (perhaps unnecessarily so–it’s just software after all) about their design pattern beliefs which makes the conversation all the more interesting and in some cases more than a tad bit irrational. Fun stuff.

As a backdrop for this discussion, I’d like to offer up the very first sentence in the introduction of the Gang of Four (GoF for short) book. I do this as a means of ensuring that none of us thinks there is a quick, simple answer to any of this.

“Designing object-oriented software is hard, and designing reusable object-oriented software is even harder.”

Now that we know we have an arduous task ahead of us, let’s get started with the discussion at hand, the shot heard ’round the world, namely the now-infamous Gateway pattern.

So what is a Gateway?

I think Brian covered this pretty well, and he made some very good points about how “Gateways” in ColdFusion don’t necessarily need to be the same thing as Gateways in Java, .NET, or any other language. I do find it funny that some of the same people that are saying ColdFusion shouldn’t become Java seem to get bent out of shape that we aren’t implementing Java-based design patterns exactly the same way in ColdFusion as they exist in Java. I suspect this may be a semantic argument at some level (how about we call the ColdFusion implementation of this pattern DataSucker and move on?), but I digress.

Many ColdFusion developers, myself included, use objects we call Gateways to encapsulate access to a database table, specifically to retrieve multiple records (meaning methods will typically return either query objects or collections of beans) from specific database tables. Is this a bad thing? Does it violate some commandment that states what a Gateway is? Let’s investigate.

To understand what a Gateway is, let us crack open the GoF tome. (Come on, admit it, you bought this book a while back but the binding still cracks when you open it. Mine sure does.) Turn to the index at the back of the book and look for the word “Gateway.” Nothing in your copy? Mine either. Must be a misprint. Let’s try “Table Gateway,” because I think I saw reference to this pattern using that name somewhere. Hmm … nothing there either. My point, specious as it may be, is that the word “gateway” isn’t even important enough to appear in the index of the Bible of design patterns. Either that or they hired a really cut-rate indexer.

We’re going to have to look through the entire book ourselves to find the mention of “Gateway” apparently. Go ahead. I’ll wait.

Didn’t find it? Didn’t think so. Me neither. That’s because–surprise!–it’s not in there. Why not? Well, my own personal theory is because the GoF are higher-level thinkers than something so specific as a Gateway as they’re currently implemented in any language. That being said, the Gateway pattern isn’t necessarily as concrete as many of us make it, but we’ll address that in a moment.

First, in the absence of any mention of Gateway, let’s look at an example of what the lofty GoF do consider to be a pattern important enough to be included in their revered book. I’ll just choose one more or less at random.

“Pattern: Bridge. Intent: Decouple an abstraction from its implementation so that the two can vary independently.”

Based on that definition, if I told you to go build a bridge right now could you do it? If you say “yes,” you’re either reading far too much into the pattern, you’re way smarter than I am, or you thought I meant for you to build a bridge for your Matchbox cars out of popsicle sticks. I suppose my point is that, in the GoF book at any rate, design patterns are so high level that while they’re very useful, the say nothing about any specific implementation details. If we could find the word “Gateway” in the GoF book, chances are it wouldn’t say anything about hitting database tables, what data types the methods should return, or anything of the sort. Their “intent” description for Gateway, if I may be so bold as to put words in their mouths, would likely be something like “Enapsulate access to an external resource.” Keep that high-level definition in mind because we’ll come back to it.

The GoF book does provide examples to accompany all their patterns, but remember these are examples that are designed to illustrate one potential implementation of the high-level design pattern. Furthermore, the examples provided are in the context of building a desktop document editor in Smalltalk. In my opinion their examples shouldn’t be (and I’ll go so far as to say weren’t intended to be) taken as gospel. The authors wisely provide examples because, as we see with the Bridge pattern, it’s a very lovely pattern but doesn’t become very concrete in the mind of developers until we see an example. Examples are indispensable to shedding light on the abstract intent statement of the design. But again, this is an example, not the example.

This brings to mind a bit of an aside, which is one of the key problems I have with the recent posts by Sean Corfield, Peter Bell, and some of the comments on Kola Oeydeji’s post (that itself was a response to Brian’s original post), namely that they seem to stop quite a bit short of being useful in any practical sense. Saying “people need to review things more” or “I do things completely differently” or “this is a pet peeve of mine” doesn’t illuminate anything. I know, we’re all busy people, but a specific example such as “I don’t use a gateway that way–here’s what I do” with some concrete details would add far more substance to the discussion.

I suppose these comments are useful in the limited sense that they urge us to think about what we’re doing. That’s fine as far as it goes. I think a lot about what I’m doing, however, and I personally find using the Gateway pattern as it has come to be implemented in ColdFusion by numerous developers quite handy. I like it. It’s cohesive, it’s easy to explain to others, it serves a clear purpose, and has a clear intent. Saying “but it’s not really a gateway” doesn’t much help the discussion, other than to cause the people who use this pattern and call it a “gateway” to defend their use of it, which as I mentioned at the outset of this post is actually useful for the purposes of facilitating discussion. Will I change my thinking on this in the future? Perhaps. It’s certainly happened before. The “it’s not really a gateway” comments don’t, however, tell me why I shouldn’t use it in this way except in an academic sense, and the validity of the academic argument itself is pretty dubious as well see momentarily.

This brings us back to the question concerning what a Gateway is. Since it’s not in the GoF book we have to look elsewhere. The obvious next avenue of investigation in my mind is Martin Fowler.

Gateway: A Definition From the Book of Fowler

For whatever reason, unlike the GoF, Fowler does grace us with a definition of Gateway, which comes from his book Patterns of Enterprise Application Architecture. I have to pat myself on the back just a bit, because Fowler’s definition is remarkably similar to the one I came up with above:

“An object that encapsulates access to an external system or resource.”

As we read on, things get even more compelling:

“Interesting software rarely lives in isolation. Even the purest object-oriented system often has to deal with things that aren’t objects, such as relational data-base tables, CICS transactions, and XML data structures.”

Relational database tables! Now we’re getting somewhere. At a high level this definition, which would be the “intent” section of the GoF book, seems to fit with getting data out of a database, and he goes so far as to give the scenario of retrieving data from a relational database as one possible implementation of this pattern. So far so good. I think at this point we can all agree that an object that retrieves data from a database, at least in Fowler’s view, is a perfectly legitimate implementation of the Gateway pattern.

Like the GoF, however, Fowler doesn’t drill down into specifics just yet. As we look further into Fowler’s pattern definitions we see that there’s another instance of the word “gateway” in his pattern definitions, specifically a pattern called the Table Data Gateway, which he defines as follows:

“An object that acts as a Gateway [he references his more generic Gateway pattern here] to a database table. One instance handles all the rows in the table.”

So now we see what I consider to be a more specific instance of the generic Gateway pattern. A Gateway merely “encapsulates access to an external system or resource,” while the Table Data Gateway pattern explicitly defines the “external system or resource” as a database table. Again, there is no real discussion of the specifics of how the Gateway achieves what it does, but Fowler does provide some sample methods that might be contained in a Person Table Data Gateway such as find(), findForCompany(), insert(), and update(). Pay particular attention to the insert() and update() methods, because they’re key to the discussion that follows.

To summarize where we are at this point, we now know that the Gateway pattern (more specifically the Table Data Gateway pattern) is indeed a perfectly acceptable way to encapsulate access to a relational database. This of course isn’t the only way to do things, however, so now we’ll examine an alternative pattern, compare and contrast, and discuss how the two might not be mutually exclusive.

Data Access Objects (DAOs)

The other pattern many of you may have seen referenced in your OO trials and tribulations is the Data Access Object (DAO) pattern. The most thorough description of the DAO that I’ve come across, and the one to which most people point when they’re discussing this pattern, comes from Sun’s “Core J2EE Pattern Catalog.” So how is the DAO pattern different from the Gateway pattern? Let’s take a look.

After describing the issues with data access in general and the goals of abstraction and encapsulation, Sun’s proposed solution is to “use a Data Access Object (DAO) to abstract and encapsulate all access to the data source.” Sounds an awful lot like the Gateway pattern, doesn’t it? Sun’s definition continues, “The DAO manages the connection with the data source to obtain and store data.” So the basic idea is that the DAO encapsulates database access.

As a point of comparison, let’s prepend the term “DAO” to Fowler’s definition of the Table Data Gateway pattern:

The DAO is “an object that acts as a gateway [lowercase mine] to a database table. One instance handles all the rows in the table.”

Even by combining the two we see that the general definition remains consistent. So at this point, and considering things only at a high level, the Table Data Gateway pattern and the DAO pattern are essentially the same. They both are defined as objects that encapsulate access to a database table, so it’s really just a difference of nomenclature. Furthermore, if you scroll down to some of the examples that Sun offers on their DAO definition page, they also show methods that deal with both single and multiple records, meaning that just as with the Table Data Gateway, you see both an insert() method, which deals with a single record, and a selectCustomers() method, which handles multiple records, in the DAO.

Table Data Gateways vs. Data Access Objects (DAOs)

At this point you might be confused. Don’t worry. Whether or not everyone admits it, you’re not alone. To review, at this point both the DAO and the Gateway, from a pattern standpoint, serve exactly the same purpose. They both encapsulate access to a database.

So why in many ColdFusion applications do we see both Gateways and DAOs, for example both a PersonGateway and a PersonDAO? To review something I mentioned above, the Gateway pattern is used by many ColdFusion developers to contain methods that handle multiple records. The DAO, on the other hand, is used in many ColdFusion applications to deal solely with single records; many ColdFusion DAOs contain only create(), read(), update(), and delete() methods (affectionately referred to as “CRUD”). To elaborate, in this scenario the PersonGateway would contain methods such as search(), getAll(), and other methods that return multiple records. The PersonDAO would contain the CRUD methods and typically nothing else.

Since in the canonical sources mentioned above (i.e. Fowler and Sun’s pattern catalog) the Gateway and DAO are in essence the same thing, where did this distinction in the ColdFusion world originate? I know specifically where it originated for me, and likely for many others as well: the Macromedia Mach-II Development Guide. Since Mach-II was the first fully OO framework for ColdFusion, I suspect that many others (like myself) learned not only Mach-II from this resource, but the relevant OO patterns outlined therein.

Because in my opinion this particular point is so tremendously pertinent to the discussions happening in the various blogs mentioned above, I’m going to quote the relevant section from the Mach-II Development Guide nearly in its entirety. This is not intended to point any fingers at anyone, least of all Sean Corfield who authored the original Mach-II Development Guide, but I do think it answers the question of where the distinction originated for many ColdFusion developers, myself included.

— Begin quote from the Mach-II Development Guide —

There are two basic patterns of access to persistent data within most applications:

• aggregated access – reporting, searching, listing multiple rows,
• per-object access – creating, editing, working in depth with a single row (object).

ColdFusion has a great built-in idiom for dealing with the first type of access  – the query – which is an efficient way to manipulate (potentially large) sets of data rows retrieved from a database (or other data sources, since you can easily create a query object and populate it with your own data). When you are dealing with aggregated access, it does not make sense to convert every row returned into a fully-encapsulated object (CFC instance) when all you are likely to do with the data is display a few fields with a link to a detail page that will focus on the selected row.

On the other hand, when you are focusing on a single row it usually does make sense to work at the fully-encapsulated object level since you are usually interested in object behavior at that point. This is also the level where you need the standard CRUD (Create, Read, Update, Delete) operations.

Recognizing these two basic patterns, you should design your components accordingly by providing separate components for each pattern. This is best explained through an example:

• If we have a business model object called Order, then we would provide an OrderGateway component for aggregate access and an OrderDAO component for per-object access (or build the per-object access into the Order object – but see below).
• The OrderGateway component would provide methods like findAll(), findWhere(), findByID() and they would all return standard query objects (even findByID() which returns a single row).
• The OrderDAO component would provide CRUD methods like store(), load(), update(), delete() and they would operate on a specific Order object exchanging data via getters/setters on the Order component or via some sort of snapshot of the Order’s data, e.g., a bean: the Order component could implement methods like getSnapshot() returning a bean and setSnapshot() taking a bean as an argument – the bean containing the core persistent data for the Order object. Sometimes a more direct data transfer between the business object and the data access object is needed for performance reasons (or because the application can ‘trust’ the components to exchange less encapsulated data, such as a struct or some opaque data structure e.g., the Memento design pattern). Such optimizations are beyond the scope of this document.

Separating out these operations from the business model object helps it stay persistence-neutral. The gateway components can be optimized for retrieving large record sets, caching etc. The DAO components can be optimized for dirty data updates, pooled object access and so on. Again, the details of these optimizations are beyond the scope of this document but providing for the two distinct patterns of data access will get you started on the right road.

— End quote from the Mach-II Development Guide —

To reiterate, the sole purpose for the inclusion of this quote is to point out where I suspect many ColdFusion developers got the idea to separate Gateways from DAOs in this way. Given the recent blog posts and comments, clearly Sean thinks differently about these things now. That’s perfectly fine. If we stop thinking about things and changing our minds about things, we stop increasing our knowledge and may as well find another line of work. I personally want to thank Sean for creating this guide not only because I learned so much from it, but because it served as a fantastic foundation on which to base an updated version of the Mach-II Development Guide (forthcoming) for the newer versions of Mach-II.

Also bear in mind that Sean’s intent with this guide was not, I assume at any rate, to attempt to provide ColdFusion developers with a design pattern Bible that should be followed to the letter, so it is incumbent upon all of us to study the design patterns on our own and make up our own minds about them. For me, the Mach-II Development Guide was a great starting point for my learning process. I’ve since learned a great deal more about numerous other design patterns and OO concepts, and I continue to learn more every day. Remember: this stuff is hard to begin with, and doing it well is extremely difficult. If we keep doing things the same way and don’t change our minds about things as we increase our knowledge, then we’re standing still as developers, so we shouldn’t see it as an error or failure on anyone’s part if they said something on their blog three years ago that they no longer agree with.

But Aren’t You Doing It Wrong?

Given everything I outlined above, specifically how both Fowler and Sun illustrate that methods dealing with single records and methods dealing with multiple records coexist in either the Table Data Gateway or the DAO design pattern, why do I still split out my data access methods into two separate objects? In short, because I can.

Blasphemy! Well, it’s actually not only because I can. It’s because I like to, and given Joe Rinehart’s recent blog post entitled “Defending the Gateway Pattern in ColdFusion,” apparently I’m not the only one.

Maybe this all stems from me latching onto Sean’s description from the Mach-II Development Guide at a formative stage of my OO ColdFusion development, but as my knowledge has grown I still think there’s a tremendous amount of validity to using the Gateway pattern for multiple records and the DAO for single records. No, you won’t find any real justification in any fancy books, but in my own real-world development I completely agree with Joe. It’s served me well, I’ve never seen any negative impact by doing things this way, I like the separation of the two concerns, and no one’s convinced me that it’s wrong to do things this way. Even if there are valid arguments to the contrary, they’re still just academic, because the bottom line is these patterns work and continue to work well in many of my applications.

Now I’ll get extremely geeky (and getting geeky while writing about OO design patterns is no small feat!) and quote one of my all-time favorite shows, The X-Files:

Deep Throat: “Mr. Mulder, why are those like yourself … not dissuaded by all the evidence to the contrary?”

Mulder: “Because all the evidence to the contrary is not entirely dissuasive.”

Ah, such wisdom from my hero of the 1990s. So why is this seemingly random quote appropriate? Because even given the definitions of such giants of the design pattern world, I’m not convinced that separating data access into two objects is “wrong.” Conversely, I’m definitely not convinced that taking these two separate objects and combining them into one is the right thing to do. Many of my thoughts echo what Joe already said, but I’ll put them in my own words and expand on them a bit.

First and foremost, I like knowing that my DAOs deal with CRUD and that’s it. Once those objects are done they’re done, and these types of objects are also fantastic candidates for code generation precisely because they are so singular in purpose and prescriptive in nature.

Gateways, on the other hand, are far more nebulous and end up changing much more during application development. CRUD operations are easy to define. As you build out an application, however, typically you’ll find yourself coming up with methods that need to be added to the Gateway that you didn’t know about at the outset. Maybe the requirements change as the application evolves, maybe you figure out different or better ways to do things, who knows. Regardless of the reason, in my development I find that I add, remove, and alter methods in my Gateways pretty regularly right up until the application is complete. My DAOs, on the other hand, are built at the outset and they just sit there doing their job.

I also like knowing that when I call methods in my DAOs, they take in an instance of the object with which they are concerned (i.e. the bean) and don’t return anything since they’re operating directly on the objects passed in. Gateways, by contrast, typically return query objects or, when necessary, collections of beans. It’s been lamented in some other recent blog posts that if we don’t return a collection of objects, again we aren’t using this pattern “properly.”

Says who? Fowler? Sun? Now maybe in Java the convention is that you return a collection of objects, but this is ColdFusion, not Java. In ColdFusion we have a wonderful little object called a query that is extremely efficient and darn easy to use. So why not use it? If all I want to do is loop over a query and display the data, and I don’t need to interact with each individual object, why on earth would I make things more complex and less efficient just so I’m better meeting another technology’s definition of how a Gateway is supposed to behave? Pure nonsense. When I need to deal with each individual object in an aggregate dataset, I can quite easily return an array of CFCs. When I don’t need to do that, I won’t bother. Some call that bad programming; I call it practical.

Some might argue that having two objects where one will suffice is a bad thing. In one of Sean’s comments to Kola’s blog post, he makes the point that by combining some of the concerns via “responsibility-driven design” as opposed to “data-driven design” (I take issue with calling the way I do things “data-driven,” but I’ll leave that for another post), that he winds up with “28-46 CFCs + Transfer compared to 100-150 CFCs which the ‘5:1’ process [meaning controller, service, bean, DAO, and gateway for each object type in the system] creates.” In some cases less is more. In OO design this argument doesn’t necessarily hold water. First of all, Transfer eliminates the need to create many of these objects because it does it for you. They still exist in some form or another, it’s just handled by Transfer.

Regardless of the details, I personally would rather have more objects with fewer methods in each as opposed to fewer objects with more methods in each, and worse, methods that may not be as related to one another as they are in smaller, more cohesive objects. I realize I’m speaking in generalizations a bit here, but I hope you see my point. The “work” of the application needs to be done somewhere. Fewer objects doesn’t necessarily mean you’re saving any work because if the applications behave the same and contain all the same business processes, then in general terms the same work has to get done regardless of how you organize the objects.

There are also those who believe that based on the strict definition of these design patterns, or maybe just their personal preference, that there should be a single Gateway or DAO as opposed to both existing and serving different purposes. In my mind there’s nothing wrong with that approach, it’s just not my preference. Again, if the only argument is “fewer objects means easier maintenance,” that’s not a valid argument in my opinion. Either approach will achieve the goal of abstracting and encapsulating data access, I just prefer having slightly more cohesive objects, and Mulder’s statement holds true: all the evidence to the contrary is not entirely dissuasive.

If I’m Doing It Wrong, I Don’t Want To Be Right

Maybe that’s overstating my case a bit, but as of the time of this writing I still like having both a Gateway and a DAO in the mix, and I have yet to see a convincing enough argument to make me change my mind. Maybe I will at some point, or maybe I’ll figure out another way to handle all this that I like even better. Then everyone can point out how gosh-darn eloquent and convincing I was in this post and call me a flip-flopper. I see a career in politics in my future.

I think the biggest problem I had with the recent discussions occurring on the various blogs I mentioned earlier is that many developers who haven’t been working with these concepts very long, and particularly those who are just getting into OO development, may be scared off. To anyone who was scared off, don’t be. Be brash. Do things completely and utterly wrong and ask for help. Use an utterly inappropriate design pattern and make someone tell you why your implementation isn’t right. Crash and burn miserably, then get up and try it again. That’s the only way to learn.

The other thing to remember is that the end goal of development is not to use design patterns “correctly,” it’s to put a functional application into the hands of your users. If your applications work, no one (especially an end user!) is going to know whether or not you’re using the “right” design pattern, and they’re certainly not going to know if you’re using something like a Gateway “incorrectly.” If the application works and the objects are cohesive and well-organized, you’ve likely done your job well, and even another developer can jump in and immediately understand your code.

If you’ve stuck with me to this point, here’s the one pearl of wisdom I was really trying to get across in all of this: overanalysis leads to paralysis. What? I just made you read several thousand words for some cliche fortune-cookie quality wisdom? You betcha. In all seriousness though, it’s very easy when reading academic discussions such as those referenced above to get caught up in the right and wrong of it all and not try anything. You’re only wrong if you don’t try.

I’ll wrap this up by saying that while none of the blog posts and comments therein caused me to change my mind (yet …), I truly appreciate the discussion. It gets all of our minds going, it makes us justify our opinions, offer alternatives, try new things, and grow as developers. And with this post you have yet another person to target if you do things differently or think that my analysis or the way I do things is utterly wrong. Comment away!

Comments