In Seattle? Into Groovy and Grails? Join the Seattle Groovy/Grails Meetup!

If you're in the Seattle area and interested in Groovy and Grails, make sure and join us at next month's Seattle Groovy/Grails Meetup on February 11. We get together every month at the Elephant & Castle, and it's a casual mix of Groovy, Grails, other technology, and whatever else comes up.

All are welcome so even if you're not using Groovy and Grails yet, come join us and we can all learn from each other. See you there!

Software Development: Doing It Scared | Our Blog | Box UK

In software, we are often faced with things that can seem intimidating. We may be assigned a project that we know nothing about. We may have to use a new tool or environment. We may be working with people who can seem intimidating. The natural temptation is to retreat to a place of safety, but that wonโ€™t get the job done. So how can we, as software developers, overcome the instinct to cower when projects seem daunting? Iโ€™ll go over a few things that have worked for me under these circumstances; your mileage may vary

This is an absolutely fantastic post outlining how to get your lazy self (hey, I’m including myself here) outside your comfort zone. When I first started working with Grails I tweeted something like, “I have no idea what I’m doing and it’s a great feeling,” and this blog posts captures that same sentiment in a more practical way. I really love the “do it scared” mantra. Words we developers should live by more often than we do.

Pragmatic Dictator ยป Tech is for Sissies

So often I see companies create job specs for engineers where the key requirement is to hire someone who can hit the deck coding like mad using whatever tools have been selected. To that end they load the specs up with endless tech hubris and at interview ask the details of this or that bit of syntax or API call. But what about the next project within the company where the tech is different? All those engineers that just got hired are now useless, they donโ€™t have the skills and we lose time whilst they learn. Or we could fire them and hire another lot?

Couldn’t agree more. The longer I’m in this business the more weary I grow of one-trick technologists. Yes, I think it’s important to spend a lot of time with a few select tools so you don’t fall into the “jack of all trades master of none” camp, but frankly better that than complete inflexibility or unwillingness to try new things.

Enterprise Java Community: Programming is Also Teaching Your Team

Programming has two goals.

One goal is to do something, of course: calculate an amortization table, present a list of updated feeds, snipe someone on Ebay, or perhaps smash a human player’s army. This goal is focused at a computing environment.

The other goal is – or should be – to transfer knowledge between programmers. This has a lot of benefits: it increases the number of people who understand a given piece of code, it frees a developer to do new things (since he’s no longer the only person who can maintain a given program or process), and it often provides better performance – since showing Deanna your code gives her a chance to point out where your code can improve. Of course, this can be a two-edged sword, because Deanna may have biases that affect the code she writes (and therefore, what you might learn.)

Nice reminder of one of the big goals of programming in my mind. Developers often get so busy that we furiously code with an eye only to the short-term end goal, while neglecting the longer-term goals of the development team as a whole. Deadlines may get met this way, but it’s damaging over the long haul.

The Passionate Programmer and Value Rigidity

I have to thank Mike Brunt for the mention of Chad Fowler’s The Passionate Programmer on Twitter (yes, occasionally worthwhile stuff crops up on Twitter …), because it’s one of the best programming books I’ve read in recent memory. It has nothing to do with coding or a specific technology, but I strongly believe it’s one of the more important and impactful books a programmer can read, regardless of their choice of technology.

Personally I love reading things that cause me to question at a very fundamental level what I’m doing and why I’m doing it. If you don’t do this every once in a while you may find yourself off in the metaphorical weeds, having a hard time getting back on track or even figuring out what direction you’re supposed to be going. The Passionate Programmer made me question something about what I’m doing and where I’m going with every chapter.

Coincidentally enough I watched the documentary TED: The Future We Will Create last week. It was inspirational throughout, but there was a particular moment that resonated with me. One of the TED prize winners that year was Cameron Sinclair, who is the cofounder of Architecture for Humanity, which advocates for “building a more sustainable future through the power of design.” One of the more amazing examples he gave in his TED talk was a building that was planted (as in using seeds), grew to 14 feet in height in a month, and then could be eaten when the building was no longer needed.

That’s extremely cool in and of itself, but another architect at TED that year was Rem Koolhaas, who designed the Seattle Public Library. I haven’t been to the library since I moved here, but after seeing this film I looked into it more, and it’s an amazing piece of modern architecture, both in terms of its visual impact as well as how the space is designed to suit the specific purpose for which the building was built.

In a brief interview with Koolhaas in the TED film, he said that the presentation that impacted him the most was Sinclair’s, because (and I’m paraphrasing here) it made him fundamentally question his approach to architecture and what he was doing with his skills. Coming from a guy who built such an acclaimed building (and I’m sure he’s done many other great things in his career), that’s rather astounding.

So back to programming. The Passionate Programmer is fantastic throughout, but what hit me like a ton of bricks, and what made me think of the TED documentary, was a chapter entitled “The South Indian Monkey Trap.” Without going too much into the reference, this chapter makes the point that “rigid values make you fragile,” and that fragility is definitely not a good thing for a programmer. Quoting the book:

“For example, it’s easy to get hung up on technology choices. This is especially true when our technology of choice is the underdog. We love the technology so much and place such a high value on defending it as a choice for adoption that we see every opportunity as a battle worth fighting–even when we’re advocating what is clearly the wrong choice.”

I’m sure folks in the CFML community recognize the tendencies here. Clearly I’m not saying CFML is a bad choice–far from it. I wouldn’t continue to use CFML if I thought it was bad. But one of the points I made in my “Best of Both Worlds” presentation at cf.Objective() this year (you can watch it here if you missed it), is that while CFML is a great choice for a lot of things, we do ourselves a disservice if we get too myopic on our technology choices. If the only tool you have is a hammer …

Now on the flip side of this, you may look around and find that seeing what else is out there reinforces your notion that CFML is a great tool for most of what you do. Even if you come back to where you started, however, there’s a tremendous amount of value in the exploration.

The other thing I love about The Passionate Programmer is that each chapter ends with an “Act on it!” section. In the case of this particular chapter the actions suggested are:

  1. “Find your monkey traps,” meaning identify your rigid values and do some soul searching on them.
  2. Know your enemy: pick a technology you hate, and build something in it. Not only does it help knowing what you’re fighting against more intimately, you add a new skill to your toolbox and might even find that the technology you hate does some things pretty darn well.

So that’s some of what’s been going through my head lately. Agree or disagree, I really do encourage you to read The Passionate Programmer because it will inspire you far more than I possibly can in a blog post overview.

Comments

Just added it to my Amazon Wishlist, thanks for pointing this out. “The Pragmatic Programmer” is another good one from the same publisher.

Posted by Ryan Stille @ 8/6/09 8:23 PM

Sounds interesting. I hope I will find the time to check out this book.

Shlomo,

http://www.shefertech.com

Posted by Shlomo Schwarcz @ 8/6/09 9:31 PM

I’ll have to check it out. On a side note, TED talks are just the most amazing thing since sliced bread. Clark Valberg turned me on to them a year ago. It’s like candy for my brain.

Posted by Ben Nadel @ 8/7/09 6:49 AM

Hi Matt,

I got the original version of the book when it first came out (My Job Went to India: 52 Ways To Save Your Job) and remember an interview with one of the prag publishers (I think it was Andy Hunt) who said he wished they’d picked a better title because they felt the title killed the sales of the book.

I’m glad they re-released it the other month with a new title, and I think it’s a great read. Hopefully with the new name it’ll get the sales it deserves (well, the new title, and of course the plug on your blog ๐Ÿ™‚ ).

Posted by Peter Bell @ 8/7/09 7:02 AM

I was looking at “The Passionate Programmer” on the Pragmatic Programmers site last week, along with “Pragmatic Thinking and Learning”. I would have bought them both by now, but my strategy for managing my spending is to keep telling myself whenever I want to buy something that “I don’t need it yet”. This post (along with the fact that today is my pay day) has convinced me to go ahead and get them.

To correct Ryan’s comment, The Pragmatic Programmers (Andy Hunt & Dave Thomas) wrote the book “The Pragmatic Programmer” but are not the publishers for it, as they wrote it before getting into the publishing business. The biggest consequence of this is that there is no eBook version available of that title. I believe Dave & Andy have made a point of selling DRM-free eBook versions of all the titles they publish, which I think is pretty darn cool.

Posted by Dennis Clark @ 8/7/09 9:46 AM

Avoid Refactoring Yourself Into a Corner

We have developer meetings twice a week at work, and we use these as a forum for people to show things they’ve been working on, ask for advice on how to approach solving a particular problem, share cool tips and tricks they learned, etc. This has been working really well and fosters what I think is the most important aspect of any development team: learning from each other. Regardless of the various skillsets and experience levels on a team, people have different areas of expertise and I learn something new or start thinking about something differently at every one of these meetings.

Today’s meeting was particularly interesting and since it’s not an uncommon situation, I thought I’d share a bit about it. A co-worker has been working on an application for the past few months and this application has a very hard deadline. Without going into all the specifics, based on the client’s own description of how things should work, this is a person-centric application, meaning everything in the application revolves around data associated with an individual person. Each person has basic demographic data but also a minute-by-minute timeline associated with them, and this timeline consists of events that may either be unique to the individual person or may be inherited from a parent group that is used to organize individual people.

We’ve discussed this application quite a bit at our developer meetings lately because it’s been an interesting exercise in developing with Mach-II, but more importantly it’s been a great case study for effective domain modeling in an OO application. It’s simple enough that it’s not overwhelming from a learning standpoint, but it’s also complex enough and has some interesting wrinkles that take it well beyond the level of the basic OO applications that we’ve likely all studied or seen in conference presentations a thousand times. It’s been great because even the people not involved with the application are learning a lot through the discussions at the developer meetings.

The basic overview of the domain model is there is a Person object, which is a typical bean, and one of the attibutes of a Person is their timeline, which for ease of use and efficiency is a query object. When a Person is instantiated they get their timeline, which is created from events explicitly associated with that individual person and potentially events they may inherit from their group timeline. This works very well in dealing with individual people and their individual timelines.

Just recently a new person on the client side was added to the mix. (If you’re saying “uh oh” at this point, you know where this is going.) This person looked at the application and from her perspective, everything was backwards. Because of her concerns with the data being managed by the application, it shouldn’t be person-centric, but rather should be timeline-centric since her role in things is to be focused exclusively on the high-level aggregate timelines as opposed to the timeline of any individual person.

Natually this was a bit of a “D’OH!” moment. While it would have been nice to have this perspective from the outset and potentially would have changed the approach taken in building the application, the reality of the situation is that we have to get the application done very soon and we have to satisfy the new requirement in order for the application to be useful. Such is life as a developer.

Since the application has been to this point entirely person-centric, my co-worker was going down the path of thinking of aggregate timelines as a grouping of all the individual timelines, which on the face of it might not seem like a bad approach. The issue is that there are a minimum of 1500 individual timelines with a potentially large number of events per timeline, and since they may change extremely frequently, caching isn’t a realistic option. Obviously instantiating 1500 objects just to get at query data would be a nice way to bring the server to its knees rather quickly.

Another great aspect of developer meetings is the chance to get a completely fresh perspective. It’s easy when you’re neck deep in an application to have your thinking get a bit rigid and not be able to see the numerous other ways in which the problem you’re facing might be solved. So you start going down straight-line path you see in front of you and figuring out ways to tweak what’s already there to handle a new requirement that may well be the antithesis of what your domain model was originally designed to address.

My co-worker described this situation as “refactoring himself into a corner” and I think that’s a really apt description of what tends to happen in these situations. Partciularly given the recent discussions about all things OO in the CFML world, this seemed rather timely because my co-worker was thinking all about the objects and the domain model and not about the data. I know, you can probably quote me from any number of my conference presentations saying that’s the way you should think about things, and I stand by that … for the most part.

In this situation, however, what we’re really after has nothing to do with the domain model (well, little to do with the domain model at any rate … more on that in a moment) and everything to do with reporting on aggregate data. These are two completely different ways of looking at things, and going down the object path when what we really want to do is produce a report starts to feel wrong rather quickly. If something feels wrong don’t ignore that feeling, because it probably feels wrong for a reason.

Luckily since my co-worker did focus on the domain model so thoroughly in the initial design of the application, the data itself was all nicely organized in the database. This was really is a side-effect of the domain model design as opposed to a data-centric approach to building the application. (Not enough time to go into detail on that point so I hope it makes sense at a high level.)

With the data sitting in the database, why worry about the Person side of things and all those objects? My suggestion was to forget about all that and focus on the end goal of what we need for the timeline/reporting side of the application. Clearly for Person and individual timeline management what has already been developed is important and will continue to be used as it always has been, but timeline reporting is a different requirement and needs a different approach.

Once you throw all the person-centric bits out the window it becomes a lot more clear. The data’s in the database, so rather than thinking of an aggregate/master timeline as being made up of individual people, why not just leverage the database to do the reporting? Maybe this calls for a view to be created, or maybe it’s a rather simple query/stored procedure, but since what we want is a big list of data, creating that from a huge number of individual objects just doesn’t make sense.

Now that doesn’t mean that OO goes out the window completely. Since this is a Mach-II application, chances are this will mean a Listener, a Service, and a Gateway in front of the raw SQL that does the reporting, which means the reporting functionality will integrate nicely right alongside the person-centric part of the application that’s already done and working well. More importantly, it gets down to using the right tool for the job. Databases are (shocker) great at managing data, so putting that work in the database will not only make things a lot more efficient than shoe-horning all the individual beans into a construct where they don’t belong, it just makes more sense overall. In short, think in objects unless you shouldn’t. ๐Ÿ˜‰

I’m not sure there’s a real moral to this story other than to point out that when you feel like you’re refactoring yourself into a corner, stop, take a deep breath, and force yourself to think about things differently. Throw out all your preconceived notions and if that’s difficult, grab a team member or three and talk through the problem with them. It’s almost better if they don’t know much about your application because they won’t have the mental baggage that you do about things, and won’t have any concept of how much rework is involved to implement their approach. And once you have them sucked in you can always get them to help you build the missing pieces!

Comments

Nice! ๐Ÿ™‚ That notion of having “baggage” that keeps you from seeing the simple solution applies to much of life. I don’t have any URLs handy right now, but I know that there’s been at least one study that shows that the more experience a person has in a given field, the less able they become to learn new or different approaches because there’s so much reinforcement behind the techniques they’ve already learned.

And in this case also dovetails rather nicely into a comment Adam Haskell made a while back about one of the nice things about Fusebox being that you don’t have to write objects for things like a report page where there’s not much behavior going on.

Posted by ike @ 11/4/08 12:36 PM

Very good read. I think the comment near the end sums up my feelings:

“More importantly, it gets down to using the right tool for the job.”

If refactoring is the tool that is called for, so be it. Just know that it isn’t the tool *in every case*.

Posted by Shawn Holmes @ 11/4/08 2:21 PM

Great piece, and definitely not a rare situation. Think it bears pointing out that just because the solution isn’t going to be use of the existing objects in the existing domain model, that doesn’t mean the solution isn’t going to be OO. As you noted, the solution to the aggregate reporting is still going to be handled in MachII. The issue really is that the domain model is being extended, right? Because it’s a new requirement, it can be handled by a new domain model, a set of objects that just happen to consume the same data as the original app is already capturing.

I think that can be such a key as well: simply understanding when a change really does impact an existing requirement and when it doesn’t. Often the ability to extend your model to add a B option to your existing A option is entirely reasonable, and that ability to extend without destroying is one of the biggest benefits to OO modeling.

Posted by Jason Fisher @ 11/5/08 5:34 AM

As I am trying to dive deeper and deeper into OO, I am definitely seeing that there is a serious difference between what one might consider as being part of the domain model and one might consider to be reporting.

Not too unlike your scenario, I was going through a thought-experiement in which I imagined a Library object. And I asked myself, Should object have a GetBooks() method that returned a collection of all books in the library (potentially millions of objects).

While months ago I might have said YES, I understand that this is a bad idea. Yes, this would probably crash the server, but that is not what actually swayed me. What made me realize the absurdity of such a method was the question:

Would this collection of data be more useful as objects rather than as a report?

When I was challenged to come up a feasible use case in which I would need to affect millions of objects, I realized that this is data that should be report oriented.

Then, I also came to the realization that there is a difference between domain-based actions and “bulk” actions. If I wanted to increase the price of all products in an ecommerce store by 10%, I certainly would instantiate each product, update the price, and then save it. I think what would make more sense is to have some object (maybe a gateway or a service) have methods that allow for bulk actions.

Posted by Ben Nadel @ 11/5/08 2:48 PM

File Manipulation on Windows Servers from Linux

This is more of a handy tip than anything earth-shattering, but yesterday I was faced with the task of grabbing all files with a particular extension from a nested directory structure, moving them all into a single directory, and renaming them with a different extension. I also had to be careful to preserve the original timestamp of the file.

The files reside on a Windows server, and needless to say the thought of remoting into the Windows server and spending the afternoon drilling into nested directories, sorting by file type, and manually moving and renaming the files didn’t appeal to me.

One of the great things about Linux is how powerful the shell is. Let me preface this with saying I’m not a DOS expert, so maybe there’s a way to do this in DOS (or PowerShell, which I’ve never tried), but I knew I could probably accomplish this entire task in a couple of commands in a bash shell.

Step 1 was to mount the Windows server drive:

sudo mount -t cifs //server.dns.or.ip/sharename /mount/point -o user=username,password=password

Note that “/mount/point” is the local directory where you want to mount the share. I tend to use something like /media/servername-driveletter because mounting everything in /media is easy to remember, and on some distros this will also cause the drive to show up on your desktop (though this doesn’t happen on Kubuntu).

With the drive mounted, I navigated to the top level of the (rather nasty) nested directory structure and ran the following:

find ./ -name “*.fileextension” | xargs -i mv {} /mount/point/destinationdirectory

What this does is traverses the directory structure, finds all the files with the file extension I needed to move, and pipes that into the move command. The “xargs -i mv {}” bit basically says “get your arguments for the command you’re about to execute from the standard input (which is the list of file names kicked off by the find command) and replace {} with the data from standard input.” Then of course /mount/point/destinationdirectory is the directory into which I want to move the files.

A note if you want to use copy (cp) instead of move (mv)–this does NOT retain the original timestamp. The cp command has a -p option that preserves the original timestamp, but this did not work for me when I was mapped to a Windows share. Apparently this is because I’m executing the command as one user on Linux and that user doesn’t have permission to change the timestamp on the Windows side. If you were logged in with the same user name on both sides maybe this would work, but I didn’t try it.

So with step one completed, I just needed to rename all the files with a new file extension, or in my case I was actually just removing a second file extension since the files were named in the format “filename.ext1.ext2” and I just wanted to remove the “.ext2” part.

After navigating to the directory into which I moved all my files, that was another one-liner in the terminal:

rename -v ‘s/.ext2$//’ *.ext2

The rename command in bash allows for the renaming of multiple files using Perl regular expressions as the criteria for the rename operation. In this case I just wanted to lop off the .ext2 bit, and apply that to all files with the .ext2 extension. The -v option is for “verbose” so I could watch what it was doing while it did it, and if you’re nervous about what might happen, you can use the -n option to have it show you what it would do with your command but not actually do it.

So a bit of research and help from a Linux guru friend, and the drudgery of file moving and renaming was reduced to two commands in a bash shell. With some clever piping I probably could have even done this in one line.

I suppose my point with all of this is when I’m faced with little tasks such as this one, I try to take the time (unless I asbolutely can’t) to find a way to accomplish the task elegantly and in a way I can use again, as opposed to blindly saying “there goes the afternoon,” shutting off my brain, and dragging files around in a GUI. Not only does this make me more productive, but I learn something in the process, and it’s something I can use and alter time and again in the future to make boring tasks a lot less work.

Comments

There is a pretty easy way to do this in Windows using pure GUI (and there are ways to do this in DOS as well.)

If you’re ever in a situation where you’ve got to use the Windows GUI, then just do a Windows “Search…” (right-click on folder, choose “Search…”) and use the extension as the filter.

It will then return every match in that directory tree. You can then select all the files, right-click and select “Cut”.

Last, just paste them into the directory you want.

I’ve used this technique to quickly clean up .tmp files from a folder and occasionally to remove all the .svn folders from a directory.

Posted by Dan G. Switzer, II @ 8/12/08 7:09 AM

The danager of mounting samba/cifs shares with the username and the password on the commandline is that during the mount operation the process listing will show the mount command AND show the username and password just as its written on the CLI. Not a huge risk, but worth noting that on a multiuser system any other user that does a process listing can intentionally or accidentally read the credentials.

Posted by Steven Erat @ 8/12/08 7:35 AM

@Dan–thanks for the Windows tip. That covers the move part at least, and I assume there’s some relatively easy file rename command in DOS that would keep you from having to do the file renaming file by file.

@Steven–good point; I guess I wouldn’t want to be in a situation where someone I didn’t trust had access to the system ;-), but that’s definitely worth pointing out. If you’re doing this from a Linux server that may have multiple people with access, you’ll want to be aware of this.

Posted by Matt Woodward @ 8/12/08 7:48 AM

FYI, the xargs command uses the environment variable space to communicate the file list from the find command. If the paths are long, or there are more than X files, xargs will blow up because it has exceeded the maximum allowed environment storage. A better choice is to use the -exec switch on your find command.

Here’s an awesome reference of find goodness:

http://www.athabascau.ca/html/depts/compserv/webunit/HOWTO/find.htm

I guarantee there’s a pile of useful stuff find can do that Windows GUI search cannot.

Posted by Jason Guild @ 8/12/08 12:11 PM

Thanks Jason–good info. So much Linux to learn, so little time. ๐Ÿ˜‰

Posted by Matt Woodward @ 8/12/08 12:28 PM