Get Your Hands Dirty at BFusion/BFlex

BFusion/BFlex is coming up on September 11 and 12, 2010 in lovely Bloomington, IN. You need to be there, and here’s why.

  • $30 for one day or $45 for both days is one of the best deals you’ll find in the conference world. (No, I’m not missing a zero at the end of those prices!)
  • BFusion/BFlex is all hands on, all the time. This isn’t a passive event by any stretch. You come armed with your laptop and you get to learn from the best people in the business.
  • Multiple all-day classes in both CFML and Flex are available. On the CFML side you can learn Mach-II from the great minds behind it (namely Peter Farrell and Kurt Wiersma), you can learn CFML from scratch from the great Adobe instructor Matt Boles, if you’re a programmer already but want to learn CFML you have none other than Simon Free at your disposal, and if you’re an open source junkie like me or just want to learn more about the free CFML engines, I’m giving an all-day in-depth class on that.
  • Great for all skill levels, from pure beginner up to advanced developers. There’s literally something here for everyone so you don’t have an excuse not to come. Just check the BFusion and BFlex schedules and tell me you don’t see something interesting!
  • It’s a nice intimate environment where you can corner speakers or other attendees and get whatever help you need.
  • Indiana University is a great venue with great facilities. Power and Internet for all!
  • Bloomington is a great town with tons to do.

I’ve been invovled with BFlex/BFusion since the first year and it’s one of my favorite conferences. I’m particularly excited this year to be able to help people delve into the open source CFML engines, so I hope to see lots of you there.

Great training, great price, great location, and great people. What are you waiting for? GO REGISTER NOW!

My CFFree Presentation from Dallas TechFest

Dallas TechFest was last Friday and although I was sad to be missing the final CFUnited, TechFest was a blast. It was incredibly refreshing to see 400+ CFML, PHP, .NET, Java, and Flex people all in the same place at the same time for some geek cross-pollination. This doesn’t happen nearly enough so thanks to all the organizers of the event, all the great speakers, and to Dave Shuck for inviting me to speak.

The talk I gave was entitled “CFFree: Building and Deploying CFML Applications on a Free Software Stack.” This was a 90-minute introduction to developing, deploying, testing, and monitoring CFML applications using Tomcat, Open BlueDragon, Railo, and a plethora of other free software applications and tools.

If you didn’t make Dallas TechFest this year and are interested in this topic you’re in luck! I gave everyone who attended my session a VirtualBox VM that has all the free CFML goodness installed on Ubuntu 10.04, and you can download the VM from my Dropbox (2.5 GB).

You can also view the presentation directly in Google Docs; just make sure to click “Actions” at the bottom then “Show Speaker Notes” to view the speaker notes since the slides alone don’t tell you all that much.

As always if you grab the VM or read through the presentation and have any questions about any of this feel free to contact me. I’m considering doing a couple of screencasts on this topic because due to time constraints I had to breeze over some of the configuration details, not to mention some of the free software goodness I didn’t get to show at all.

If you want to learn more about free CFML in person, next up on the calendar is a full day hands-on session that Adam Haskell and I will be doing at BFusion in September, and then of course three full days at OpenCF Summit in February 2011. Hope to see you at one or both of these events!

CFML Advisory Committee Officially Dead: My Version of the Story

Since Adam doesn’t have comments open on this post (what’s up with that?), I feel compelled to clear the air and discuss the probable ending of the CFML Advisory Committee from my perspective. (And yes, all are welcome to comment on this post.)

I really regret having to spend the time writing this since in a lot of ways it will be airing dirty laundry. But since Adam is grossly misrepresenting a great deal of what went on during the effort of the CFML Advisory Committee and what led to its eventual demise, I think it’s only fair that people don’t take Adam’s version of the story as gospel. It’s far from it.

Preamble

First, I’d like to get some semantics out of the way. I can’t bring myself to refer to this now-defunct effort as “Open CFML” because it’s such a huge misnomer. Frankly, we never referred to the committee as “Open CFML” so I suspect the use of that term in Adam’s blog post is merely FUD/scare tactics that suit his purpose. (Saying Open CFML is dead sure has impact, don’t it?)

The CFML Advisory Committee was anything but open. There was no communication with the CFML community as to what was going on with the effort, there was no way for the community at large to participate, and suggestions by myself and others to update the wiki regularly with our progress and have an open mailing list that anyone in the community could at least read were continually shot down. Given all this it’s not surprising the effort ultimately failed, because openness always wins in the end.

Now for the timeline of the CFML Advisory Committee and why things went wrong from my perspective.

In the beginning …

When I was invited to join the CFML Advisory Committee I was excited because I originally thought the effort was important for the future of CFML. The CFML landscape changed dramatically with the introduction of two open source CFML engines, so it seemed logical that the interested parties would want to get together and discuss a certain level of standards for the language. It’s also something that the CFML community was clamoring for, at least initially, because of concerns over code portability between the various engines.

After getting up to speed on things and having discussions via the closed mailing list as well as a couple of conference calls, it became quite clear to me that the effort wouldn’t be able to survive the politics. Even given my valid pessimism, I decided to continue on in earnest, hoping for the best but expecting the worst.

Here comes a bit of dirty laundry, but I think it’s necessary for people to understand what was really going on behind the scenes. Adobe’s participation and level of effort during their involvement with the committee was incredibly lackluster. The committee had conference calls monthly in the beginning, but after Adam didn’t show up for three calls in a row, it seemed rather pointless to continue to schedule them. The email traffic on the closed mailing list would go in fits and starts, and in many cases months would go by with no traffic at all.

To cite but one example of Adobe’s lack of effort, the way the committee was attempting to organize the language was to use a Google spreadsheet, with columns for the CFML tags and functions on the left, and then a column for each engine indicating whether or not it had the tag or function. There were then columns for each member of the committee to indicate if they thought the particular tag or function should be “core,” “extended core,” or “vendor specific” as far as the eventual language specification was concerned.

As you can imagine simply organizing the spreadsheet took a great deal of time and effort, to which many members of the committee contributed. Each vendor was tasked with entering their engine’s tags and functions. OpenBD and Railo provided their information quite promptly, and I think it was Rob who did the Lion’s share of getting things organized initially, but Adobe’s column was incomplete for weeks. Finally I took a day or so and added/corrected all the Adobe functions and tags. That at least got things to the point where we could vote.

When it came to voting, Adobe failed to complete this effort as well. To put things in perspective voting took about an hour, but Adobe couldn’t seem to find that hour to input their votes. First it was the CF 9 release, then it was MAX, then … who knows. Bottom line is it never got done. Everyone involved with this effort is incredibly busy, yet everyone but Adobe made the time to vote.

Lack of Accountability Means Lack of Commitment

After weeks turned into months and the voting was still waiting on Adobe, I believe it was Peter who made the modest proposal of setting deadlines for votes. If someone didn’t meet the deadline then we’d simply indicate the person didn’t vote and proceed with the votes we had. Everyone paid lip service to having deadlines, but even while discussing deadlines Adobe reserved the right to ignore the deadlines if they were “busy.” Again, we’re all busy. Either the effort is worth your time or it isn’t.

I also suggested repeatedly that the whole specification and voting process be open to the public. The members of the committee were selected to represent the CFML engines and the CFML community, but we don’t necessarily have all the best answers, and at a minimum the community at large should have been able to see the process as it unfolded. But that would have meant both commitment and accountability on the part of everyone involved, and in the case of certain members of the committee the commitment clearly wasn’t there. Having accountability–let alone public accountability–wouldn’t work in their favor.

In the end the incredible lack of progress led everyone to agree to simply codify what was in Adobe CF 8/9 so we could get past the voting roadblock and move on. (Keep that decision in mind; it backs up an important point I’ll make later.)

Conspiracy Theories Debunked

Here’s where things get really bizarre, so please don your tin foil hats and try to stay with me.
Adam mentions the CFML Conventional Wisdom group in his post, which is an effort Alan Williamson started in order to foster public discussion of the CFML language. Personally I think an open discussion list should have been an arm of the CFML Advisory Committee from the start.

To quote Adam’s post, “At the very least, this explained why Peter so abruptly resigned.” I’m not going to speak for Peter, but this is not why he resigned, and his resignation wasn’t exactly abrupt. He debated it for a long time, but to call it abrupt reminds me of a scene from “Fletch“:

Dr. Dolen: “Shame about Ed.”
Fletch: “It was. Really a shame. To go so suddenly.”
Dr. Dolen: “Oh, he was dying for years.”
Fletch: “Sure, but the end was so sudden.”
Dr. Dolen: “He was in intensive care for eight weeks.”
Fletch: “Yes, but the very end, when he actually died, that was extremely sudden.”

Why Adam chose not to take Peter’s eloquent resignation email at face value, but instead drums up yet another conspiracy, only serves to further his agenda.

Back to our story. So why didn’t I mention the CFML Conventional Wisdom list to the committee?
Two reasons.

First, it’s a public mailing list, and I found out about it like everyone else did, on a blog post or Twitter or who even remembers where. Alan may have even sent me an email directly to make sure I was aware of it, but it wasn’t a big secret. There were no back-room dealings on any of this. Alan saw a need, created the group, and then announced it publicly. Simple as that.

Second, and more importantly, I saw this as a completely separate effort from what the committee was doing. The committee never had a public forum to discuss CFML, and the powers that be on the committee didn’t want one. So this provided something the committee didn’t, and giving the community a voice in helping to define the language they use is something vitally important in my opinion.

Ultimately the Conventional Wisdom list could even serve as a benefit to the committee. Loose ideas could be hashed out, sample code usage could be created, and the results could be submitted to the ivory tower that was the committee.

And a bonus reason: at the time when the Conventional Wisdom list started, the committee was in no way, shape, or form in the business of discussing enhancements to the language. We weren’t even able to codify what was already in the language, so for Sean, Peter, or me to bring a CFLOOP enhancement discussion to the committee wouldn’t have made sense. The committee wasn’t the time (or the place, frankly) to have those sorts of discussions.

The way Adam phrased all of this makes it sound like Alan, Peter, Sean, and I got together and were all conspiring against the committee, and maybe Adam even believes against him personally. Nothing could be further from the truth. Sean, Peter, and I found out about the Conventional Wisdom group when everyone else did, and since people on the committee are supposedly aware of what’s going on in the CFML community, I assumed everyone on the committee would have seen it as well. The implication that we were keeping a publicly announced, publicly available list hidden from the committee simply doesn’t hold water.

As for my better mouse trap header above, ultimately I think having an open discussion group is the way things should have been from the start. Why not seek input from the community at large? Why not have the discussions out in the open? Seems like a no-brainer to me. If Adobe wants to have enhancement discussions with their customers and among the members of their private programs that’s great for Adobe, but that doesn’t let the legions of CFML developers in the wild participate in the process.

Again, openness always wins.


Where I Agree
Adam did make a couple of points with which I agree.


“In the end, the community isn’t losing much at all with the demise of the OpenCFML board.”

I completely agree with this statement. Though I was honored to be a part of it I questioned the effort from the beginning because at the end of the day, Adobe ColdFusion sets the de facto standard for the language. I made that point in my last email to the committee list (which for a very long time was the final email on the committee list), and I don’t think anyone would disagree.

I take issue with Adam’s follow-up to this statement, but life’s too short to nitpick every last detail of what he said. The real beneficiary of this effort was supposed to be CFML developers, but somehow that ideal got lost along the way.

Bottom line here is that CFML developers don’t have anything to fear due to the loss of the committee, because the practical value of the end goals were rather dubious from the start. It’s in the best interest of the open source engines to be largely compatible with Adobe CF, which they are. We haven’t had an official CFML standard before and the world hasn’t come to an end, so the cf_apocalypse won’t come now.

“Innovation and progress in CFML is driven exclusively by the ColdFusion community.”

I suspect Adam and I have very different ideas of what constitutes the “community” and what community really means, but ultimately this is correct. The odd thing here is that before the Conventional Wisdom list was started, there was no single public forum where members of the community at large could discuss what they want to see in the CFML language. So if innovation and progress in CFML is truly driven by the community, they now finally have an outlet.

Setting the Record Straight

Now, for the corrections.


“Matt claimed the OpenBD team was too unorganized to submit tags like CFJAVASCRIPT and CFSTYLESHEET (tags I had hope to include in CF9).”
Pure FUD on multiple levels. I may have said something along the lines of “These tags were added on a whim last week,” but that’s how open source projects work. If Alan or Andy get an idea, or need something for a project they’re working on, they’ll create the tags and release them in a nightly build. We then get feedback from people actually using the new features and refine as needed. That in no way means the OpenBD team is disorganized. It differs from proprietary software development, sure, but calling that process “disorganized” would be the equivalent of me calling Adobe “lazy” because they only have a new release every two years. Apples and oranges.

CFJAVASCRIPT and CFSTYLESHEET were added to OpenBD, and Adam’s response when he found this out was apparently to take it as an affront because we didn’t run the idea by the committee first. Adam seemed to continually and conveniently ignore the fact that OpenBD is an open source project, so Adobe or anyone else can see what we’re doing every step of the way.

Furthermore, it was agreed upon by all members of the committee that tags and functions developed in specific engines would be brought to the committee only if their creators wanted the language enhancements to be considered as “core” in the CFML specification. We didn’t consider CFJAVASCRIPT, CFSTYLESHEET, and some of the other innovations in OpenBD as “core” CFML, so we didn’t officially submit them. But again, the source code is open. If Adobe saw something in OpenBD they thought made sense to be core to CFML or to add to ColdFusion, they could have brought it up and we could have discussed it.

And let’s talk about what “submitting a tag” to the committee even means. I’m not sure what exactly I was even supposed to “submit”–the tags were out there in the OpenBD source along with documentation as to how they are used, so wouldn’t it make sense that if Adobe was interested in considering those tags for ColdFusion they could read the docs and ask about it on the committee list? If you get the sense from Adam’s post that there was some rigorous formal process that OpenBD was somehow subverting, don’t. There was no such defined submission process in place.

Finally, the “tags I had hope[sic] to include in CF 9” bit is completely bogus. Yes, Adobe did make some changes to CF 9 (specifically CFSCRIPT if I remember correctly) based on the committee’s recommendations, and I applaud them for that, particularly given that CF 9 was nearing the end of development at that point. They ignored plenty of other recommendations, and the reason for that–the real bottom line here–is that CF 9 was largely baked by the time the committee was even having these discussions. So to point a finger and say CFJAVASCRIPT and CFSTYLESHEET didn’t make it into CF 9 because I failed to make an official submission to the committee is false on numerous levels. We didn’t consider them to be core to the language, the code and spec were available from day one, and adding this new functionality to CF so late in the development process likely wasn’t going to happen anyway.

“Sean claimed that Railo wanted to wait a version (or two) to see how new Railo tags were accepted by the community before making a formal recommendation.”
Again, this is how open source projects work. Did Adobe expect that the open source engines will stop following the process that is best for their projects and users, and hit the brakes on the rapid pace with which new features are added, to ask the committee’s permission before making enhancements to their engines? Does it not make sense–and ultimately benefit Adobe–if the open source projects introduce new features to CFML and let the community kick the tires before finalizing the feature? That saves Adobe time in the future if they decide to implement these features, because a lot of the questions surrounding the new features and how they should work will already have been answered.

I’m at a loss to see what the real issue is here. OpenBD and Railo are both open source, both have nightly releases, both have public roadmaps … everything’s always available for anyone to view. I think ultimately this comes down to a culture clash between open source projects and proprietary products, which was probably partially to blame for the failure of the committee’s efforts.

“As a community, we never needed the OpenCFML board to guide or document feedback.”

I give this one a “thumbs sideways.” We didn’t need the CFML Advisory Committee specifically to serve this function, but I don’t see Adobe doing anything to foster open language discussions in a public forum either. Perhaps that isn’t their job; they focus on finding out what their paying customers want, and no one can fault them for that. In similar fashion the open source projects get continual feedback from their users and make changes accordingly.

I do think it’s important that there be some avenue for people to discuss CFML as a language in an engine-agnostic way, however, but that was as simple as creating a Google Group. It’s a much better solution and will yield far better results than the committee ever would have.

“I really want to thank Ben, Rob and Ray for the work they put into the OpenCFML.”

… but no one else. Not Sean for making valiant efforts to keep things going even when the committee was at its lowest points, and not any of the rest of the people who participated far more than anyone from Adobe, and particularly Adam, ever did. If anyone questioned Adam’s real attitude towards the committee that statement pretty much sums it up for me. He was never interested in setting politics aside and working collaboratively to create a CFML language spec, and apparently the only efforts that matter are the ones made by the people in his camp. “You’re either with us or you’re against us” has been proven to be a very divisive attitude, and it’s one that ultimately isn’t good for the community as a whole.

“As far as I am concerned, the ColdFusion ACPs will be the CFML Advisory Panel for ColdFusion X and beyond. We’ll be asking them to review and sign-off on all of our language enhancements (very soon).”


So this at least defines what “community” means to Adobe, and helps to illustrate that they never had any interest in helping to define CFML as a language outside of their own product. That’s fine, and makes perfect sense coming from a proprietary software company, but leads me to the conclusion that the CFML Advisory Committee was a PR stunt to begin with. Adobe’s heart was never in it, they were just trying to get out in front of the open source CFML revolution.

In Summary …

The goal of Adam’s post is patently obvious: to blame everyone but himself and Adobe for the failure of the CFML Advisory Committee. His rhetoric sounds almost McCarthy-esque. Get people focused on a common enemy–real or imaginary–and they’ll become oblivious to little things like logic and the truth.

Let me be clear: Adobe’s lack of participation and paranoia over the supposed malfeasance of Railo and OpenBD are, from my perspective, a massive part of what killed the CFML Advisory Committee. I’d have a lot more respect for everyone involved if we could have collectively decided to end the effort, made a joint statement, and parted ways amicably. But for Adam to post a truth-challenged version of events out of the blue, while not surprising, is in very poor form.

And That’s All I Have to Say About That
I’ve said my peace and then some at this point. Feel free to comment, and if you have any specific questions you think I can answer, I’ll be happy to do so. If, however, your comment is of a “he said/she said” nature or intended solely to inflame the situation further, don’t be surprised if I don’t respond. Adam gave his version of the story, I’ve given mine in abundant detail, and everyone can continue to speculate on what went on behind the closed doors of the committee. If you care to, anyway. I’d certainly hope you have much, much better ways to spend your time. 😉

To me, it’s not worth dwelling on all of this. The CFML Advisory Committee will go down as an interesting side-note in the annals of CFML history, and people looking back will wonder why we even tried in the first place. I’ve been asking myself that very question for quite some time.

Has Oracle been a disaster for Sun’s open source?

As others have noted, this is a good demonstration of the fact that open source projects are effectively “immortal”: provided there is sufficient interest among users, they can always be forked. It should also serve as a reminder to Oracle that they are the guardians of the open source projects formerly managed by Sun, not the owners (well, they own the copyright, but that’s not quite the same.) If it fails to move the projects forward in the way that many users would like, it may well be faced with more forks.

The problem is that Oracle is naturally trying to optimise its acquisition of Sun for its own shareholders, but seems to have forgotten that there are other stakeholders too: the larger open source communities that have formed around the code. That may make sense in the short term, but is undoubtedly fatal in the long term: free software cannot continue to grow and thrive without an engaged community.

This is a really insightful analysis by Glyn Moody of what the Oracle acquisition of Sun may mean for important open source technologies big and small, from things like OpenSSO and OpenSolaris to OpenOffice.org, MySQL, and even Java.

The mention of the “immortality” of free software and open source projects is really key in my mind. A lot of people wondered about the future of MySQL when Sun bought it, and many are outright panicked now that MySQL is in the hands of Oracle.

But that’s what’s so great about free software: none of these things are truly in Oracle’s control. They belong to the communities around them. So ultimately it wouldn’t be wise for Oracle to piss off the communities that made these projects what they are.

That doesn’t mean Oracle won’t do something terribly unwise but if they do, the code’s out there and the free software community will ultimately do what’s right and best for the project in the long run, even if that means forking.

Cutting Waste by Reforming IT | The White House

While a productivity boom has transformed private sector performance over the past two decades, the federal government has almost entirely missed this transformation and now lags far behind on efficiency and service quality.  We are wasting billions of dollars a year, and more importantly are missing out on the huge productively improvements other sectors have benefited from.

Quite simply, we can’t significantly improve the efficiency and effectiveness of the federal government without fixing IT.

Some rather astounding (albeit not surprising) numbers on some of the horrendously wasteful projects in government IT. I hope this starts a similar movement in the other branches of government, because all told this would make a massive difference in government spending, and that’s rather critical right now.

Using free and open source software for all projects where it’s feasible would certainly help as well.

Open Source Bridge – The Story of Spaz: How to Give Away Everything, Make No Money, and Still Win

Edward Finkler http://getspaz.com @funkatron http://funkatron.com

  • non-technical talk–“why I do what I do”

    • learned how I define what success and happiness are

  • Spaz is a personal project — about 3 years old

    • twitter client — “your special twitter friend”

      • based on original logo people thought he was making fun of mentally retarded people
      • actually is a picture of clay aiken from early american idol

  • Early 2007

    • work started on Spaz
    • had done a couple of open source PHP libraries in the past
    • had only done open source for other developers, not for end users
    • started work in RealBasic on Mac

      • also the author of LameBrain, wrapper for LAME

    • started getting interested in twitter API
    • not many other Mac clients for twitter at the time
    • article written up in ars technica about it
    • pownce came along about the same time

      • came out with desktop client, written in apollo (now AIR)

    • RealBasic wasn’t set up to make it easy to make it look non-standard or skin things
    • silverlight was also being released about this time, also Java FX
    • particularly interesting aspect of AIR was that you could make apps in HTML/CSS/JS in addition to Flash

      • looked at using Flex, but not a fan of monotheistic technology
      • Flex community is a lot like the MS development community–one official source of information, not a lot of community sharing going on

        • quite a contrast with something like PHP where everyone shares in a bunch of different ways

      • didn’t like having to go through the official channels to try and figure things out

        • really hard to simply search for things on the internet and find examples with Flex

    • twitter was really flaky at this point–had to write the code very defensively
    • learned a ton by digging into this–crash course in javascript
    • driven because he was interested in it and enjoyed working on it

      • “I liked what it did, and I used it”

  • November 2007

    • Adobe had AIR Developer Derby in mid-2007

      • submitted Spaz on a whim
      • got a call in October 2007 from Adobe saying he won

        • got to go to Adobe MAX conference, lots of PR around it

    • Adobe didn’t give a rat’s ass that it was open source

      • they liked it because it was pretty
      • only got about 5 submissions on the HTML side of things

    • was at the AIR booth–people asking tons of questions that frankly Adobe should have been answering

      • turned into evangelist of sorts

    • cool that Spaz got known in the wider world

  • 2008-2009

    • started to get emails criticizing the name–“why do you hate the disabled?”

      • happened quite a bit

    • tons of twitter clients released in 2008

      • lots of them written in AIR
      • a few clients were a lot better with marketing themselves — biggest example is TweetDeck

        • people spending full-time on it, went after venture capital, etc.
        • cool if you want a twitter client that fills your whole screen, but wasn’t what he wanted to do

    • seeing success of tweetdeck was hard

      • started seeing comparisons of spaz vs. tweetdeck–lots were kind of nonsensical

    • the thing I like about open source is creating something and getting ideas from people, hearing about how they’re using the product, etc.
    • lots of end users were saying stupid things like “Spaz sucks”

      • big disconnect–creating products for end users is pretty different from creating developer libraries
      • people were dismissive and mean — made it not fun
      • eventually learned to respond with humor to baseless criticisms

        • helped to not get as upset about things

    • started seeing companies that were hanging off twitter and providing services that twitter doesn’t provide

      • twitter has started to stomp out all the ancillary business around twitter

    • started getting offers from companies around spaz

      • people offering money to add specific services to spaz, e.g. twitpic

    • wanted to continue to provide an open source, transparent twitter client

      • in one case took money as a donation to offset overall development time into spaz
      • wasn’t interesting in doing this in general

    • this underlined a specific way that he wasn’t willing or able to compete with other clients that were out there

      • would sacrifice transparency around the project
      • realized how screwed up and evil tech journalism is–fundamentally wrong

        • actively manipulate how people perceive things, don’t give coverage to things that don’t play their game

    • started to realize he couldn’t compete with other clients because of all the extraneous junk that goes on

  • palm webos — jan 2009 – june 2009

    • got call to see if he was interested in working on apps for the new webos
    • palm wasn’t really known as an open source company
    • said up front he wanted to keep the app open source–palm didn’t object
    • web development as a rule is pretty open, but palm didn’t really have this understanding in their DNA
    • had to sign really strict NDA
    • palm was rushing things out, documentation was terrible
    • went to a developer event at palm–could talk about things, but couldn’t share the code he had written–against the NDA
    • this six months sucked because couldn’t make something and share it with other people
    • palm didn’t tell him another company was also working on a twitter app for webos

      • found out from someone else–no information sharing going on
      • bunch of secret stuff–wasn’t fun

    • didn’t get a development device–emulator ran 3X faster than the actual phone

      • didn’t write specifically to their proprietary platform so it ran slow
      • another twitter app that did write to the proprietary bits ran faster

    • got email from palm asking not to release source code

      • had told them first thing he was going to open source it
      • lot of wtf moments
      • decided he would never work for another company like this under an NDA
      • palm didn’t get the sharing culture–they get it much better now

    • two guys from ajaxian got hired to better interface with the development community

      • immediate changes for the better–hope HP doesn’t screw it up

    • reminded me of lessons i should have learned the first time around
    • friend recommended book — “ignore everybody and 39 other keys to creativity”

      • the thing that you really like doing, if you rely on it for income, it’ll become less enjoyable and it will change how you interact
      • when spaz became something that was being defined by standard definitions of success, the process became less enjoyable
      • had to redefine what success was

  • july 2009

    • spaz statement of purpose

      • can keep going back to this and referencing it to remind himself of why he’s doing what he’s doing

    • spaz was built for the sake of building it. it is not a means to an end. however, creating it has had several good consequences.

      • point of it was to build something and build something good

    • spaz demonstrates that making things is good, and sharing how you make them is better.

      • creative endeavor in and of itself is good
      • sharing is what makes me happy

    • spaz is a necessary counter to closed, hidden technologies. spaz must always be open.

      • source code must always be open and have to be transparent about the intentions of the project
      • important that we have options, even if they aren’t the most popular or defeat other projects

    • the value of spaz does not lie in the judgments of others, but in the process of building it, and the enjoyment derived by those who use it.
    • we welcome anyone who wishes to participate in the spaz project with open arms, as long as they understand and respect the purposes of the project.
    • the spaz project values clear and open communication between participants.
    • having the statement of purpose helped keep things going when it was frustrating

  • 2nd half of 2009

    • not a lot happened
    • worked on getting webos version better
    • started on spazcore to share common things between all the different platforms
    • discovered didn’t have time to try to keep multiple versions going
    • started working on community building

  • diversity statement

    • women are under-represented — 10-15% in IT as a whole, open source it’s more like 2-3%
    • worked with a group called phpwomen
    • largely lifted from python’s diversity statement

  • rest of 2010 — cultivating a community

    • initially was using google code for everything
    • have found that github works better

      • a lot of people use git (particularly js people), more social aspects oriented around git (easier to fork, etc.)
      • potentially attracts more people

    • tenderapp.com and lighthouseapp.com

      • tender oriented towards end-user support — offer free accounts for open source projects
      • lighthouse more oriented towards developer issue tracking, setting milestones, etc. — also have free accounts for open source projects

    • if you only have roadmap, etc. in your head it’s very hard to get people to help you

      • need to break things down into issues of various size
      • schedule people’s time for them in the sense that you give them bite-sized tasks

    • hackathon

      • great way to get more people involved
      • helps to have a schedule
      • done online with irc

    • have someone who’s more or less dedicated to doing community related tasks

iizip – Hacking Together Your Own Dropbox

Ben Dechran, Sputnik Agency (Australia)

  • works for a marketing agency, lots of photoshop files thrown around
  • file server which works great most of the time unless …
    • people forget to put the files on the file server
    • people forget to rename a file before putting it on the file server and delete the old file
    • people don't put a file on the file server and go on vacation
  • dropbox nice solution, but has issues
    • works well for small companies, small groups, unconferences, etc.
  • problem #1 — storage space
    • in our case, needed 3TB of space
  • problem #2 — need internet connection
    • if connection goes down, you have the files but you lose the sync
  • problem #3 — dependent upon third party
    • what if they go down?
  • problem #4 — all or nothing
    • every person replicates everything
    • dropbox supposedly doing selective sync
  • problem #5 — user management
    • users can't be managed by your company's infrastructure
  • problem #6 – big bad world
    • security and privacy
    • it's encrypted, but you're sending data out to someone else's servers
  • other options
    • ifolder – open source project by novell, written in .net
      • tricky to set up
      • slow
      • packaged for windows, mac, opensuse
  • decided to roll my own–needed …
    • local files
    • monitor for changes
    • versioning
    • conflict resolution
    • transparent — install and forget
  • attempted solutions
    • tortoisesvn — people forgot to check in, etc.
    • cron job to push changes to svn
      • some of the changes were hard to detect
      • if someone deletes a directory on their hard drive, deletes the metadata
    • tried git — since it's distributed it solved some of the issues with svn
  • for synchronization
    • inotify, inotifywait — in the linux kernel
    • notify-send — could use this for bubble type notifications in dropbox
  • for conflict resolution
    • grep the git status
  • iizip is the combination of all these various tools
    • currently the project is a bunch of scripts; eventual goal is to have a package that would be deployed/installed on local machines
    • iizip-init — creates git repository and iizip directories
    • the scripts mostly map to the git commands
      • e.g. send checks in locally and also pushes to remote repository
  • someone in the audience suggested looking into using couchdb in similar fashion to how ubuntu one works
  • what's in the pipeline?
    • partial checkout so not everyone has to have every file locally
      • idea of subscribing to specific directories–IMAP already does this
    • push changes from other machines
      • currently there's a cron job that runs on the local machine
      • dropbox immediately pushes changes to all machines
      • considered using xmpp to handle this, but IMAP has persistent connections and also supports directories
    • multi-user
      • currently can't run more than once on the same machine since the directories are hard-coded
  • unix tools philosophy — many small tools linked together to achieve a larger goal
    • also makes it easier to port to other operating systems
  • unison — would in theory do all of this so worth checking out
  • potential issues — git doesn't track empty directories
    • if it's empty you may not care

Open Source Bridge – Relational vs. Non-Relational

Josh Berkus, PostgreSQL Experts Inc.

  • overview focused on choosing what type of database you need vs. investigation of any specific database
  • up until a few years ago there were only a handful of options for open source databases
    • most were sql/relational
    • a few written in java
    • only really exciting thing going on in the relational world is postgres vs mysql
  • today there are many more open source databases
    • as many as 5 dozen now?
  • databases for lots of different purposes, but lots of people want to lump a lot of the new ones under the "nosql movement" label
    • not so fond of this term
    • has implication that every database that doesn't have a sql interface is more or less identical
    • all non-relational databases aren't the same
    • have graph, document, key-value, distributed, hierarchical … quite different from one another
    • some of the non-relational databases have sql interfaces
  • all relational databases aren't the same either
    • embedded, oltp, mpp, streaming, c-store …
  • mythbusting
    • "revolutionary" is bandied about a lot but database technology goes back a long way
    • not really any new database designs in terms of fundamental architecture
    • are new implementations and combinations of design
    • last "new" thing was map-reduce in 2002
    • even couchdb is largely similar to Pick, which was created in 1965
    • when looking at new databases, don't look for revolutionary concepts, look for good implementations
    • what's going on right now is actually a renaissance of non-relational databases
  • myth: "non-relational databases are toys"
    • google – bigtable
    • amazon – dynamo
    • facebook – memcached
    • us veterans administration – pick, cache
  • myth: "relational databases will become obsolete"
    • xml databases were supposed to replace rdbms ca. 2001 — didn't happen
    • rdbmses evolved to include xml functionality
    • one of the things we'll see out of the current non-relational innovation is that some of the implementations will hybridize with one another
  • myth: "relational databases are for when you need ACID transactions"
    • transactions != relational — orthagonal features
    • robust transactions without relationality: berkelydb, amazon dynamo
    • sql without transactions: mysql isam, ms access
  • myth: "users are adopting nosql for web-scale performance"
    • sometimes it is, sometimes it isn't
    • performance test done by myyearbook.com
      • benchmark of key/value storage and retrieval
      • only real difference in performance is between databases that guarantee durability and those that don't
    • horizontal scalability
      • some non-relational databases are built for horizontal scalability and some aren't
      • complexity of implementation rises with the ability to scale out to a massive number of nodes
  • myth: "one ring theory of database selection"
    • "what's the best database to use?" – wrong question
    • don't need to use only one database
    • choose the db that meets your applicaton's goals, or use more than one together
    • use a hybrid
      • mysql ndb
      • postgresql hstore
      • hadoopdb
  • but what about choosing between relational and non-relational?
  • relational oltp databases
    • transactions: more mature support
    • constraints: enforce data rules absolutely
    • consistency: enforce structure 100%
    • complex reporting: keep management happy!
    • vertical scaling (but not horizontal)
  • sql vs. no sql–sql promotes …
    • portability
    • managed changes over time (ddl)
    • multi-application access
    • many mature tools
    • but, sql is a full programming language and you have to learn how to use it
  • no sql promotes …
    • programmers as dbas
    • no impedance mismatch
    • fast interfaces
    • fast development and deployment
    • but, man involve learning complex proprietary APIs
      • in some cases not easier than sql
  • main reason to use sql-relational databases
    • "immortal data"
    • your data has a life independent of this specific application implementation
    • important to be able to access data accurately and consistently forever
  • how DO i choose?
    • define the problem you're trying to solve
      • what is it that my application wants to do with this data and how does it want to do it?
      • i need a database for my blog, i need to add thousands of objects per second on a low-end device, etc.
    • from the definition you create you can define a database shopping list
      • define the features you ACTUALLY need
  • fit the database to the task
  • "I need a database for my blog"
    • use anything!
    • no open source databases that wouldn't support someone's individual blog, flat files would even work
  • "I need my database to unify several applications and keep them consistent"
    • high-end sql-relational database best choice for this
    • "PostgreSQL: It's not a database, it's a development platform"
  • "I need my application to be location-aware" — geo applications
    • PostGIS – geographic relational database
    • queries across "contains" "near" "closest"
    • complex geometric map objects
    • couchdb spatial and spatialite are now available as well
  • "I need to store 1000s of event objects per second on a piece of embedded hardware"
    • db4object — embedded key-value store
    • others: berkeleydb, redis, tokyocabinet, mongodb
    • db4object
      • project was german train system — records data every few milliseconds
      • low-end embedded console computer
      • simple access in native programming language (java, .net)
  • "I need to access 100K objects per second over thousands of connections from the web"
    • memcached – distributed in-memory key-value non-persistent database
    • use: public website
    • typically supplements another database
    • alternatives: redis, kyototyrant, etc.
  • "i need to produce complex summary reports over 2tb of data"
    • luciddb – relational column-store database
    • for reporting and analysts
    • large quantities of data
    • complex olap and analytics
    • used along-side oltp running production apps
  • "I have 100s of govt documents I need to serve on the web and need to mine the data as cheaply as possible"
    • CouchDB
    • storing lots and lots of government documents that didn't have a consistent format (don't know content or structure)
    • used in combination with postgres to keep structured metadata
    • couchdb is also great for mobile apps
  • "I have a social application and I need to know who-knows-who-knows-who-knows-who"
    • surprisingly hard question to answer with a normal db
    • use a graph database — neo4j is most popular open source one
    • social network website
    • 6 degrees of separation
    • "you may also like"
    • type and degrees of relationship
  • "I get 1000s of 30K bug reports per minute and I need to mine them for trends"
    • used on mozilla firefox crash reports
    • hadoop — massively parallel datamine
    • hadoop + hbase
      • reports are then put into postgres for viewing
  • conclusion
    • database systems do better at different tasks
      • every database feature is a tradeoff
      • no database can do all things well
      • need to make tradeoff decisions when picking databases
    • relational vs non-relational doesn't matter
      • pick the database(s) for the project or task

Questions

  • how difficult to migrate from something like couchdb to postgres?
    • depends on how much data and in what form
    • since couchdb works with json that's pretty easy
    • if you want to take the document structure out of something like couchdb and put it into a relational model, the decomposition process will be complicated

Open Source Bridge – Thursday Keynote: Mayor Sam Adams on CivicApps

Sam Adams, Mayor of Portland

  • CivicApps for Greater Portland
    • interaction at last year's OSB kicked off regional effort to put Portland in the forefront of open source and open data
  • update on how Portland is leading in the area of open source
    • asked for city government to be an early client of open source work that is happening locally
    • also asked for financial assistance with open source projects
  • civicapps can change the relationship between the government and the people they serve
    • give people new tools to get at what they need
    • greater accountability
    • example: iphone app where people can take pictures of potholes–this gives the photo and the exact coordinates of the pothole
      • submitters get notification when the pothole they reported is fixed
      • idea of government reporting back directly to citizens is huge source of satisfaction
      • now the challenge is to provide this type of customer service on more platforms
  • civicapps has gotten a lot of national attention because it's regional–isn't specific to one city
    • portland isn't that unique–60K people who work in portland but don't live in portland
    • lots of other open data apps are specific to cities–not as realistic to the daily life of people
  • contests to develop apps around open data
    • area that needs a lot of work ("significant mystery to citizens") is the inner workings of the decision making process
      • e.g. city council issues being considered
      • city council agendas can have 60-150 items on them
      • mystery to citizens how the work actually gets done
      • need to make actual decision making easier to access and more understandable to citizens
  • have tried to surround these efforts around an economic development strategy
    • digital development is now one of the four targeted industries for economic assistance
    • investigated what local assets match growth trends regionally
  • portland's value proposition
    • not silicon valley and don't want to be
    • where portland can compete and win on a proportional basis is that there's a more diverse group of technology folks in portland
    • clearly is smaller, but especially around software, there's a greater diversity of different talents and approaches than anywhere else in the world
    • portland technologists are also very agile
    • passion in portland in these areas is huge–everyone involved tends to work together very well
  • how can portland's government help technologists succeed?
    • recession has hit some businesses, but software/technology has overall survived better than other industries
    • has been a lack of access to capital to bring best ideas to market
    • started portland seed business fund — direct result of requests from software industry
    • http://mayorsamadams.com — more details on this fund
    • portland development commission — funded "Portland Ten"
      • bootcamp that works with 10 startups
      • access to capital, connection to very best thinkers from around the world
      • "tough love"–helping ten startups get to a million dollars a year in revenue
    • survey will be sent out to people participating in all these efforts–really need the feedback to make sure we stay on track
  • "I want you to take over the world from your base here in Portland"
  • 25 people involved with civicapps here today to help people hack on data and apps

Rick Nixon – project manager for civicapps

  • first ever Civic Code Day
  • lots of people here responsible for lots of data for the city, trimet, civicapps web site
  • role of govt: supply data
  • role of citizens: do something with the data
  • seeking to transfer data collection effort to citizens as well
    • open source the web site, number of other efforts going on

Open Source Bridge – How Two Fools Made Themselves Indispensible From Their Basement Office

Chris Chiacchierini, Mason Bondi – Oregon College of Oriental Medicine

  • 50 staff, 250 students
  • university originally chose proprietary commercial cms for $15K
    • used for marketing site
    • only one of 5 people in the world using it
  • attended open source conference and learned about open source cmses
    • "holy shit, we just got snowed"
    • chose Joomla! for ease of use
  • didn't want to give up on open source cmses
    • went to library, asked if they'd be interested in putting up a site using oss cms
    • did this all under the radar, wanted to wait until it was too late to go back before anyone knew
  • through this a lot of institutional knowledge is being built, and since people had freedom they were starting to get more creative
    • class materials being put online
    • financial information about student loans
  • lots of people and departments were using the open source cms
  • once it was opened up to the students, it really started to take off
  • two parallel forces going
  • got call while he was on vacation — marketing site got hacked
    • could now go to executive council with security concerns and propose switching over to the open source cms
    • was getting hacked every couple of days
  • executive council asked how much it would cost
    • couldn't believe cost would be $0
  • 4-5 departments already using the open source CMS loved it
  • what about the rest of the community?
    • every single office at the university has a stake
    • lots of departments only updated static information once a year
      • weren't interested in doing anything with the CMS, or are too busy
    • e.g. grades–wanted to put grades online
      • would be vastly more efficient than mailing, but hard to make people change
  • a project management cycle that works
    • opportunity -> needs analysis -> cost benefit analysis (proof of concept) -> project plan proposal -> approval -> design, build test -> implementation & training -> review
    • most often the people with the money aren't the ones who have the technological know-how so there's a lot of politics involved
    • a lot of the project management cycle is done in stealth mode
    • technical people tend to want to take the ball and run with it and tell everyone to get out of the way, but this doesn't work without a sponsor
    • need to get a sponsor at the executive level
      • often best to get the person who has the power to pull the rug out from under you to be the sponsor–they get part of the credit
    • need to sell the idea to a sponsor in the opportunity phase
    • still selling hard in the needs analysis phase
      • can do some of this in stealth–pick people you like and trust to get a little bit of insight into what the needs are
    • after this, set up a needs analysis meeting to ask for feedback–make it clear this is the one chance people have to offer feedback
      • better than going office to office–gets everyone in the same room
      • people see what the impact is outside of their own department
      • also lets people gripe if they want to
    • in cost/benefit phase have to make sure the proof of concept is in place
      • can't only look at cost as cost of the software–also involved is implementation, training, cost of support, etc.
    • project plan proposal–where people who gave feedback see what will actually be implemented
      • even though it won't be exactly what each person wanted, they'll see the bigger picture
    • approval — doesn't only mean budget, but once something's approved there's no turning back
      • if you're not sure you can follow through, stop before approval
      • after approval you HAVE to deliver
    • implementation & training
      • second tier thing they implemented was putting course materials online using moodle (course management software)
      • no way could get everyone involved and get all the instructors to get all their materials online
        • students weren't excited at first either, but once they used it they won't go back
      • had a goal of rolling out three classes per quarter as a pilot
        • chose instructors that taught enough classes that most of the students would be involved
        • none of the courses are exclusively online but there are pieces of the course you can only get online
      • followed faculty around for three weeks, constant contact, asked for feedback from both instructors and students
        • important to get everyone comfortable
      • what they wanted to happen started to happen–students didn't want to deal with printed materials anymore
        • students started talking to faculty members whose courses weren't online
      • building things out slowly builds inertia, but always keep the costs in mind
      • as far as timing goes, as long as you can keep showing progress, specific deadlines aren't that important
      • if possible getting isolated focused time with people is ideal for getting things ramped up
        • can avoid ongoing and future training, also gets people thinking more creatively
    • review
      • constant cycle
      • some formal review, some informal
      • important to make incremental changes so the solution continues to meet the needs
  • before the cms project started, the IT guys were seen as a necessity
    • now people are looking at the IT department as something that can help with solutions
    • creating a lot of work for the department
  • through a successful project you can gain "street cred"
    • create a working relationship with other departments