Sansa e280 and Rockbox on Ubuntu 9.04

This post probably has a very limited audience, but what else is new. ;-) I'm finally getting around to de-Appleizing my music collection (more on that in my forthcoming Amahi chronicles), and yesterday I installed Rockbox on my Sansa e280. The e280 is a great little player with a ton of features, and since it's a bit older now you can pick one up dirt cheap if you're interested in playing around with any of this. (Accessories for it are amazingly cheap too--I got an extra case for 49 cents from Amazon.)


For those not familiar with Rockbox, it's an unbelivably cool open source firmware replacement for several different makes and models of digital audio players, including iPod, Sansa, and others. Think of it as a new operating system for your DAP.


What Rockbox does for you is unlocks a bunch of features that the manufacturer of the DAP didn't include. A big one for me is the ability to play Ogg Vorbis files, but it supports 15 codecs as opposed to the usual two of most DAPs. It also adds a lot of niceties that aren't in many of the original models, lets you create and upload your own visual themes to the DAP, and since it's open source it fosters a community of theme and plugin developers around it. For more info you can read Why Rockbox? on the Rockbox wiki.


If you're interested in installing Rockbox on your DAP, just make sure to check the specific model and firmware version of your player to see if Rockbox supports it. In general, the older the player the more likelihood there is that Rockbox will support it. For example, my e280 had the first-gen firmware on it, so it's supported, but the second-gen e280s, even though they're pretty old at this point, are currently not supported.


The installation process is dead simple; you plug your player into your Windows, Linux, or Mac machine, download the Rockbox installer, run the program, and it installs the new firmware to your DAP.


On Ubuntu 9.04 (Jaunty), however, I ran into an issue. When I plugged my e280 in with the USB mode set to MSC it would mount and then immediately unmount, and of course the Rockbox installer won't work unless the device is mounted.


There's a lot of information available about this problem and numerous potential solutions, but only one worked for me. Jaunty ships with a config file (specifically /usr/share/hal/fdi/preprobe/10osvendor/20-libgphoto2.fdi) that has a bunch of digital audio players "definitions" in it, and that was causing a conflict.  Once I removed all the references to Sandisk players in that file, the Rockbox installer saw my e280 fine.


Rockbox is fantastic! If you have a supported player I really encourage you to give it a shot.

Open Source Bridge: Running an EDU on OSS

Michael Alan Brewer

Franklin College Office of IT, University of Georgia

US PostgreSQL Association



• what we'll cover

    • oss applications that help to run the administrative side of edus

• we'll not cover

    • oss in the curriculum

        • lots of groups working on this

    • oss on the desktop

        • at UGA the english dept requires openoffice for document interchange

• motivation: why use oss?

    • obvious answer is money

    • reliance on PEOPLE, not code

        • value is in the people

        • the people are the resource

        • good way of keeping good people there, and keeping people good

    • ability to own data

        • HIPPA/FERPA

    • ability to make code work for our own needs

        • UGA: academic advisor assignments--DegreeWorks didn't handle multi-majors

• proprietary vs. oss

    • major proprietary players

        • SunGard

            • Banner

            • DegreeWorks

        • Oracle

            • PeopleSoft

        • Blackboard

            • Blackboard Learning Systems (formerly WebCT)

        • SharePointLMS (.NET based)

        • JoomlaJMS

            • based on Joomla, but this product is closed

    • lots of oss options

        • home-rolled

            • mostly based on LAMP

        • framework-based

• lots of acronyms

    • CMS, LMS, LCMS, multiple definitions for each

    • in general these all refer to online learning communities

• running an institution required you keep track of

    • students

    • applications

    • forms

    • test scores

    • academic records

    • grades

    • papers

    • performances

    • digital media archiving

    • medical records

    • evaluations

    • etc etc

    • tracking teachers has its own set of things to track (salaries, payment, etc)

• edus need most of the standard business-type tools

    • cms

    • calendar (huge)

    • event management

        • here, open conference works comes in handy

• non-standard business-type tools

    • rules/work process/paperwork

        • hippa/ferpa

        • human subject testing

        • hazardous chemical inventory

        • research animal inventory

    • library tools

        • e.g. Evergreen, Koha -- currently not much traction in higher ed

• area-specific tools

    • PostGIS

    • Music

        • ear training

        • notation

    • UGA

        • jExam for chemistry and math

        • English

            • openoffice

• common LMS tasks

    • class rolls

    • student/faculty interaction (message boards, wiki, mailing lists, chat)

    • shared calendar

    • tests (creation, distribution, grading)

    • file sharing

• Moodle

    • CMS

    • "Modular OO Dynamic Learning Environment"

    • "designed to support a social constructionist framework of education"

    • two editions of an O'Reilly book available

    • GPL licensed

    • PHP/MySQL, PostgreSQL, MSSQL, or Oracle

    • tasks

        • class management

    • support

        • forums, conferences, mailing lists, paid support vendors

• Sakai

    • named after an "iron chef"

    • group of lg universities

        • Indiana, MIT, Berkeley, Michigan, Stanford, etc

    • course management

    • Mellon Foundation Grant

    • educational community license

        • is an open source license, compatible with GPLv3

        • Java based, MySQL or Oracle db

    • very similar to Moodle in terms of what it does

    • support

        • forums, conferences, mailing lists, paid support

• Kuali Foundation

    • name comes from malaysian for "wok"

    • started in 2004

    • Indiana, Hawaii, but more business focused

        • corporate partners on their board (IBM, Sun/Oracle)

    • educational community license

    • Java + Oracle (Oracle only)

    • tasks

        • admin/financial software

        • not a course management system

        • being expanded to research management and web services

        • also starting work on student management

    • same support options as sakai and moodle

    • can pay to join the foundation and get better support

• .LRN

    • "Learn, Research, Network"

    • Based on OpenACS (1998)

        • ArsDigita Community System

    • GPLv2

    • architecture

        • AOLServer instance

        • Tcl

        • PostgreSQL/Oracle

    • tasks

        • class mgmt

        • also inherits from OpenACS framework

            • ecommerce

            • expense tracking

            • photo albums

            • presentation software (wimpypoint)

    • same support options + active IRC channel

• big trend now is towards third-party hosted apps, specifically google

    • google apps education edition

        • email, calendaring, etc. is hard--let google do it for you

    • have no idea how google is getting past ferpa and hippa

    • ASU is running all their mail off google, other large universities

    • "cartoon physics roadrunner principle"--if you keep running and don't look down you won't fall

• survey tools

    • surveymonkey

    • google forms

    • again, outsourcing university data to a third party

• facebook apps

    • haven't seen this yet, but get ready

    • students, faculty -- many already on facebook

    • logical to say "why not do a class message board on facebook!" -- short leap

    • again, legal issues about outsourcing data to facebook

• Second Life

    • at least 2 courses that are taking place in Second Life

        • "I don't even know what that means!"

    • who owns the data?

• resources

    • moodle.org

    • sakaiproject.org

    • kuali.org

    • dotlrn.org

    • openacs.org

Open Source Bridge: Trust the Vote - An Open Source Digital Public Works Project

Matt ? (didn't get last name)

Director of Communications, Open Source Digital Voting Foundation



• video

    • proprietary voting systems still being used

    • got past hanging chad, but seeing more problems, not fewer

    • security concerns invite election fraud

    • little chance machines are going away--companies make lots of money

        • expense to companies for increasing transparency doesn't make them money

    • need to trust our votes will count

    • cannot and should not expect the private sector or govt to fix the problem

        • needs to be a grassroots movement

    • need to shift away from proprietary companies controlling voting machines

    • need to move from black box voting to glass box voting

    • OSDV isn't just a think tank

        • technology professionals teaming with virtual community of volunteers

        • building actual voting machines and software

• pop quiz

    • true/false--there are fed guidelines for how votes are counted

        • false--elections are handled at the local level

        • not all states have counties--very irregular system

        • most of the election controversy that we've had have been over how ballots are counted

    • T/F--absentee ballots are always counted

        • false--"ballot of last resort"

        • predicted that very shortly this will change--court case out of TX being heard by the supreme court, which may overturn section 5 of the voting rights act

    • T/F--major voting vendor systems rely on current commodity hardware/software

        • false--not current

        • many run on windows 95 or windows ME

        • hardware can be a z80 processors (late 70s, early 80s)

• how america votes is the cornerstone of democracy

    • buzzword compliant, but true

    • market for voting machines is a malformed market

        • little incentive to innovate

        • process of becoming certified can run $6-7M

        • not a huge market--very limited customer base for selling voting machines

        • if it's a public company, responsibility to shareholders

            • if conflict between public and shareholders, companies go with shareholders

        • this is all bad from a democracy standpoint

• critical infrastructure--what do "we the people" own?

    • we own roads, etc.

    • needs to be critical democracy infrastructure

    • shouldn't be left to companies, something the people should own

• players

    • premiere (formerly diebold)

    • hart

    • sequoia

    • ess

• problems

    • decertification by states of a lot of machines--not trustworthy

    • osdv briefed justice dept--strong sense that only 2 or maybe only 1 of the voting machine companies will even exist by the end of the year

• measures of success

    • modular, adaptable

    • trustworthy, repeatable

    • usable for voters and people running elections

• imagine a clean slate--how would this work?

    • freely licensable software for voter registration, ballot casting, and election mgmt

    • needs to handle the whole election lifecycle

        • certification at federal level mandates this

    • has to interface with external systems (DOJ, INS, etc.)

    • need to have a process with public review

    • need to have a new set of licenses for how oss can be accepted by municipalities

    • need to have a roadmap for development and adoption

    • need to accommodate special needs for voting

• why not the commercial sector?

    • could a commercial but altruistic company be created to handle this?

    • barriers to innovation

        • expensive to get in the game

        • ROI is very low

        • need to have defensible IP so we can make money

        • if it's transparent, anyone else can copy it

• why shouldn't the govt do this?

    • too many potential political issues

    • development by committee

    • everything takes far too long

    • "NASA of voting systems"--development could take decades

• transparency

    • other advocacy groups out there talking about voting

        • open voting consortium, electronic voting machine project, academics

        • govt itself also talking about it, bills being sponsored, etc.

    • what makse OSDV different is they're working on actually building stuff

• digital public works project is solution

    • lack of transparency -> open source

    • inconsistent behavior -> fault tolerant

        • can't have the machine shut down if there's a problem with one ballot

    • conflict of interest -> non-profit

• open source is a provocative term

    • can mean licensing, can mean how development happens

    • osdv focusing on the former

    • IP is owned by the people of the united states

• osdv is producing systems that people can see, touch, and try

• in some cases are rules in place that prohibit technology being used

    • e.g. blackberries didn't used to be allowed on the senate floor, but the rules have been rewritten after the fact

    • easier to build first, work on certification process and adoption after the fact

• development process

    • core team (CTO + 5 core team members)

        • db services

        • voter intelligence services

        • user experience

        • security/privacy

        • infrastructure

    • rfc service

        • design congress

        • made up of elections directors for all of the states

            • these people know the most about how elections are actually run

        • rfcs turned into user stories

        • user stories become design specs

    • end result is voting technology

    • after it's built, federal and state certification

• public technology repository

    • public draft standards & specs database

    • election markup language (XML)

    • voter registration system

    • public test harness

    • commercial deployment license and public development license

    • commercial vendors can license the software, or govts can use public dev license

    • independent testing and certification

• projects

    • digital voter registration system

    • ballot design studio

    • ballot casting and counting

    • election management system

    • operating system platform

    • down the line--open source hardware running the open source software

    • OSDV is a 501c3 (charity)

    • strong coroporate support (HP Labs, Oracle, Linux, Mozilla Foundation)

    • working with Rock the Vote to provide registration systems

        • Rock the Vote reg systems currently on 22,000 web sites

        • RTV accounts for 70% of new registrationg

        • target to have 90% of all new voter registration in 3 years

• Technology

    • using Ruby on Rails, Linux, Apache, MySQL

• Learn more

    • http://trustthevote.org

    • twitter: @TrustTheVote @icount

    • source: github.com/trustthevote

Open Source Bridge: Ask Forgiveness, Not Permission

Emma McGrattan

Sr. VP of Engineering, Ingres

Board Member, Eclipse Foundation



• if you ask permission, the typical answer you get is NO

• much easier to ask for forgiveness

• budget challenges

    • 80% of budget goes to operations -- license fees, subscriptions, power, etc

    • 20% -- new investments/innovation (rarely higher than 30%)

    • as people being asked to reduce budgets, they're cutting off new investments or even canceling existing projects

        • this cuts off the ability to innovate

        • "if you stick your head in the sand you're exposing another part of your anatomy"

    • you can cut operation costs so new investments don't get hit so hard

    • how do you maximize the impact of budget in innovations? open source

    • people often cut heads without thinking about the real impact

        • service levels, unique skills/knowledge--very hard to get this back

        • demotivating for the people who are left

• old economics of IT

    • what is vendor lockin?

    • architectural lockin

        • lockin to proprietary stack

        • no access to source code

        • even if bits of things are open source, the stack as a whole is proprietary

    • commercial lockin

        • high upfront fees

        • logic is buy more and save!

        • yearly maintenance and support fees

        • annual costs NEVER decrease

    • can often be expensive to get off of proprietary stacks

• evidence of vendor lockin

    • oracle, sap, microsoft raised support/subscription prices 15-20% last year

        • they aren't giving 15-20% return on this in terms of better support or a better product

        • they raise prices because they can

    • in databases specifically, little innovation from early 90s to today

        • pay huge prices in some cases for a commodity

        • same with operating systems -- not a lot has changed in the last 15 years

• commercial lockin

    • if one company buys another company, the proprietary licenses of the company being acquired cannot be transferred

        • e.g. bank that got $20M bill for database licenses when they bought another bank

    • with proprietary software, if your organization shrinks your fees don't necessarily come down

• breaking vendor lockin

    • technology -> open standards

        • right product for job

        • high portability

        • reduce vendor monopoly

        • commodity pricing

        • move from premium to commodity

    • commercial -> open source

        • no license fees

        • no internet access fees

        • pay per use

        • access to source code

        • flexibility and choice

• why open source?

    • enabler of innovation

    • can sneak new products in under the radar

        • added benefit of vendor independence and flexibility

    • gartner study--biggest reason people look at oss at this point is cost

    • many people concerned about security

        • proprietary companies often spread FUD about oss from a security standpoint

• this has been done before

    • late 90s

    • e.g. release of XP forced upgrades of hardware

        • forward-thinking CIOs adopted linux to save money

    • as software is proven and support improved, replaced business critical systems

    • save millions moving from premium to commodity pricing

• the next step - evolving your IT portfolio

    • reduce costs in fixed cost side of business

    • take a look at the entire portfolio of applications

    • look at projects that can be delivered in a short time and have a big impact

    • can't impact operations in a big way

    • if a conversion project is successful, easier to talk about new development

    • applies across all levels of the stack--db, app server, etc.

    • focus on stuff that's easy to move initially

• benefits to oss but also hidden costs

    • no license fees -> selection/audit fees

    • pay for what you use -> internal support

    • no license mgmt -> skill transfer and training

    • no ongoing maintenance fees -> internal dev costs

• countering objections to oss

    • security

    • availability of support

    • some people still see oss as niche

    • open source is unregulated and anyone can contribute code

        • need to educate people around oss development practices

    • open source if for hobbyists and cannot support mission critical apps

• dispelling myth of security through obscurity

    • software should be secure by design

    • vulnerabilities are typically found through reverse engineering, not through reading source code (e.g. MS IE)

        • even in firefox, bugs are typically found through using the product, not through reading the source code

    • linux kernel has been found to contain 1% of the bugs that a typical closed source project would contain (Carnegie Melon)

• professional level support

    • enterprises want 24/7 support

    • with oss, more than 1 vendor to choose from

    • open source model drives comapnies to provide superlative support

    • choice between professional support and community support

        • can choose depending on needs of project

• open source in the enterprise

    • eclipse

    • ingres, mysql, postgres

    • linux, bsd, opensolaris

    • apache

    • tomcat, jboss

    • open office

    • firefox

    • thunderbird

    • alfresco

    • jaspersoft

    • many of these are seen as the leader in their space over proprietary solutions

• anyone can contribute myth

    • need to educate people about oss development process

    • meritocracy, not everyone can commit, etc.

    • quality is better--more eyes on the code

    • security built in from the ground up (or can be)

• not suitable for mission critical myth

    • huge number of examples of mission critical apps running oss

• summary

    • move from premium to commodity pricing where you can

        • no license fees, can pay for only what you need, etc.

        • frees up money for innovation

    • move from lockin to flexibility and choice

        • reduce vendor monopoly

        • can use threat of moving to another platform as leverage with vendors

        • access to source code

    • don't ask permission, ask forgiveness

        • use open source software and free time to get things done

Open Source Bridge: 5 Things to Know About MySQL if You Don't Have a DBA

Sarah Novotny

Blue Gecko (MySQL and LAMP Services - http://bluegecko.net)



• changing ecosystem

    • MySQL -> Sun -> Oracle

    • Monty Program AB

        • Maria variant of MySQL

        • bringing together contributors and service providers around MySQL/Maria

    • OpenDB Alliance

    • no need to have concern--tons of community support, forks, Drizzle, etc

    • mysql.com packages/rpms handy when you don't need specific patches

• versions

    • 5.0 - stable

    • 5.1 - fresh feature set

        • many open bugs

        • partitioning (pretty stable)

        • row based replication (40 open bugs on this right now)

        • mysqlslap

        • 5.1 is also part of the new licensing setup

            • split license -- community edition and enterprise edition

            • many replication fixes already in enterprise version

            • community edition only released every other version

    • 5.4 - 5.1++ google patches

        • scalability improvements -> multicore

        • better concurrency with innodb

        • changed the locking code

        • enhanced monitoring / diagnostic

        • sane defaults for my.cnf out of the box

    • 5 is still the best version to use if you don't need what's in 5.1

        • know that you may have problems with 5.1 esp. with replication

• a short diversion

    • mysql has 3 real components

        • client - libmysql

        • server

        • storage engines - you can choose your storage engine based on your needs

    • High Performance MySQL 2d ed is a great reference

• storage engines

    • innodb

        • preferred in many cases--has transactions

        • row-level locking as opposed to data-level locking

    • myisam

        • default--doesn't support transactions

    • merge

        • "poor man's partitioning" -- can look at multiple tables as a single view

    • blackhole

        • does nothing but log to the binary logs

    • archive

        • append only engine--great for logging

        • can replicate to another server and change the engine type to do queries

    • federated

        • hack around some of the limitations of mysql

        • can access tables on remote servers

• locking

    • lock contention is one of the biggest scaling hurdles

    • innodb has row level locking, myisam has table level locking

    • innotop is a great tool to show what's locking

• backups

    • take them and test a restore

    • mysql has a lot of ways to be backed up--make sure you can meet mean time to recovery

        • time to recovery will dictate backup strategy

    • mysqldump on a running server is now always a valid backup

        • if you put mysqldump in a transaction you lock the entire db during the dump

    • replication is easiest and gives you a warm standby

        • can also take cold backups off of this at night

        • can use for testing, but then not a valid standby

        • can use it for backups without affecting IO on production server

    • innodb hot backup

        • closest thing to an oracle backup that mysql has

        • log and file based

    • xtrabackup

        • free version of innodb backup (?)

    • mylvmbackup

        • lvm snapshots are a great way to take backups

        • can be taken from replication server or production server

        • set of scripts that locks the db, takes the snapshot

    • backups lead to clones

        • if you have a replica you can move to a test server and play with things

• scale

    • when it's time to go big what are the options?

    • separate reads from writes

        • even if they aren't going anywhere different early on this opens up scaling opportunities later

        • makes for a simpler, cleaner, more cost-effective way to scale

        • write 1 / read many

    • partitioning / sharding

        • sharding is generally across multiple servers

        • partitioning is generally on a single server

        • no standard interface for sharding at this point

    • multimaster / rings

        • multimaster = ability to write to many dbs, have changes synchronized to all slaves

            • difficult -- can be done, but it's difficult

        • rings

            • very brittle -- if something breaks fixing it is difficult and manual

    • memcached

        • can be used for transient tables

    • wafflegrid

        • taking memcached and using it to spread the innodb buffer pool across multiple machines

        • larger btree indexes can be kept in memory--faster inserts

        • patchset that can be applied for mysql

• another short diversion - to swap or not to swap

    • turning off swap gives you different risks

    • with no swap, out of memory errors are catastrophic

    • with swap, as memory fills up it can get unusably slow, but it doesn't crash

    • question with swap is what happens it he OS swaps out something like an innodb buffer?

        • easiest solution is to use large page files

• free tools

    • innotop

        • show queries, query statistics, replication status, can kill queries, etc.

    • maatkit

        • used to fix replicas that get out of sync

        • maketablesync, maketablechecksum

    • mysql proxy

        • handy for doing load profiling

        • good for understanding what's going on between your application and the db

        • mysql proxy sits between the app and the db so you can see what the app is sending to the db

        • can be thrown into a production system with a couple of firewall rules

    • monitoring/trending

        • e.g. nagios but any will do, just do it

    • cacti templates

        • provides graphing for mysql metrics

• additional resources

    • irc.freenode.org

        • #mysql

        • #maatkit

    • mysql.com

    • High Performance MySQL 2d ed

Open Source Bridge: Thursday Keynote

Sam Adams, Mayor of Portland



• intimidating with so many people online, so i'll fight back--let's see if i can twitter and give a speech at the same time

• city govt has been unnecessarily cloaked in proprietary software

    • laggard in the use of open source

    • going to do everything i can to change that

    • city govt will be a laboratory for local efforts to push innovation in area of oss

• portland has 25 year record of investing $4M/yr in a system of citizen involvement

    • history -- started with neighborhood associations -- supporting logistics for that

        • asking opinions/input on what the city govt should be working on

        • still important--90+ neighborhood associations and numerous business associations

• govt should be open, transparent, and aggressive in getting people's opinions

    • given this, ironic that we're so behind in terms of open source

• have opportunity to help our local efforts because we're big spenders in area of digital media

    • if we can partner with open source advocates more, will make efforts more successful

• recently Vancouver BC passed resolution to make their govt as open as possible

    • Portland has "friendly competition" to out-open-source Vancouver BC

• if you're frustrated with govt efforts in this area ...

    • when i was elected as commissioner i wanted to have a blog where people could leave anonymous comments

    • wasn't allowed to do it--legal and tech reasons

    • set up "pirate" web site and ran it for 4 years

    • when elected mayor, lo and behold, they found a way to let me blog

• in the months ahead i hope you'll partner with us to change the way we do business

• have very talented technology folks working for the city

    • but still trapped in the historical force of the bureacracy

    • 5400 employees, $2.4B budget

• Portland recently came in second place behind Washington DC in terms of best online presence

• Portland currently doing RFPs - http://www.portlandonline.com

    • create applications that will take the amazing amount of data we have and make it relevant, real, and compelling for citizens

    • DC is taking "boring lists" of things like traffic crashes, etc. and is making them come alive

    • important for the basics of democracy--this data is important for making decisions, but it's not accessible to decision makers or citizens

• portland known as a place of sustainability and open, transparent govt

    • keeps govt more accountable

    • more data can be made understanable to citizens, the more benefit will come from it

    • seen cases where once data was available, decision makers can change their minds radically

    • scary that so much good information is inaccessible

• software isn't just important to govt, it's one of the four core industries for this region

    • sustainability

    • technology

    • outdoor gear

    • software and digital media

• govt needs to improve, but tech industry needs to improve as well

    • sometimes difficult for non-tech people to get a handle on things

    • tech people need to continue efforts like this so govt can figure out how we can best help you

    • govt needs to create a strategy that will help technologists every day

• goal is to work with open source advocates to grow portland's reputation as a hub for oss

    • draft of proposal for more use of oss in city govt

    • oss advocates need to speak up, show their numbers in this effort



Ward Cunningham, CTO of About Us (aboutus.org), Inventor of the wiki


"Innovations in Teamwork"



• as humans we naturally cooperate, but it never seems easy

• agile development corrects dysfunctional behavior resulting from decades of misunderstood risk

    • lots of people make software development more difficult than it needs to be

    • traditional concern of software managers has been that the project is going to be a disaaster

        • pour lots of money into something but get nothing in return

    • to address this problem, managers seem to follow the model "when developers program they screw stuff up, so let's keep them from doing anything"

    • overplanning is dysfunctional--90% planning, 10% actual work doesn't succeed

    • agile says "let's start in a direction and change direction as needed"

        • even with the best planning, things are going to change anyway

• agile teams learn to serve the previously invisible needs of their customers

    • teams need to know how to talk to each other AND to the customer

    • what's cool about agile is that by using the software to talk, it brings up issues that the customer couldn't articulate without seeing things in action

    • sounds hard until you do it, then you realize it's easy

    • "shared experience of watching software come to life"

• organize by Location

    • put teams close together so we can facilitate communication

    • pair programming--"radical colocation"

• remember we're doing this for the customer

• Agile

    • Concern: Risk

    • Barrier: Plan

    • Team: Location

    • Serves: Customer

    • agile basically stands this on its head

    • people who finish writing applications get to talk about what it's like and how you do it, not people with some abstract "expertise"

• wiki was invented to support a world-wide community

    • activist thing--desire to change the way software was done

    • in this process, "accidentally" reversed the traditional approach to knowledge

    • Concern: Knowledge

    • Barrier: Privilege

    • Team: Attention

        • idea is the people who have "been there" share their knowledge

    • Serves: Reader

• Open Source - how did open source people get so smart?

    • Concern: Property

        • people putting money into software think they need to hold onto it tightly

        • people almost seem to think if someone else "takes" their software and uses it, that's bad

        • people using your software is good!

    • Barrier: License

        • notion that you can own software is crazy

        • more appropriate patent law to apply would be like mathematics

            • you can't patent a formula

    • Team: Merit

        • assemble teams, and commit rights are based on merit

    • Serves: Developer

        • much open source comes out of a developer scratching their own itch

• Go out and innovate

    • so much opportunity to study model above and apply it

    • reverse the traditional models

• work together--it's harder than working alone, but more fulfilling, end result is better

• look for texture

    • e.g. pair programming--just pointing at the screen can say a lot

• you have the freedom

    • time and opportunity is there

    • opportunity to create is immense with all the tools and libraries we have available

• share what you discover

    • anything you give away comes back to you 10 times over

Open Source Bridge: Work With the Government for Fun and Profit

Deborah Bryant

OSU OSL, Runs GOSCON



• today's talk

    • snapshot of where os is being used in govt today

    • why govt needs your skills (or is just about to find out they do)

    • govt procurement systems including culture

    • brief intro to hoops

    • ideas that can get you working on govt projects

• extra stuff

    • what's happening in DC

    • with big federal agency will be the first to give into floss

• about the speaker

    • public sector communities manager / goscon.org director

    • former state deupty cio

        • first egovt platform in 2000

    • hopelessly optimistic

        • democracylab.org

        • osdv.org

            • national spec for open source voting systems

        • open health tools

• where is oss in govt today?

    • infrastructure - extensive

    • database and back office systems, gis, cms

    • increasingly replacing legacy systems

    • collaboration tools, wiki, blogs, vertical app development

• why government work?

    • once established, can provide a long-term source of income

    • projects tend to move slower, provide greater workload flexibility

    • govt agencies always pay their bills

    • add credibility to your client portfolio

        • basic "litmus test" passed

• why not govt work?

    • long ramp to establish oneself as a contender

    • formal rfps can be challenging to respond to

    • cost of doing business with may exceed your threshold

    • procurements can be pulled back at any time

• procurement

    • govt is not monolithic

    • oregon example

        • 135 agencies, boards, and commissions

        • 35 counties

        • 250 cities

        • plus myriad special districts (water, sewer, utility)

• all govt shares common values

    • public, open and competitive

    • equal access for all vendors to information

    • equal consideration based on objective criteria

    • increasing formality as costs increase

    • must have no local preference by policy

    • can prefer a minority business

• oregon has a public procurement system online

    • can be informed when bids are open

    • can see who's winning bids

• common procurement strategies

    • personal services agreement

        • low threshold ($5K), limited use

    • competitive pricing

        • call around for three prices

    • requests for information (RFI)

        • trolling for better info to develop an RFP

        • vendors who would actually respond to the RFP don't respond to RFIs

            • reveals strategy to competitors

    • master services/regional collaborative contracts

• get started strategies

    • sign up for procurement notices

    • go talk to someone who works for government

    • get on approved vendor lists

    • identify companies winning IT bids today and partner with them

        • e.g. find a company getting contracts that don't already have your skills

    • if you have more money than time, hire a professional to get you started

• rfps: what to expect

    • merciless deadlines

    • ruthless rule-following

    • information holes large enough to drive a motor-pool car through

        • but you get to ask questions

    • no dickering with terms and conditions

• safety tips

    • read solicitations carefully! (x3)

    • if you can't stand reading long documents, pay a smart person to do it for you

• if you're lucky enough

    • be flexible in your business model

    • don't let the scope creep

    • all the usual best practices to ensure you're on track

    • leave the organization behind you stronger

        • don't just create opportunities for you to get hired again

        • knowledge transfer is important

• deb widsom--relationships

    • be respectful--could YOU do their job?

    • be transparent

    • ask for advice

    • be helpful

    • don't buy lunch

    • don't spam or contact during an official procurement process

        • follow official channels

    • always deliver what you promise

• crystal ball: opportunities

    • local govt wants: training!

    • oss-savvy project management

    • writing rfps or grant requests intended to encourage oss

    • health IT, esp. electronic health records

        • legislation on the table related to this already

    • public safety/first responders

        • data interop, GIS

        • huge opportunity for open source in local govt--not everyone can afford an ESRI license

    • govt doesn't mind paying for services but they don't want to pay for licenses so much anymore



Questions

• if you have a product the govt needs but they don't know they need it, how do you pitch it?

    • have to find someone in govt who thinks they have the problem

        • if they don't think it's a problem, they won't bite

    • give it away the first time

        • ask about what the process would be

Open Source Bridge: Assholes are Killing Your Project

Donnie Berkholz

Gentoo Linux

http://dberkholz.wordpress.com/



• what do people want to get out of this talk?

    • how handle other than banning or feeding the trolls

    • how take negativity and turn it into something

    • how do you help people realize that they're assholes?

    • what if the key leaders in your floss projects are assholes?

        • shouts of "FORK!"

• reasoning for giving this talk

    • been contributing to gentoo for 6 years

    • what you hear about gentoo a lot is "it's falling apart, it's dying"

        • that sentiment has been around for 5 years, but it's still around

        • damage done by a small amount of poisonous people

        • damages not only the community, but your POTENTIAL community

    • want to learn what i can to do what i can to save the projects i care about

• community is THE MOST IMPORTANT aspect of your project

    • contributors -> results -> reputation (cycle repeats)

    • if reuptation suffers, you lose people and the cycle breaks

    • if repuatation isn't good, people may pass on the project based on negativity that has no basis in reality

• what makes a great community? what makes a horrible community?

    • people's social abilities on a project fall on a gaussian curve

    • big middle chunk of people who have decent social skills, don't save the day, don't make trouble

    • outliers--"the best"--both technical and social (rarely the same person)

    • on the other end of the outliers you have the assholes

    • need to get quantitative and figure out things like commits, mailing list posts, etc.

    • harder to get metrics for social aspects

        • can measure things like do posts go down after an asshole posts?

        • how quickly do people unsubscribe?

    • to make a great community and avoid a horrible one, you need some sort of numbers

        • otherwise how do you know you're improving?

        • better to have an imperfect measurement than no measurement at all

    • need to look at both sides of the gausian curve--have same # of outliers on both ends

        • do they balance each other out? unfortunately no

    • how much does it hurt people when you're a jerk, vs. how much does it help people when you're nice?

• what isn't an asshole--definition

    • conflict is good--just arguing doesn't mean there are assholes involved

    • problem with assholes is that if people feel like they're going to get insulted personally, they won't say anything

    • focus on IDEAS in conflicts, not personal attacks

    • Intel offers classes on how to have conflicts with people in a productive way

• what is an asshole? two tests

    • after talking to the asshole, does the target feel oppressed, humiliated, de-energized, or belittled?

    • does the asshole target those less powerful? (or, is everyone targeted equally?)

        • being equally mean to everyone doesn't really make it better

    • people who study incivility in the workplace have found it the power issue doesn't really matter, but more often than not attacks are made on people less powerful

    • trolling doesn't target those less powerful--can go both ways

    • first test is really the key one

    • executive director of the gnome foundation on twitter: realize i've been in a bad mood because of an email i got hours ago--impact is lasting

• patterns

    • who's always having a "bad moment?"

    • if bad moments happen 90% of the time, you're probably an asshole

    • EVERYONE will be a jerk every so often--non-assholes realize what they've done and apologize

    • can be quantitative with this--ask others on the project how they feel after interacting with the suspected assholes

        • problem is there is no baseline, so you'd have to ask about everyone, so this is an imperfect measurement, but at least it's a measurement

    • psychologists say it's "states vs. traits"

        • temporary asshole vs. just an asshole

    • problem is over time, people build up a tolerance and don't realize the true impact of the assholes on new people

    • e.g. debian development mailing list--some of the assholes get put on ignore by active developers, but new subscribers see the bad stuff while the developers have their head in the sand

    • need inflow of new people not being scared away by the environment of your project

• problems assholes cause

    • does the good balance out the bad?

    • does one insult balance out one compliment?

    • it doesn't balance out

    • how many positive interactions does it take to balance out one negative one?

        • negative interactions are 5X more powerful than positive ones

        • less than 1/6 of people on a project need to be assholes to barely break even

        • if you want to build a better community you have to beat this

        • your competitors are also trying to build a better community, so you can't stop dealing with this problem

    • huge gender difference in how people react to this behavior

        • in general, men fight back, wheras women run as opposed to retaliating

        • already a low ratio of women in technology--this makes it worse

    • assholes affect the project as a whole, and the leaders of the project get blamed

• problems assholes cause: targets

    • 48% decreased their effort

    • 47% worked less time

    • 38% dropped their quality

    • 66% declined performance

    • 80% lost time worrying

    • 63% lost time avoiding

    • 78% became less committed

    • 25% quit; 20% of WITNESSES quit

        • hard to track--people who quit might do so up to a year afterwards

        • witness issue is very important--on open source projects there's a mailing list so everyone on the mailing list is a witness

        • to put it another way, if you get 4 new contributors, 1+ will quit due to assholes

    • this is HUGE in a volunteer project

• cascading effect amplifies the problem of assholes

    • targets often retaliate against other people

    • very easy to be a jerk to someone you don't know, so this is a big problem on mailing lists

    • anything you can do to make things more personal will decrease the levels of hostility

• problems assholes cause: projects

    • reduced innovation and creativity

    • reduce cooperation and cohesion

    • cost of targets' retribution toward project

    • impaired cooperation from external projects & people

    • impaired ability to attract the best & brightest

    • recruiting more assholes

        • people tend to hire/attract people like themselves

        • in an open source project, everyone's a recruiter

        • can wind up with clusters of assholes

    • key is constructive conflict

        • reject the idea, not the person

    • e.g. from "no asshole rule" book

        • company's best salesman--after firing, productivity went up 30% because everyone else became much more motivated

        • if someone's an asshole AND they're contributing, this is what makes things tricky

• TCA: total cost of asshole

    • time spent by team lead

    • time spent by developer relations team dealing with complaints

        • multiplied by # of members

    • time spent by project leaderhip

    • time spent recruiting and training new developers

    • time lost by targets

    • on gentoo--leadership spent 3 months discussing a small cluster of people who were causing problems

• an asshole is incompetent

    • they may be a very good coder, but a project is not just about coding

    • it's about being able to work with other people

    • politics is how decision is handled as a group and how people work with other people

    • social ability is important

• how do you fix it?

    • personal interactions--easiest and best way to deal with it

        • even informal chats on im/irc can help

        • get people in the same room

    • model what a good interaction is like

        • don't be an asshole yourself and you're setting an example for others

    • show people what the community norms are--create a code of conduct

        • takes a long time to figure out a culture when nothing it documented

        • problem is that once something is written down, assholes find the loopholes

            • then the argument isn't even about right or wrong anymore

    • need to have a specific way of dealing with assholes

        • where do the targets go when they have a problem?

        • have to have a place to report the problem and encourage people to report it

        • also, something needs to HAPPEN when something is reported, otherwise it's pointless

        • need to respond quickly to everyone involved

        • remember that one instance is not a pattern--have to count things over time

        • BUT if something's total egregious, one instance may suffice

    • reducing smaller instances can help avoid the larger ones--things don't escalate

• all projects have a mission

    • project mission isn't "make these assholes better people"

    • we're trying to write software, not fix people

    • on the project, need to focus on specifying and enforcing what's ok and what's not

    • if you've tried 3 times with someone and they don't change, they're never going to get it

• how do you prevent it? tomorrow's assholes

    • code of conduct--not a set of rules, but generalized cultural guidelines

    • recruitment standards--usually can spot the assholes early

        • do research on people you're bringing on a project

        • have a probationary period--figure out whether or not someone belongs in your community

• resources

    • The No Asshole Rule

    • The Cost of Bad Behavior (releasing July 9)

    • Getting Past No

• in the long run dealing with assholes is never worth it

Open Source Bridge: Information Security for Open Source Projects

Kevin Kenan

www.0xc0deshop.com



• definition

    • information security succeeds when it keeps business disruptions caused by unauthorized access to an acceptably low level

• disruptions include

    • downtime

    • customer dissatisfaction

    • derailed strategies

    • lawsuits

    • investigations

    • PR issues

    • fines

    • extortion

    • loss of competitive advantage

    • theft

• in terms of resource allocation, if you aren't having any of these problems, how do you determine LOE on this?

    • why aren't you having any issues?

        • might just happen to have excellent security already

        • might not have been targeted

        • might not have noticed attacks that did happen

    • justifying spending on security is more difficult if you haven't had any problems yet

    • think of it like insurance--just because your building hasn't burned down doesn't mean you don't need to worry about fire

• question: regarding the fire analogy, are there guidelines like building codes?

    • some are out there, but are largely descriptive

        • PCI, NIST, OWASP, HIPPA, FERPA, etc.

• security is not a binary condition

    • even the most secure businesses can still suffer breaches

    • just because you've had a breach doesn't mean you're not secure

    • information security is a risk management issue

        • mitigate risks, determine where for the business the comfort level lies

    • make your business more resistant to attack

• what can you do?

    • assign someone the authority to make them accountable for security

        • otherwise it's just lip service

        • accountability doesn't mean "if there's a breach, fire them"--this is about defining a program

    • create an operationalized production environment

        • access control, configuration control, change control

        • without this, hard to keep track of who's accessing what when

        • security is built on a CONTROLLED IT environment

        • itpi.org - "Visible Ops" - step by step procedure for getting to sanity

            • follow-up: "Visible Ops Security"

    • build a detection and response team

        • pyramid - bottom up:

            • logging -> aggregation and filtering -> daily review -> response

        • with these metrics in place you have something concrete to show mgmt

            • don't make any of this a skunkworks project

            • don't monitor individuals if not authorized

    • build a security assurance team

        • operating systems

        • networks

        • malware defense

        • data protection

        • applications

        • identity management

        • encryption

    • implement a vulnerability management program

        • art of finding weaknesses in the computing environment

        • this can be time intensive

        • think about segregation of duties--people configuring systems shouldn't be doing the vulnerability testing

    • implement a product security program

        • vulnerability response plan

        • customer guides and tools

        • security code reviews

        • secure coding standards

        • coverity code analysis

            • if you have open source code coverity will scan it for free

            • great thing to show customers

        • source code control

        • security testing

• open source and security

    • perception that proprietary code is more secure

    • proprietary vendors are spreading FUD about the security of open source software

    • is important for open source businesses to demonstrate that they take security seriously (see product security program above)

    • make all your security information and testing open

        • can be run by third-party companies

        • challenge proprietary companies to do the same

Open Source Bridge: Layers of Caching - Keys to Scaling Your Web Site

Lance Albertson (Lead Sysadmin/Architect for OSU Open Source Lab) and

Narayan Newton (Partner, Tag1 Consulting)



• importance of caching

    • ram is fast and cheap

    • utilizes resources more efficiently

    • improves site experience for the users

    • serve high traffic sites with fewer servers

    • has become increasingly important with rise of dynamic, user-driven pages

        • there is a limit to how quickly you can throw out a page that hits a database

• cache layers

    • application caching

        • e.g. drupal works with blocks on a page--can cache content in specific blocks

        • pushing small caching bits like this throughout the app can make a big different

            • don't even have to cache for long to make a big difference

    • query caching

    • opcode caching

        • caching compiled php/python files so code doesn't have to be compiled over and over

    • reverse proxy caching

        • squid, varnish

        • cdns list akamai

            • make sure http headers are being set properly to allow caching

            • many frameworks don't set http headers correctly for caching

• application caching

    • most frameworks include caching internally

    • optional modules/plugins add more caching

    • research application/framework before deploying

        • many apps/frameworks don't have caching enabled by default

        • some apps can be set to write out static html files

    • database driven vs. file based

    • preventative caching

        • caching an object, page, section of the page so it doesn't need to be built ever time

        • might have to think about this at development time depending on the framework

        • how stale can this content be?

        • remember that there's a limit to everything--speed, size of server, etc.

• query/object caching

    • memcache/apc

        • most frameworks have support for some kind of object cache

        • used to prevent repeated database queries for frequently requested data

        • latency vs. coherency

            • e.g. amazon -- network latency between web nodes

            • memcache can be on a single server or spread among multiple servers, but typically it's a single bucket

            • in some cases people use multiple memcache buckets to decrease latency, but this also decreases coherency since each cache could have a different version of the data

• opcode caching

    • reduces cpu cycles typically

    • apc (alternative php cache) or eaccelerator

        • bug - segfaults apache on occasion

    • compiled apps in memory == good

    • apache modules

        • mod_wsgi - python

        • mod_passenger - rails

        • mod_perl - perl

• reverse proxy caching

    • framework/backend agnostic (http based)

    • squid, varnish, mod_cache

    • pros

        • cached content served without hitting web server

        • extremely fast

    • cons

        • difficult to use with dynamic web sites

    • lazy cookie/session initialization

    • can use http headers to split cache on a specific "vary key" (vary header)

        • e.g. split cache between users who are logged in and those who aren't

        • users who aren't logged in get cached data that's still valid

    • squid2 most stable, performs better than squid3

    • varnish2 performs better than squid, but doesn't do well with large objects like isos

        • varnish is only reverse proxy, whereas squid does forward proxy as well

• VPS tweaks

    • for many of the tips above you need lots of ram

    • VPSes don't always have enough RAM to do everything mentioned above

    • memcache is probably out

    • preventative caching

        • eaccelerator -- writes to disk when it runs out of RAM

        • reverse proxy caches

    • low memory footprint

        • nginx/lighttpd

        • MyISAM

        • strip out modules in apache that you don't need--they all take up RAM

• edge side includes

    • squid3 and varnish support it

    • can tell the reverse proxy what parts of the page to cache--can expire individual parts of pages

• cdns

    • limelight and panther are much cheaper than akamai

    • amazon s3 is a "crappy" cdn, but it can function as a cdn

• load testing tools

    • siege and jmeter

    • "users complaining it's too slow"

    • apachebench

Open Source Bridge: Getting Started in Open Source

Cat Altman and Leslie Hawthorn, Google Open Source Programs



• why do people in the audience care?

    • more creative approach to problem solving

    • better quality software

    • cheap!

• why should you care?

    • gain new skills

        • lots of opportunity to be surrounded by people that are smarter than you

    • gain new contributions

    • professional networking/career opportunities

        • public, reputable body of work visible by anyone

    • professional gratification

    • share your values

        • sharing work, sharing knowledge, making the world a better place

• to be involved in open source you don't have to be einstein

    • don't even have to be a coder

    • soft skills matter

• soft skills

    • testing

    • localization

    • documentation

        • people new to projects are ideal to write documentation--they're figuring it out themselves

    • release engineering

        • e.g. current release coordinator for SAMBA is a nurse, not a developer

    • UI/usability

        • pretty is nice, but useable is VERY good

    • user support

        • if you're a skilled user, you can help a lot

    • marketing

        • articles, how-tos, tutorials

        • graphic design

        • advocacy

        • event organization/staffing

        • projects look for end users--share your enthusiasm

    • fund raising

        • bandwidth, server space, etc. all cost money

    • community manager

        • "make the community a useful place for other people to be"

        • osbr.ca - article about being a community manager at google

• how do you get started?

    • find a need that matters to you

        • technical challenge - in open source you can make things work the way you want them to

        • social considerations

            • e.g. openmrs (medical record system)

            • footprint.org / allforgood.org - os aggregator of volunteering opportunities

            • "code as public service" - contribution to social good

    • find a community you are comfortable with

        • enough communities that you can find one that matches your personality

        • more interested in a brand-new project, or a pre-existing well-known one?

    • ask your friends

    • please note - limit your scope

        • easy to burn out

        • open source needs you! pick one or two things that you're passionate about

        • total understood in os that after you've made the contributions you want to make, you're going to move on to something else

• after you pick a project, know the history

    • there is the opportunity to change the world with floss

    • great way not to get flamed

    • review the project website/wiki

    • read mailing list archives/forum topics

    • look for/ask for noob documentation

        • if there is none, are new contirbutors welcome? or does this just need to be created?

    • check out the source code

    • do other research

    • you will feel like an outsider for a while

    • show that you have respect for the time of the people who've been involved with the project for a while

• observe the present

    • lurking has value

    • review mailing lists, irc, etc

    • how have people gotten involved in the past?

    • calls for volunteers

    • is the community tone welcoming?

    • is the project a place you'd enjoy contributing?

    • it's OK to walk away if it isn't a good fit

        • you won't have enthusiasm if the social aspects aren't a good fit

• create the future

    • encourage newcomers to ...

        • file a bug report

        • submit patches--be lenient here!

    • volunteer to help

        • make useful suggestions to potential contributors

        • if you're volunteering, don't say "I'll do it" and then have to have your hand held

• mailing list etiquette

    • don't rehash old discussions

    • don't reply to every message on a thread (me too!)

    • only reply to relevant text of a previous message

    • when in doubt, don't top post

        • problem with mobile clients

    • "me too" posts are a big no-no

    • no filibustering

    • if you just dove in, don't participate in +1/-1 voting right away

• communication annoyances--ways to look less professional

    • silly nicknames

    • multiple nicknames in different media

    • overusing CAPITAL LETTERS

    • using excessive punctuation or none at all

    • overuse of LOL type abbreviations

        • caveat: if this is the lay of the land in the community, then fit with the group style

    • assume you should be formal in the style of how you'd address a potential employer

        • don't know who's on the other end of the message or who might be reading it

• hazing is unproductive

    • there is a way to correct people who need correction

        • "I know you didn't realize this or you wouldn't have done it, but ..."

    • your internet behavior is eternal

    • "an eye for an eye makes the whole world blind"

    • think about how you want to be perceived

    • don't post drunk, tired, or mad

• more things to avoid - cluelessness

    • failure to pick up on the "mood"

    • not understanding common goals

        • .e.g svn was designed to be a replacement for cvs. it's their mission statement.

            • guy jumps on list berating people that they should work on cvs instead

    • asking rtfm questions

• avoid hostility and non-cooperation

    • angrily demand help

        • if you don't get a response in a week, re-read what you posted and see if you can improve it

        • if you still don't get a response, the project may be dead or there may be another reason

    • deliberately riling people

        • don't feed the trolls

        • donotfeedtheenergybeast.com -- good for people who are actually trying to help the troll

    • willing to complain but not fix

    • failing to accept criticism well

    • it's ok to walk away

• don't be scared!

    • open source community in general is a friendly place full of excited, smart people who want to make the world a better place

    • it's worth the effort to get involved

• summary

    • everyone was new once

    • remember how much you know -- and how much you don't

    • share your knowledge

    • contribute to the growth of your project

    • enjoy!

• resources

    • cathedral and the bazaar (esr)

    • how to ask questions the right way (esr)

    • free software foundation

    • karl fogel - producing open source software http://producingoss.com



Questions

• how would you answer the question "why do you do this without getting paid?"

    • there are many things in life we do that we don't get paid for--same motivations

        • love of it

        • may pay off in other ways (jobs, connections, services around software, etc.)

        • for the social good

Open Source Bridge: Open Source Microblogging With Laconica

• Evan Prodromou

    • wikitravel

    • laconica

• "I'm not Twitter. That's OK."

    • 100% of the money made on the web today is made by companies who aren't twitter

• passionate believer in open source, open data, open content, open standards

• we're moving our software and our life into a cloud that's dominated by a few companies

• facebook 2009 = aol 1992

    • microsoft = microsoft :-)

• this is bad for the web, for business, and bad for society

    • tim o'reilly, ibm, the economist also have this opinion

• autonomo.us

    • working on these problems from a number of angles

        • legal, advocacy, code

    • Franklin Street Statement (find url)

    • best way to have open cloud

        • floss + open data + open standards

• where to start?

    • found myself looking at twitter every morning--great place to start

    • status matters on the web today

• microblogging

    • shot notices, social network, multiple channels

    • mix of synchronous and asynchronous

    • mobile and pc

    • twitter grew 900% in the last year--mainstream is focused on it now

• uses of laconica

    • enterprise

    • community

    • public broadcast

    • government

• everything's gotta be connected

    • we don't see this currently--status isn't shared between systems

    • monolithic systems suck, walled gardens suck

• what we have now is one big twitter

    • isn't distributed

    • need more of a pub/sub model -- should be able to use different kinds of software/systems to get at statuses in other systems

• thesis: success depends on one simple protocol and one good server

    • has happened with email, web servers

    • hasn't happened with IM (xmpp close)

    • we need open source microblogging software

• requirements

    • thousands to millions of installations

    • installable anywhere, esp. commodity hosting

    • big, fast-moving development community

    • this all points to php + mysql

• other requirements

    • themable

    • extensible

    • programmable

    • scalable -- this is a big challenge

• scale has different aspects

    • people seem to focus on # of users

    • need to support both small and large scale deployments

    • ideally software would change not at all or only incrementally across the scale

• what i did:

    • laconica (http://www.laconi.ca)

    • web app written in php, uses mysql or postgresql

    • cloned the twitter rest api

        • got lots of twitter 3rd-party dev community on-board

    • lots of offline processing -- STOMP servers

    • Support Jabber for IM

    • email gateways for sms

        • being in the sms space is really expensive

    • hooks-based plugin architecture (like mediawiki)

        • events happen in the code, hooks available for these events

• openmicroblogging.org -- "good enough protocol"

    • http-based protocol

    • users on one server can subscribe to notices from users on another server

    • oauth used for authorization

    • subscription == authorizing a remote server to push notices into your inbox

    • version 0.1 - push text and profile data

    • version 0.2 (coming soon) - uses atom entries and vcards

    • identities are urls

    • don't define a syntax for the text in notices, but working with microsyntax.org

        • working on developing a standard for microblogging syntax

• distributed microblogging -- how do you do @ replies?

    • @nick -> guess

        • subscriber

        • more info = distributed over microblogging network

        • @nick@ex, @nick%ex

        • server-defined aliases, user-defined aliases

        • similar to email

• other nice things about laconica

    • use openid for authentication

    • twitter bridge

    • facebook application and facebook connect (logging in and posting)

        • facebook TOS are more strict that twitter so no facebook -> laconica yet

    • use hash tags officially

    • push data out to public downstream users (xmpp, ping, sup)

    • default install uses creative commons BY 3.0

    • also support groups, multimedia, sharing, conversation tracking (tree of notices vs. stream of notices), comet, user themes

• how are we doing?

    • launched 7/2008

    • 50+ contirbutors

    • dozens of plugins

    • 200+ public sites

    • many private sites: Sun, SAP, Motorola, Mars Inc.

    • goal: 1M sites by 2013

    • full-time job for evan, 7 employees

        • installation, maintenance, service, support, customization

        • software as a service offering - http://status.net (private beta)

    • venture funded, revenue positive

• who's going to use this software?

    • enterprises

    • brands and broadcasters

        • e.g. oprah's driving millions of people into twitter with on benefit to her

        • why not host own site, pump data out, and get people back to your site

    • communities

• there's a great future for the open web. are you part of it?

• evan@identi.ca



Questions:

• implementation of xmpp for mobile access

    • single jabber user (update@identi.ca) sends and receives notices for the entire system

    • system is a little bit complicated--streamlining needs to be done

    • sign up on web site, put in jabber id, user receives confirmation code, and then preferences can be set

    • various commands available through xmpp interface

• what's STOMP?

    • interface for queue servers

    • not a piece of software, it's a protocol - identica uses rabbitmq

        • activemq also supports stomp

    • when a notice is posted to identica, needs to go out to N number of users within 30 seconds

        • this CAN'T be real-time, but semi-real-time works well with queuing

    • author's own inbox is updated immediately, rest of the world sees it a few seconds later

• what other kinds of non-microblogging uses are you finding for laconica?

    • e.g. questioner says he uses it to push out environmental conditions

    • evan: "status sharing more intereting than microblogging"

    • school districts--send notices out to parents (snow day, school lunch, etc.)

    • laconica server in data centers--servers update laconica with server status

• current status of project? what's stable vs. not?

    • queue servers--becoming more important, particularly with move to being a service provider

    • file sharing - images, audio, video, word docs (more for enterprises)

        • not looking to be imageshack or youtube, but would like to use multimedia to augement status sharing

    • location/gps (coming in future release)

        • ability to search notices by location

• filtering?

    • definitely thinking about it -- probably plugin first, more core later

    • more a problem for people following thousands of people than people just following a handful of people that they actually care about

    • ideas:

        • muting (ability to turn someone off temporarily)

        • baeysian filtering/thumbs up-down

• file sharing--how not turn this into a mess of copyright violations and porn?

    • does take monitoring--admins first, later crowd sourcing

    • ability to take down inappropriate material

    • for sites just using the software, isn't really laconica's problem--would be the responsibility of the site owner

    • not a new problem--same thing with blogs, etc.

• @ replies--is one more popular than another?

    • not really--many differernt formats work

    • jaiku will be supporting openmicroblogging format

    • crossover with openid--both microblogging and openid identities are urls

        • shouldn't be limited to openid, but perhaps identitiy could be delegated

• openmicroblogging standard -- implementation follow protocol or vice-versa?

    • in parallel -- protocol didn't drive implementation blindly; was more interactive

    • very simple protocol--some criticism for that, but 0.1 protocols are the only ones that tend to get used

        • if it gets too big no one implements it

• how compare with google's opensocial protocol?

    • big point of opensocial is to let you build something like a facebook application and put that into a bunch of different social networks

    • what isn't built into that system is any federation between the social networks

    • opensocial does have a cool status mechanism so lacnoica could tie into that part of opensocial

Open Source Bridge: Wednesday Keynote

• 400+ attendees

• yahoo developer network--partnering with sunlight foundation to provide new egovt tools

    • new tools available later today

• attendees/speakers

    • ward cunningham

    • creator of usb 3 driver for linux

    • rasmus lehrdorf

    • goscon creator

• who are you?

    • 1 in 3 speakers are female

    • 25% of attendees are speakers

    • 92% of attendees involved with floss for > 1 year

        • 1400 years of os experience total at conference

    • 17% are vegetarians

• what can you do?

    • talk to each other, learn from one another

    • "jointly solve problems"--collaboration is open source's strength

• how open source software spreads cyborg culture (amber case - cyborg anthropologist)

    • first they ignore you, then they laugh at you, then they fight you, then you win

    • floss is now winning
<