Hack Day Random Technology

I can't believe it's all over!

God, what an extraordinary weekend! Even after a good night’s sleep, I’m completely exhausted. The last of us left the venue around 11pm last night, tired but pretty euphoric. It’s difficult for me to know what to talk about at this point. Many of us have been working on the project for months, but I think it’s only been in the last week where it’s all come together. And even then, much of the weekend was put together on the flyfixing problems, making sure everyone was okay, dealing with the unexpected.

The thing that sticks in my head most over the last few months is the passion of the people involvedand I mean everyone involved. From the inception of the idea to have a Hack Day for Europe in London through every stage of its implementation. Everyone who gave up their time to do everything from logo design, agree to speak, organise the venue, deal with the legal work, heft boxes around, gently herd the cat flood of contributors, construct and look after the websites, find the money, organise the band, organise the filming, get the staging set up, work out the lighting rigs, talk to security, get the microwave antenna on the roof, sort out the wifi, make sure we had food, source all the schwag, look after the press, sort out health and safety, get the presentations running smoothly, get the blimp crew more heliumI mean, dear God! They deserve bloody medals.

Collectivelyin my opinionall these people managed to create between them the right kind of event. Something that would mean that some of the people we admire most in the world (creative, industrious, fun future-creating technologists) would not only want to come to, but would be able to bring alive. And you did! Hundreds of you came along and demonstrated your ingenuity, your sense of fun, your enthusiasm and your patience. You made rockets, you made widgets, you made blimps. You hacked Nabaztags, tracked satellite flares and fashioned web services out of doodles on paper. You twitched net curtains, made things out of lego and mashed stuff together so that we could all see that it should never have been apart in the first place. You mated television with technology in ways that surely should be illegal. It’s enough to restoreno, that’s wrong, massively amplify anyone’s faith in humanity. Well, mine anyway.

This is already a bit of a sentimental post, but I wanted to catch how it feels right now before I lose it, so sorry if I go off on it all a bit more. It’s going to get long now. Leave while you can…

For me, there are dozens of things that stick in my head. Talking to Matt McAlister on the train from Sunnyvale to San Francisco where he first said, “We should do a Hack Day in London” and where I said, “Yeah we totally should” even while in my head I was thinking, “Ha! We’d never pull that off!”. The first visit to the venue in February with Glenn Bishop, Matt Locke and Matt McAlister (Cashmore got stuck in traffic because of the weather). Announcing the event on my site. Ploughing through infinite applications with Murray Rowan. The revelatory appearance of Anil Patel into the whole proceedings. Meeting the amazing Elaine Pearce! The super-long and pivotal meeting with Amber McCasland, Sarah Mines and Matt Cashmore. The effortless way the speakers came together (and the brilliant work they’ve done). Talking to the London perl mongers and the London Ruby User Group on IRC to explain the event. Endlessly just missing doing an interview for Vitamin. Thefirst time I did a site visit and could tell people why. Seeing Micah Laaker‘s finished logos for the first time. Matt McAlister showing me the Gobos.

And then there was the event set-up and the event itself. Turning up at Alexandra Palace on Friday morning having been juggling Fire Eagle work until midnight the day before, looking at the venue in a certain amount of terror. Watching the stage go up. Participating in the human chain of beanbags. Tom Croucher and Kent Brewster’s epic bag fight. The schwag bag filling production line. Missing the party in Camden. Hefting sofas around. The stunning late-night party at the venue the night before we opened the doors, full of moody lights and Cashmore’s bloody awful Queen / 90s rave music. Writing my Fire Eagle talk at midnight at Tom Loosemore’s house. Seeing people start to arrive on the Saturday morning. Publically showing off Fire Eagle for the first time. The venue being ****ing hit by ****ing lightning in the middle of our talk! The chatter on the radios when we heard. The roof opening and the rain coming in. Staring at the umbrellas dumbstruck. Moving everyone out into the courtyard. Ash Patel handing around sandwiches. All the hackers being absolutely incredibly patient and totally bloody indomitable. Resetting the room and bringing everyone back in again. The hum of activity as people finally get their heads together with functional wifi. Late night widescreen Doctor Who with three hundred of my closest friends. Making people move around too much. Frantically running around before the presentations to troubleshoot, make sure everyone knew what they were doing and make sure that Chad was okay. Trying to set up an amateur faceball tournament on stage so that the judges could have a bathroom break, only to discover that they were all fine. Co-ordinating the timings behind the scenes as we tried to work out if the talks were over or under-running. The ‘gah’ moment of realising we’d screwed up one of the big changes and the incredible work of everyone concerned (but particularly my favourite three guys who sorted out the P.A. system and whom I owe a big fat pint) to get the judging and prizes moved into the food room. The incredible positive energy of all the hackers. The waves of goodwill. And the band! The bloody band!

But weirdly for me, if you were looking for the heart of the event. If you were looking for the absolutely best time that made it all worthwhile for me, it was overnight. It was the period between nine and two am where everyone was doing precisely what they wanted to do. Where the lighting was atmospheric, where the coding was focused and everyone seemed to flow, where the room was gently buzzing with key-strokes. And the experience of all of those people turning around to the stage and running like kids to watch Doctor Who on a huge screen with a hundred of their peers and friends for one of the most extraordinary cliff-hanging episodes of the series was just amazing. It was more like being at home than being at home is.

Well, that’s me done. Thanks so much to everyone involved from the BBC, from Yahoo! and most importantly from everywhere other than the BBC and Yahoo. I’ve made a lot of new friends, caught up with a lot of old friends and been a part of something I don’t think I’m going to forget in a hurry. I couldn’t be prouder of all of us.

Hack Day Technology

Fifty new places to Hack Day…

So we hit our initial limits for people to invite to the Hack Day a little while back but a few people have dropped out and we’re feeling adventurous, so we’ve decided to open up another fifty places. So if you’ve been thinking how much you wished you were coming, but were convinced that you wouldn’t be able to get in, now you’ve got a second chance.

And to tempt you along a little more, Matt McAlister has posted a few of the companies that are coming along as ‘sponsors’ (really they’re organisations who wanted to contribute in some way and be a part of the day). So we’ve got people from Second Life, Arduino, Nabaztag, IBM DeveloperWorks, Make Magazine, Moo and O’Reilly coming along. And just glancing through some of the attendees, it looks like we have people coming along from Nokia, OpenStreetMap and all kinds of other places.

To sign up go to the Eventwax registration page and fill in your details. But do it quickly as spaces are disappearing. Also tell your friends!

Hack Day Random Technology

Less than a week to Hack Day!

With only a few days to go now until Hack Day and my free time evaporating like lead on the surface of the sun, I thought I should duck back online briefly and give you guys an update on what we’re up to.

Firstly, we’ve got a Hack Day backnetwork set up and running. A backnetwork is a site where people who are coming to the event get to generate their own impromptu social network, meet each other beforehand, chat and find out information about the event. It’s also a hub for aggregating content around the event, including blog posts and flickr images.

If you’re coming to Hack Day, then you should be on the backnetwork. If you don’t think you’ve received an invitation then check your spam filters today and if you still can’t find anything, then IM me on plasticbagUK (AIM) or tom.coates (Y! messenger) and I’ll resend your invitation out to you. Please remember though, I probably won’t be free enough to chat. Please don’t think I’m rude if I run off screaming after a few minutes. This whole thing is quite time consuming.

Next up, the band! On Sunday night we’ve got an exclusive performance by next big thing The Rumble Strips. They’ve been described as one of the bands to watch for 2007 by the BBC and are currently touring the country as one of NME’s hottest bands of the moment.

If (like me) you’re too old and unhip to know the band particularly well, then you should probably start off by visiting their MySpace page, their page and maybe buying a couple of their bouncier singles at or on iTunes. With any luck you should be able to tell your friends that you were there when it all started.

On the other hand, maybe your friends should be there too! If you’re coming to the Hack Day, we’ve got a bonus for you. You can invite up to three of your friends to come and see the band with you! From 7pm on Sunday those friends will be able to join you inside the venue at Alexandra Palace and experience the band with you live in concert. I’m quite happy about that. To get them to come, you’ll need to join the Backnetwork. There’s a part of the site called ‘Party Invites’ and that’s where you put the names of your friends!

Now, to the other communities that you should be joining or keeping up with. Alongside the Backnetwork we’ve set up a Flickr group. There’s already an active Facebook group up and running. Obviously the Unofficial wiki still ticks along nicely. And there are presences on Jumpcut and Upcoming. I think we’re trying to get something cool going with Instructables too, but I’m not so sure about that one yet.

What else do I need to tell you? The tags we’re using for the day mostly will be hackdaylondon. If you want to tags stuff on Flickr using machine tags then the correct term is upcoming:event=173371. That’ll do for now.

Hack Day Technology

Who is speaking at Hack Day?

I’ve just posted a really long piece over on the Hack Day blog about the various people from Yahoo and the BBC who are going to be coming to Alexandra Palace in a little over two weeks. Sorting out the speakers has been really surprisingly easy and good fun because everyone’s been so keen to come over and play. Nearly everyone I’ve wanted to come is coming, and will be around not just for their talk on the Saturday morning but for the whole weekend.

Some names and projects really stand out for me. Jonathan Trevor‘s going to be talking about Yahoo! Pipes which is a project that emerged out of the TechDev party of Yahoo that until recently I was working for. I remember Pasha Sadri mentioning this idea he had when Simon and I first joined the place, and it’s been really gratifying to see it come to fruition. Jonathan, who is coming over to speak, has been heavily involved in the whole project from really early on, and did an enormous amount of work on the incredibly stylish engine for manipulating feeds. Having him come over is really cool.

I’m also really excited to have Tristan Ferne and Chris Bowley from BBC Audio & Music Interactive coming. Tristan runs the my old R&D team at the BBC. We overlapped for a while, and one of the results of that time was the Annotatable Audio project which has now become Find Listen Label. He’s going to be talking about ‘Things to make and do’ at those places where media and technology collide. Very excited about that. Looking forward to seeing what they’ve been up to.

Another session that I’m personally really thrilled about is Aaron Straup Cope and Dan Catt—both of whom work with Cal Henderson at Flickr—doing a session on Machine tags. I don’t know that Machine tags have really got the press they deserve, and I was particularly excited to get them coming to talk about them because they’re (as far as I know) totally non-proprietary. It’s really just a concept that Flickr have pushed further than most other organisations. I’m particularly fascinated by it because it’s all about taking the machine-readability of tags and extending it massively, with lightweight and community-generated schemata allowing any object to have an almost infinitely extensible pseudo-database of information attached to it. I’m not explaining it very well. If you want to know more, you should read up on Flickr’s implementation of Machine tags.

Speaking of old-friend and fellow (ex) UK Blogger Cal Henderson, he’s coming to Hack Day as well. He’ll be doing his barn-raising rendition of ‘Why Flickr’s Web Services Rule’ (or something suitably kitten filled). So that can’t help but be awesome. I’m hoping for a few rousing games of Faceball too.

Er. Who else have we got? Obviously‘s main culprits Matt Cashmore and Ian Forrester are going to be performing. We’ve got Christian Heilmann and Nate Koechley talking about YUI, Kent Brewster doing a Yahoo/BBC mash-up in an hour and Bradley Wright talking about Yahoo! Answers APIs. And if that’s not enough for you, if you’re into Geo stuff we’ve got Mirek Grymuza talking about Yahoo! Maps APIs and Mor Naaman from Yahoo! Research Berkeley, who are responsible for such projects as Zonetag and TagMaps.

Oh! Which reminds me. There’s a project that Paul Hammond and Simon Willison and I started working on late last year with Mor and the YRB crew that I’m also hoping we’ll be able to show off at the event. I’m not yet clear on whether we’ll be doing a session on that yet. I guess I’ll see how the next couple of weeks goes.

There’s more news to come about some of the other people coming to the event, but some of that’s in the air at the moment. Hopefully we’ll have more to talk about on Monday or Tuesday next week. Keep watching the Hack Day blog and in the meantime…

Hack Day Technology

On Alexandra Palace and Hack Day…

The first time I went inside Alexandra Palace was a couple of months ago, while we were still trying to work out if we were going to be able to carry off this whole Hack Day concept in the UK. It was snowing insanely hard and I got completely lost between the tube and the venue. I was wearing canvassy trainers. They got pretty much immediately soaked through leaving me with numb lumps of uncooked meat at the end of my legs. I kept slipping all over the place and falling over as I tried to get my way up the hill. In the end I got a cab. There were kids everywhere throwing snowballs at passing cars and sledging down the hill. Matt Cashmore—who had thought of the whole Alexandra Palace venue in the first place—couldn’t even make it to the venue because the roads were so slushy and clogged. I took a couple of pictures that day and posted them to Flickr: The view from Alexandra Palace, International Woodworking Convention. But even then walking into the venue I remember getting entirely over-excited by the whole thing.

Last week, I went back for the first time since February. The weather was completely different – beautiful, clear skies. Beating sun. Everything was green. But the effect was pretty much the same. This is going to be great. I’ve put some pictures below to whet your appetite, but if you really want to get the whole feel for it in these last few weeks, I’d recommend checking out the Flickr set I’ve put together: Site visit to Alexandra Palace. And if you’ve not applied to come yet, then there’s still time.

Hack Day Technology

Who are we hoping will come to Hack Day?

Since Yahoo and the BBC announced a couple of weeks ago that they were doing a Hack Day in London’s Alexandra Palace, I’ve had a lot of people contacting me with questions and comments. Of all the questions, the most common one by far tends to go something like, “Hey there, I’m really excited by the Hack Day thing, but I’m not sure I’m the right kind of person for the event. Should I come?”

The truth of the matter is that the event in Sunnyvale was really exciting because there were a whole lot of people working in really different areas, and I’m really hoping the same thing happens in the UK. Personally I’m hoping that alongside people working with the web and widgets and desktop software that we get a fair few students and creatives working (or looking to find someone to work with) in the cutting edge of real-world product, hardware design or even fashion design. I think the day has the potential to be a real creative melting pot.

That may sound a bit weird, but let’s have another look at some of the projects that were made at the US event. In fact let’s look at the group who won the day. Known as Black Box Nation, they were a group of three women working with sewing machines, soldering irons, nokia 6682s, pedometers, Flickr and the Zonetag API. They collectively put together a handbag that took a picture with a camphone every few steps and used Zonetag to upload it to Flickr and turn it into a life-recording weblog. It even geotagged the photos as it went. You can see the Flickr stream of the handbag to this day and there’s a site about it the whole project at Blogging in Motion. Does that kind of thing interest you? Do you want to get together a project about future fashion with built in computrons? Then Hack Day really could be a great place to do it, with access to a whole range of people who’d be able to give you information about some of the technology side.

Winning Hack team

Another project that did really well at the event was called the YBox, a piece of hardware that called itself Konfabulator for your TV. This project was a whole bunch of technology inside a sweet tin that you attached to your television. Once you’d done this, it gave you access to simple plain-text useful information pulled off the internets by computer magic. This kind of project could be a really good match for the UK Hack Day. If you’re interested in this territory, I can reveal that we’ll definitely have at least one speaker who’s very knowledgeable about building and developing around set-top boxes and interactive television.

And this is just the start. The other projects ranged from the sublimley useful to the ridiculously fun. Cal Henderson knocked up a digest-maker. You could go to a web page, type in a word and get back a digest of information about that subject that would automatically print itself out of a nearby printer. That one used Flickr, and Wikipedia. Another guy called Mo Kwaken made something called Blabber which unfortunately I can’t find any more information about online. It was supposed to be able to use absolutely any face from Flickr but in the end just used Patrick Stewart. You drew a line where the mouth was and then whenever you talked into a microphone, his mouth would open like Canadians on South Park. So you could pretend to be Patrick Stewart, which was fun.

There was an awesome Upcoming hack called gutentag which added a different social layer to the service, and an awesome flickr-based project called The Color Field Camera which would identify the colour of the thing it was pointed at, access Flickr and source pictures that matched that colour using the Flickr Colr Pickr. That was pretty extraordinary. Here’s a picture below:

Basically the day was tremendously creative, and hopefullyif you’ve read some of this postyou’ll realise that you don’t actually need to be working in Python to be able to come along. If you’ve got a great idea that you can’t build along, then first off I’d recommend trying to find a team of people to work with you. Ask your technologist friends or come up with an idea you could build together. If you do have a team, make sure you let us know though when you sign up, so we can make sure that you all get to come along. Or if you’re a designer and you want to come along to help out other people, and are prepared to get involved in their hacks – that’s good too. If you know people who build and create things in different and unexpected territories, then let them know about the day. This really is a day where you can show off your creativity.

I’m hoping that we’ll get a whole bunch of people coming who are interested in building mash-ups or new sites and services. I’m hoping that people will come along and build the features that they’ve always wished their favourite sites had, using all the gear and technology that Yahoo and the BBC have available. But I’m also definitely hoping that we’ll get groups along from people interested in weird interactive art projects like the ones that Regine Debatty talks about at We Make Money Not Art or some of the really awesome people who make stuff at the RCA’s Interaction design course. I’m hoping that people interested in Everyware and the kinds of physical stuff that Schulze & Webb and ThingM put together will also think about coming.

If any of this is exciting to you, then sign-up now and spread the word around. You can also read more about what people are saying over on the YDN Hack day blog. Andas usualif you have any questions, please feel free to ask me in the comments here or directly at tom {at} the name of this website.

Hack Day Technology

A Hack for Europe!

Right then. This is going to be quite a big deal so pay attention! A few months ago you’ll remember Yahoo! put together a Hack Day on their campus in Sunnyvale and invited several hundred designers, developers and engineers to come and crash the place. People camped in the grounds overnight, had access to a whole range of speakers and technical experts from Yahoo!, Flickr and Upcoming, and used the time to build a whole range of bloody fascinating things including a handbag that posted pictures to Flickr and a ybox. You’ll also remember the live entertainment, the piles of pizza, the donuts in the morning, the bleary-eyed pioneers and the heady stench of unwashed technologists on the Saturday evening.

Well, I don’t know about you, but I think we can do better than that. So it is with great pleasure that I’m going to direct you all to and encourage you to sign up for the first Hack Day to be held in Europe. And this time it’s not only a Yahoo! event, because it’s a partnership with BBC Backstage. And it’s not at a campus, it’s at London’s bloody Alexandra Palace! On June 16th and 17th! Bring a kite!

Everything else remains the same. It’s a two day event, starting first thing on Saturday morning and running through to Sunday evening. We’ll have a whole bunch of speakers from Flickr, Yahoo! and the BBC to start us off. We’ll have food—mostly flat—to meet the specialised needs of our guests. There may be booze. I’m not telling. If you want, you can stay awake all night or crash out in a corner in a sleeping bag. The only requirement or restriction (except for the legal ones, which you should probably read) is that you come to the event and try and build something, ideally using some of the stuff that the organisations hosting the event have to offer. Did I mention it was free?

You can build robots if you’d like, or things involving televisions or tagging or photos or smart dust. There will be prizes for the best stuff made. And judges! And probably a limited number of free Flickr badges! And yes, there will probably be a band. And no, it probably won’t be Beck.

Even with space for hundreds of creative h4X0rs, an event this awesome will get oversubscribed, so I’m afraid not everyone who applies will get a ticket. And in order to makes sure that the people who actually make things get to come, I think there’s going to be a process where we weed out obsessive business networkers. So be patient once you’ve appliedwe’ll let you know as soon as we can if you’re invited or not. If you want to improve your chances, stick a couple of links to stuff you’ve made into the ‘Special Requirements’ field of the sign-up form. People who build get priority.

Anyway, that’s your lot. I can’t tell you how excited I am about this event and how great it is to be a small part of the incredible group of people from both organisations who are helping to make it a reality. I think it’s going to be really awesome and I’ve wanted to talk about it for a long time. Hope to see you there! Sign-up now!

Net Culture Politics Technology

The Open Rights Group Party…

As some of you know, I’m on the advisory board of ORG, the British Open Rights Group which is a non-profit organisation focused on making sure that British people have the right to use technology in reasonable ways and that their traditional rights aren’t compromised by companies using technology in restrictive ways. The organisation acts as a balance to some of the vested interests in business, government and elsewhere. It’s there to try and fight people who would rather compromise your rights to protect out of date business models than they would innovate and change. It’s there to make people aware of the abuses or errors that might result from electronic voting schemes or huge databases of information on UK citizens. It is, generally, a pretty honourable little organisation and a smaller, bottom-up version of the EFF.

But that’s not important right now. The important thing is that they’re having a party for supporters and people who are interested in their activities on Shoreditch High Street in London next Wednesday, and that you should find some way to come along.

There will be ‘public domain’ DJs celebrating the importance of publically owned culture, remixed visuals and free culture goodie bags as well as the appearance of Danny O’Brien, long time NTK nerd, EFF activist and general wit. It’s free to attend, although obviously we’re hoping it’ll result in a few more people signing up to fund ORGs activities.

The event will be at Bar Kick at 127 Shoreditch High Street, London E1 6JE from 6pm until 11pm next Wednesday 11th April. You can find more information and register for the party on the support ORG page which also includes a map and instructions for how to find the venue. Hope to see you there. There’s also a raffle

Design Social Software Technology

Methods for the social archiving of mailing lists…

Imagine you’re on a mailing list that archives URLs that people share in some form, and that this creates indirectly some kind of archive or directory. Imagine that this archive has generally been maintained by hand and in a formal taxonomic structure. Imagine that the weight of maintainance started to get the list owner down and they decided they could no longer justify the time they’d need to spend on it. How to distribute the work effectively? How to maintain the utility of the directory without killing the people upon it? What follows are some freeform, stream of consciousness-style notes written off the top of my head. Better out than in.

Your most obvious territory for thought might be the categorisation scheme and how to dismantle the formal structure in favour of something lighter and less complex to maintain. The most obvious direction change you could make here would be to move towards a folksonomic tagging approach. But a true folksonomy must emerge from the overlaying of many people’s efforts otherwise what you’ve got is a personal, informalor just plain badtaxonomy. So straightaway you may end up having proliferated the work rather than reducing it. It may be more distributed, but is it any more likely to get done? That’s a difficult question to answer.

Skipping away from the question of annotation for a moment, let’s look for a moment at how to get the first order objects (links) in the database in the first place. one approach would be to put every unique URL sent to the list directly into a database. Conceivably you could organise those URLs by tags imported from other locations – for example you could just go and get the folksonomic information for that URL from

There are problems with this approach of course. For a start, you have then a repository of information about links that’s completely editorial free and doesn’t necessarily represent the context in which the URL had originally been shared. That is to say, you don’t have any of the original posters thoughts on the link, just the link and some tags. You could apend the whole e-mail to the URL, but then you you’re stuck with what happens if the list is private. Obviously then you’d be stuck.

An alternative: when an e-mail is sent to a list containing a URL, why not get the server to reply immediately to the original poster with a post containing a link to the place they could annotate or categorise their link to be added to the directory. That way the link originator could take responsibility for their particular piece of maintenance and the directory could grow through the individual actions of multiple individuals. Conceivably, links could be added to the database immediately they’re sent to the list, but not made ‘public’ until they’ve been annotated by either the link originator or the list owner. Because you’d be able to track the originators of the e-mails, you could then easily create a queue of URLs to subsequently annotate or approve.

There’s still a problem here, although it’s not a big one. If you take the folksonomic approach to categorisation then you’d have to rely on the individual’s personal taxonomy rather than on the wisdom of crowds bubbling up ‘correct’ categorisations. So then you have to ask yourself whether there were ways that you could usefully allow other people to enhance these URLs with more information after the originator or site owner has done the initial work. One option is to mine or another social bookmarking site as I proposed above. The other might be to allow other users in the mailing list to add their own annotations and tags to the link concerned. A server could usurp all e-mails containing links and add in additional link to a place where they could be annotated subsequently. The readers would automatically see the original link and then a link place where they could annotate the item. My big concern here is that individuals would be compelled by the software to move the conversations about links off-list and thus deform or split the conversation more than necessary.

One sideline… Of course you don’t necessarily need to get people to follow a URL to add in their information about a link – particularly if they’re the originators. Another approach might be to send the originator an e-mail (as above) with an identifiable string in the subject. Then simply replying to that e-mail with a message only containing a paragraph of text or a few Flickr style tags could add those tags and that annotation to the database. One anxiety there might be people incorporating accidentally great tracts of their previous e-mail into their annotations. Not ideal. Too fragile.

On the other hand, instant messaging in the Twitter model might provide some good options. Imagine if all users on a mailing list added their IM details to their profile, and added a bot to their IM friends charged with handling their mailing lists. When a message was sent to the list with a URL in it, the originator could be sent an IM request to describe the URL and everyone else on the mailing list could be sent the URL without comment. Once they’d observed the link, they could simply reply with their own comments or annotations which would then be saved to a database. Easy. If they didn’t want to keep getting URLs, a simple ‘off’ command could cease the flow…

The most obvious problem there would be if another URL came in as you were typing or if there were substantial communication delays, but I suspect these could be resolved one way or another.

Another option: individuals could choose to categorise URLs within the mailing list by hand using a third party service like which could then be aggregated by a local piece of software. They could either use their own personal accounts and mark things ‘for:{name of list}’ or they could use a shared account. This way you could bootstrap off other tools rather than build everything yourself. The most obvious problem: Is this work that people would want to do? If it is, would using (and conceivably then having to change accounts if you were already a user or having to mix in other people’s links with your personal linkstream) be a greater impediment than another approach? Tricky one.

A few other approaches leap to mind, but I think I’ll leave it there. If anyone has any other ideas, I’d really appreciate hearing them. A good way to think around the territory would be to think about which groups of people could do the various tasks associated with saving or annotation. In some models it’s likely to be the posting user who does everything, in others their peers take on spotty bits of work and all the annotation. In still others you can imagine a dedicated admin doing all the work, and in others still, people off list completely could be categorising and annotating what people on-list are doing. Finding the correct approach will rely on working out where the motives for contributing might be for each group of people and how to build something that meets that particular groups needs. Any thoughts?

Social Software Technology

Social whitelisting with OpenID…

My ex-colleague Simon Willison has recently been doing some profoundly good work out in the wilds of the Internet promoting and explaining OpenID. In fact, the best articulation I’ve seen anywhere on the Internet of the OpenID concept is his screencast which I think neatly sums up the value of the concept as well as how easy they are to use.

You’re going to need to understand OpenID before I go much further, so if it’s an area that’s new to you, this is the point where you need to either go and watch that screencast or follow carefully the simple description of the service that follows…

OpenID—fundamentally—is a solution to the problem of having a million user accounts all over the place. Instead of getting hundreds of user names all over the place you go to a site that provides OpenIDs and choose one username and password. These sites then give you a pretty simple web address which is probably easiest to think about as a profile page for you. Then when you want to sign into any other site on the Internet with an OpenID all you do is type in the address of this profile page. The site you’re on wanders over to that address, the other site asks you for your password, you tell it your password and then you’re bounced back to the original site where you are logged in and can get on with your business unfussed. Sometimes the local site will ask you if you want a different user name. That’s all there is to it.

Having the same ID across a number of sites can also make a number of other things possible. You could hook up all the stuff you do over the internet really easily, and aggregate it and get a handle on it. You wouldn’t have to share your passwords with lots of different people either. All good. From my perspective, given my long term interest in technologies of moderation and social software, Open ID also provides one super-significant thing – relatively stable, repurposable identities across the web as a whole that can develop levels of trust and build personal reputations. But more on that in a moment.

Of course with new solutions come new problems and the most obvious problem associated with OpenID is ‘phishing’, which is to say that a site could ask you for your OpenID and then pretend to be your central provider. You type in your password thinking that you’re safe, but in fact you’re giving out your details to a rogue third party, who now can use it across all of your registered sites and services. This—let us be clear—is a very real problem and one that Simon talks about again in his piece on OpenID phishing. I’ve heard some really interesting ideas around how you might do this stuff effectively, but I’m still not completely sure that I’ve heard one that I think is totally convincing. This isn’t such a problem for the phishing-resistant old dogs like the people who read my site, but could be an enormous problem for real people. In the endif such a project is to take offI suspect this problem is going to be solved by a combination of design and education. People are going to have to get their heads around web addresses a bit more. Or we’re going to have to build something into browsers that handles distributed identities a bit more effectively.

The area that I really wanted to talk about today though was social whitelisting, which Simon and I were discussing the other day and which Simon has already written about on his site. This emerged out of some conversations about a very weblogger-focused problem, ie. comment spam. I’ve written about problems that I’ve been having with trackback and comment spam before, but every single day it seems to get worse. I get dozens of comment spams every single daysometimes hundreds. And this is even though I use extremely powerful and useful MT plugins like Akismet. And the spam is profoundly upsetting and vile stuff, with people shilling for bestiality or incest pornography, or apparently just trying to break comment spam systems by weight of empty posts.

Over the last year or so, it’s stopped being a problem that I’ve been able to deal with by selectively publishing things. Now every single comment that’s posted to my site is kept back until I’ve had a chance to look at it, with the exception of the few people that I’ve marked as trustworthy. It has now very much become a problem of whitelisting for meof determining which scant number of users I can particularly say it’s okay to post. And if this is where I am now, with my long weblog history and middling okay technical abilities, I can only dread where everyone else is going to be in a couple of years time. This is unsustainable and we have to change models.

Which is where the social whitelisting concept comes in. Most whitelisting has been around approving specific individual people, but this doesn’t scale. A large proportion of the people who post to my sites are doing so for the first time and may never post again.

The solution that Simon and I came up with was really simple and sort of the opposite of Akismet. Jason Kottke deals with a hell of a lot of comments every day. So does Techcrunch and GigaOM. Every day they approve things that people have written and say that it’s okay for them to be more regular posters. So each of these people is developing their own personal whitelist of people that they trust. More importantly I trust Jason and Techcrunch and GigaOM along with Matt Biddulph and Paul Hammond and Caterina Fake and about a thousand other people online. So why shouldn’t I trust their decisions? If they think someone is worth trusting then I can trust them too. Someone that Caterina thinks is a real person that she’s prepared to let post to her site, I should also trust to post on mine. This is one of the profound benefits of OpenID – it’s more reliable than an e-mail address that people can just spoof, but it’s just as repurposeable. You can be identified by it (and evaluated and rewarded for it) all across the whole web.

So the idea is simplicity itself. We switch to a model in which individual sites publish lists of OpenIDs that they have explicitly trusted in the past. Then individually, site owners can choose to trust anyone trusted by other site owners or friends. People who are trusted by you or your friends or peers can post immediately while the rest are held back in moderation queues for you to plough through later. But with any luck the percentage of real comments held back over time would rapidly shrink as real people became trusted and fake people did not.

Another approach to this idea would be to create a central whitelisting service with which you could share your specific trusted OpenIDs and associate them with your weblogs. Through a central interface you could decide to either accept a generic trusted set of whitelists from the top 100 weblogs on the planet to get you going, or add in the specific weblogs of friends, family and colleagues that you know share the same interests or readers. And of course individual weblogs can be rated subsequently for whether they let through people who subsequently turn out to be troublemakers, or rewarded for the number of real people that they mark as trustworthy. I want to make this particularly clear – I’m not talking about one great big web of trust which can be polluted by someone hacking one whitelist somewhere on the internet. I’m not talking about there being one canonical whitelist anywhere either. I’m definitely and specifically talking about you deciding which site owners (or groups of site owners) that you trust and that being the backbone to your personal service. People that your peers trust may be different to the people that my peers trust. And so it should be.

There’s even a business model here. I’d pay (a small amount) for any service that allowed me to have vibrant and enthusiastic conversations on my weblog without having to manually approve every single message. I’m sure other people would too. And of course, much like OpenID itself, there’s no reason that there should only be one such whitelist provider online. There could be a whole ecology here.

So what do people think? Does this have legs? Is it a sufficiently interesting idea to play with further? Where is the OpenID community on this stuff at the moment? Could social whitelisting of OpenIDs be the thing that rescues distributed conversation from death by spam? There’s a discussion over on Simon’s original post on the subject, or feel free to post below (but be warned that it may take me a while to approve your messages…)