Vancouver Open Data Hackathon: DTES Cloud

Nearly two dozen people attended the Vancouver Open Data Hackathon today to bump heads and code. After a round of introductions we forgot names but somehow remembered those with common interests for what to do with all the data that the City had made available to us.

Vancouver's open data catalogue has mostly geographical data: the locations of street lamps and where the nearest school can be found. There are some possibilities here but the available information felt limited.

So instead, people talked about high level ideas around RDF and ontologies, health cards and Open Electronic Health Records (EHRs). A lot of big ideas brewing. People also flocked to other datasets such as Translink's API and, for my part, online news feeds.

I worked with the W2's Honey Mae to create a visual impression of Vancouver's Downtown Eastside. In a couple hours, we managed to mash together content from multiple news sources and produce a tag cloud.

Since we only had a couple hours we made use of free, online tools for everything. The RSS feeds were pulled from the Vancouver Sun, the Province, the Georgia Straight, Megaphone Magazine, and the Vancouver Observer.

The news was then fed to Yahoo Pipes which did most of the digestion work for us. We filtered the news items, keeping any entries with the words "downtown eastside" or "DTES." Then, we extracted all the words from the articles and passed the list on to another application, Wordle.

Wordle calculated the number of occurrences of each word and then produced for us a nice little tag cloud!

The cloud doesn't have much data and there's a lot more we'd like it to do but it's a good start. Ideally, we'd like to see more data go into it. Say, the last 10 years of newspaper articles and what they had to say about Vancouver's favourite neighbourhood.

It'd be great to put the information on a timeline and see the words shift, maybe assign sentiment values to different words and plot the general positive or negative attitudes at any given time.

I'd also like to work more with the data that Vancouver released. If we can build something with it, it'll encourage them to release more.

I'll be brainstorming what might be done with neighbourhood boundaries and locations of water fountains. If you have any ideas, please post them below!

Oh, and many thanks to the Vancouver Archives for hosting, and for the Hackathon organizers. Great event!