"Example of routed travel distances used on walk score."

If Paul Butler can create pieces of art out of playing with data, then I can… aspire to try. Having gotten access to a firehose of data, I set out to play with it over the weekend from a local coffee shop.

With some analysis, exploration, and photoshop, I ended up telling a story about the city of Seattle, the two tech giants established in the city, and the local public transit system. Below is the resulting map of how far one can get via public transit and walking from the centers of Amazon (left) and Microsoft (right) tech headquarters within 25 minutes. It takes a minute to get from one outline to the next. The visualization is as if the the entire area was a pond, and one was to drop rocks at the locations of transit stops, at the time one can arrive there. The resulting ripples fill in the space at walking speed.

"Amazon vs. Microsoft transit time map: 25 minutes"

It’s pretty clear that for a given company, having a reasonable commute will dictate exactly what side of the Lake Washington one would have to reside on. It’s also interesting to see how parts of Kirkland are outright inaccessable, while sections of Bellevue and Queen Anne are underserved. Of course this map doesn’t tell the complete story, as both Amazon and Microsoft also offer private shuttle services, and Microsoft has many sattellite offices throughout the city. This has been brought up in the comments on Reddit; through by far the most upvoted contribution was this derivative piece of art.

"Flaming guy catching a kicked ball."

All of the underlying data is available via Travel Time API and can be used to make the same kind of travel maps. Walk Score already uses this technology for finding hotels closest to multiple meetings, apartments closest to a particular location (school or work), and travel maps. Some other interesting ideas that I would like to see someone do in the future would be around mobile locations that quit making assumptions that I can jump across river/highway/barriers to get to their local deals. Programmable Web is likely the place to track API mashups for just such applications.

Update: there has been an addition to drive time api examples to include a rush-hour drive time estimates mode. Neat!

Drive Time API example

No related posts.

New bell ad: stop the Belligerrence; from @ubbtor

UBB (Usage Based Billing / “Metered Internet”) is a hot topic right now, with many Canadians angry over the failures of CRTC and the subsequent forced price hike of the internet, even when using small competing ISPs.

There were arguments about online games (Steam):

Thanks to UBB, I can no longer use Steam in Canada thanks to your ridiculously low caps provided by my ISP. This is total bullshit as I am now forced to use a brick and mortar store to purchase my pc titles rather than committing a little under half of my bandwidth to downloading one new game title.

There were arguments about keeping in touch with one’s family:

I’m in the US military and my wife’s Canadian. During my last deployment, one of the easiest ways for her and I to keep in contact while she finished up her University degree in Ontario was webcam. She had Roger’s internet and they’ve had this bullshit implemented for some time. So, she had some crap 25 Gb max rate and we hit that most every goddamn month just trying to stay in contact. (Yes, in hindsight we should have switched and whatever. That was then, this is now)

And of course many arguments over economic impact on consumers:

I’m about to lose an extra $200 a year just because some large corporate giant says so: Recently, Bell Canada has had the Canadian Radio-television Telecommunications Commission (CRTC) make a decision that aims to take more money out of the pockets of Canadian consumers and ruin online services that are competitors to Bell Canada.

A quick video intro to what’s happening: Strombo Talks About The Impending Metered Internet

And many many more arguments about why metered internet is a bad idea. Here, I’ll talk about how Usage Based Billing will disrupt (in a bad way) Computer Science education, innovation, and as a result, the software industry.

– Coffee shops (traditionally hubs for exchange of ideas and birth place of innovations) will no longer be able to offer free wifi, in a fear of punitive per-GB charges. The stereotypical image of the next Silicon Valley entrepreneur starting a company out of a coffee shop will vanish, and with that a pool of innovation.

– Students will be limited in exposure to new technology. Proposed 25 GB download caps are just too small. Software downloads can take up a large chunk of the allowed quota: iPhone SDK is 3.5 GB, the latest version of Linux (Ubuntu) is 0.7 GB (plus all of the updates!). Tinkering with new types of operating systems and making applications for one’s phone is the inspiration that’s necessary to get students interested in the pursuit of technological education, but such can easily hit the allowed bandwidth cap, and will likely be discouraged in low-income households.

– Online learning resources will take a hit. TED.com is an amazing resource that delivers ideas that are persuasive, ingenious, inspiring, and informative. Of course video streams quickly add up to the bandwidth caps. Schools have just started moving towards the internet to better teach the material, but now rich media (videos, interactive applications, etc) will likely be reconsidered, in other to keep the resources equally available to all students.

– The Cloud is the bleeding edge of technological innovation. As companies like Amazon, Google, and Microsoft innovate new ways of thinking about communications, data, and computer resources; Canadians are getting locked just to their local machines. If we can’t backup our data (Dropbox) and stream music (Pandora), then we are not in an environment where we can create such companies.

– Limit of research with large datasets. The Netflix Prize is the first that comes to mind where open experimenting lead to development of better technology. Twitter offers even larger data streams, offering insights into everything from geopolitics to the stock market. A Canadian example would be the the Goldcorp Challenge, when in 2000 Goldcorp has released its geological data to the world in an attempt to save a gold mine in Red Lake, Ontario from closing. Many submissions came in, including those from students. The mine ended up finding over $6 billion (that’s billion with a B) worth of gold. Open access to large datasets has clear impact on innovation that results in new technology, insights, and creation of resources. There’s more data than ever available on the internet now, but exploration of such will be discouraged with metered bandwidth fees.

– Similar to research on large data sets, contributing to distributed scientific research will become discouraged as well. Folding@home is a distributed computing platform, researching diseases “such as Alzheimer’s, ALS, Huntington’s, Parkinson’s disease, and many Cancers”. Bandwidth caps will discourage contributions to this and similar research models.

The initial counter argument might be: “just pay more to get as much access as before”, but this is not that simple. Price-conscious University students will start second-guessing if they should explore their innovative ideas, when data consumption is met with punitive fees. High schools students often have limited say over the connections in their household, and low-income families might get into a conflict between providing an open learning environment and paying extra costs.

It’s clear that telecommunications companies such as Bell and Rogers fear technological innovations that was brought by Skype, YouTube, and Netflix; but pricing such innovations out of the market will have a far greater impact on the environment that allowed for such innovations (and many many more) to be developed in the first place.

What can you do? http://www.reddit.com/r/canada/ is our battleground; a lot of information and ideas are posted and discussed there. Excellent list of actions one I take. Sign the petition. Attend a rally in downtown Toronto. Let everybody else know how this change will affect them. Join the discussions: @OpenMedia_ca, @ubbtor.

Related posts:

1. Wishing for cake through fibre optic cables

"New CD browsers" on Flickr, by San Mateo County Library

Recently, a lot has been happening in the peer-2-peer space, at least from the legal perspective. The Pirate Bay trial, plus various torrent websites shutting down; TorrentFreak is an excellent news source for all that goodness. While a lot of people scream that RIAA/MPAA (and their international equivalents) are not keeping up with the times and technology, we are not really any closer to having an entertainment distribution system that takes advantage of available technology to make it convenient for consumers and one which fairly compensates the industry that creates the said content. So I propose starting a discussion, by presenting a scheme that pulls p2p technology and current copyright laws closer together.

There are two basic premises:

1. Music is purchased as a license for an individual to indefinitely enjoy a particular piece of entertainment.
2. It is permissible to lend someone a physical music CD, provided that it is original (such as store bought), and not a copy.

The latter point is what allows one to sell their used CDs (imagine that, there’s an actual business that does that, with 11 locations in Ontario — The Beat Goes On), borrow CDs from a public library (the Toronto Public Library has 59 838 titles (in multiple copies!) available), or to lend a CD to a friend (just try to outlaw that, and the general public will actually start taking interest in the application of copyright laws, and the next election).

The former point stipulates that a physical CD is simply a token, showing that the current holder has a license to enjoy the media; the recordings on the disk are simply a convenience factor.

So the natural step, at least as it seems to me, is to digitalize this license tokens, and let them be shared over p2p technology, as you would have shared a physical CD with a friend. Here’s what I have in mind:

Scheme illustration, put together with free icons from IconsPedia

All of the music is readily available for download (similar to current .torrent approach) and copies could be cached on your device (to save bandwidth), but it is not playable unless a license token is also available on the system. The license tokens will act as unique digital keys, which could be borrowed and released back into the cloud of p2p.

If one had purchased a set of license keys for the latest music album, but isn’t listening to this particular music items at the moment (school, work, sleep.. there are many reasons why ones entire music collection isn’t played 24/7), those keys are available to be given to someone else. Similarly, one could temporary take possession of someone else’s license, while such is available.

To prevent leechers, and make for a fairer sharing experience, a model similar of current private torrent trackers could be applied — those who contribute more keys will get priority status. Bonus points for contributing keys to high-demand or rare media. Maybe preference for own social graphs.

The net effect of automating request-play-release cycle is that a) sharing music will be completely legal, b) ease of use would be on par with current technology, and c) popular content (ones with more simultaneous key usages) will see a proportionally larger share of revenue. Just think back to days of mixtapes, and trading CDs with friends; except that it’s now done every 3 minute, through a wire, and you are friends with the world.

The caveat is that, yes, one would actually have to pay for some of the licenses to take part in this sharing community process. Though ones choice of which artists to support and generalizing the use of granted licenses should make it a fair deal. iTunes and Amazon’s MP3-Downloads show that a lot of people are still willing to pay for very specific music purchases online.

But now, given the fact that “pirates [are the] biggest music buyers”, via Ars, do we really have to implement such an explicit model?

Those who download “free” music from P2P networks are more likely to spend money on legit downloads than those who are squeaky clean, according to a new report out of Norway.

No related posts.

Not quite like this... yet.

TorrentFreak reports that an Israeli ISP, Bezeq, is actually modifying the .torrent files, as they are downloaded by their customers.

Looking back almost a year, I’d just like to say — I told you so.

It works as follows. When a Bezeq International customer downloads a .torrent file the ISP will intercept it and add (!) a new tracker to it. The additional tracker is only accessible for Bezeq International customers and it connects to a high speed web-seed hosted on Bezeq International’s network.

Instead of blocking / throttling / capping / sending-threatening-letters or finding other ways of annoying their own customers for actually using the services they pay for, Bezeq caches popular downloads. The ISP saves on costs for having less traffic go outside of their own network, and customers actually end up with faster downloads. Still, this involves intercepting and modifying downloads, which makes me feel uneasy, as this could easily be turned the other way.

This concept will actually lessen the server load of hosting companies that host the file itself, well good for them.

No related posts.

Crop of 'My Cyber Social Map' by frankdasilva

It seems that people are beginning to realize that what they say, or what is said about them, online, could actually matter. More so, if it shows up on Google. Which is exactly what happened to a certain individual, who was recently looking up his own name, and found something he did not like. A forum post, from 2004.

“I found said post as hit #7. The absurd comments contained within could be classified as nothing other than slanderous, if not criminal. My name is not one that is common so I am taking these accusations personally and I ask that they are removed immediately so that further action is not required.”

Among many problems with the above claim, is the fact that the post in question was about some kid, who happened to go by the same name, doing stupid things in high school. If the age discrepancy or the absurdity of the context doesn’t click in, perhaps the fact that it’s an opinion posted by an anonymous users on a forum should.

It is unreasonable to take everything posted on forums or blogs as facts.

Besides, the forum itself, and it’s operators (myself and Dan) are just a carrier, and can’t be held responsible for the actions of 3rd party users. Still, some seem to be under the impression that big words and threats are the way to go.

So we’ve started a wiki page of Legal Threats that we’ve received; it will be updated as more show up. That’s right, we are doing it The Pirate Bay style — all threats will be publicly posted. We even have enough time to reply!

A few lessons learned:

1. Your name will be searched online. Internet content should be considered permanent, so try to do something noteworthy.

2. Failing to do the above, start a blog. If anything, it should rank higher than absurd 4 year old forum posts about random kids.

3. If you plan on threatening someone and make it sound all serious — you have to take it seriously as well. Otherwise that nonsense will just end up on the internet. See point #1.

No related posts.

Given the popularity of the IRC protocol for communication between the infected computers, I thought it might be an interesting thought experiment to consider other means of communication. On a locked down corporate network, blocking all but a few essential ports, HTTP is typically let through; I guess people need to access web pages for research and other business needs. Without setting up any of our own control servers, lets see what existing web services one could use to spawn a web of online mischief!

Botnet over Twitter

Twitter's social structure makes it almost too easy to set up networks of bots.

Twitter might prove to be an ideal service, as it is already meant for post-on-web social communication not just between humans, but bots also.

Twitter’s API makes it very easy for software to access and post all the vital information.

The service itself comes build-in with the concept of following specific accounts — allowing one to setup the network layout entirely within the webservice itself.

The functionality for replies, direct messages, and private profiles are just gravy.

Botnet over Reddit

A clever use of cats and captions might let Reddit submissions live long enough to be read by the bot army

Just like every social-network website, Reddit comes with a set of “friends”, and even private custom reddits that can be used as “channels” to communicate in.

Simple and clean HTML markup makes it easy to parse the contents of the page.

Though the zealous community is quick to point out any suspiciously spammy activity; And Proggit members will likely hijack any control in place.

Botnet over GigPark, et. al.

GigPark could be used to request and find recommendations for Handyman, Doctors, and Denial of Service Attacks

The above ideas could be generalized to any Social Network website. GigPark’s feed of trusted recommendations is already filtered to 2 degrees of separation between the linked nodes, making filtering and discovery of new bots much easier. Friend-lists, messages, and favourite “recommendations” (which could function as a queue of tasks) rival Twitter’s toolset.

Actually GigPark’s innovative Suggested Friends feature will sync the “friends” from other social networks such as Twitter or Facebook, allowing for redundancy across multiple neworks.

Next Steps

Some downsides to this, obviously hypothetical, method involve the fact that too much reliance is placed into the host network. The webservice might be in an advantageous position to identify all the nodes; perhaps more so than an IRCop discovering the IRC channel where bots have gathered to communicate.

Another is the issue of information persistence. Web applications will typically keep the entire history of commands online. While privacy options that some social networks supply might hide some (or even all) of the activity from the public, some extra work needs to be done to hide the information from the host itself. Obfuscation, encoding, and the liberal use of “delete” options will scatter the data though the access logs, making it reasonably more difficult to trace the activity, rather than simply taking a snapshot of the database. Some bots don’t enjoy being studied by security researches, so they might be more exposed here.

Finally, going back to the issue of security and the corporate firewall — there will likely be a proxy server filtering access to certain websites. Some might be blocked because they distract employees (Facebook, MySpace, etc), others might match on some content. Though with a simple goal of communication, one just needs to find an online service that is trusted-enough to be widely accessible, and some means of getting it to display your supplied information.

This doesn’t even have to be accomplished by online POST requests, but now we are getting into the XKCD territory of Cuils.

For smaller numbers he has to SAVE lives. The birthrate channel is even more of a mixed bag.

No related posts.

I feel that sugar deserves a spot as an honourary member of the Computer Science field, for its integral part in the culture. It’s up there, right next to the coffee. The thing is, our brain is a powerful machine, and according to Popular Science, kicking things into the thinking gear costs 1.5 Calories per minute. That’s not as much as during a physical workout; though trying to sustain active thinking throughout the 8 hour workday will add up to 720 Calories. Keep in mind that 2000 is what is recommended for daily intake.

No wonder I find myself snacking on chocolate and loading my coffee with sugar at work — sugar is a form of an easily available energy. And taking an excerpt from a certain TED Talk — hackers have gotten in trouble for the first time over the consumption of sugar, as seen in a letter below from the director of Princeton to John von Neumann, who was building the first electronic computer at the time.

Though while I’m at no risk of obesity as all that sugar is actually burned up, there are other side-effects to watch out for — most notably tooth decay, as I recently found out from my dentist. I am not yet prepared to cut down on sugar, coke, or chocolate; so it’d be interesting to hear what others snack on during the workday. Any special measures to stay healthy?

No related posts.

Rogers, one of the two major Canadian Internet Service Providers, has been busy exercising their position of power. Again. After having been injecting content into HTTP webpages for half a year, Rogers has moved on to hijack DNS as well, replacing “not found” responses with pages full of ads. Though why stop there? Internet comes with many more communication protocols; plenty of opportunities to disrupt expected responses and inject unwanted ads into someone else’s content. All in the name of extra profits.

This is a hypothetical visual study of possible implications. All images below have been edited. It is not the point to single out Rogers; it’s just that they’ve started the trend that users disagree with. The point is that users should disagree with such actions. While not everybody might care about occasional unresolved lookups, there are other services that could be targeted next.

MSN, Yahoo, XMPP (and other) IM protocols

This one might be hitting a lot closer to home, for some, than seeing ads on “missing webpage” responses; even though it’s a similar concept. Only this time in your Instant Messenger. Every once in a while one would send a message just as the contact goes offline… more commonly it’s busy/away/whatever status… so many options to take advantage of!

WHOIS

WHOIS is a query/response protocol used to look up information about a domain. It would be easy to inject some plain-text advertising, along with the requested information. While only a small subset of the internet population uses this protocol, it is also a very specific demographic to be targeted with offers regarding domains and other web-related products.

RDP — Remote Desktop

While technically the data is encrypted, as it should be, it seems that older clients might use weaker protection schemes and be vulnerable to man-in-the-middle attacks. ISPs often happen to be in the middle. If one’s entire desktop is streamed over the internet, why not supplement some of the experience with sponsored offers?

BitTorrent

Actually a lot of ISPs already treat BitTorrent traffic in a special way. Though instead of annoying users with injected downloads and making money, ISPs often unfairly throttle this protocol, annoying users for free.

And more!

• POP3/STMP — inject ads into emails!
• RTP — manipulate streaming audio/visual content.
• IRC — similar to Instant Messaging.

The good news is that we are not there, yet. The bad news is that not enough people seem to be aware of this creeping trend. The worse news would be similar tactics catching on in other communication mediums.

In a way an Internet Service Provider is very similar to a Postal Office. Their services are purchased in order to deliver data. Now imagine Canada Post (or FedEx or UPS) operating in a similar manner. It would be like reverse censorship — instead of taking content out, they’ll be putting more in, along with your original package. I’ll doubt they will get away with such practices for long.

Original image by arbyreed

Is it because snail-mail letters are sealed in envelopes, that we expect a them to be delivered without tampering (HTTP page content injection)? And similarly expect a certain level of privacy, instead of getting a package full of ads on topics of “return to sender” mail (DNS hijacking)? And that we expect any letter, regardless of content, having paid equal postage, to be delivered at a similar level of service (BitTorrent throttling)?

Perhaps we’ll need to start placing our internet packets in envelopes as well. Some people think we should encrypt all internet traffic. Besides implementation and adaptation difficulties, this would also add additional strain to the networks… but at least it would help ISPs to stay more honest. I just hope that we will not be forced to do it this way.

Related posts:

1. Rogers ISP and their lack of neutral net
2. ISP modifying .torrent file contents

University of Waterloo’s iconic Computer Science sculpture has gone missing, from in front of the Mathematics & Computer building. The metallic blue “abstract art” sculpture that spelled out “CS”, supposedly from a variety of angles, has not been found to greet the Math students, on their way to class this morning. It feels like a small part of me has died today.

It looks like the sculpture has been stripped from the base, and the concrete base itself has been slightly moved as well. There’s a lot of construction going on throughout the campus, and it’s likely that the “CS” has been moved to be preserved while the service tunnels to the Nanotech building site are put into place. Though for now, the whereabouts of the Waterloo’s Computer Science symbol is unknown.

Before

Original image by Jim Wallace

After

Original image by Tony Targonski

Related posts:

1. University of Waterloo Tour + Wings 2007

There seem to be some misconceptions about Apple computers. Some of the common questions that come up, at least when thinking about for-University laptops, are along the lines of “Will I be able to write programs on a MacBook?” and “I thought Macs were just for arts…”. I will try to challenge that view.

Note: This is not a OS X vs. Windows vs. Linux argument. There are plenty of those just about everywhere else. If you really feel like repeating any of the arguments that have already been mentioned many times before — try the forums instead.

The use of Apple computers among the software developers is actually quite common. And with a plethora office snapshots available on Flickr, this calls for a visual case study!

Adobe

Original photo by: tskdesign. That’s totally a Mac. See the original image for higher quality. Though having made Photoshop, the Apple application, there isn’t much surprise here.

Digg

Original photo by: Mrs L. Digg’s Kevin Rose loves his MacBook. There’s more of this at just about every Diggnation episode.

Facebook

Original photo by: Zach Klein. This Facebook office comes with a couple of Apple workstations.

Flickr

Original photo by: roozbeh-a-brojeni. Searching for Flickr on Flickr is so meta…

GigPark

Original photo by: GigPark @ Flickr. GigPark is all Mac and iPhone. Those aren’t even available in Canada yet! (Also, is that Facebook.com on the screen? Slacker.)

Google

Original photo by: Anil Chopra. I don’t understand this, but one of Google’s offices has a boat for a couch. There’s also a MacBook.

LinkedIn

Original photo by: LinkedIn Blog @ Flickr. Look, it’s iMac. From another angle it’s actually a Cinema display attached to a MacBook Pro.

Microsoft

Original photo by: jeffwilcox. That’s right, a MacBook Pro on Microsoft’s campus. Actually those are not nearly as uncommon as one might think, as Microsoft develops plenty of software that runs on Macs as well.

Mozilla

Original photo by: stuartp. Mozilla’s new Toronto office is all Mac (and a cardboard box for a coffee table).

Twitter

Original photo by: kohtzy. I suspect that just about all of the computers at Twitter are Macs. Though other than the few marked in the image, event at high resolution cited, I can’t say definitively.

So there you have it — a bunch of Apple computers used by real developers at real software companies (likely using real programming languages). There are, obviously, a lot more tech companies, but I’d leave them to the comments section.

Related posts:

1. The silliest reason to get a MacBook