Adventures (in code) - Alastair Coote

"Silicon Valley is high school. But it’s only the smart kids, and everyone has lots of money"

Wed, 04 Apr 2012 16:06:00 -0400

““Silicon Valley is high school. But it’s only the smart kids, and everyone has lots of money””

- It’s official: Silicon Valley is The OC.

Bravo has apparently commissioned a show called “Silicon Valley”, which will follow people “on the road to becoming techie superstars”. If you care to see a preview, skip forward to the 1:30 mark on the video below:

link

It’s difficult to put into words just how awful this is. I suppose I should have seen it coming- The Social Network made tech startups “cool”, and I suppose this is just a natural progression of that.

The supposed saving grace of this is that it’s produced by Randi Zuckerberg- Mark’s older sister. But from impression given off by promo, I’m not seeing much of the actual reality of the startup world. Think back to the TechStars reality show- that was done with the full involvement of TechStars, and it still came out with major issues. This looks to be in a whole other league.

So, consider this your early warning. When you introduce yourself as a web developer at parties you’re going to start getting some very strange questions. And everyone is going to assume that you’re a millionaire.

Living in the Big App-le Where There's An App for Almost Everything

Mon, 26 Mar 2012 10:05:00 -0400

Living in the Big App-le Where There's An App for Almost Everything:

New York Daily News article featuring Taxonomy:

nycdigital:

Here are the top mobile applications to put the city in your pocket

Why are posts about the Geeklist controversy being removed from Hacker News?

Thu, 22 Mar 2012 15:06:00 -0400

Long story short- Geeklist had a public falling out with a female developer over a promo video they made. A storify of the tweets was made, and started rocketing around the internet. Unless you’re on Hacker News, in which case it never happened.

Well, it depends when you visited Hacker News. If it was this morning, then you had the chance to join in on a lively debate on the topic. It got to the third place on the home page- but then mysteriously disappeared. No matter- if you logged onto Hacker News later on in the morning, you might have had a chance to look at the second post. Oh, until that one was deleted as well.

What’s going on? Are people really flagging these stories to get rid of them? A story about Geeklist’s apology lives on, hiding out in the third page of stories- if the apology is relevant content to Hacker News, why is the backstory to that apology not relevant? If the HN community is flagging and deleting these stories then we’re making a statement of our own about sexism within the tech industry, and I would appreciate some clarity on what’s happening here.

So, the City wants a taxi smartphone app...

Fri, 16 Mar 2012 14:35:00 -0400

Well, file this one under “interesting news”. Today the Taxi and Limousine Commission uploaded a Request for Proposals (RFP) for a smartphone app allowing users to pay for their journey using their smartphone. Oh, and they welcome any applications which include extra functionality, sort of like the stuff Taxonomy does.

OK, I’m not getting carried away just yet.

I’m not deluded. Taxonomy is an entry in an app-making competition that hasn’t even been judged. It isn’t even a company yet. There will be larger, more established companies applying for this opportunity, and they’re probably far more experienced in submitting proposals to government than anyone involved with Taxonomy would be.

But still, it will at least be a good exercise. Proposals are due in June, so there is time to assess feedback from NYC BigApps and craft a decent, effective proposal. I’ve spent a long time thinking about what is needed in a taxi app, and I’ve got extensive experience working with online payments. At the very least it’ll be a fascinating framework to plan the future of Taxonomy around. At most, well, it could be something very interesting indeed.

The view from behind Kobayashi as he set his world record.

Tue, 13 Mar 2012 15:52:56 -0400

The view from behind Kobayashi as he set his world record.

Once my StartupBus stuff was over, I put my yellow jacket back...

Tue, 13 Mar 2012 15:51:00 -0400

Once my StartupBus stuff was over, I put my yellow jacket back on and officiated some world records. This is by far and away the biggest- Kobayashi’s grilled cheese record. I’m shamelessly reblogging this from Alex, someone who did a way better job than I did of blogging their StartupBus experience- and by coincidence also witnessed the record. Great job blogging, Alex!

southbysouthalex:

Kobayashi breaking the world record for most grilled cheeses consumed in a minute (Taken with instagram)

So, this week is going to be interesting. Tomorrow morning at...

Mon, 05 Mar 2012 13:12:00 -0500

So, this week is going to be interesting. Tomorrow morning at 4am I get on a bus bound for Austin, and SXSW. On my way there, I’ll be making… something with a group of other New York technologists.

Expect updates.

NYC BigControversy

Mon, 05 Mar 2012 09:57:00 -0500

Over the weekend, Joshua Brustein of the New York Times published an article about the NYC BigApps program. Boy, did it strike a nerve. It discussed the perceived failure of previous winners Roadify and Sportaneous and the effect the contest has had on other government transparency initiatives, while not mentioning BigApps success stories like myCityWay. As an entrant in this year’s competition, I’d like to tackle each of those issues in order.

Iterate, iterate, iterate

The Social Network made it look like a startup is the shortcut to easily millions- coding in a dorm room then being mobbed by investors while you take over the world. The reality is that the very few startups succeed, and fewer still succeed doing exactly what they originally planned to do. They change- or, pivot- and adjust the idea, or sometimes switch to an entirely different one. That’s the process Roadify is in now- they took an idea and ran with it. Over time, they’re tweaking that idea to fit the market, recently launching a v2.0 with a whole raft of new features.

Thousands of companies are going through this process as we speak (did you know that Pinterest first started in 2008?)- it’s just that BigApps winners get a lot more publicity than other companies their age. That’s a double-edged sword, because it means that outsiders expect immediate success, and take any change from the “winning” plan as an admission of defeat.

And let’s be clear- your average BigApps entry is not a business when it is entered. The guys that made Roadify did the same thing I did when I made Taxonomy- found a problem they and their friends experience while living in New York, and explored ways that technology can help solve that problem. Converting that into a successful business is an entirely different, longer process. Business is difficult.

Of open government

The other charge leveled at BigApps was that many of the entries were not civic in nature, and not in the spirit of open government. I’m not sure I’m convinced by that argument (I’d say that many entries like Taxonomy combine both civic and non-civic elements), but I also think that it ignores a larger reality- that my time is not free.

Don’t get me wrong- I like to think of myself as very civic-minded, and I’m happy to donate my time to city causes. I participated in the Reinvent NYC.gov hackathon a few months ago- it had no tangible benefit to me personally, but it was a fantastic experience. I worked with some of New York City’s best development and design brains to imagine how the city web site could be better. But it only lasted one weekend. My BigApps entry took around three months of my spare time to create. I didn’t enter the competition for the prize money (though it would be very nice), I entered it to start a business. I want to take my idea and turn it into a New York success story.

If BigApps was limited to only civic-based entries, I’m confident that the quantity and quality of entries would decline, as people like myself decide that the cost/benefit ratio has changed. I think there is a case for a civic app competition (ideas like VoterFriendly are fantastic, and deserve recognition) but that isn’t the problem BigApps is trying to solve. It’s supporting the ever-growing New York startup scene, and giving each of us that participate a new sense of entrepreneurial spirit.

I don’t think that’s a bad thing.

Edit: as a quick footnote, I want to make it clear that I’m not trying to attack Joshua Brustein in any way. He was, I think, just reporting on the opinions that are out there. My reason for writing this was that I think the world of startups is often misunderstood, and that in the same vein, NYC BigApps has been misunderstood.

Windows 8: highlighting the stupidity of Hulu and Spotify's business models

Wed, 29 Feb 2012 12:06:00 -0500

Windows 8, Microsoft tells us, is going to be everywhere. My PC, my tablet, my phone… all running the same OS, providing a consistent experience on all. Your laptop will even have a Bing Maps app, just like a tablet. Wifi connections are also everywhere these days, too. And laptops with built-in 3G connections are surfacing all over the place.

So, fast-forward a few months and imagine me with my Windows 8 laptop sat next to my Windows 8 tablet-toting friend in a coffee shop. We both feel like listening to an album on Spotify while we work, so we both look at our identical start screens, hit the ‘Spotify’ button and load the app. I select my album and start listening. My friend, however, is told “you must have a premium subscription to listen to Spotify on mobile”. Maybe we both want to catch up on the latest episode of Modern Family. Again, look at our identical start screens, load up IE, hit hulu.com, load up the video, and… damn. Hulu Plus subscription needed for the tablet.

Of course, Windows 8 isn’t out yet, and we don’t know how Spotify and Hulu are going to react. But that’s how it works right now- arbitrary lines have been drawn in the sand by music and video providers about who can and can’t access their media for free, and developments like Windows 8 are making that approach look very strange indeed.

Why can my Macbook watch Hulu videos while my iPad can’t? Because it has a keyboard? As our devices converge, these rules are making less and less sense- and these companies are going to have to change. I’m looking forward to a future where the limits of my internet connected device are decided by the abilities of the device, not outdated thinking.

My app has to dodge around government regulation. Should it have to?

Mon, 27 Feb 2012 11:47:00 -0500

Update: It seems from some of the discussion that this post has been misintepreted: it wasn’t intended as a complaint about the government. Rather, I wanted to highlight that small startups don’t have the capacity to lobby for government change, so we have no choice but to dodge around regulations. That puts us behind larger corporations- is there anything we can do to change that? We work together in co-working spaces, should we… co-lobby?

Taxi travel is stupid.

Well, OK, the actual travelling part isn’t. But the part before that- standing on a street corner in the rain, mindlessly flapping your arms around trying to attract the attention of a driver… that’s stupid. We’re living in the twenty-first century and we’re still getting taxis the way we did in the 1940s.

I have to admit, it’s (not) getting better

Now, improving taxi travel interests me. It interests me so much that I made an app with the express aim of making every part of a yellow cab journey in New York better. It helps you find destinations, estimates costs, tracks your journey and lets you share the information with your friends. It doesn’t find you a taxi. Because it can’t. You can’t book yellow cabs- the city doesn’t allow it. Now, I understand that there were a lot of reasons for creating this rule a long time ago. I understand that if I want to book my trip to the airport tomorrow morning in advance, I can use a private livery service. But if I want the nearest car to take me home right now, is that really a booking? Or is it just the modern equivalent of a street hail?

Well, according to the law it’s still a booking. So, what do I do? Well, for Taxonomy the short-term answer is simple, if unsatisfying- switch to using private car services. But imagine my fantasy future where everyone in the city is using Taxonomy to make immediate bookings of cabs every night. The yellow cab drivers have been shut out of the market by city regulations. And they pay the city a lot of money, so they’d probably have the right to be a little angry about that. They deserve to get customers. And the customers deserve to have as many cars available as possible.

Of laws and law-making

It’s not like my idea is alone in this, either. Uber, the car service that works along similar-but-different lines to my idea, has been threatened with bans in DC and San Francisco. AirBnb has a cloud of questions following it around. Finance, education, transportation… anyone working in these areas can tell you that there are acres of regulations and red tape designed to make it difficult for a young upstart to do something interesting. So what do we do about it? I’m a techie learning my way into the business world, but do I also have to become… a lobbyist?

But lobbyists are evil! Hmm. Refusing to bring money into the process might make it more legitimate (it’s not like I can afford it anyway) but there’s just a bigger barrier getting in the way.

Time = Money

Law changes move at a snail’s pace. In fact, they usually move backwards a few times along the way, too. Vested interests will be resistant to change (really, an understandable human reaction to an outsider deciding they want to change the way you make your living), so you’ll have to negotiate with them too.

The simple fact is that we don’t have enough time. When you’re trying to kick-start a business on a shoestring budget, every minute is valuable. And disappearing down the rabbit hole of government regulation is a great way to ensure that you won’t move forward. So we dodge around the issues and keep going. But it hurts our products and it hurts the experience for the end-user.

Surely there must be a better way? Discussion is, as always, welcomed on the Hacker News.

This post is part of a series about the creation of Taxonomy, a taxi assistant app. It is an entry in an app-creation competition called NYC BigApps. There’s a public voting segment, and I’d really appreciate your vote. The irony of BigApps being a city-run app competition is not lost on me.

CartoDB Blog: Taxonomy: Your taxi experience in NYC

Wed, 22 Feb 2012 10:28:06 -0500

CartoDB Blog: Taxonomy: Your taxi experience in NYC:

cartodb:

Alastair Coote, a web developer currently living in New York, is writing a series of posts about the development of Taxonomy, a taxi assistant app he made using a free dataset of taxi drivers’ names and IDs, available at the NYC Open Data website.

Taxonomy uses the…

What's next for Taxonomy?

Mon, 20 Feb 2012 13:08:00 -0500

Taxonomy is in the App Store, and the NYC BigApps contest is in full swing. The public vote has been nothing short of insanity, and Taxonomy has next to zero chance of making it within the top two and winning the public vote prize. But that’s fine by me- there are eleven other awards up for grabs and it’s kind of liberating to be able to renew my focus on them.

So, with that in mind, I thought it was time for a quick update on what I’m working on right now, and where I want to take Taxonomy in the future.

App Updates

As per the rules of the BigApps contest, I can’t release an update to Taxonomy until the voting period is over. Very frustrating, as there is some stuff that badly needs to change in the app. I’m readying two updates right now:

1.0.1 Bug fixes

The app was created with an iPod Touch as the test device. It turns out that it has a few significant differences compared to an iPhone, and that a couple of performance-killing bugs have crept into Taxonomy as a result. Now that I’ve had a chance to test with an iPhone, I’ve started putting some fixes together that’ll make sure the app works great on all devices.

1.0.2 Speed improvements

Now this I did know about, but didn’t have the time to fix- Taxonomy is slow. Not all the time, but at various points it just doesn’t have that “slick app” feel. A lot of this has to do with the underlying jQuery Mobile framework the app uses- I’m making some low-level changes there with the aim of making the app a lot more responsive.

It’s quite likely that both these updates will be rolled up and put into the App Store together, but if I start to run low on time, the bug fixes will be rolled out independently.

A pivot to profit

In the longer term, I’m looking beyond the BigApps competition. How could Taxonomy become a viable, fully-fledged app instead of a competition entry? Neatly, the answer involves fixing the biggest hole in Taxonomy’s arsenal of taxi weapons: getting a taxi.

As I mentioned in the introduction post, Taxonomy can’t find you a taxi. There’s a legal reason for that- the city forbids making bookings of yellow taxis- but it’s easily the most painful part of taxi travel. I’d love to make my case to the city about how taxi travel needs to change (surely an instant trip with the nearest taxi is more of a 21st century street hail than a booking?) but that’s probably not a viable tactic for running a business.

So, the future more than likely lies in private car services. There are dozens of then in New York, but no-one I know really uses them except for trips to the airport. But there’s clearly excess capacity- you see them driving around Manhattan trying to pick up street hails (which they’re technically not allowed to). So why don’t people use them? Phoning to book. Uncertain cost. Lack of trust of the company. Unknown arrival time. These are all reasons that have been given to me for why people don’t use the private services, and I think that Taxonomy could solve them all.

Imagine a “Seamless for taxi travel”, for want of a better comparison. Listings and reviews of taxi services, as well the existing fare estimator and car tracking. There’s certainly some competition out there, but I don’t think anyone has done it right yet. Taxonomy could fit that space.

But first, BigApps. Even though I won’t win the public vote, I’d still love your help in earning semi-respectable placing.

Think all government data is boring? Think again: a showcase of location data you want to use.

Fri, 17 Feb 2012 12:00:00 -0500

There’s a ton of government data out there- you probably already know that. But it’s no good to you, right? Education statistics, financial reports… very important stuff, of course, but kind of… dry. Not exactly “every day use” sort of stuff.

Well, I’m going to try to challenge that perception a little. I signed up for a contest called NYC BigApps, which is a city-run competition to encourage use of city data. I was pleasantly surprised by the data I found. So, I thought that for a little bit of Friday fun I’d run through some of the more interesting and unexpected geo datasets. At the end, I’m going to discuss how I’m using NYC open data in my BigApps entry, Taxonomy.

The embedded maps you see are provided by CartoDB, which rocks. Seriously, go check it out. Each of the markers are clickable, giving you a sample of what data is available. The data, of course, comes from the NYC Open Data web site. I’m sure other cities have similar data out there- and if they don’t, urge them to release it.

WiFi hotspots

link

An absolute no-brainer, this one. Everyone needs a WiFi connection every now and then, and this dataset shows you their locations, and whether they are free or paid (in the map above, green is free, orange is paid). The world is crying out for ratemywifi.com - how’s the connection speed? Does the guy behind the counter get angry when you only order one small coffee every four hours? Do you get a cool IP address or some lame 192.168 handout? What’s the hacker to amateur novelist ratio? Surely a billion dollar IPO awaits.

Filming Locations

link

File this one under “I never even thought of this before”. A ton of stuff gets filmed in New York every year, and of course permits need to be approved before the cameras can roll. It turns out that the New York government has made that data public, going all the way back to the 60’s and before. Maybe next time your friends visit you can assemble a walking tour of your favourite movies. Fun fact: NYC BigApps entry Scene Near Me uses this data to text you when you check in near a filming location.

Public Bathroom Locations

link

OK, OK, this isn’t exactly the sexiest dataset going, but don’t try to tell me you wouldn’t use it. Whether you’re in Central Park or just out shopping, no-one wants to buy a coffee just so that you can go to the bathroom- it’s only going to make you need to go again in half an hour. Never was there a greater resource for the public good - bathroo.my, anyone?

Parking Locations

link

I don’t own a car, but I’ve heard they’re quite popular. I also heard at the BMW iVentures launch party that parking in the city is an utterly miserable experience. So it was pleasing to see that the city has an open directory of parking garages, even including the number of parking spots in each one. As you may be able to tell from the map above, there are quite a few.

How I’m using NYC Open Data

As I said at the start of the post, I’ve used NYC Open Data myself, in my NYC BigApps entry. One sits at the heart of the app- it’s the taxi driver dataset. I use it to allow Taxonomy users to review their taxi driver- the idea is that over time we collect feedback on the best and worst drivers, as well as tracking which ones take the most efficient routes, and which ones don’t know their way around.

I made another- more fun- data mashup, though. I wanted the map in my app to be dark rather than the usual Google Maps bright and bold, so I made custom map tiles. As part of that process, I used the following datasets:

Building Perimeter Outlines
Street Centerline
Roadbed

(from the NYC DoITT GIS downloads section) to create the higher zoom-level maps. I’m very happy with the way they turned out, and I’d encourage you to give them a try next time you’re making a map-based app or site:

Taxonomy is available in the App Store, and even has a web site for you to find out more.

Vote for Taxonomy in the NYC BigApps contest!

Thu, 16 Feb 2012 15:21:34 -0500

Vote for Taxonomy in the NYC BigApps contest!:

Hey folks. My taxi-tracking app, Taxonomy, is entered into an app creation contest. There’s a public voting section, and it’s currently tied in 9th place. I need to push ahead! I’d appreciate your votes:

http://bit.ly/yURqU8

HTTP referrers: a good thing or a bad thing?

Thu, 16 Feb 2012 12:17:00 -0500

During my daily browsing of Hacker News, I found a submission lamenting the forced move from browser to app when consuming content on a tablet. Too true- it’s infuriating. But it’s worse than that- it breaks one of the little underpinnings of the web that almost all web devs use, but rarely consider in their own projects- HTTP referrers.

It reminded me of a very unexpected- but happy- event that occurred earlier this week. My post on making custom maps had been a hit on HN, and I was watching my live Google Analytics data (which has become an obsession for me since it launched). I noticed that I had one, then two, then three referrals from “chat.stackexchange.com”. Intrigued, I dug a little deeper and found out that someone looking to solve their own mapping problem had stumbled across the post, and people in the ‘Fake Programmers’ chat room in Stack Exchange were discussing it. We ended up having a great discussion on mobile mapping, offline routing and other such ideas.

I’d like to think that it benefited all parties involved. But it wouldn’t have happened if they were chatting inside an app. Or, even, using IRC. WebSockets and speedy JavaScript engines mean that creating an HTML5 chat room is now an entirely feasible project, but it seems these technologies arrived too late (or were mis-marketed) and allowed apps to swallow the majority of mobile usage.

But, of course, what of privacy? A lot of sites mask their referrer data- Twitter hides behind t.co links, and Facebook sends every link through one “l.php” file (though that is changing drastically with Open Graph- a possible future post to follow on that topic). Indeed, many chat room users might not want page owners to be aware when they’ve sent a link out.

Where’s the line?

As a developer, I’d love for browsers to provide some kind of referrer hook that would allow, say, and IRC app to load a page and set the referrer as “irc://someserver”. But as a user, I want my privacy protected so that unscrupulous site owners can’t analyse everything I do. I’d be interested to know people’s opinions on this, and how we can balance the two viewpoints.

This post is part of a series about the creation of Taxonomy, a taxi assistant app. Taxonomy is an entry in an app-creation competition called NYC BigApps. There’s a public voting segment, and I’d really appreciate your vote. Thanks!

Fun with analytics: pitting Hacker News and /r/programming against each other

Wed, 15 Feb 2012 11:56:00 -0500

I noticed a weird anomaly in my analytics today. It started shortly after midday EST on Monday, and lasted for approximately 24 hours. What was the shape of this crazy UFO-style disturbance? Something like this:

I call it the snapped hockey stick.

The anomaly was my post about making custom maps for my app, Taxonomy. I submitted it to both Hacker News and /r/programming, and to my surprise it was a hit in both. So, delving through the figures, I thought we could have a little fun, and compare Hacker News to /r/programming- the figures, the discussions, and which of you are nice and voted for my app.

How it started

After writing the post, I immediately submitted to HN and /r/programming (or proggit, to its regulars). There was no immediate traction in either. I also sent out a tweet, cc-ing MapBox, the makers of the TileMill software I discussed in the article.

You’ll notice that I sneakily included a link to the HN submission. I was retweeted by MapBox, which started to bring traffic to the site. Looking at my analytics real-time stats, there were around thirty users on the site at any given moment.

Soon, traffic picked up- and fast. My HN submission had gotten about four votes- if your submission is recent enough, that’s all it takes to climb the board rapidly. Meanwhile, the Reddit submission had also taken a life of it’s own and arrived on the proggit homepage. At peak, I had over 350 people browsing the site at any one time. So, did including the HN link in my (retweeted) tweet help? The jury is out- the proggit link did just fine without it.

The figures

So, first off, let’s take a look at the raw numbers. It may not surprise you to learn that Reddit trounced Hacker News in most main metrics:

link

So, for those of you still unaware: Reddit (and this was only the programming subreddit) is a traffic building machine. As for the distribution over time, the two started off similarly, but changed as time went on:

This resembles (but does not match) the chart positions. Both entries shot up relatively quickly- peaking at around 9th place on Hacker News, and making it all the way to first place on Reddit. But it started to slide back down the HN board relatively quickly, leaving the front page by the end of the day. The Reddit post, however, remained in the top spot for nearly twenty four hours. It definitely didn’t see a traffic peak on day #2 however, suggesting that being in the top spot for a long time isn’t as beneficial as you’d imagine- presumably, all the proggit members that were going to read the article already had.

The small bump in Reddit traffic towards the end corresponds with a crosspost I made (after it was suggested to me) to /r/geospatial. A much smaller sub-reddit (1,900 readers vs 356,000), it only attracted four upvotes- but at it’s peak wasn’t far off the peak HN traffic.

Engagement

So, that’s the raw traffic. But how can we examine user behaviour in a little more detail? To start, let’s take a look at average time spent on the page:

link

None of them exactly seem stellar when put like that, but proggit definitely comes out as the bad boy at the back of class here- 16 seconds is surely barely enough time to read anything. The ‘other’ category was made up primarily of Twitter traffic- it stands to reason that @MapBox followers would be more interested in the post than the average programmer, so it’s interesting to see that HN almost matches it.

Who is my best friend?

So what about the users that actually performed an action on the site? I did have a call to action, of sorts. My app is running in a competition that has a public voting segment, and I had three links on the page urging people to vote (did you know that it’s still open? And that you can vote every day? Uh-huh.)

It’s not really a traditional call to action. Normally, you would be using your page to persuade the user that it is in their personal interest to click your link- that they will get something out of it. My voting link can’t really promise that- I’m asking people to inconvenience themselves for absolutely no material benefit. So it’s interesting to see how everyone behaved. Annoyingly, I only managed to plug in vote tracking at around 4pm EST, so these results aren’t so accurate (in the interests of fairness I’m only taking traffic from after 4pm, too):

link

So- proggit may give you traffic, but HNers are almost twice as likely to click. That said, I got more than double the traffic from proggit, so it still wins in overall clicks.

You’ll notice that I say clicks and not votes, because I can’t actually guarantee that someone clicking the link will go on to vote- indeed, according to my stats I ought to have got 271 votes from HN and proggit combined, but that’s actually less than my overall vote count after a week of voting. I estimate that I got around 30 votes out of the whole thing- pushing me up two whole places to 9th, but suggesting that ChallengePost have a low conversion rate on their voting pages. Still, thank you to everyone that did vote. (did I mention that you can still vote? And that if you already have you can vote once per day? Oh, I did. Sorry.)

(If anyone did go to the voting page them subsequently not vote, I’d be very interested to know the reasons why- let me know.)

Other observations

That’s the numbers more or less crunched. There isn’t much statistical analysis to be done of the comments, but there are some interesting anecdotal points. The comments between the two sites were varied- proggit had an in depth discussion about the name Taxonomy, something that didn’t appear once in Hacker News. For it’s part, HN had some detailed feedback on specific roads that looked wrong, and labels that were difficult to read. In the world of “popular off topic discussion”, HN was peeved by the persistent toolbar that my blog had, while Reddit was preoccupied with getting me to kiss the guy who works down the street from me (both posts were voted second and first, respectively).

One thing that didn’t happen was my post going viral. When I’m watching stats with RecordSetter, a link will often be featured on a large site (The Daily Mail, or HuffPo, for example), then pop up on ever-decreasing sizes of site for days afterwards, creating a sort of outward ripple of traffic that lasts for days. I saw nothing like that with my post- no-one linked to it (aside from a few Facebook and Google+ inbound links I saw), and the traffic was all but dead shortly after 24 hours had passed. I have no insight into why this might be the case, but it’s an interesting data point.

Conclusion

Stats are fun. Now I’m off to submit this link to Hacker News and proggit so that I can examine that traffic, in a beautiful, never ending circle of numbers.

Vote for my app! Daily!

Tue, 14 Feb 2012 11:25:00 -0500

Vote for my app! Daily!:

Tumblrators:

I’m really happy that everyone enjoyed my article on making custom maps. The app I made them for is entered into a competition, and you can vote for it once per day. So if you have a few spare minutes, I’d be very grateful:

http://bit.ly/yURqU8

I had no idea how to make custom maps, so I learnt by doing. You should too.

Mon, 13 Feb 2012 11:19:00 -0500

Lately, I’ve been making an app called Taxonomy (that desperately needs your votes …sorry). It’s designed to make your taxi experience better- a traveling companion, if you will. Going by the logic that a lot of people (in New York, at least) only really travel in taxis when out at night, I decided that it ought to have a dark colour scheme. Not morbid dark, but, y’know, “mischievous things happen at night” dark. I mostly succeeded- styling my logo, headers and buttons in a dark monochrome with some bright yellow highlights (it is a taxi app, after all), but every time I wanted to show the user’s current location this ridiculously bright Google Map showed up and ruined my carefully cultivated style. Something had to be done.

I played around with the styling options Google Maps now provides. It’s powerful, but it still wasn’t enough. The fonts were wrong, the options were difficult to tweak, and- as I was quickly discovering- the Google Maps API doesn’t even perform that well on mobile devices. I remembered back to a Hacker News post I’d seen some weeks earlier about TileMill, and the beautiful maps I’d seen created with it. Now, I’m just a web developer- I’ve done enough projects to know what latitude and longitude are, but that’s about the extent of my knowledge. Despite that, I decided to dive in and give it a try. How hard could it be?

Very hard. Also, very easy.

TileMill really is a joy to use. The installation (on my Ubuntu VM) was a simple sudo apt-get install, and the front-end is entirely web-based. Within five minutes I had it running, and was panning and zooming my way around a map of the world. But before I could get started on my Taxonomy maps, I needed some data. And that’s where it started to get a little complex.

OSMs and SHPs and DBFs, Oh my

First off: data sources. If you’re doing what I’m doing and making a roadmap, OpenStreetMap is your best bet (I actually used NYC government data for parts of mine, but it would take about five blog posts to cover that). I didn’t want to download the entire planet from OSM (all uncompressed 250GB of it), but luckily there are numerous sites out that that serve up pre-chopped, pre-compressed datasets for you to download. For me, Teczno served very well- it had both a SHP file of the New York coastline, and an OSM file of all the roads contained within.

The SHP is imported very easily, but the OSM has to be converted using a tool named Quantum GIS, which has an OSM plugin allowing to import OSM data, then export a shapefile.

Back to the mill

So far, so complex. But now we’re back in the cozy arms of TileMill, and everything feels a little easier again. Once you’ve imported all your layers, you just need to write some CSS. Well, not exactly- TileMill uses a language called Carto, which is syntactically identical to CSS (or, more like LESS/SCSS as it allows nesting), but uses different properties. You refer to layers by id or class, both specified on import.

OSM labels roads by category, allowing you to use different widths for different road types. For example:

#osmroads[highway='residential'] {line-width:5;}
#osmroads[highway='primary'] {line-width:10;}

Should you want to look through the road types, you can click the magnifying glass icon on the layer selector to open up a data view, like so:

Sadly, there’s no way to click on a road on the map and see the data on it- exasperating, as there are more road categories than you think, and it’s difficult to track them down. One fun solution I did find was uploading the dataset to the always amazing CartoDB (their developer Javier Torre is always on hand to give tips and advice) which lets you click on individual roads and see the associated data.

Say goodbye to your life

This is an awkward part of the blog, because it’s the part where I tell you that I’ve endangered your life. Put simply, tweaking these maps is addictive. You’re literally never done. Some of it is because the language is still a little fiddly- for instance, there’s no variable for the current zoom level, so if you want to vary widths you have to make multiple rules like:

#osmroads[zoom=10] {line-width:2;}
#osmroads[zoom=12] {line-width:4;}

and so on, and so forth. But it’s mostly because making maps is an art, and if you’re anything like me, you’re nowhere near the master of it. So every little tweak you make is an exploration in map design- what works, what doesn’t, when it’s too busy, when it’s too empty… TileMill is an enabler- it makes the tweaks so easy (and reflects them instantly in the preview) that you’ll probably never stop. Don’t say I didn’t warn you. Luckily for me, I was working to a deadline, so I had to stop. I’m not perfectly happy with the tiles I’ve got, but they’re good enough for the v1.0 of Taxonomy.

Putting it all together

So, once you’re happy with your maps, you need to export them. Get ready for a long wait- tile crunching takes time. To produce my set of NYC tiles took around 11 hours- multiply that by two because I had to make a second set for iPhone retina displays. It’s not a killer, but it does mean that you can’t get instant results- prepare accordingly.

Once all my tiles were exported into the TileMill .mbtiles format, I installed its stripped-down server counterpart, TileStream, on my Amazon EC2 server. With a CloudFront CDN instance in front of the server, it makes for a great distribution method- much easier than putting and deleting hundreds and thousands of images on S3. It’s worth pointing out that the makers of both these products, MapBox, also provide some very hassle-free hosting, should you be interested.

For the final integration into my app, I used leaflet.js in a web view. I was astonished at just how good Leaflet is- the Google Maps API had led me to believe that HTML5 maps would never match their native counterparts, but that’s not true. It’s allowed me to make Taxonomy 95% HTML5-based (just the nav bar components are native), and provides me with an extremely simple path to an Android port in the near future.

All done

If there is a conclusion to this post it’s that if you’ve ever considered making your own custom maps and are unsure- you need to try it out. I think they’ve been a major value-add for Taxonomy, and I’ve enjoyed every minute of working on them. If your site/app operates worldwide the time and disk space involved in crunching the tiles (my retina version of just NYC is 1.7GB) might prove prohibitive, but otherwise you should give it a shot.

If you want to take a virtual tour around New York using my maps, take a browse in the embedded map below. Discussion is welcome on the Hacker News post and Reddit post.

This is the first in series of posts about the creation of Taxonomy, a taxi assistant app. Taxonomy is an entry in an app-creation competition called NYC BigApps. There’s a public voting segment, and I’m way behind. I thought that writing some original content was preferable to spamming my friends- if you found this post useful, I’d really appreciate your vote. Thanks!

Introducing... Taxonomy

Wed, 08 Feb 2012 22:31:00 -0500

I made an app. If you’d like, you can go straight to it’s NYC BigApps page (and give it a vote!). Or, I can tell you a story.

State of the Taxis

There are over thirteen thousand taxis in New York City. Chances are, if you’re not taking the subway to your destination, you’re in a taxi. Unfortunately, our experiences with them aren’t always that positive. I took a particularly memorable trip a few months ago that went through the wrong side of Central Park and ended with me sprinting down the street to my (y’know, important) destination. As I sat down- apologising profusely for being forty-five minutes late- I thought to myself: there must be a better way. But I had no idea who that driver was, nor whether he was bad at his job or just having a one-off bad journey. Taxis are a totally unknown quantity- where are they, how good are they, how much will my trip cost… there’s just no way to know.

Enter, BigApps

The NYC government is into the tech industry in a big way. One of the ways in which it shows it’s affections is the NYC BigApps contest, which is in it’s third year this year. The premise is simple: make an app that uses some form of open city data, win prizes and promotion. To kick off the 2011/12 round of BigApps, they held a weekend Hackathon at Pivotal Labs, and I went along. Browsing through the available datasets, I stumbled upon a dataset of taxi driver names and IDs and- thinking back to The Longest Cab Ride Of My Life- began to formulate a plan.

The hack I submitted that weekend wasn’t complex (you can read more about it in a previous post, if you’d like)- it took your taxi driver’s ID and tracked your journey through the city, allowing you to review your trip once it was complete. End result: a growing dataset indicating which taxi drivers know their way around (or, to be more skeptical, which ones deliberately go the wrong way), and what their customer service is like. I’m pleased to say that it picked up two awards at the end of the hackathon, and enough positive feedback to convince that I was not alone in my taxi frustration and that I ought to make a full app out of my hack.

Progression

As I sat at my desk with my hacked together webapp in front of me, I realised that it wasn’t enough. Reporting bad drivers is a great incentive for the civic-minded, but if I wanted this app to be used by everyone it needed to be more compelling than that. I got to thinking about a taxi journey from start to finish, where the points of frustration are, and what I could do about them.

Starting at the start- where am I going? I might have the name of a bar, but I’ll have to look that up. Perhaps I have a street address, like “129 E15th St”… but I don’t know where that is. Neither will my taxi driver. So let’s make a destination search engine that allows you to search by name or street address, and that returns meaningful street intersections (“East 15th and Irving Place”) you can use in a cab.

OK, fantastic, so I’ve found my destination. But how much is this taxi going to cost, and how long is it going to take? How about throwing in a custom route planner that works out a suggested route, and tells you the information you need to know about your trip. Bonus points: let’s take all the information we’re receiving from other users (you’re travelling down Broadway at 5mph?!) and use it in our calculations, so you get the most accurate estimate possible.

Now that I’ve gotten a cab, I need to tell my friends that I’m on my way. But what if the journey takes longer than I thought (say, forty-five minutes longer)? Let’s track my journey and make it shareable. Let my friends get up to the minute updates on my current location and destination. While I’m in the cab, why not make some use of the dead time? We both know I’m going to forget to check in on Foursquare when I arrive, so let’s enter that information now. The app can send it in once I’ve arrived at my destination.

But let’s not forget the roots of the app- once my journey is over, we’ll provide options to send off a review of your experience. And we’ll still log what route your driver took to see if they are going the right way.

Oh, and Google maps are boring. Let’s make them look nice.

Taxonomy

The result of these thoughts and three months of “in my spare time” development has created Taxonomy (I hope you’ll agree that it’s a catchier name than “NYC Taxi Tracker”). It’s available for iOS in the App Store, and an Android version is on the way. I also made a promo web site at www.taxono.my for your viewing delight.

The future

You’ll notice that I missed out a step back there- actually finding a taxi. There’s a good reason why Taxonomy can’t do that right now- it isn’t legal to book a yellow cab in New York. So, Taxonomy’s future beyond NYC BigApps belongs with private car services. There are dozens of private car services in the city, and no-one I know uses them unless they’re going to the airport. Yet, you see the black cars out on the streets on a Friday night trying to pick up street hails (which they’re not technically allowed to do).

So, let’s make private car bookings more convenient. Just think what services like Seamless did for ordering food, and ZocDoc did for doctor’s appointments- get rid of the horrible phone bookings, post public reviews to let the best rise to the top- and tie it all into your current location.

Consider this NYC BigApps entry a taste of what is to come- making every taxi journey simpler and faster. And with less arm waving on rainy street corners. So, do me a huge favour and go to Taxonomy’s NYC BigApps page and give me a vote!

Dear Tech Community, We Have A Communication Problem

Wed, 18 Jan 2012 14:03:00 -0500

Today is the big day. Wikipedia is out. Google is doodle-out. I wanted to play my part, so I headed down to the NYTM protest outside the offices of Senators Schumer and Gillibrand. It was a fantastic sight- the turnout seems to have been huge, and the majority of people were carrying signs and placards.

Sadly, I couldn’t stay for long (indeed, the protest is still going on as I write) so my experiences should absolutely be treated as anecdotal- not a reflection of the protest as a whole. But then, that’s the exact experience a passer-by gets when walking past on their lunch break.

What I saw worried me. We’re not making the right points. Or when we are, we’re not saying them right. For example, a conversation I overheard between someone giving out flyers and a member of the public:

“So what are we all protesting?”

“SOPA and PIPA.”

“What?”

“SOPA and PIPA. It’s an anti-piracy law that…”

Argh. You’re using a SOPA-supporter talking point, then disagreeing with it. But it’s too late, you’ve already lost him- because who wouldn’t be in favour of an anti-piracy law? Despite the way they’ve been framed (and named), SOPA and PIPA are not anti-piracy laws. They are laws that will allow the government to control your access to the internet. That is what we need to be communicating- not what the law is intended to do, but what it will actually do.

Meanwhile, the speaker at the front was describing how “Hollywood is trying to prop up their broken business model so that they can buy themselves Porches for a few more years”. Also not good- if you were just wandering past you might think it was a rant about wealth distribution. We should be focusing on:

We are a success.
The tech industry is one of the few economic bright spots in the country right now. It’s a tired phrase, but we are job creators.
We- the job creators- are fighting against intrusive government regulation.
The proposed laws would add a huge administrative burden upon tech companies, and could drive them out of business. People would lose their jobs. (You’d think every Republican Presidential candidate would be falling over themselves to endorse us, wouldn’t you? Strange, that…)
We are anti-piracy.
Piracy is a legitimate problem, and the internet is an enabler of it. There are existing provisions that work very well (DMCA) and we are open to discussing whether further measures are appropriate. But SOPA and PIPA go too far, and were created with zero tech consultation.
Innovation is better.
We are the people that created Netflix and Spotify- both services that have helped to dramatically reduce piracy by providing content to users in a form they are willing to pay for. We are innovative, but SOPA and PIPA will dramatically restrict our ability to continue to be innovative in the future.

This has the potential to be a huge milestone in activism within the tech industry. We’re coming of age. Today’s blackouts have shown that we are full of will, but we need to have the same razor-sharp focus around messaging that older industries have developed. If we’re not careful this is going to be portrayed as “tech companies want to make money from piracy”, and it’ll all be for nothing.

Don’t get me wrong- I don’t wish to write off the protest as meaningless. Seeing that many people out protesting for our cause was fantastic. I just want to make sure we do this right.