updates @ m.blog

The Year in (Mostly Stupid) Side Projects

2012 was a very busy year for me, as things really ramped up at Bitly, getting settled in on a different coast of the country, etc. However, I managed to find some time in the margins for side projects, and I thought it would be useful to reflect on the things I built and what I learned.

Like many geeks, side projects are often more about having a task (it doesn’t really matter what) to explore learning a new technology, rather than building something out of a fervent desire it must exist in the world. Thus, my side projects tend to veer towards single-serving jokes, but hopefully there is something in the underlying tech that could be useful to others.

Note: While 2012 is the first time I feel like my code is approaching the point where it might actually be useful to other people to learn from, I’m by no means an actual software developer – so I’m probably a bad source for “the right way to do things” and possibly a better source for “what it looks like when a part-time hacker screws around making something.”

Anyhow, enjoy?! On to the projects (in chronological order)…

The Projects

personalappeals

Not much of technical note here, a simple Rails app doing Hot or Not with those ubiquitous annual Wikipedia banners. Pretty much notable only in that it annoyed Jimmy Wales – is there a badge for that?

screenshot-pa

Most important thing I learned: db:seed is really handy. Jimmy Wales doesn’t like it when people sell Trucker Hats with his image on them.

web | source

star dot ws (šŸŒŸ.ws)

On most browsers/OSes you won’t even be able to see the name of this project, let alone get to it.

At some point I found a loophole to allow registration of emoji domain names. That’s mostly it, that’s the punchline – just that they exist now, and that you can do that. Needless to say, once I figured that out, I went on a bit of a domain shopping spree…

stardotws was a quick something I put up for one of the emoji domains I registered. Since it’s a website that you can pretty much only get to from an iPhone, it’s not really optimized to run on anything else. I used some quick CoffeeScript for the animation.

screenshot

Most important thing I learned: While Heroku is an amazing place to host quick hacks, the Cedar stack really isn’t a terrific place for static content – I believe I spent more time messing around with various ways of trying to replicate simple page caching than I did creating the rest of the page.

web | source

pigstream

A simple program to power the GloatingPig twitter bot, which is a reply-bot that taunts people who can’t beat a level on Angry Birds (when I first wrote it, Angry Birds was a much more popular game).

Most twitter reply-bots have to constantly poll Twitter, and maintain some sort of state to avoid spamming the shit out of everything. Instead, I decided to use the Twitter Streaming API here, so there is no polling needed, as well as no state to maintain – Twitter simply lets us know when there is an event we care about, and we can act upon it in realtime. This not only makes pigstream very efficient, but also a lot more fun: as a reply-bot that responds fairly instantly catches people in the visceral, emotional moment that caused them to tweet to begin with, thus their reactions are better.

screenshot

8 months later, GloatingPig has interacted with hundreds and hundreds of people on Twitter, and not a day goes by that he doesn’t pleasantly surprise someone, often generating replies, favorites, and retweets. This is my favorite kind of project – “perpetual motion” side projects – simple concept, low effort of implementation, and it lives on and keeps going on its own without requiring maintenance, upkeep, or marketing.

Most important thing I learned: Whenever you can, it’s beneficial to avoid having to handle state.

web | source

goodvsevil

A quick hack I did mostly to teach myself Redis. Uses the Twitter streaming API to keep a constantly running tally of whether cats or dogs are mentioned more on Twitter. I meant to replace it with a less trivial example, but never came up with something interesting before I moved on to other projects. Fits my “perpetual motion” hack criteria so it lives on today.

screenshot

When cleaning up things later on, I decided to go back and add a “streaming” mode using SSE, to see what things would look like in even-more-realtime. Ultimately I didn’t like the feel as much, but it does mean this project could actually serve as a useful template for someone who wants to see either how to subscribe to and monitor via the Twitter streaming API with Redis backing, and/or how to monitor that in realtime via a web UI using SSE or polling.

Most important thing I learned: Redis is a-maz-ing and easy to fall in love with. Also, if you use Oj for JSON parsing you can do an insane amount of throughput even on a single free Heroku dyno.

web | source

PetChecker

My dog walker uses a service called Pet Check Technology, which logs their walks with GPS along with notes, etc. Somewhat cool, but unfortunately: 1) their website is atrocious to interact with, 2) there is no iOS app for the clients, only the walkers.

Thus, enter in a lot of screen-scraping and tcpdump to reverse engineer their website and private API… and many painful hours later, PetChecker was born:

ss-list ss-details

Boy, do I have a lot of respect for people who do iOS development – despite having some experience with Obj-C (dating way back to the NmapFE days), this was painful. Things that should be simple involve so much boilerplate code, it’s ridiculous. I’m definitely planning to check out RubyMotion should I ever decide to do an iOS app again.

I want to make sure there are no legal complications releasing this one to people other than myself, so screenshots only for now unless someone really wants it which will motivate me to get to the last mile.

Most important thing I learned: While Cocoa is a painful environment for quick hacky development, it can become even more immensely painful when dealing with messy data/APIs where you might have to hack around things not working the way they should in an ideal state – which happens a lot when reverse engineering something not “intended” to be a web service. Most of the Cocoa convenience libraries I worked with failed miserably when you have to deal with things like HTTP servers not returning the proper Content-Type or status code. RubyMotion (and BubbleWrap) may be a more hacker-friendly alternative to investigate for this sort of project.

lolcommits

The project that took on a life of it’s own. Simply put, lolcommits takes a snapshot with your webcam every time you git commit code, and then archives a lolcat style image based on that. It started as a quick joke when I was trying to think of something amusing to do when I figured out you could programmatically access the webcam on a MacBook, and has since morphed into a project with thousands of users, dozens of contributors (who quickly helped port it to both Windows and Linux), and a whole lot of lulz.

lolusers

This project involved a lot of firsts for me. While I’ve spoken at many conferences and given many talks, presenting lolcommits at Hack and Tell was the first time I’ve given a “tech talk” on code I’ve personally written, which was a special moment for me.

This project was also when I really had to begin writing a full test suite for my software. Since lolcommits has to work across all sorts of variations of operating systems, Ruby versions, library dependenciesā€¦ it’s really easy for a change to break something in an unexpected way (often on a platform I didn’t have access to test on myself). Having a good test suite helps minimize that, although it’s still not perfect. When you’ve been in the business of web development for a long time, going back to shipping versioned desktop software can be a shock to the system. Us web guys have it easy – browser incompatibilities are nothing compared to the unpredictable state of system configuration on a random desktop workstation. (On that note, travis-ci is fucking awesome for automated testing across multiple builds. Can’t wait for them to add multiple operating system support.)

Most important thing(s) I learned: When managing a project with multiple open-source contributors, a functional test suite will save your bacon. Having your software be dependent on ImageMagick will put you in configuration hell for maintainability across systems – unfortunately there are zero good alternatives I was able to find.

web | source

pullcrusher

There are lots of great utilities out there to losslessly optimize images, but most people forget to run them. Pullcrusher makes it easy to optimize the images of any GitHub repository, and contribute the optimizations back to the maintainer as a pull request – in a single command. Possibly the only actual useful software I produced this year, and ironically, thus the least popular!

screenshot

Super useful, yeah? Guess that’s why no one cares. :-)

Most important thing(s) I learned: git has so many environment and configuration variables that can modify operation, that when developing software that extends and depends on git functionality, it’s helpful to use a clean development VM to ensure things will work on other systems. Vagrant is a great way to do this.

source

canalstintegers

2012 was the year of artisanal integers. Since Aaron is a dear friend, I couldn’t resist the chance to poke a little fun at an already fun meme, thus canalstintegers was born. Not sure how well the joke works outside of NYC, and it certainly only is funny to people who are familiar with the artisanal integers meme, but hey, it has an API!

ss-canal

Most important thing(s) I learned: I care far more about HTTP status codes than celebrities.

web | source

What next?

I’d love to work on some more mobile projects I think, but in general I’m going to keep up my modus-operandi of exploring new technologies with silly ideas. Hoping for a productive 2013!

Comments