Tuesday, October 11, 2011

On Kottu development, and oxygen for ideas

Kottu, after the 8.0 redesign

The thing with designing a software system you have no idea about is that you don't know how deep the water is, and you never will, until you jump right in. When Indi contacted me back in July with the proposal to join him at Kottu, I was stunned. It was truly a dream come true. I loved Kottu, Kottu was every young blogger's best friend. And it was where people could find good content, and sometimes even love

Coming back to the jumping in the water business: there is no way you can get something as complex and dynamic as Kottu right first time around. And I jumped right in, rewrote the whole code base from scratch. Every software best practice that I learned went flying out the window, and I cobbled up the least complicated thing that worked. No OOP, no comments, no nothing. Getting a working system out (no matter how ugly the code looked) was first priority. And only after going public with that did everything else flow along: documentation (still partial), code refactoring (check out the Posts class, which is basically what the whole back-end of Kottu looks like now) and more features.

When Kottu 7.8 beta first launched, we had no thumbnails, no categories and a bunch of those buggy Javascript social widgets that made loading the home page take ages. Only user feedback and time fixed those issues, and I don't think it is humanly possible for it to have happened any other way. Good systems don't just happen. Of course, there was UNIX, but let's not try to pretend we're Richie and Thompson, shall we? ;)

Right, then. The back-end of Kottu hit version 8.0 with some nice little refactoring (basically a revamped utils folder.) There are some development goals that I hope to achieve before we hit version 8.0 officially. (Note: These are not long-term goals like Indi has described here, but more of short-term, little, code-based goals)

  • Refactoring the front end

    There is a lot of code that gets duplicated in index.php and search.php and elsewhere in the front end. I hope to unify everything into a PageGen class to make generating pages easier and the code much cleaner.

  • Caching

    Kottu stupidly generates every page dynamically... every.single.time. I plan to put an end to this madness and store cached copies of several frequently accessed pages (or options within Kottu, say for example "Sinhala + Popular Today"). This would make me feel less guilty about running long and costly queries to give users a bit more of the beautiful data we have at Kottu. Don't tell anyone, but there will be graphs! ;) Shh!

  • Better documentation

    If you ask people why many open source projects fail, they would say that the two primary factors are developers losing interest, and lax user documentation. I've been a typical programmer, and been slacking off when I was supposed to be writing the f@#%ing manual (couldn't resist, sorry!) :D Yeah, there is a nice little markdown file on our Github, but we need more, including better on site documentation (not about the code, obviously, but think of a better About Us page).

  • Not-so-active blogs, what is?

    Okay, so there are active bloggers, and there are not-so-active bloggers. Thing about Kottu is we're dealing with limited resources, servers that might melt sooner or later, and other real world problems. We currently have 907 blogs listed on Kottu (wait, I'm sure that number was higher... dafuq?). What does FeedGet.php do? It takes 50 blogs (least recently polled), goes to each of those RSS feeds and adds any new posts to our database. Now, FeedGet.php is cronjob'd to run every 5 (!!!) minutes, the minimum limit allowed by our hosting provider, but still it takes approximately 1 hour 30 minutes to repoll a feed. So, if we polled your feed just before you posted a blog post, then sorry.com, bro. You will have to wait for one and a half hours for your blogpost to appear on Kottu. (We can't poll more that 50 feeds at a time due to the fact that PHP is a freaking crack addict that gobbles up insane amounts of memory without de-allocating any)

    What do we do now? What do we do now that doesn't get me banned from the next blogger meetup? Maybe we should poll not-so-active blogs less frequently, giving priority to active blogs - which are more likely to have new posts. And as soon as a less-than-active blogger makes his comeback, his blog is made active again, and gets polled in the usual way. It's sort of evil, yes, but necessary if we're to increase the response time of Kottu. And it also encourages bloggers to be more active. Whaddaya think?

So, finally, don't let the version numbering and jibber jabber fool you. Kottu is pretty much a work in progress, and Indi and I will continue to tinker and attempt to make it better and faster and cuter. And hopefully, I will realise my dream of a happier, friendlier, active little Blogosphere, like we had back when I first started in February 2009. Those were the days, maaan! :')

P.S. The title of the post is from an awesome awesome article by the guy behind Wordpress. READ IT! It totally changed the way I view software, and contributed to numerous bugs on Kottu, and I highly recommend it for anybody, geek or Greek. :D