After updating his iPhone to iOS 4 and upon coming back from a trip abroad, my friend found his device unable to connect to any website or mail server. Calls and SMS worked, but not data, except when connected to a wifi network. Stranger still: putting the SIM card in another phone, made it all work again (indicating that the SIM and all subscription services were fine). No need to mention: Softbank support was thoroughly useless in helping with that, merely suggesting it was a hardware problem and the phone had to be changed.

It wasn’t, of course.

Apple’s iOS 4 update shipped with a “bug” of sorts (possibly intentional) that, under certain conditions, will remove your current, valid, “APN settings profile“. Without this profile, your phone has no way to access the data plan provided by your operator. You need to restore valid Softbank APN settings.

There are many ways to fix that on the iPhone, one is to use the brand new iPhone Configuration Utility, an even faster one if you have access to a Wifi network, is to connect to the following address from your iPhone (using Wifi, since 3G doesn’t work): http://www.unlockit.co.nz/

Click on Continue > Custom APN.

Then, depending on the actual colour of the SIM card Softbank put into you phone (silver, for older phones, black for recent ones), fill up the form using these info:

For black SIMs:
APN: smile.world
Username: dna1trop
Password: so2t3k3m2a

For silver SIMs:
APN: open.softbank.ne.jp
Username: opensoftbank
Password: ebMNuX1FIHg9d3DA (unconfirmed)

Then click on Create Profile, click Install on the screen that comes up. You should be done!

Happy browsing.

PS: the reason the iphone OS update screws up this setting in the first place seems linked to past attempts at jailbreaking your phone… Apparently, even if you subsequently had your phone properly reset to non-jailbroken factory settings, the mere fact that you once had it jailbroken is enough to trigger that bug during the update. The old adage about malice vs. stupidity applies here, but it is hard not to think that this could be somehow deliberate on Apple’s part.

Post originally published on: Dave's Blog (please leave your comments over there)

Softbank iPhone connection issue with iOS 4

Ninety minutes of the funniest, wittiest and most insightful rant you will ever hear on Parrots the Universe and Everything:

Post originally published on: Dave's Blog (please leave your comments over there)

Douglas Adams’ Talk at UCSB

1) A fairly decent interface. A newsfeed that was actually designed to intelligently filter stuff of interest to you while hiding the the rest automagically (instead of requiring you to constantly click through endless moronic application notifications, courtesy of your bored-friends-at-work).

2) Everybody was/is on Facebook. Even those kids you used to share your milk with, back in first grade… Facebook is the ultimate “where are they now” tool… If somebody born within your lifetime is not on Facebook, chances are they are either dead or building pipe bombs in a secluded cabin somewhere deep into the woods. All you need is a full name and/or school attendance year.

3) Advanced privacy features meant that people used their real names (a necessity to make point #2 worth anything), while allegedly keeping private stuff away from your boss/exes/crazy Google stalkers etc.

Here we are now, a couple years later and point #1 has died a long and painful death at the hands of a dozen asinine “interface redesigns” plagiarising any other Web 2.0 service with an ounce of popularity, all the while bringing server cost down (yes: turned out, all those great intelligent filtering tools were so intelligent they did not scale at all… oops).

Point #2 is more valid than ever: it is only a matter of time before even dead people have their Facebook page (never mind: they already do). But let’s be honest: once you’ve looked up all your friends from kindergarten and realised you did not share much beside reminisced fondness for crayon drawing and shared hatred of afternoon nap time, once you’ve made sure the asshole bully from Junior High is now assistant manager at Taco Bell and once you’ve found out that secret High School crush Susie now has three kids, two dogs and a suburban house, and is (according to her status) feeling bloated after that huge KFC meal they just all had at the mall… Once you have satisfied that bit of morbid curiosity about every single living soul you have ever interacted with during your life… You just want to go back to hanging out with people you actually chose to be friends with, preferably at an age where your common interests involved more than making watercolour handprints and trying not to pee your pants in public.

Do I even need to go over point #3?
By repeatedly telling its users to deal with the loss of privacy and shut up already, Facebook is only making official a situation that existed since the explosion of so-called Facebook “Applications”, a couple years ago. It is clear Facebook never seriously thought that the common low-lives that created these “applications” (and took over your newsfeed with invitations to become a vampire, check out friends’ cabbage patches or find out which cooking utensil best represented you) would ever respect the laughably unenforceable Terms of Service and politely stay away from your private data (and that of your friends, and your friends’ friends…), even though they were given full access. Rest assured that if you ever installed one of these waste-of-software on your Facebook profile (or, in most cases, if just one of your friends did), all your Facebook data has long been sold and re-sold to a thousand shady Russian “entrepreneurs” and sits on hundred of servers around the globe.

People like this guy (whose fastidious, nigh unreadable, yet mysteriously popular, prose nearly makes me feel good about the level of verbosity of this present rant) are either incredibly naive or very dimwitted, if they woke up today and realised that Facebook was not the best thing that could happen to privacy: “So we have this company who is in the business of gathering people’s private data without billing anything for the huge server cost… Gee, I wonder what they ever could sell to make money”…

And without these three points, what are you left with? A vaguely creepy mega-corporation that is trying to build their own subpar Internet ersatz, like so many before them (AOL, Microsoft, MySpace yadda yadda yadda, please take a ticket and a seat over there) and likely: many more after them.

Facebook wants to become the Internet for lazy people and computer-illiterate schoolkids, because they know that’s all it takes for the rest to follow: what are you gonna do once everybody you know uses Facebook to communicate? Tell them they are totally being played by the Man, delete your account, retire from the world and go back to reading books at home on your own? I didn’t think so either. By the way, when did it become normal to use Facebook messages to contact people whose email address you have known for years? Is it really more convenient to use a shitty webpage interface to send a message that results in an actual notification e-mail being sent to my inbox which I need to click in order to be taken to Facebook’s homepage where I finally get to see your message? Where is the added value or simplicity, beside the extra page-count for Facebook? Is your e-mail client so crappy that Facebook’s messaging interface is preferable?

It would be illusory to think you can “boycott” Facebook, even assuming such a boycott would really reach any sort of momentum: the kind of people who understand, let alone care about, such issues have all long spoken out… with the harsh consequences we’ve all seen on Facebook’s dwindling userbase of 400 million accounts…

To be honest: I don’t care enough… I do not like what Facebook is trying to do to the internet, but I have better things to fight for (and I still have some hope they will shoot themselves in the foot, like so many before).

But my calculated apathy has its limits: located somewhere around the point where every single detail of my curriculum vitae becomes marketing fodder at the hands of a shifty corporation headed by a morally bankrupt fratboyish entrepreneur…

Which is why as of today, my Facebook profile only contains the bare minimum in relevant and/or accurate information, and nothing that I would not feel comfortable publishing on any other public internet forum.

It is mostly a (useless) principled stand: the web never forgets anything and I have long lost my online anonymity anyway… But it only took me a couple minutes fiddling with my Facebook account settings, and I have always dreamt of people actually calling me Baron von Umlaut, so why the hell not.

Post originally published on: Dave's Blog (please leave your comments over there)

Facebook and Me

Seriously, who still buys these things?

My guess is: people who also just purchased a brand new Sony Minidisc player¹ and/or will only use devices that bears the same comforting look as the pocket calculator they had back in High School.

I don’t see why else anybody would willingly spend up to twice the price of an iPod Touch on a tool that will, at best, do roughly what any iPod/iPhone does… minus the thousands of non-Japanese-related features.

Trust me, I am very receptive to the argument of the simple tool that does one thing and does it well, without the clutter and confusion of a myriad peripheral features… But if that’s what it takes, buy an iPod Touch, forget it can be a music player, a web browser or a gaming platform and use it solely as a Japanese study tool: you will still be getting a better deal than with one of these ridiculously overpriced/underfeatured denshi jisho.

In case you are considering such a purchase, or if you already own an iPhone/iPod Touch and wondered what apps you should get in order to turn it into the ultimate Japanese studying tool, here are my three picks:

#1 Kotoba!

Kotoba! is the best mobile Japanese dictionary, bar none. It uses Jim Breen’s public Japanese dictionaries for its data and provides absolutely every single lookup method you could ever desire, combined into a sleek and intuitive UI. If you factor in Apple’s built-in handwritten Kanji input², you have a Japanese dictionary that far outpaces any handheld electronic toy out there.

With all that and despite being a commercial-grade application, Kotoba is absolutely free! Not “free”, not free³: absolutely free, as in freeware, born out of the dedication of its author, Pierre-Philippe di Costanzo. So get it now and go make a donation right after.

#2 KanjiBox

Yes, I am ever so slightly biased on that one…

But there is a reason I set out making KanjiBox for Facebook many years ago: none of the other kanji-studying application came even close to what I wanted in such a product (smart drilling, stats etc.). This is still mostly true with iPhone apps.

To me, a Japanese study app should offer, at the minimum:

1. Intelligent questions: not just throwing a couple random kanji and asking you to pick one (“Gee, I wonder which between 大 and 嘛 could mean big“)…
2. Smart drilling: what’s the point of using a software if it cannot store my answers and use the data to adjust to my level at all time.
3. Progress feedback: I don’t know about you, but knowing where I stand in my studies is a huge motivation factor. Sure, it’s a stupid bar chart, not a perfect representation of my actual everyday fluency, but it still helps setting studying goals for myself (and knowing whether I am somewhat ready for JLPT).

These, and a couple less critical, but still thoroughly enjoyable, features, such as the competitive edge brought by scoring (and comparing with your friends), are what I expect from language-studying software in the 21st century. And you would be surprised how few (if any) of the apps out there do that.

Which is why I have no qualms recommending my own KanjiBox to all serious Japanese students.

For those who can’t get themselves to purchase KanjiBox for whatever reason, I suppose my next choice would be Kanji, which fails at practically all the points mentioned above (it really is the most basic type of flashcard app), but at least has a nice clean UI and won’t set you back too much with its one-buck price tag (probably still too much for what it is). As for KanjiBox’s lesser doppelganger, Kanji Flip: I would perhaps recommend it (although I am no fan of its UI and self-quizzing method), but at $6, it is ridiculously overpriced (that’s twice KB’s price for half the features).

I am sure there are a few free applications that do most of what the ones above do (after all, Kotoba is by far the best dictionary app, and it’s free). But frankly I haven’t seen them yet. And you do know what they say about getting what you pay for.

#3 Japanese Podcasts

Alright, I lied, it’s not an app.

But you now have a pocket-size device with a powerful dictionary and a versatile study guide: it would still be silly not to take advantage of the fact that it can play sound and music (huh, some would even say it is its primary purpose).

I have personally never found Japanese dictionary with audio cues and other attempts at replacing a basic classroom teacher, to be very useful, for two reasons: 1) you will still need to spend time with a real native Japanese speaker anyway (especially around the beginning). 2) Japanese pronunciation is really straightforward: if after a couple hours’ worth of studying you cannot pronounce every single word in a dictionary, you probably need to take on another hobby.

On the other hand, nothing (and certainly not rote learning of entire lists of kanji and vocabulary) will ever replace listening and conversation practice.

Better than limited phrasebook apps, I think using the iPod for what it was meant to do —download and play music/podcasts — is a nicer way to get your regular dose of Japanese practice. As for which podcasts: there just isn’t a way to recommend something that fits all levels and interests.

For beginners, Japanese 101 has a nice array of podcast material (not so convinced by their apps, on the other hand), if you are at a slightly more advanced level, I would suggest directly checking out real everyday-life Japanese podcasts, in a topic that you like. The important part being to find materials that are engaging and updated regularly…

If you really don’t know where to start (and have a sufficiently advanced levels), check out NHK’s podcasts: they have a variety of feeds covering many different topics from daily news to pop music (they also have Japanese lessons, but I don’t know what they are worth).

Whatever you do, stay the hell away from those “Hi, I am 14-year old Mike from Arkansas, and I will be teaching you Japanese: KONNECHEVA!” podcasts… No offense, but I have a couple Astrud Gilberto records at home: you don’t see me trying to teach you Portuguese over the Internet…

頑張って！

And that’s it. For $201.99, you have the ultimate Japanese learning and reference device. And it even plays music for that price!

Why only three recommendations when there are literally hundreds (if not thousands) of Japanese-related apps in the iTunes App Store? Because as mentioned above, there is a beauty to simplicity. Redundant tools rarely improve your experience, they just scatter your attention (and waste your money). That being said, feel free to explore more apps (and send me your recommendations, if you think they deserve being added here⁴).

Post originally published on: Dave's Blog (please leave your comments over there)

3 Apps to Turn your iPod into a Japanese Study Tool…

1. “fit up to twenty tracks in your pocket!”
2. ok, not quite Kanji: Chinese Hanzi… but for most purposes, it will really work just the same. Just make sure you set it with Traditional Chinese, not simplified
3. “Free… for a limited time. Purchase the full version for only $19.99″
4. Need I mention that “WRITE Your Name in JAPANEESE” and other “60 KANA YOU MUST KNOW” apps, really do not need to be brought to my attention.

Last week, upon glancing at my message logs by chance, I realised it decided to send half a dozen empty messages to the first contact in my address book. It then topped that series with an audio SMS: 30 seconds of muffled sounds from whatever crowded bar I must have been in, that night.

All that while using a throwaway number on a prepaid German SIM card (thus unknown from most of the people in my address book).

Meanwhile in Europe, my friend Abigail is probably ever so slightly worried by that mysterious German caller who sent her all these creepy empty messages.

Post originally published on: Dave's Blog (please leave your comments over there)

My Cellphone is a Creepy Stalker…

When it comes to services and subscriptions (cellphone, ISP, banks, heroin dealer…), I am a company’s wet dream customer: one that never leaves for a competitor. Not that I develop any particularly fuzzy feeling for whatever nameless corporation happened to have a branch on the right street-corner on the right day, but when it comes to going through endless paperwork again, moving my account data, updating everything: I just. can’t. be. arsed.

Which is why I have been a faithful customer of AU for over 5 years: not because they are great (Docomo is cheaper, Softbank has better phones…) but because I will always endure a sizable share of customer abuse and groundless fees, rather than having to track all my friends and acquaintances to send them my new contact info (and when you think of it: these things have a price too, so I am not doing it all out of pure apathy).

Why won’t I be a customer of theirs for another 5 years, then? Well, read on and learn how a company loses a customer without even noticing it.

This morning, I landed in KIX after a three months research stay abroad, coming home to a mailbox stuffed with a dozen increasingly angry bill payment reminders from KDDI.

Now, as you can imagine, I did not just leave the country in December, hoping my keitai bills would just take care of themselves during my absence: back then, I dutifully went to the AU shop and sat through yet another endless paperwork session to set up automatic bill payment through my bank account. I also informed them I would be out of the country until March (no way to put my account on hold, of course) and made sure there were ample funds on my bank account.

Just around the time I did that and probably while I was busy getting on my way to the airport, AU decided to send me my monthly bill for December. Which arrived a few days after I had left for the Winter, and sat in my mailbox until my return.

To be honest, I had suspected this would happen. Even envisioned the possibility that, by some unfathomably dysfunctional bureaucracy, it would take a while (and perhaps a few sent reminders) for a human at KDDI to check my account and realise that they had all the info needed. Meanwhile, the prompt automatic payment of all further bills would show even the most stubborn software program that I wasn’t on the lam.

This, of course, is not what happened.

When I showed up at the local KDDI AU shop, holding a dozen reminders for my unpaid December bill and two mail receipts confirming that the money for January and February had been successfully charged to my account, I was informed by a very apologetic – yet unmoved – employee, that the line had been closed and the phone number released. No amount of pointing out the ridicule of the situation and expressing polite indignation at the fact that my account was still being charged for two extra months of a line they terminated (with fees, of course) did get me anything more than a few look of deep embarrassment.

And just like that, I was on my way to the Softbank store.

Not complaining here: while perfectly content with my two year-old keitai, I can certainly find a use for a brand new iPhone. You would just think that AU, as a company, would kinda try to keep their customers (its stagnating line of phones sure won’t).

Post originally published on: Dave's Blog (please leave your comments over there)

au KDDI: Bureaucracy FAIL

… at 4 terabytes…

… per user.

Post originally published on: Dave's Blog (please leave your comments over there)

You know you are in a Bioinformatics lab when…

If your talk lasts 25 minutes and goes into the minutiae of protein-protein interactions with regard to oncogenic pathways, maybe spending half of it convincing an audience of biologists and bioinformaticians that cancer is a bad thing that needs curing is not the best use of presentation time.

Post originally published on: Dave's Blog (please leave your comments over there)

Bioinformatics Conference Pro Tip

KanjiStory.com is a website geared towards people studying Japanese kanji (and, I guess, to a lesser extent, Chinese… but it probably needs some tuning for that). It provides a simple yet powerful interface for people to write kanji mnemonics in the form of a simple story.

The best way to see what I mean is to go register (10 seconds, one click), read a few stories and finally: take a stab at writing your own. Allowing users to contribute stories is at least 90% of the point of this website at this stage, so please do not just go, check out the dozen sample stories and call it a day without trying the editor.

The current version of the website, although quite spiffy code-wise, needs a huge amount of work to be called a proper beta. And then, there are two million cool features just waiting to be added. However, given my very limited time resources, I figured I would first check to see how much interest (and active participation) in the project I can raise, before committing any more time working on it.

Do not hesitate to post your comments and suggestions below, but keep in mind that this is all very early-stage development and that many new features will come, once (if) this ever takes off the ground as a community project.

As an aside, if you are a reasonably experienced PHP dev with an interest in contributing to this project: get in touch (use the address: “zedrdave” at Google’s mail).

If you want to be kept informed of future KanjiStory-related news, easiest way for now is to sign-up as a fan on the facebook page (until I set up a proper forum and RSS feed on the website).

Post originally published on: Dave's Blog (please leave your comments over there)

Kanji Stories…

Aside from breeding deadly mutant monkeys to serve in my army of evil minions when I kickstart the world-domination part of my plot, that is.

Before I go any further, let me remind the casual reader that: 1) it is most likely nice and sunny out there where you live and you would be considerably better off looking at squirrels running through the trees 2) if you have even the slightest inkling of formal mathematical/computer science training, you will be better served foregoing this edulcorated version in favour of one of the 10 million tutorials and entries on bioinformatics available throughout the internets (Wikipedia being a good place to start). The entry written henceforth is geared at some hypothetical grandparents who would care to know what the fuss with modern Science is all about (for instance mine, were they not already perfectly content in the sole knowledge that the good Lord has put all these tiny amino-acids together in the best possible way of all worlds and that modern genetics are the work of the Devil¹).

In last month’s episode, we laboriously learnt that Biology abounds with really, really, tough problems. Two major points were:

1. For all practical matters, NP-Complete problems are all in the same bag: finding a way to solve one efficiently would mean you can solve any other in roughly the same order of time.

2. Once you have proved that a problem is NP-Complete, trying to find an exact solution for a real-life set of data, is about as meaningful as trying to take down the Everest with a toothpick. There are however plenty of ways to find an approximate solution. Proving NP-Completeness is your cue to start looking for approximation algorithms; and thus the fun begins.

Today, instead of going straight onto the myriad fun ways in which mathematicians solve biology problems, and which one of those I am actually connected to, another digression and an illustration everyone has heard of: genome sequencing.

Full genome sequencing (mapping the entire DNA of a given organism) is one of the earliest application of modern bioinformatics techniques, a seminal example: it starts off as a rather straightforward bio-chemistry problem, soon runs into pesky matters of size, complexity and intractability, goes through a difficult phase of alcohol and substance abuse, but is ultimately saved by the power of Love and Mathematics.

Before I go into the gory details, allow me to dissipate a common misconception about DNA sequencing: it is nowhere as easy as you might have been led to believe by your TV (most people’s preferred source of Science™ facts). Hearing of “DNA tests”, “DNA crime database” and other everyday life DNA-related techniques might make it sound like sequencing is as easy as sending your saliva swab to the lab and waiting a couple days for the results. In reality, despite serious advances, actual full genome sequencing is still a multi-year, multi-million-dollar affair. When people talk about DNA in a forensics or medical context, they are usually looking at a single base nucleotide, located at a precise location on one gene, out of the entire genome. Even cases that require a larger sample of such observations (e.g. DNA matching, when it actually uses sequencing altogether) are still somewhere in the lower hundreds (if that). That’s a mere 100 bases to look at, against 100+ million for the first organism fully sequenced, 10 years ago (make that 3 billions for humans). Quite a difference in scale. And, of course, this is one of those problems where solving twice the size requires much more than twice the time (hopefully by now, this does not surprise you, otherwise you might want to go back and read episode 1 again).

OK, let’s start:

The Easy Part

Long before full-genome sequencing arose, there have been many efficient methods to map small fragments of DNA. How small depends a lot on how efficient, how fast and how reliable you want the result, but 100 bases is a good figure to go with (could be 100 or 1000: it would not matter much when compared to aforementioned billions bases of the human genome). The methods used to do small-scale sequencing are quite nifty, but since they merely rely on boring² bio-chemistry stuff, I will let you google around (Wikipedia’s entry on the topic is not the greatest introduction, for once). For our sake, let’s just say we have biological tools that can accurately map segments of DNA of 100 bases (or smaller) in a fairly short time with near-perfect accuracy.

Other magical tools that we have at our disposal in our bio-chemistry toolbox allow us to somewhat painlessly:

1. replicate a sequence (of any length) an arbitrary number of times.

2. cut sequences in smaller bits at arbitrary positions: usually upon encountering a specific pattern in the sequence and with a certain frequency rate (e.g.: upon encountering the pattern ‘CCCGGG’ anywhere in the sequence, the cut will occur 1 out of 6 times).

3. work on a subsequence at a specific position (as if it was cut from the rest): provided you have an exact pattern where you want to do it (and the pattern is long enough to be unique in the sequence), you can isolate a subsequence from the rest with absolute precision and accuracy. However, engineering such an artificial cut is infinitely more complex, long and costly as using some of the “ready-made” cutters of 2.

The same with more science and less words, you ask? Sure, here you go: DNA primers, DNA polymerase, PCR, restriction enzymes, kangaroos³.

For the sake of keeping things simple (ha!) we will blatantly ignore the fact that DNA is not a single strand, but two complementary sequences tied together that need to be periodically pried apart or rebuilt during these operations. In the end, it does not matter much to the problem at hand.

Equipped with such tools, how should we go about processing entire chromosomes (usually north of 10k bases for simple organisms, 50 million for humans)?

The Straightforward Method (aka The Doomed Approach)

“Primer walking” is a (comparatively) simple method that more or less consists of focussing on segments small-enough to fit our small-scale sequencing method: start at a given position (see tool 3. above), map a few hundred bases, use the chunk just mapped to find the next starting position, lather, rinse, repeat.

There are two major problems with this method: As mentioned, “starting at a specific position” on a sequence is a very ad hoc and therefore costly (in money, in time…) operation. That operation furthermore requires knowing where to start, which means having processed the previous segment (so we can use the end of this segment as the pattern to start at in the following step).

The entire process therefore requires sequentially executing a series of operations, each of which requires a certain time. Surprisingly no NP-Completeness or exponentials involved here: only a very, very long task with no way to parallelize, which is only theoretically better. In practice, this makes this method viable (albeit slow and costly) for sequences of up to 10,000 or so bases.

On longer sequences, and particularly on our friendly multi-billion genomes, that would be a Fail.

The Smart Way (aka The One That Gives Headaches)

From the above, it seems obvious the only way to stand a chance at decoding entire genomes in less than a lifetime is to tackle many small segments, possibly all of them, at the same time. And this is exactly what shotgun sequencing does: after making a good number of copies of the original sequence, we sic a bunch of ready-made sequence cutters on them. The resulting mixture is a horrible mess of small fragments, that are small enough to be each sequenced easily. Which is great, except you then have no idea how to put them back together so as to get the bigger picture.

Not clear? Need another silly analogy? Sure:

Imagine you have a (very very long) text printed on a piece of paper: make a dozen photocopies and leave them in a room with your 5-year-old nephew and a good pair of scissors (arranging the pages in such a way that each is cut separately, in its own unique way). Put all the resulting confetti in a bag, shake a few times.

Now put the original text back together. Go ahead, I’ll wait.

Funny enough, this is near-exactly what DNA sequencing entails: once the first phase of sequencing is done, you are given a large bag of short strings (long sequences of ‘A’, ‘G’, ‘T’ and ‘C’ instead of common words in our analogy, but the same for every other purpose). You are looking for the text that is made of these strings in a specific order (and repeated over, as many times as there were initial copies). An “easier” problem is to find any text on which you could fit each little fragments (with lots of overlap, since there are many copies, each cut a different way) without worrying too much about whether this text really is your original. In practice, to get the real solution, all you have to do is find the shortest such text⁴: the one where you have no “superfluous” bits of text (not covered by any fragment) and as much overlap as possible (ideally in such a way that each bit of text is covered exactly as many times as there were original copies).

This problem is known as the shortest common superstring. And guess what: remember that little list I mentioned last week? Yea, bummer.

Given a candidate, merely verifying whether you have found the true Shortest Common Superstring is an NP-Complete problem. Finding that string from scratch, belongs to an even harder class of problems known as “NP-Hard” (where, as the name lets on, real badass NP problem kids hang out). Given the size of your data (millions of tiny fragments), finding an exact solution is obviously not gonna happen in this lifetime. Neatly enough, solving this problem turns out to be exactly the same as solving a particular brand of Traveling Salesman Problem (remember?). But as I mentioned last week: where standard algorithm fail and exact solution cannot be found, it is usually still possible to find a decent approximation in a fraction of the time. Sometimes this solution even turns out to be the exact solution, or so close that it doesn’t matter. This is the case here.

And excited as I am sure you were all, to jump into the actual mathematics of solving this particular problem, a sense of human decency and the already astronomical word count on this entry conjointly force me to skip to the end and suggest that the morbidly obsessed bravest among you peruse their local Google to look for some fascinating readings on reducing Shortest Common Superstring to Asymmetric Traveling Salesman Problem and/or finding approximation algorithms for the Shortest Common Superstring problem.

One of the trick behind Shotgun Sequencing, beside the actual approximation algorithm used to rebuild the original sequence, is the clever use of statistical redundancy to reach a guaranteed level of accuracy. That’s how we are able to say with certainty that the accuracy for the current Human Genome project is “no more than 1 error every 100,000 bases on average” (which sounds an awful lot better than “fuck if we know, but we dang sure hope it’s enough to tell it apart from them monkey DNA”).

The thankfully brief conclusion

And there we are, I have exposed, in nearly less time than it takes to kill a grizzly with a feather⁵, how Mathematics and Algorithmic can save the day in Genetics and help ensure that we will all have a bright future made of beautiful blond, three-eyed babies.

Of course, given that Shotgun Sequencing in its earliest incarnation, dates back about 40 years, there is very little left to be done for young upcoming bioinformaticians in this particular direction. And none of my work has anything to do with common sequencing problems.

But next week month soon, I will maybe talk of more recent challenges in bioinformatics, perhaps even some of mine!⁶

Post originally published on: Dave's Blog (please leave your comments over there)

What is Dave doing in Bioinformatics? Pt. 2

1. I know: one is not supposed to capitalise the name of God’s evil nemesis, but I am going on the assumption that Satan is a vindictive bastard and one can never be too prudent in courting the good graces of major players of the afterworld.
2. just kidding, fellow biologist friends: keep looking after those test tubes and churning out my data. kthxbye.
3. Have fun with your friends: can you spot the famous Australian marsupial among these terms related to DNA engineering?
4. the only reason this is not 100% true is the pesky problem of highly repetitive sequences. Something unlikely to arise in a real book, but very common in some DNA.
5. Pro tip: go for that spot under their foot, grizzlies are extremely ticklish there.
6. … in part 205 to 300 of this entrancing 1400-part series!

The short answer is that I do research in Bioinformatics, which is where Mathematics (along with Computer Science and a dozen other disciplines) meet with Biology and Genetics in a dark back-alley, and do all sorts of indescribable things to each other in the hope of: creating a better world, curing cancer, breeding the next race of eugenic übermenschen or making a few bucks for Big Pharma… whichever comes first.

But that sort of answer, while technically correct, does not really tell you why such an unnatural coupling of disciplines was warranted in the first place. Allow me to start at the beginning. Way at the beginning.

[open long semi-relevant digression that can be advantageously replaced by a thorough read on Complexity Theory, if you feel up for the more sciencey and truthy version of things]

The scientific problems of this world tend to fall in either of two categories: those you might eventually solve with a good computer and some time… and those you will never solve exactly, no matter how much crazy sci-fi supercomputing power you throw at them.

This “solvable” vs. “not solvable” demarcation might sound like a tautology, until you understand the full meaning of “never” in the above statement: these, are not problems that might be solved one day, when science progresses far enough or computers get ten, a hundred or a million times faster. These are problems whose solutions require calculation of a complexity that is proven to be beyond the reach of any conventional means of computation in any foreseeable future (“unconventional means” would begin with the discovery of heretofore unknown laws of Physics: in other words, unlikely in your lifetime. at best).

By and large, the mathematical complexity of a problem, is the order of time (or computing power) it will take to solve it, relative to its size.

Without calculating the result of a certain task, it is often possible to predict whether producing this result could or could not be done in a reasonable amount of time (where “reasonable” usually means “in less than the age of the universe, assuming the use of every single computer on earth”, or somesuch).

There are countless examples of tasks falling in the first category, “easy” tasks that can be solved quickly, regardless of how big they are. For example, anybody past kindergarten age can presumably add two numbers of practically any size with a piece of paper and a pen. You just add each digit one by one (and, yes, carry the one) and adding two 100-digit numbers will take barely more time than adding two 3-digit numbers.

Now consider a different task: say you are a traveling salesman who needs to plan their next sales route. You have a map of the region, with the towns you must visit and all the distances between them, given in kilometers. How do you find the absolute shortest route that will take you to each city at least once without wasting gas or time?

More to the point: how difficult do you think finding that route will be?

Sure, it sounds easy enough: pick a starting point, follow every roads that go from that city to another one, then onto the next etc. Keep the shortest distance you’ve found. Can’t be that tough, right?

Let’s say there are five cities: you pick a city to start from, then check all remaining four, and from each four, go onto one of the remaining three etc. etc. In total, that’s 5x4x3x2x1 = 120 different paths to compare (that product can also be written using the factorial function: n! = n x (n-1) x … x 3 x 2 x 1. e.g. 5! = 5x4x3x2x1). Not so bad.

What if there are a few more cities… for instance, two times more: 10 cities. That’s 10! = 10x9x8x7x6x5x4x3x2x1 = 3,628,800 paths to look at. Huh, that might take a bit longer to do by hand. No worries: somebody will write a computer program that gives you the answer in a couple seconds.

Except that, you guessed it, each time I double the number of cities, the difficulty does way more than just double.

For 20 cities, the number of paths to look at is: 20! = 2,432,902,008,176,640,000.

For 70, cities, there are 70! (that’s factorial of 70: 70x69x68x…x3x2x1) possible paths to check one by one. That number has exactly 100 digits. This is (very) roughly the number of particles in the entire universe. Assuming you were to put every single computer in the world to work on this, you likely would not be done by the time the Sun explodes.

Of course, you might suspect there are better ways to do this than looking at every possible path one by one. There are. But they are neither easy, nor significantly better. In the end, no matter what method you come up with, it has been proven that it will always require in the order of 2ⁿ operations (where n is the total number of cities). Although it is smaller than n!, 2ⁿ is still a ludicrously huge number, for a large-enough number of cities (“n”). In practice, this puts the Traveling Salesman Problem in the “unsolvable” category: by putting enough cities on a map, one can always ensure that no amount of raw computing power and mathematical genius will ever solve it in a reasonable amount of time.

Another common example is the Knapsack Problem: given a knapsack that can only be filled up to a certain maximal weight, and given a set of items, each with a specific weight and value, how difficult is it to pick the combination of items that will have the highest value, while remaining under the total weight limit. To the despair of jewelry thieves the world over, this problem is equally impossible to solve easily: as soon as the number of items to choose from gets big enough, the number of possible arrangements to look at shoots through the roof (and it is very hard to carry your own supercomputer while robbing a jewelry store).

These two problems (and many others) belong to a class of problems known as NP-Complete. The only two things you might ever want to know about NP-Completeness and NP-Complete problems (beside the fact that unicorns and kittens are considerably more fun to look at) are that:

• NP-Complete problems are not just problems that “seem” tough until somebody figures a better way to do them. They are tough. Mofo tough. And provably so. No matter how fast computers keep getting (assuming they keep improving at that rate), they will always fall short of solving this type of problem, for big enough data.
• All NP-Complete problems are interconnected. In fact, this is how you usually prove that a problem is NP-Complete: by proving that solving it is as difficult as solving another NP-Complete problem. If by a mathematically implausible stroke of genius you found an easy way to solve one NP-Complete problem, you would have found a way to solve them all.

Most importantly: NP-Complete problems are everywhere.

If the plight of traveling salesman and jewelry robbers throughout the world does not seem to justify wasting your brain cells, keep in mind that their family-friendly formulation is a mere varnish on top of some very ugly raw math definitions, that make them the standard building blocks of countless science and everyday-life problem modeling.

[end digression]

Which finally brings us to Bioinformatics. By way of Biology: Biology is full of hard problems. And by hard, I do not mean “take a long time, need a big microscope and a big computer”, I mean problems that somehow tie to NP-Complete problems. You could say that a lot of things in the real world, end up toward NP-Complete problems, but there are two particularities that make biology/genetics problems more likely to:

1. Biological things love exponential growth. Pretty much every Living Thing in Nature grows exponentially. This tends to greatly affect problems related to the science of Living Things, otherwise known as Biology.

2. In order to be useful, biological problems have to be solved on humongous data sets: whether it is sequences of billions of amino-acids, graphs with thousands of vertices and edges or experimental results involving thousands of genes at once…

If you read the digression above and throw in these two points, you might start seeing why biologists have started running into lots of walls over the past 20-30 years… The universal approach to solving Life Science problems (buying bigger computers or waiting a tad longer for them to pop out the results) no longer works. Identifying protein similarities in a DNA sequence database is not just a needle-and-haystack problem, it is is a needle with a near-infinitely large haystack problem.

Of course, I did omit a major detail in my coarse overview of NP-Completeness above: provided you are fine with an approximate solution, most NP-Problems can be solved very nicely. However, the mathematical tools to solve problems approximately are often very different from the more intuitive (but impractical) techniques that lead to exact solutions. At the sight of the barbaric math involved, most traditional wet lab biologists will usually recoil in horror and tell you to call them when the algorithm that does it is done developing and implementing… Quite understandably, biologists are most happier handling their test tubes, cutting open innocent mice and manning their cool expensive equipment to produce ton after ton of raw data: they didn’t sign up for the boring math and computer stuff that is increasingly necessary to interpret or use these data.

And this, ladies and gentlemen, is how a guy with a background in Math, Computer Science and Artificial Intelligence, who has not touched a microscope since high school, ends up working on protein sequences and drug interaction prediction.

Next week(*), I might even tell you how…

(*) margin of error: +/- 3 weeks.

Post originally published on: Dave's Blog (please leave your comments over there)

What is Dave doing in Bioinformatics? Pt. 1

As it turns out, however, they were not talking about an all-expenses paid tour of Asia and America’s best nightlife spots.

For mild entertainment and posterity value, a few frackload of random tidbits gleaned over the past 10 days and 25,000 miles (counting):

Boston is a nice city. Somewhat nicer than I imagined (was perhaps one of the only major US city I had never been in). At least in the middle of July, when the sun is warm and rain had apparently stopped pouring, just in time for my arrival there. But weather concerns apart, it feels like one of a rare breed of US cities, where you can live (fine) without a car. Which automatically puts it toward the top of my book. It also has lots of nice tree-lined avenues with cute little houses, and plenty of coffeeshops with semi-witty names and lovely US-style breakfasts (baaaacon…) that nearly make up for the filtered sock juice they call coffee…

Coincidentally, and with no bearing on the above statement of appreciation: Everybody in Boston is a 20-something upper-middle-class white person who only wears pastel polo shirts. Really: everybody. Even Asian people there are white. And they wear pastel polo shirts. On their way to one of the 259 Ivy League universities within walking distance of Fenway park.

I am told there are black people living in Boston too.

Upon confirmation, my sources insisted that they really meant African-American, not Indian (Indians are also white in Boston, but some of them do wear white short-sleeves instead of pastel polo shirts).

In between a lot of half-mumbled droning and powerpoint pr0n, centered around the life and death of proteins, worms, projectile vomit-inducing bacteria and many others, I did learn a couple cool things.

Did you guys know that the differences in some pathways of circadian cycles, from one individual to the next, have been scientifically linked to sleep dephasing issues? In Time Magazine-talk: Science has proved that there is such a thing as “morning-people” and “evening-people”. I know, it’s neat, isn’t it.

In fact, all that revolves around circadian cycles, and the incredible chemistry and mathematics involved in creating multiple near-perfect body clocks, each running on their own period, is quite awesome and fascinating.

That being said, scheduling a talk on circadian cycles in the middle of a hot Summer afternoon, following half a day of presentations and one heavy american lunch, is as close an attempt at humour as one may get in such places.

If you have no idea what that paragraph above was about, even after reading this: congratulation on most likely going through High School without getting beaten up every week after PE.

Guess who the computer at Kansai International Airport selected as the lucky winner of a voucher for a once-in-a-lifetime, super-extra-fun, “additional security controls and full body check before boarding” prize.

Guess who, out of a rough 50 speakers presenting their work along the week, had the honour of giving the very last presentation of the last day of the conference, hot on the tail of a mere 8 hours of talks.

Hint: those were both the same person.

I am still not quite sure what miraculous circumstances were involved in yours truly not being booed off the stage of Boston University’s School of Biomedical Engineering Sciences. But the favoured hypothesis revolves around liberal use of the mesmerizing “blow up” transition effect in his Keynote presentation.

My one and only regret about the patent-pending “chocolate pretzel” finger food made available in the conference rec room: not bringing back any, so I can present them to the next person who dares criticising British food. Say all you want about the blandness of lamb in mint sauce: no sane Britton has ever willfully tried to make food by dipping a bread-like substance in salt, then chocolate.

I am still open to the idea that the whole thing was an industrial mistake (“Damn, we completely forgot to use unsalted pretzels for the chocolate batch”) unloaded at competitive prices to cost-conscious conference caterers.

Around the top of the List of People Your Do Not Want As Your Neighbour on a 12-Hour Flight: 1) Morbidly obese North-American man with lax body hygiene standards. 2) Bratty 12-year-old kid on serious Ritalin withdrawal with no conception of personal space boundaries. 3) Japanese salaryman drowning his existential fear of death (aka looming retirement to a life at home with his wife) in mini-bottles of cheap red wine and lukewarm beer, drank as fast as JAL hostesses will bring them, before taking on the task of becoming your best friend by asking you the same 5 questions a dozen times each.

Great thing about being on the 6-flights-in-7-days program, is that it is perfectly possible to enjoy all of the above, and still have room for a couple boringly normal flights, merely spent trying to find the best way to fit your knees under your chin, while seriously suspecting your seat’s back of being less than 90º in so-called “upright” position.

Northwest sucks. Tracey (and Co) rules!

Good to know: when they screw up your domestic NRT-KIX connection and leave you stranded in a closing airport at 9pm, Northwest Airlines will give you the option of riding Shinkansen (the following day) at their expense. As for accommodation: you better have awesome friends living in Tokyo, or hope that the weather is warm above the sidewalks of Shinagawa station.

Also good to know: JR counter guys will not, unless specifically asked, give you a receipt for your overpriced 16,000 yen Nozomi + Airport shuttle ticket. You will usually realise that somewhere halfway between Tokyo and Osaka, shortly before you also realise that the last ticket gate eats all your tickets, leaving you with absolutely no proof whatsoever that you ever rode shinkansen on that day, and a very useless Northwest refund claim form.

To my lovely Bostonian who described Boston Summer as an object of fear, on account of unbearable heat and stickiness: do not visit Kyoto, Tokyo or Singapore during the Summer. Ever.

Oddly enough, the extra couple degrees of Singapore weather aren’t so tough, when experienced from a luxury condo overlooking the residence’s 24h olympic swimming pool and door-to-door taxi service to the convention center downtown. I always knew it would pay off one day, sharing a couple school years with the future masters of the finance world (the smart ones: those who still have a job today).

Running one of the largest worldwide conference in your domain and requiring from your speakers that they run their slides exclusively through “the presentation rooms’ laptop [...] equipped with Windows XP SP3 and Microsoft PowerPoint”… makes me both extremely irritated and smugly satisfied that I have moved on to another domain ever since.

For no good reason I can think of, I am absolutely, utterly, in love with Singlish… “And if these not fit, you bring them back tomorrow lah. Can can lah!“

Post originally published on: Dave's Blog (please leave your comments over there)

On being a traveling salesman of science…

KanjiBox for iPhone

By far the best way to spend your Summer while improving your Japanese (whether at the beach or on a crowded Tokyo subway, wedged between two sweaty salarymen). This application works on all iPhone and iPod Touch models (provided they run iPhone OS 3.0 or later) and is entirely offline (doesn’t use any internet connection at all).

More info and screenshots on KanjiBox’s website…

And of course, for our ipod-deprived brethren, there is always KanjiBox for Facebook: free and full of fun multiplayer goodness!

Post originally published on: Dave's Blog (please leave your comments over there)

Study Kanji on your iPhone

Insulin I – by Rattus Norvegicus

Post originally published on: Dave's Blog (please leave your comments over there)

Abstract Art or Science?

Dear fellows at the HUGO Gene Nomenclature Guidelines Committee,

It has recently come to my attention that you have decided to do away with names deemed “inappropriate or offensive” found in the existing international gene nomenclature.

I couldn’t agree more: I always thought that the officially registered name for gene kill-all-the-Jews-and-drink-their-blood was a bit politicised for a scientific setting.

Whatever you do, however, keep your hands off gene SHH, otherwise officially known as sonic hedgehog homolog (Drosophila) gene. We like it the way it is (blue, spiky and running very fast). Beside, what better way to break the news of some potentially fatal gene mutation disease to a kid, than by introducing a beloved computer game mascot!

PS: and for chrissake, drop the Comic Sans font: it makes your world-class gene database website look like it was coded by a 1st year CS student in 1991 (yes, I know: it probably was).

Love,

Post originally published on: Dave's Blog (please leave your comments over there)

Save the fastest running gene in the galaxy!

Since about age 5, I have come to grasp with the notion that staring at an oven timer doesn’t make it cook a cake faster.

And yet, I still seem to think that staring at the live output of my painstakingly slow linear optimization program is gonna make it spit its final value faster.

Post originally published on: Dave's Blog (please leave your comments over there)

Can’t help staring…

Can’t wait to see how fast Unreal Tournament runs on a cluster of CRAYs.

Now we’ll see who gets that top score on the SETI@Home project.

“This will greatly help me compute substrate cleavage point predictions for this new set of data in reasonable time, thanks.”

Post originally published on: Dave's Blog (please leave your comments over there)

New Toy

In the last installment of our Boring Geek Updates™ series for the year and as part of my general strategy of complete WordPress disengagement, I finally decided to do something with that long-neglected pet project of mine: WP-plugins.net

Namely: sell it.

As you can tell from the ‘beta’ sign and the outdated reference to an ongoing update process, to be completed ‘any time now’, this site has long shifted out of my personal field of interests. Don’t really have the motivation, definitely don’t have the time.

But, maybe you do!

If so, now is the time to make me an offer.

In a nutshell:

1. For whatever price you pay, you get: full control of wp-plugins.net (I’ll have it transfered to your registrar), all the code that sits on it and currently powers both frontend and backend, plus the entire plugin database.
2. What you won’t get: hosting (you gotta have your own server), full ownership of the code (some of it is already GPL’ed, so I can’t do anything about it, but you can still use it however you like) and I will most likely remove all plugin authors’ email information from the database, unless you can make a compelling case for me not to.
3. I would like it if: you have plans for the site that involve a certain continuity with the existing service. Whether you keep the entire thing as is and just maintain it or entirely overhaul it with new and exciting features, I’d just like to be sure this domain doesn’t become a cybersquat in a month or two. Basically, if you can prove some loose level of involvement with the WP development community (really: anything will do… You are not interviewing for a job here), I’ll be much more likely to take your offer into consideration.
4. I have frankly no idea how much this domain/site should go for. Which is why I’m asking you to send me an offer, by email (update: domain sold, please do not contact me about it), with whatever amount you think fair (along with a two-line plan of what you intend to do with it, see above). For reference, just note that I have routinely gotten (and declined) offers for advertising deals running about $1000/year… Which pretty much sets a starting price (but then again, make a convincing case if you can’t afford that much and maybe the spirit of christmas will overcome me and I’ll let it go for less than that for a good cause).

Update 12/19:

• Credits: whatever you do with the domain (as said previously: hopefully something for the common good), I would appreciate credit (name + link anywhere on the site will do). Then again, if you have a strong case not to, just take it to me and we’ll see: this is a “would like”, not a “must”… Either way, please let me know.
• Deadline: I’ll accept email offers until Monday 8pm GMT. At which time I might contact a few of you for additional information or clarifications on your plan, then let everybody know shortly afterward.

Anyway, all interested parties please send your short and honest offers to the following address: wpplugins4sale@gmail.com.

Thank you and happy eggnog-drinking binge!

Post originally published on: Dave's Blog (please leave your comments over there)

WP-plugins.net domain name for sale!

]]> http://unknowngenius.com/blog/archives/2008/12/19/wp-pluginsnet-domain-name-for-sale/feed/ 1 http://unknowngenius.com/blog/archives/2008/10/19/yet-another-facebook-application/ http://unknowngenius.com/blog/archives/2008/10/19/yet-another-facebook-application/#comments Sat, 18 Oct 2008 17:54:12 +0000 Dave http://unknowngenius.com/blog/?p=1733 Post originally published on: Dave's Blog (please leave your comments over there)

Yet Another Facebook Application
]]> If you belong to the great Facebook family and don’t know what to do with your weekend, why don’t you go spend it playing with my latest production: a kickass personalized trip-recommendation application.

Post originally published on: Dave's Blog (please leave your comments over there)

Yet Another Facebook Application

]]> http://unknowngenius.com/blog/archives/2008/10/19/yet-another-facebook-application/feed/ 1 http://unknowngenius.com/blog/archives/2008/09/07/its-4am-do-you-know-where-your-children-are/ http://unknowngenius.com/blog/archives/2008/09/07/its-4am-do-you-know-where-your-children-are/#comments Sat, 06 Sep 2008 19:25:07 +0000 Dave http://unknowngenius.com/blog/?p=1719 Post originally published on: Dave's Blog (please leave your comments over there)

It’s 4am, do you know where your children are?
]]> It’s Saturday night, just past 4am.

Right now, I could be:

• In my bed, sleeping.
• In my bed, not sleeping.
• Somewhere in the world’s largest city, getting drunk, dancing to loud music, watching girls in Japanese schoolgirl outfits wrestle in jell-o. Possibly all of the above.
• In the middle of the desert, doing drugs and discovering my inner totem animal.

Instead: I am in the middle of an empty office lab overlooking deserted downtown-Tokyo, staring at a monitor, running Support Vector Machine simulations.

Something, somewhere, went very, very, horribly wrong with my life.

Post originally published on: Dave's Blog (please leave your comments over there)

It’s 4am, do you know where your children are?

]]> http://unknowngenius.com/blog/archives/2008/09/07/its-4am-do-you-know-where-your-children-are/feed/ 4 http://unknowngenius.com/blog/archives/2008/09/03/lets-proceed-scientifically/ http://unknowngenius.com/blog/archives/2008/09/03/lets-proceed-scientifically/#comments Tue, 02 Sep 2008 16:42:40 +0000 Dave http://unknowngenius.com/blog/?p=1716 Post originally published on: Dave's Blog (please leave your comments over there)

Let’s proceed scientifically…
]]> Pros and cons of loading a 120-ft scroll of tracing paper into my typewriter and going on a frenzied 48h non-stop benzedrine-fueled Masters thesis writing session:

Pros:

1. It worked for Jack Kerouac.
2. I’d be done 3 days earlier: more time to edit and correct typos.
3. I’d be done 3 days earlier: more time to spend drunk in some seedy Golden Gai bar.

Cons:

1. Actually, it took him three weeks to write it. I don’t have three weeks.
2. Sleep deprivation-induced typos probably likely to include: entire paragraphs written in Urdu, random obscene expletives, obscure references to isomorphic transformations in alternate planes of reality…
3. Comparability of Beat literature masterpiece with Natural Language Processing project involving Machine Learning parsing of Rhetorical Structure Trees: dubious.

It’s a toss, really…

Post originally published on: Dave's Blog (please leave your comments over there)

Let’s proceed scientifically…

]]> http://unknowngenius.com/blog/archives/2008/09/03/lets-proceed-scientifically/feed/ 4 http://unknowngenius.com/blog/archives/2008/07/14/spam-karma-is-gpl/ http://unknowngenius.com/blog/archives/2008/07/14/spam-karma-is-gpl/#comments Sun, 13 Jul 2008 18:29:04 +0000 Dave http://unknowngenius.com/blog/?p=1697 Post originally published on: Dave's Blog (please leave your comments over there)

Spam Karma goes GPL
]]> Geek news warning: sane people and anybody for whom such acronyms as PHP or GPL merely evocate some brand new drugs the kids might be into these days: you are probably better off skipping this one.

I’ll try to keep it short.

Spam Karma 2 is now released as GPL v.2. This essentially means you can do anything you want to it, except claim you made it (copyright and attribution notice must remain there). You should also note that any attempt at deriving some ill-deserved profit from it through harebrained web marketing schemes will earn you both my long-standing scorn and a nut-shriveling decrease to your actual karma.

I suppose another angle to that post’s title could be:
Officially discontinuing Spam Karma’s development: so long and thanks for all the fish
as this is what this truly is about.

But, such a title would be slightly misleading (and no doubt heavily quoted out of context): Indeed, I am hereby officially announcing that I will no longer support, maintain or further develop Spam Karma (beside some very occasional, very limited poking, until the transition to a self-maintained project is completed). However, thanks to the magic of free software, all the unsung heroes of the Open Source world will soon rise to take over and bring you a stronger, better, more closely supported version of Spam Karma!

Okay, what’s more likely to happen is that nobody will really bother taking over, except perhaps a handful well-intentioned but utterly clueless beginner coders who will quickly find themselves overwhelmed by the task and next be seen running away screaming at the top of their lungs. Hey, I’m not blaming anybody: I wouldn’t waste my time on a non-paying, open-source community project either…

But on the off-chance that you would (and trust me it won’t do anything to help you get laid either), I have set up a Google Code repository, which could become the jumping point to some magnificent community-based development effort (or not). If you are interested in participating in any way, contact me (mail or contact form) with a *brief* description of who you are, what you can do and what you wanna do. I don’t need a resume (I am not hiring), just a very quick idea of what level of responsibility you’d be willing to take on the project. I’ll put in the first couple people that seem to know what they are doing (and do not sound like they’ll be selling everything to Russian mafia-owned spam sites) as administrators of the project, and hopefully from there on, things will work by themselves…

If you think you’d like to tackle any aspect of SK2 development (including possibly porting it to other platforms), here is your chance. Speak now or go back to more fruitful and life-rewarding endeavours forever.

Oh, and as for the “reasons”, well, here they are:


1) Life.
Much as I love the challenge and excitement of coding an anti-spam filter and thinking up new tricks to defeat parasitic life-forms of the web, I just don’t have the time anymore. And to be honest, if I did have the time, I probably would have other challenging, exciting new projects I’d rather tackle. I’m fickle like that.

2) WordPress
I will really try to keep that one short, because I could probably write a novel of that. And it wouldn’t be a very interesting read.
In a word: WordPress kinda sucks nowadays. Its retarded upgrade rate makes it nearly impossible to keep up, in turn making it a constant security threat on my servers. And each time I finally cave in and install one of those “mandatory security upgrade”, it also installs 600 Ko of other theme compatibility-breaking fluffy crap that I never asked for in the first place. Usually setting the ground for the next cycle of security-exploit-rushed-upgrade. To sum up, it’s become incredibly bloated and tedious to support. Replacing it on my own servers is very high on my list of things to do (which means somewhat in the first 1000 items).

Having no interest for WordPress anymore, I have thus very little interest for WordPress-related development.

As for WP coming bundled with its own anti-spam plugin, I could also go on for hours on that. The fact that a community-based open-source project is used to distribute a commercially licensed piece of software doesn’t make me particularly happy. But frankly I haven’t cared and still don’t care enough to even raise a stint. At any rate I know lots of people (me included, obviously), aren’t convinced by the way Akismet works and are happier doing the filtering on our own servers, so there is definitely room for SK2-like plugins out there.

Anyway, thanks everybody for your support all these years and let’s gather a round of applause for our brand new Spam Karma GPL Edition!

Update: in addition to the Google Code-hosted project, there is now a dev mailing list set up on Google Groups, go check it out and feel free to sign up if you are interested in SK2′s future development!

Post originally published on: Dave's Blog (please leave your comments over there)

Spam Karma goes GPL

]]> http://unknowngenius.com/blog/archives/2008/07/14/spam-karma-is-gpl/feed/ 125 http://unknowngenius.com/blog/archives/2008/07/08/nifty-japanese-input-trick/ http://unknowngenius.com/blog/archives/2008/07/08/nifty-japanese-input-trick/#comments Tue, 08 Jul 2008 06:15:07 +0000 Dave http://unknowngenius.com/blog/?p=1695 Post originally published on: Dave's Blog (please leave your comments over there)

Nifty Japanese Input Trick
]]> One little-known feature of the Japanese Input tools on OS X is the ability to easily access a whole lot of unicode symbols without having to go dig through the Character Palette each and every time. If you enable Japanese Input (also known as Kotoeri) on your mac, hitting a keyboard shortcut (apple-space by default, I think) will toggle kana input on and off, whereby you can type japanese words in kanas and press the spacebar to pick a matching kanji (followed by ‘enter’ to validate the transliteration).
The nifty bit comes from the availability of UTF-8 characters that are not kanji, but nonetheless useful in a lot of situations. Just as any other kanji, typing a kana sequence (usually the name of the symbol in Japanese), followed by a press of the spacebar, will automatically let you insert the desired symbol.
Note: Apparently, most of these work equally well on Windows Japanese Input system, but I haven’t tested it.

For example, any Japanese girl knows all too well how to obtain the following cutesy icons:
おんぷ[onpu]　→　♬♩♪♫
ほし[hoshi]　→　☆★

On a more pragmatic note, you can also choose from a very complete set of arrows:
やじるし[yajirushi]　→　↑↓☝⇔ etc.

And one of my personal favourite: european currency symbols that would otherwise take half an hour to find on a standard US keyboard:
ゆーろ[yuro] →　€
ぽんど[pondo] →　£

Another very cool set for your scientific paper-writing needs:
すうがく[suugaku]／えんざん[enzan]　→　√∃∀≠±∇

Not to mention the entire greek alphabet:
あるふぁ[arufa]　α
べーた[be-ta]　β
がんま[ganma]　γ
でるた[deruta]　δ
しぐま[siguma]　σ

etc. etc.

You will find even more of these in this large (albeit probably not exhaustive) list of special character shortcuts.

Post originally published on: Dave's Blog (please leave your comments over there)

Nifty Japanese Input Trick

]]> http://unknowngenius.com/blog/archives/2008/07/08/nifty-japanese-input-trick/feed/ 8 http://unknowngenius.com/blog/archives/2008/05/03/this-is-all-getting-really-boring/ http://unknowngenius.com/blog/archives/2008/05/03/this-is-all-getting-really-boring/#comments Sat, 03 May 2008 16:53:32 +0000 Dave http://unknowngenius.com/blog/?p=1668 Post originally published on: Dave's Blog (please leave your comments over there)

This is all getting really boring…
]]> Don’t you think?

I mean, the alcohol, the drugs, the neverending nights of feral sex, the uninspired blogging… it gets old, really.

Alright, so maybe not the booze, drugs and sex part. But the blogging part: definitely. I don’t mean the part about writing inane crap that nobody in their right mind should care about, in between two intense navel-staring sessions. I don’t think I’ll get tired of that part any time soon (I’m trying though). I mean, the sterile format that this blog has come to follow.

Oh, trust me, I am very aware of it. Sure, I have many excuses as to why my posting rate has dwindled to the levels of Bangladesh’s strawberry production on a bad monsoon year… Work, life, love (or pursuit thereof), happiness (idem) etc. But we all know there’s more to it. Truth be told, blogging here bores me, most of the time. There are a couple reasons for that, chiefly among them are:

• This blog started out on the wrong foot.
  When I decided to open my first self-hosted online space, it started as a bastard mix of for-friends-only (“hey guys, long time no see”) news reports and travel-journal (“lookee all the whacky things they have here”)… Both rather boring genres in the long run, neither something I really felt like doing much. But I have been pulled toward these roots ever since.

• It is read by all the wrong people.
  Quite expectedly (although I originally never intended it to be), this has become the place where all people who either have invested some DNA into me, or were court-ordered to stay at least a continent away, come to get their life update on Dave. Knowing that both your genitors (hi Mum! hi Dad!), extended family, past love interests (and potentiall future ones) are all reading this, puts a serious cap on any attempt at spontaneity.
• And therefore… I write elsewhere.
  Yes, I know, it hurts, but I have been seeing other people. In other locales, other languages even. Usually with completely different style and contents. Don’t even try to search the web: believe me you won’t find it. Those other writings are all that this isn’t: personal, fun, hyperbolic, unauthentic, uncensored etc.

Then why bother?

Good question. I suppose because this still serves a purpose for some writings, in some contexts. Also because I hate giving up. And closing that blog before I turn 50 would feel like giving up.

But things need to change. Not sure what, but they do.

Still working on details. I technically have about 10 days before the official 5 year anniversary of this blog. Do not expect grand announcement or sudden changes, just be warned.

Sorta.

Post originally published on: Dave's Blog (please leave your comments over there)

This is all getting really boring…

]]> http://unknowngenius.com/blog/archives/2008/05/03/this-is-all-getting-really-boring/feed/ 2 http://unknowngenius.com/blog/archives/2008/04/14/roadmap/ http://unknowngenius.com/blog/archives/2008/04/14/roadmap/#comments Mon, 14 Apr 2008 19:22:45 +0000 Dave http://unknowngenius.com/blog/archives/2008/04/14/roadmap/ Post originally published on: Dave's Blog (please leave your comments over there)

Roadmap
]]>
5. Vita
   • Find a home.
   • Get my Gaijin Ausweiß.
   • Resume working out.
6. Academia
   • Finish all unfinished uni projects.
   • Get seriously started on my research.
   • Make up my mind on the whole PhD thing.
   • Pick a subject.
   • Pick a country.
   • Pick an advisor (not necessarily in the above order).
7. Technica
   • Fix WP Plugin DB.
   • Ditch WordPress.
   • Drop Spam Karma (see item above: sorry folks, we had a good ride).
   • KanjiBox port for iPod Touch/iPhone.
8. Et caetera
   • Sushi.
   • Beer.
   • Music.
   • Art.
   • Et alia caetera.

Post originally published on: Dave's Blog (please leave your comments over there)

Roadmap

]]> http://unknowngenius.com/blog/archives/2008/04/14/roadmap/feed/ 5