launchpadblog

A couple of days ago I posted that Launchpad had been assigning too much karma for certain types of translation work.

Basically, if you imported translation files — PO and POT files — into Launchpad from an Ubuntu package or a Bazaar branch then you got vastly more karma than if you simply translated directly through Launchpad’s web interface or uploaded PO files as ‘user uploads’.

Of course, that was unfair so we decided to adjust the inflated karma that Launchpad had awarded. The way Launchpad calculates karma is pretty complex, so the manual reduction we made to karma awarded for those translation imports has also affected karma given for other types of work — such as bugs, code, or blueprints — even for people who never did anything in Translations.

So, what’s going on?

Launchpad treats all its applications as equal in terms of the karma they can give out. Let’s call that the “total karma pool per application”. This means that if the total sum of all karma assigned in Blueprints to all users is X, the total karma assigned in Bugs or Translations will also be X.

However, X is determined as the largest of the actual sums of karma values all applications have. So, if Translations is handing out the most karma in Launchpad, all karma values for other applications will be scaled up to match.

There’s more in our explanation of karma calculation on the help pages.

So, what does this mean? After we fixed bug #286359 to send import notifications to Ubuntu packagers, we also started giving out translation import karma for them. An unproportionally large amount of karma was given out compared to other stuff in Translations, and that caused the total pool of Translations karma to grow, so X started growing for all applications. People noticed that their Translations karma was getting bigger and bigger, and reported bugs such as bug #337645. We also got a few comments on IRC. However, nobody noticed — or at least, they didn’t complain 🙂 — that their other karma was growing along with the growing total karma pool for Translations. Suddenly (or not that suddenly, the scaling factor increased over time), people had about 2–3 times more karma then they were supposed to.

Finally, 6 weeks after we introduced the problem, we’ve normalized translations karma so uploaders do not have an unfair advantage over actual translators. That meant that the total pool of Translations karma went down, and with it, X went down and everybody’s karma across the system went down, regardless of the application. Note that everybody’s non-Translations karma got proportionally scaled down, so relative karma scores are still the same (i.e. if you had more karma than your friend Terry a week ago, you’ve still got more karma than Terry today, unless Terry has subsequently gone on a Launchpad work rampage).

What can we do to lessen the consequences of any karma normalization in the future? We’ve got to be careful when introducing or modifying existing actions. We’ve got to normalize karma points before we work on a feature which actively gives them out. And we’ve got to better communicate changes like this.

So, what actually happened is: karma that was never meant to exist disappeared. Many parallels to the economic situation are easy to envision. Launchpad was The Karma Bank. 🙂

This entry was posted by Данило Шеган on Thursday, April 16th, 2009 at 9:06 am and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.