Karma — where did mine go?

A couple of days ago I posted that Launchpad had been assigning too much karma for certain types of translation work.

Basically, if you imported translation files — PO and POT files — into Launchpad from an Ubuntu package or a Bazaar branch then you got vastly more karma than if you simply translated directly through Launchpad’s web interface or uploaded PO files as ‘user uploads’.

Of course, that was unfair so we decided to adjust the inflated karma that Launchpad had awarded. The way Launchpad calculates karma is pretty complex, so the manual reduction we made to karma awarded for those translation imports has also affected karma given for other types of work — such as bugs, code, or blueprints — even for people who never did anything in Translations.

So, what’s going on?

Launchpad treats all its applications as equal in terms of the karma they can give out. Let’s call that the “total karma pool per application”. This means that if the total sum of all karma assigned in Blueprints to all users is X, the total karma assigned in Bugs or Translations will also be X.

However, X is determined as the largest of the actual sums of karma values all applications have. So, if Translations is handing out the most karma in Launchpad, all karma values for other applications will be scaled up to match.

There’s more in our explanation of karma calculation on the help pages.

So, what does this mean? After we fixed bug #286359 to send import notifications to Ubuntu packagers, we also started giving out translation import karma for them. An unproportionally large amount of karma was given out compared to other stuff in Translations, and that caused the total pool of Translations karma to grow, so X started growing for all applications. People noticed that their Translations karma was getting bigger and bigger, and reported bugs such as bug #337645. We also got a few comments on IRC. However, nobody noticed — or at least, they didn’t complain :) — that their other karma was growing along with the growing total karma pool for Translations. Suddenly (or not that suddenly, the scaling factor increased over time), people had about 2–3 times more karma then they were supposed to.

Finally, 6 weeks after we introduced the problem, we’ve normalized translations karma so uploaders do not have an unfair advantage over actual translators. That meant that the total pool of Translations karma went down, and with it, X went down and everybody’s karma across the system went down, regardless of the application. Note that everybody’s non-Translations karma got proportionally scaled down, so relative karma scores are still the same (i.e. if you had more karma than your friend Terry a week ago, you’ve still got more karma than Terry today, unless Terry has subsequently gone on a Launchpad work rampage).

What can we do to lessen the consequences of any karma normalization in the future? We’ve got to be careful when introducing or modifying existing actions. We’ve got to normalize karma points before we work on a feature which actively gives them out. And we’ve got to better communicate changes like this.

So, what actually happened is: karma that was never meant to exist disappeared. Many parallels to the economic situation are easy to envision. Launchpad was The Karma Bank. :)

5 Responses to “Karma — where did mine go?”

  1. Ploum Says:

    “This means that if the total sum of all karma assigned in Blueprints to all users is X, the total karma assigned in Bugs or Translations will also be X. ”

    I’ve never understood the rationale behind this implementention choice. It means that, from a karma perspective, it’s better to work on specification than on bugs and translations just because a lot less people are working on specification. Just doing small changes in specifications might give you a lot more Karma than days of bug hunting or translation.

    It also looks like, as a consequence, work on a bazaar branche is not really karma rewarding : a lot of people are using them so karma is low for that.

  2. oliver Says:

    Thanks, that explains why my karma suddenly went up like hell last weeks without much work on my side – it’s back to normal levels now :-)

  3. Данило Шеган Says:

    @Ploum: Blueprints is the biggest culprit now, I’d say. I’d agree that what we have now is far from the perfect system, and we’d be happy to improve it further. Please file a bug where we can discuss and figure out the best way forward. It might be something simple like dividing the total value by the amount of distinct people using the application (i.e. having any karma in it): if all applications have roughly the same proportion of active/inactive users, we should get much better results.

    On the down side, any change will cause “jumping” karma (up or down, depending on the change), and more questions from people.

  4. darkhole Says:

    Jajaja, funny… Many thanks for the work!!!
    My karma is low again, but equality is better.

  5. cprofitt Says:

    I am happy with whatever system gets used… I just enjoyed the roller coaster ride we had…

Leave a Reply