The Importance of Metrics
Revolutionary movements require revolutionary progress. However, at the start of a Movement, such progress may not be immediately evident to those whose views of progress have been tainted by commercial software, where progress is measured by feature enhancements, quality improvements and user satisfaction. These are false idols and the shallow view of progress they support are irrelevant for true free software.
Rejecting the repressive capitalist view of progress-as-production and production-as-consumption, and the doctrinaire emphasis on results-oriented metrics, we instead adopt the dialectic of progress-as-being and being-as-becoming, with metrics illustrating not what is produced, but what is willed. Rather than galley slaves, prodded by whip lashes to “row harder!”, our motto shall be: “row louder!”
As Mark Twain famously wrote, “There are three kinds of lies: lies, damned lies, and statistics.” Our aim in developing metrics is to avoid the taint of this condemnation by avoiding the statistical sciences entirely.
Two metrics that any free software marketeer must be able to produce are:
- A measure of the size of the community, in order to demonstrate the overwhelming strength and vigorous growth of our Movement.
- A measure of the diversity of the community, in order to demonstrate that it is a true People’s Movement with a broad base of support.
Community Size
A metric that can never disappoint is “cumulative number of developers” on the project. Calculating this metric is simple: count the total number of individuals who have contributed to the project since it started. You are not concerned with the number of contributions, the quality of contributions, whether the person did one patch and then was never seen again, or any other extraneous detail. You’re just counting names. Any project can use this metric to show month-after-month steady improvements.
The genius of this metric is that it can never decrease. It is mathematically impossible! Once a name is added to your list, it is never removed. One a contributor, always a contributor, at least on paper. Worst case, even if an asteroid should hit your next project conference and kill every one of your developers, you could still report that your numbers were flat. Unfortunately, your developers would be as well.
Advanced marketeers should also note the following additional techniques, which can be used to generate even more impressive numbers without additional effort:
- Don’t just count those who are writing code. Almost anyone can be called a “developer” these days. Translators (“Localization Engineers”), build lab guys (“Configuration Management Engineers”), testers (Software Quality Engineers), etc. Include all of their contributions.
- Don’t limit yourself to those who actively contributed code to your project. You can also include developers who contributed to precursor versions of your code, or who contributed to 3rd party modules. They might not even know that their code is in your product, but that doesn’t matter. We appreciate their “contribution” nonetheless and should acknowledge them as an integral part of our thriving community, in spirit at least.
- One you have made the novel step of counting non-project members as developers, there is nothing in principle against extending this even further to the developers of the tool set and libraries you are using. Credit them all, living, dead and legendary. Larry Wall, Dennis Ritchie, Alan Turing, and St. Hubertus, patron saint of mathematicians, can all be listed as developers on your project.
- With some advance planning your project’s development team can help increase these numbers even more. Have them think of some simple text manipulation tasks that can be done by volunteers with little knowledge or understanding of the code. For example, have them do things like reformat code, spell check comments, alphabetize include directives, ensure that every C/C++ file ends with a newline character, etc. Although this will not improve the product for your users, it will enable a much larger group of contributors to get involved so we can add their name to our list. And once on the list, it stays on the list forever. I am told on reliable authority that such code changes are perfectly safe and cannot have a negative impact on product quality.
But be warned: use of these advanced techniques might open you up to criticism of promoting numbers that are meaningless, that they saying nothing about the actual size or composition of your development community. Such criticism can easily be dismissed by feigning to be insulted and responding along the lines, “How could you possibly suggest that we not recognize the hard-working {translators | build lab guys | testers | dead mathematicians | patron saints} who {contributed to|had stuff we took without their knowledge for} our project?”. This argument will almost win, since no one will want to appear ungenerous, especially when intercessory saints are involved.
Community Diversity
Another metric that you might be asked for is a metric to illustrate how much of the project is dependent on any one company, such as the project’s main sponsor. A common error is to look at the contributions made to the project and report how much was apportioned to which individuals and companies, based on recognized software metrics, like # of commits, lines of code, function points, etc. You must not fall into that trap!
A much better metric, if you want to be seen as diverse, is to count the number of developers in each company. Do this without regard to whether the person made a one line change 2 years ago and was never heard of again, or whether the person works 14 hours a day on the project every day. Just look at the number of developers. If you have already followed the advice outlined above, and increased your “cumulative number of developers” metric, then you are perfectly positioned to also demonstrate your diversity.
For example, suppose you have 400 developers, and 10 of them do 90% of the work, and they are employed by a single company. Avoid the naive mistake of saying that one company was responsible for 90% of the contributions. Instead say that the company’s developers (note we’re emphasizing people not work) represent 10/400 or only 4% of the project. That is the genius of this metric. You can have almost all the work done by a single company and still claim that your community is diverse.
No related posts.