Just like with best practices and professional certifications, most of us have a love/hate relationship with key performance indicators (KPI). It's certainly compelling to be able to measure how good we are at delivering projects. But the non-repeatability and variability of projects makes it elusive at best. Nonetheless I decided to give it a try in order to better structure the software development audit methodology I'm currently working on.

First let's have a look at the characteristics that every KPI should have. The SMART criteria really help:

• Specific (= objective)
• Measurable (= easy to measure)
• Attainable (= appropriate range)
• Relevant (= directly linked to business goals)
• Time-bound

For example, "the number of projects over budget vs. total number of projects, during the last month" would make a decent KPI.

Second, we categorize the KPI. I've found that the balanced scorecard approach is very useful. Instead of applying it to an entire organization, we can apply it to the department/team delivering software solutions, or even to a single project. The balanced scorecard is composed of four perspectives. In our case, we assume that the overarching goal of the organization is to achieve sustainable profitability, therefore we place the finance perspective at the top. In order to achieve this goal, we need to satisfy - better, to delight - our customers (whether they are internal or external to the organization), which is represented by the customer perspective. Assuming that the quality of results tends to reflect the quality of processes and the quality of people, we understand that the internal processes and learning and growth perspectives make that possible.

Third, we identify KPI for each perspective:

Financial

• ROI, short-term
• ROI, long-term
• Cash flow

Customer

• Value: value is more to the point than scope. Scope refers to compliance to a project contract, whereas value refers to how well the delivered projects does what it's suppose to do in terms of business value. Plenty of projects deliver the agreed-upon scope but fail to deliver the value they were supposed to.
• Quality
• Timeliness
• Relationship and communications

Internal processes

• Quality management
• Cost, schedule, and scope management: although being on-cost, on-budget, and on-schedule does not garantee project success (see also the value KPI), I believe that these should be managed properly. Note that this KPI is about management and not compliance.
• Productivity
• Collaboration and communications

Learning & growth

• Team satisfaction
• Process improvement
• Technical excellence

That's the easy part. Now we should identify indicators that are directly linked to each KPI. Indeed these KPI are high-level. We need indicators that are closer to reality and easier to measure. If we look at the ROI KPI for example, we might define underlying indicators that measure how well we calculate ROI with standard methods, at the right time, and compared to other projects in same or other organizations. The ROI KPI would be composed of these underlying indicators.

It's a work in progress. More on this in later posts.