Portfolio intelligence: analytics and visualisation

Written by Bryan Fenech, Director - PPM Intelligence.

Charles Minard's Visualisation of Napoleon's Retreat from Moscow

More often than not the implementations of portfolio management that I come across do not deliver on their promised benefits. The reason for this is almost always the same: portfolio governance boards and portfolio management offices (PMOs) do not make sufficient use of the plethora of data available to them to support decision making about portfolio selection, monitoring and controlling.

Organisations are particularly weak in the areas of portfolio performance, risk and benefits management. The most common approach followed when portfolio governance boards review their inflight portfolio is to run through a list of projects and programs with health ratings (eg traffic lights – red, amber, green) for a variety of performance indicators (eg overall, budget, schedule, resources, risk, and benefits), and some basic commentary. If you are doing this you are wasting your time.

Firstly, it is ineffective. This approach is incapable of creating learning and facilitating decision making that improves structural and systemic problems and inefficiencies regarding the organisation’s portfolio management capability and capacity. If something is going wrong with one of the projects or programs in the list the best that the portfolio governance board can do is authorise actions to be taken by, or with respect to, the project or program concerned. But, secondly and worse still, this is merely duplicating effort that has already taken place at Steering Committee level; the portfolio governance board is doing little more than delegating back (with interest) what the Sponsor has escalated up to them and is already doing their best to deal with. This is not a good use of anyone’s time and energy.

Portfolio Analysis

An alternative approach is to give your portfolio data to a data analyst with project management knowledge and ask them to turn it into valuable information that tells you something about portfolio performance, risk profile and benefits position as a whole, rather than project by project. Here are a few examples of what you might ask them to provide you:

• What types of risk are the most common across the portfolio? Is there a particular type of resource bottleneck that presents regularly? Is vendor management a recurring problem? Is there evidence that project managers are consistently demonstrating forms of cognitive bias in their estimation, such as being overly optimistic?
• Are there particular “break points” in the portfolio (ie, projects or programs upon which there is a high degree of dependency by other projects and programs)? What is the health of the most significant break points? What is the combined value of 'at risk' investment dollars or benefits dependent on these break points?
• How does the health of tier 1 projects and programs compare with less critical projects and programs? Is there a point of increasing complexity at which portfolio performance and throughput tends to 'fall off a cliff'? Is the organisation allowing sufficient schedule and budget contingency in its tier 1 projects for these realities?
• What is the combined value of investment dollars or benefits by overall traffic light rating? Is the position improving over time?

The information derived can be a snapshot at a point in time or historical.

Portfolio Visualisation

This information is far more valuable than a list of projects and programs with health indicators. It facilitates evidence-based decision making for steering the portfolio and resolving problems relating to capability and capacity, and provides the business case for action to address systemic problems.

This is a more appropriate focus for portfolio governance boards.

However, it is critical that portfolio analytics are presented in a way that decision makers can make immediate sense of them. Data visualisation techniques can help to present an incredibly rich picture of your project portfolio. Bringing together graphical representations of portfolio information in a portfolio info-graphic facilitates immediate practical application by senior decision makers. This approach delivers vastly superior outcomes in terms of the quality of learning and decision making.

The power of this information in the hands of senior management is significant. It can be applied to everything from which risk categories to address first, to where to focus recruitment efforts, to how to tailor project management training needs to get the most value for the organisation.

Roles

This has implications for PMOs. In particular, the role of the portfolio analyst needs to become more analytical and versed in data visualisation techniques to be able to surface key messages and trends out of the detail of portfolio data. In most cases, PMO personnel come from a delivery background, and adopting portfolio analysis and visualisation techniques and approaches requires a significant degree of upskilling.

This in turn highlights the importance of the role of independent portfolio assurance. Assurance, through health checks and other reviews, of your most important projects and programs remains essential. However, a portfolio intelligence expert can give you immediate access to the relevant analytics and visualisation skills needed to bring your portfolio information to life. Such services are a highly cost effective use of your assurance investment, resulting in improved portfolio performance and knowledge transfer.

For the purposes of this article I have created some simple visualisations of risk and benefits data using a basic tool. But there isn’t one size fits all for this and there are very powerful tools available to help you explore your data with greater precision and expressiveness. While there are common techniques, it is worth developing your own rich visual representations of your portfolio data. This information is what is unique to your business and it can be enriched so that it becomes one of your most valuable information assets and a source of competitive advantage.

To kill a project ...

Written by Bryan Fenech, Director - PPM Intelligence.

"Action expresses priorities" - Mahatma Gandhi

When to kill off a project, and the decision criteria for doing so, has been a prominent discussion topic among my colleagues lately. It seems that it doesn't happen nearly as often as one would expect. And, perhaps surprisingly, there is not a lot of science that goes into such decisions.

What the theory says

The theory says that you should kill a project when its business case no longer makes sense. There are 2 reasons why this may be the case. Either:

1. Costs are higher than expected because of delay, poor estimation of effort, error and rework, or resources have become more expensive than planned, or
2. Benefits have been compromised due to delay, overly optimistic estimation of new revenue or cost savings, competitors getting to market first, downturn in market conditions, or disruption of the market due to external conditions.

The availability of other projects that represent a better return on the investment of organisational resources should be a determining factor also. Why persist with a project that is no longer expected to deliver valuable outcomes when there are other more attractive options?

What happens in practice

In practice these principles are a good basis on which to make a case to wind up a failed project. More often than not, however, I find its much simpler than that and if there is a major project failure - usually a significant delay or cost blow out - no one will want to be associated with it.

However, statistical studies such as the CHAOS Report suggest that, while about 25% of projects are failures that are killed off at some point, about 50% continue on through to completion even though they are on average around 200% over budget or late. The latter figure suggests its harder than expected to kill a project, and not always a rational decision.

Human factors

This accords with feedback from some of my colleagues from the MBITM Program at the University of Technology, Sydney who work in the field. Consider the following 2 quotes which I think put the matter succinctly:

"It seems to me that many projects don't refer back to the Business Case often enough. And that human trait of wanting to salvage something (anything!) from the investment in the projects leads to a reluctance to kill it off." 

"I've not seen any project 'killed' in my experience. What I've experienced is that when a project fails to deliver, rather than killing it then and there, more resources (money, labour etc.) are thrown at it to 'make it work' - nobody wants to be seen or associated with a failed project. Or it is re-scoped to such an extent that it really bears no resemblance to what was originally envisaged or sign(ed) off on. Successful projects have many parents, while a failed project is an orphan."

Killing a healthy project

I have heard it sometimes said that the maturity of an organisation's portfolio management capability can be measured by its ability to kill projects that are not troubled.

The reasoning is that an organisation with a high portfolio management maturity level will be so in tune with its resource-supply/project-demand equation that it will be able to respond to new opportunities and make space for them by stopping or deferring less valuable in-flight projects.

But is it really that easy? In my experience, not only do organisations not have the tools to do this with any level of rigor, but there is no agreement on the critical issues involved.

For example, when comparing the value of a new opportunity with an in-flight project should we take into account the sunk cost expended to date in the latter? If we do a straight NPV comparison at the time of decision the in-flight project has already expended some of its costs so its NPV at that point will be hard to beat? Alternatively, should we include this sunk cost in the NPV equation? But if we do that then it is no longer an NPV that we are measuring but a present value of both past and future cashflows.

Another problem is that the estimates for in-flight projects are likely to be more accurate than those for a new idea. Should we discount estimates, or apply a risk factor, to account for the different levels of confidence?

What does best practice say?

There are no hard and fast rules here. Both of the key industry standards - Management of Portfolios (Axelos) and The Standard for Portfolio Management (PMI) - are silent on these questions. This is curious given that killing off projects when appropriate is often cited as one of the key objectives of portfolio management.

This area is ripe for field research to survey and gather data on how organisations are approaching these issues.