Evaluation of Project Requests

The Portfolio Management can provide the most value for choosing the best projects out of a big number of project submissions. Without a managed, transparent process the risk that less valuable projects are going to be executed is high, eating away precious resources for better choices.
Like for other evaluation processes the criteria which are driving value creation and are hence the most important for an organization must be defined, agreed and made public.
The most common approach is to have a criteria evaluation table in place where each criterion can get a defined score that is weighted according its importance for the organizational value creation. Such criteria could be

  • ROI/NPV, one of the most common financial criteria
  • Adherence to strategy so all projects are going into the right direction
  • Innovation presumably the biggest motivation for executing projects
  • Secure the business, before an organization can think about new horizons the current state must be secured and run stable

The portfolio manager can usually not evaluate all criteria and so he or she needs to include in the process single points of contact (SPOC) from various teams. This is not always easy as such SPOC can come from organizations that have requested a project themselves. It is only natural that in these cases the corresponding SPOC has potentially a conflict.
Guidance and governance of the process is provided by the portfolio management ensuring that everybody understands what is expected and what she or he has to do. Important is that the SPOC get support from subject matter experts (SME) concerning the project request. Ideally these are coming from the project requesting teams themselves.
The goal of the exercise is to have finally a ranked list of projects, the most valuable are on top and the ones providing the least value at the end. The costs are estimated for each of the requests and summing them all up from top to the bottom will show the total cost of the portfolio. Usually there is not enough funding for all projects and so it is necessary to introduce a cut line to determine which ones get the scarce resources and which ones not (for the moment).

The table below shows a simplified example of such a scoring table as a real example would not fit on this page.

ppm scoring table

Important is to that the scoring follows the model and not vice versa. I observed many times that people start to adjust for example the weighting of the scores to see at the end the list they always had in mind. This should not be the process of scoring. Once the model is defined, presented and agreed it should be used as such. This is the reason I don't show weightings in the presentations as this leads to discussions concerning the model and not the project requests.

The advantage of involving SPOCs and SME using a predefined criteria table is that the process is transparent and repeatable and reproducing the same results under the same circumstances. This is a core asset of the portfolio management.

Managing Uncertainties

One factor that has an strong impact on the value side of the portfolio is uncertainty, which means the unknown risks or opportunities that cannot be evaluated in the identification phase until the project is planned in detail and executed (and even then we deal with this kind of uncertainty which it mitigated through contingency reserves in the project). This means that in the Identification process group only estimations of magnitude can be made assigned with a big portion of uncertainity.

Still the portfolio managers need to have a tool to estimate if a project is likely to deliver -50% or +100% of its value.

This sounds very complicated but isn't a hard thing to do. Basically uncertainty is defined by only two components.

  1. Where to go? This parameter defines how aligned and agreed is the scope of the project request. Is it absolutely clear what has to be done and what the success criteria are? Or is the project a kind of bue sky where everything should be possible? Or maybe there is no good alignment between the stakeholders which will lead to endless discussions.
  2. How do we go? This question tackles the technology where it is a different thing if you are using something you have already worked with or something brand new where you don't have the experience. In addition new technologies are often not bullet proof and stable. The lack of experience with the base-technology and issues concerning the stability are risk factors.

By using the scope and technology as parameters a two dimensional diagram can be drawn to estimate the allover uncertainity as shown below.

CRD v2

(Please note the sectors and risk percentages are examples and can deviate in different organizations)

The levels of uncertainty are specific for different businesses so the portfolio manager needs to analyze and establish a "gutt feeling" for her- or himself by using statistical methods. With enough experience the portfolio manager can already establish a first draft of the diagram and the risk levels to start. Important is that the model is known and aligned with the key stakeholders so it won't be challenged afterwards.

By plotting all the projects in the portfolio into the risk estimation diagram the portfolio management can also give a good overview of the risk distribution in the portfolio.