Framework: An Integrated Approach to Portfolio, Program and Project
|[AACE HOME] -- [TCM HOME] -- [BUY THE TCM FRAMEWORK] --[DOWNLOAD PDF VERSION]|
[TOC] [PREVIOUS] [NEXT]
II. STRATEGIC ASSET MANAGEMENT PROCESS - CHAPTER 6 - STRATEGIC ASSET PERFORMANCE ASSESSMENT
6.1 Asset Performance Assessment
The asset performance assessment process analyzes cost accounting and performance measurement data to identify asset performance problems, opportunities, and risks for which the requirements for a solution can be assessed. Performance problems with existing assets (which are also improvement opportunities) are generally identified by analyzing variances between planned and actual performance. Through internal and external benchmarking and intelligence gathering, the assessment process also identifies new business opportunities. Identified asset portfolio performance improvement opportunities and risks are assessed and managed through the requirements elicitation and analysis and asset change management processes (Section 3.1 and Chapter 6.2, respectively), which close the strategic asset management cycle loop. Findings and experiences from the performance assessment process are captured in the asset historical database (Section 6.3) for use in future asset management.
The primary inputs to the process are internal asset cost accounting and performance measurement data (see Sections 5.1 and 5.2, respectively). The quantitative measurement data are analyzed for variances against the asset performance requirements (Section 3.1) and the investment decision basis (Section 3.3). Measurements of the attributes and performance of other enterprises assets are another basis of comparison; these are generally obtained through benchmarking methods. Root cause analysis and lessons learned methods identify improvement opportunities and risks (generally from a more qualitative perspective) for immediate assessment and for future requirements assessment when captured in a historical database.
The asset performance measurement and assessment processes are part of a continuous effort to evaluate and improve investment decisions so that the enterprises business objectives, as reflected in requirements, are more successfully achieved (i.e., business problems are solved).
.1 Requirements and the Investment Decision Basis Establish What to Assess
Requirements are conditions or capabilities that must be met or possessed by an asset to solve a business problem. They are the basic unit of communication that links the needs of the customer to the design of the solution. When a realized asset meets requirements, we have a solid basis for evaluating the strategic success of that asset. As discussed in Section 3.1, asset requirements can generally be categorized as either a functional requirement (i.e., what the solution has to do), or a constraint or non-functional requirement (i.e., qualities or attributes that a solution must have).
Some might add business requirements (i.e., enterprise or operations missions, goals, or drivers) as a category, but in general, these can often be categorized as constraints. Another requirement to assess is whether the requirements themselves are solving the business problem as intended. Asset performance is unlikely to be considered successful if either the requirements are not met or requirements are not adequately addressing the business problem or opportunity.
Profitability analysis is the key method for assessing performance against general business requirements. It recognizes that the most important business requirement for most enterprises and the basis for their asset investment decisions, where applicable (see Section 3.3), is a measure of economic return on investment (ROI) or equivalent; ROI is a single measure that expresses in monetary terms the value of the investment over its life cycle to the enterprise.
Cost of quality analysis is the key method for assessing performance against functional requirements and constraints. Cost of quality refers to the cost of error-free, conforming, and not conforming (i.e., at variance) with these requirements. Costs of quality are generally analyzed in the following five categories: error-free, prevention-related, appraisal-related, internal failure, and external failure. Prevention and appraisal costs, such as employee training or product testing, are essentially designed into the asset or process during asset planning (i.e., costs of conformance that are fixed or controllable by design), so performance assessment tends to focus on the "resultant" cost of failure or variance during the assets use (i.e., costs of nonconformance are more variable by nature).
A business strategy map and its companion balanced scorecard is another popular methodology used to first communicate the executive teams strategy to employees, second align the organizations work and priorities with the relevant measures (referred to as key performance indicators or KPIs) that will achieve the strategy, and then assess performance variance between the actual and target for each KPI. Balanced scorecard seeks to ensure that business strategy and deployment look at performance from a "balanced" perspective of non-financial measures (such as on-time delivery performance) measured during a period that ultimately contribute to the financial measures reported at the end of the period (when it is too late to act). Specifically it recommends that metrics from multiple perspectives be considered, such as a financial, customer, business process, and learning and growth perspective. Some organizations add additional perspectives such as environment or safety or prefer to use the seven core value and concept elements of the Malcolm Baldrige National Quality Award.
The important point is to apply metrics in a linked framework that reflects the strategic intent of the executive team. After KPIs are cascaded from executive and middle manager levels into the core processes, then the number of requirements established, and hence to "score," can be quite large. Therefore, enterprises ideally focus on the vital few (rather than the trivial many) key performance indicators that are measures believed to most directly correlate with successful achievement of business objectives within the inter-related perspectives. A most important KPI from a financial or cost perspective is ROI (or its equivalent) as assessed using profitability analysis. Other key indicators may include target costs and so on. However, elements or drivers of cost of quality that focus on the customer and business process perspectives can be used as KPIs.
6.1.2 Process Map for Asset Performance Assessment
As was discussed, the asset performance assessment process analyzes measurement data to identify asset performance problems, opportunities, and risks for which the requirements for a solution can be assessed. As shown in Figure 6.1-1, the process centers on steps that analyze performance variances from a cost perspective and identify performance improvement opportunities and risks.
Figure 6.1-1 Process Map for Strategic
The following sections briefly describe the steps in the strategic performance assessment process.
.1 Plan for Performance Assessment
The process for performance measurement starts with planning for the assessment. Strategic asset management is an integrated process, so initial planning starts with development of the requirements (Section 3.1) and continues through development of the investment decision basis or business case (Section 3.3). As the basis for assessment, the requirements and investment decision basis are the key inputs for the planning step. If only a few KPIs will be measured and assessed, these need to be identified when planning the process. Also, because performance measurements are done to support the assessment methods, planning for measurement (Sections 5.1 and 5.2) and assessment must be done together.
Much of the performance assessment work will be ongoing and may be somewhat automated as part of the enterprise business reporting cycle. This ongoing effort may require little planning. However, profitability analysis of products, service-lines, types of channels, and types of customers (preferably calculated with ABC/M principles), cost of quality, benchmarking, and lessons learned may be routinely reported from a repeatable computer system or optionally undertaken on a special study or cyclical basis. These methods typically rely on analytical software modeling tools and data that must be managed to support the process.
Specific roles and responsibilities for assessment need to be planned. Some of the assessment work (e.g., operations analysis) is the responsibility of ongoing operations and functional management or equivalent and may require little planning. The role of cost engineers is most significant for performing cost-based assessments on a special study basis such as ROI and ABC/M analyses.
Another aspect to plan is the use of information technology for assessment. Enterprise performance management systems expand on enterprise resource planning (ERP) systems by constructing an enterprise information platform (EIP), which is a common database from multiple disparate data sources for all performance measures. This platform, commonly referred to as a data warehouse provides the capability to assess measures against established performance criteria (e.g., balanced scorecards, "KPI dashboards," etc.) as well as extract, transform, and analyze information for decision support. This is the growing field of "business intelligence" (BI).
Planning ensures that the measures captured in and reported by various measurement systems are either consistent with the assessment tools (e.g., ABC/M), or that processes and procedures are in place to translate measures to the assessment basis. In addition, the interaction/interface of various reporting systems within the enterprise or with external suppliers, if any, must be considered and addressed in performance assessment plans.
.2 Analyze Performance Variances
With the assessment basis established and assessment plans in place, performance assessment begins with analyzing quantitative performance variances. The methods used for analyzing asset performance generally mirror the methods used for asset planning and investment decision making.
Investment Return and Profitability Analysis
As discussed in Section 3.3, the most common economic valuation method used to support decision making is net present value (NPV). NPV calculations convert cash flows to a lump sum (i.e., single point) equivalent financial value at the present time by discounting future cash flows assuming a cost of capital rate that recognizes the time value of money. Most analysis problems examine the financial return for an investment for which there are both cash inflows (i.e., revenue or benefits) and outflows (costs). The alternative with the highest positive NPV is usually the one selected. The ROI for an investment alternative is that interest rate for which the NPV is zero.
At the time of assessment, the ROI or other profitability metric is updated with actual revenue and expense cash flows instead of estimates depending on where the asset is in its life cycle. The initial actual expense is generally the cost invested in the project to create the asset. The longer the asset has been in use, the greater the confidence in the investment return and profitability analysis.
Profitability will generally be assessed for each asset investment as well as for all or parts of the enterprise (e.g., by organization, programs, portfolios, etc.). To appropriately determine the ROI on a particular asset or cost object (e.g., facility, product, product line, sales channel, distribution channel, customer, process, etc.), profitability analysis may apply activity-based costing/management (ABC/M) accounting and analysis methods (see Section 5.1) since different work activities are first consumed by different cost objects. These cost objects in turn consume each other, such as customers purchase a basket mix of products as well as consuming non-product-related costs-to-serve.
An enterprises project system is an asset and process for which performance is assessed as well. Requirements for project system performance usually focus on project cost and schedule performance. These measures are usually assessed project by project (see Section 10.4) and are reported to the asset management system for portfolio analysis. However, the ultimate success of a project (i.e., investment) is its profitability and this cannot be assessed with confidence until some time after the project has been completed.
Cost of Quality (COQ) Analysis
As was mentioned, cost of quality highlights the location in a business process and magnitude of the costs of not conforming with requirements. It also sheds light on prevention and appraisal costs, such as testing, to avoid non-conformance costs. Cost of quality analysis is nevertheless a cost separate from simply processing outputs the first time error-free. Figure 6.1-2 illustrates the cost of quality concept with costs varying with the degree of quality attainment. The typical cost of quality categories are prevention, appraisal, and failure (internal and external failures are summed in the figure).
Figure 6.1-2 Cost of Quality Concept
The cost of prevention, which is usually a function of asset planning, will tend to increase as more physical and/or process "perfection" is built into the design. The cost of appraisal tends to decrease as the need for inspection and monitoring is reduced. Finally, the cost of failure tends to decrease as the need for rework decreases and customer satisfaction increases.
Performance assessment tends to focus on the cost of varying from requirements (i.e., cost of nonconformance or "failure"). Failure costs are usually sub-categorized as internal or external. Internal costs tend to relate to process nonconformance and include items such as scrap materials, additional labor (poor productivity), and rework. External costs tend to relate to physical nonconformance and include items such as warranties, returns, and lost sales, as well as difficult to measure costs such as customer dissatisfaction or lost goodwill.
Cost of quality variance analysis consists of comparing planned failure costs (which should be minimal) versus actual costs for the asset in question and relating the variances to the requirements that were not conformed to. The requirement nonconformances that have the most cost effect are then given priority as opportunities for improvement.
The cost of internal failures is easiest to measure and assess (e.g., labor, material, etc.). Much of an organizations COQ is "hidden costs" beyond the obvious and easily trackable costs of an inspection department or person or scrap or reworked material. Various parts of a workers day involve testing or reacting to unplanned mistakes. ABC/M is an ideal method used to "unhide" these costs because it not only assigns and traces (rather than allocates) indirect costs effectively, but also allows all activity costs to be classified by the five COQ categories (and optionally indented deeper). That is, nonconformance work activities often originate from the performance of non-value adding activities in a process that can be studied with ABC/M. (Scoring and tagging each work activity cost by its COQ classification using ABC/M "attributes" is referred to as an additional dimension of costthe "color of money"). Some external costs are also easy to measure (e.g., returns), but others such as customer dissatisfaction must be estimated.
While the cost assessment focus is generally on the failures, the total cost of quality must also be assessed to determine if the planned balance of prevention, appraisal, and failure cost was achieved.
If the assessment finds that the requirements were met, but external failure costs are still high or increasing (e.g., the customer is dissatisfied), then the requirements need to be reassessed in the asset change management process (see Section 6.2) because they may not be addressing the customer needs or wants.
Benchmarking is a measurement and analysis process that compares practices, processes, and relevant measures to those of a selected basis of comparison (i.e., the benchmark) with the goal of improving performance. The comparison basis includes internal or external competitive or best practices, processes, or measures. As Watson states, "the process of benchmarking results in two types of output: benchmarks, or measures of comparative performance, and enablers (i.e., practices that lead to exceptional performance)." Therefore, though benchmarking is one process, Figure 6.1-1 shows benchmarking twice to reflect its use in performance variance analysis ("benchmarks") and identifying improvement opportunities ("enablers).
The benchmarking process has four basic steps that follow the Shewhart or Deming "plan, do, check, and act" PDCA cycle (i.e., the basis of the TCM process as a whole) as follows:
plan the study
collect the data
analyze the data
adapt and implement practices
In the first step, the asset or process to be studied is defined, performance measures are identified, and the external enterprises or internal organizations to include in the study are determined (i.e., what and whom should be benchmarked). In the second step, data are collected from applicable sources (e.g., direct contact, questionnaires, public data, etc.). In the third step, performance measures of the enterprises are compared and the enablers (i.e., improvement opportunities) that facilitated the best performance are identified. The last step, which is included in the strategic asset planning process in TCM, is to adapt the enablers to the enterprises situation and implement the practices as appropriate.
In terms of analyzing performance variances, benchmarkings role is to provide an external basis of comparison to supplement the enterprises internal basis. Its role in identifying improvement opportunities is covered in the following section.
.3 Identify Opportunities and Risks
In many cases, findings from quantitative performance variance analysis can be used to immediately correct asset or process function or operation when the cause of the variation is obvious and readily corrected. However, long term and continuous performance improvement for the asset portfolio generally requires further analysis of opportunities and risks.
As shown in Figure 6.1-1, the step for identifying opportunities and risks is shown both in parallel to and receiving input from the variance analysis step. This is intended to reflect the fact that while the steps serve different purposes and can be performed somewhat independently, they are best done together in an integrated way.
Lessons learned is qualitative information that describes what was learned during the performance of a process, method, or tool. Lessons learned are typically elicited through the use of subjective surveys, narrative descriptions, interviews, or formal lessons learned workshops. Lessons learned are captured in a database to support ongoing development or improvement of processes, methods, and tools. Lessons learned activities are focused on increasing the asset or project management teams understanding of asset or project performance.
Lessons learned are a subset of the evolving field of knowledge management, which seeks to create and share knowledge in an organizational or social context to support making the right decisions. While shown in Figure 6.1-1 as taking place after the asset planning and performance measurement processes, lessons should be captured as they are learned. Once captured, the lessons can be reviewed by the asset or project management team, usually in group settings, and analyzed to identify and document potential asset or process improvement opportunities or risks. Lessons learned assessment and analysis tend to be subjective in nature.
Root Cause Analysis
A more objective method of assessment is root cause analysis. As generally practiced in a quality management process, root cause analysis investigates and identifies the most basic reasons for non-conformance with requirements. Methods such as cause and effect diagrams are used to identify and classify causes so that corrective actions can be identified.
Risk Performance Assessment
Each of the performance assessment methods described above may identify imminent or occurring risk factors, as anticipated in risk management planning (see Section 7.6) or otherwise. The risk management plan includes plans for monitoring anticipated risk factors. Risk assessments are considered in the requirements change management and assessment processes as appropriate.
As discussed in Section 6.1.2, the benchmarking process identifies not only "benchmarks," but also "enablers," which are practices that facilitate best, or at least improved, performance. It is important that organizations follow a common definition of items, resources, or costs when benchmarking to assure comparability and avoid flawed conclusions from "apples-and-oranges" comparisons. Along with market research and general business intelligence gathering, benchmarking also identifies asset concepts, qualities, attributes, or plans that the enterprise may consider in its own requirements assessment and asset planning.
.4 Review, Document, and Communicate Performance Assessments
The asset management function (e.g., strategic or capital planning, etc.) regularly checks that the basis of performance measurement and assessment is appropriate and in alignment with the current requirements assessment, asset management planning, and performance measurement basis. Assessments are also reviewed to ensure timeliness and accuracy. Most performance measurements are not made directly by those responsible for the performance assessment; therefore, those with performance assessment responsibility should spot check the performance measurements to some extent (i.e., audits, inspections, questioning operations management, etc.) to ensure that the data being received and assessed are reliable, appropriate, and understood.
After review, asset improvement opportunities and benchmarks are documented and communicated to those responsible for the requirements elicitation and analysis process (Section 3.1) for consideration in ongoing management of the asset portfolio. Also, any assessment findings that the requirements are not appropriately addressing business problems or opportunities are communicated to those responsible for the asset change management process (Section 6.2).
Finally, performance assessment information is captured in a historical database (Section 6.3). The information is used to help improve assessment tools such as software, models, procedures, and so on. Assessment data, particularly lessons learned, are used for planning future asset investments.
6.1.3 Inputs to Strategic Performance Assessment
.1 Requirements. Quantified requirements define some performance measures to be assessed (Section 3.1) as well as methods to be used for the assessment process.
.2 Requirements Changes. Changes to the baseline requirements and asset plans are identified in the change management process (see Section 6.2). Requirements changes may result in changes to performance assessment.
.3 Basis for Measurement. Planning assumptions and constraints incorporated in the basis of the investment decision (Section 3.3) define some performance measurements to be assessed.
.4 Performance Measurement Plans. The measurement and assessment processes are usually planned together.
.5 Cost Accounting and Asset Performance Measures. These measures are the basis against which quantitative variances will be analyzed.
.6 Stakeholder Input. Lessons learned are captured from all applicable parties that have a role in managing the asset.
.7 Risk Performance Assessment. The risk management process (see Section 7.6) identifies potential risk factors to track and assess.
.8 Other Asset Performance Attributes and Measures. Internal or external asset performance measures and practices information is gathered for benchmarking.
.9 Historical Information. Successful past performance assessment approaches are commonly used as future references. Historical performance data and lessons learned are also captured.
6.1.4 Outputs from Strategic Performance Assessment
.1 Asset Improvement Opportunities. Asset performance problems, opportunities and risks are identified so that requirements for a solution can be assessed (see Section 3.1).
.2 Performance Benchmarks. Internal and external measures obtained from benchmarking may be used in requirements assessment and project planning (e.g., competitive targets).
.3 Information for Change Management. Assessment findings that a requirement is no longer addressing the business problem are reported to the change management process (see Section 6.2).
.4 Historical Information. The performance assessment approaches used are captured in a database (Section 6.3) for use as future planning references. Assessment findings, particularly lessons learned, data are captured as well.
6.1.5 Key Concepts for Strategic Performance Assessment
The following concepts and terminology described in this and other chapters are particularly important to understanding the strategic asset performance measurement process of TCM:
.1 Requirements. (Section 18.104.22.168 and 3.1).
.2 Investment Decision Basis/Business Case. (Sections 22.214.171.124 and 3.3).
.3 Balanced Scorecard. (Sections 126.96.36.199, 3.1, and 5.2).
.4 Key Performance Indicator (KPI). (Sections 188.8.131.52, 3.1, and 5.2).
.5 Investment Return and Profitability. (Section 184.108.40.206 and 3.3).
.6 Cost of Quality. (Section 220.127.116.11).
.7 Benchmarking. (Section 18.104.22.168).
.8 Lessons Learned. (Section 22.214.171.124).
Further Readings and Sources
The strategic performance assessment process is generally covered in business and strategic management texts, including those that cover quality management and benchmarking methods. The following references provide basic information and will lead to more detailed treatments.
Amos, Scott J., Editor. Skills and Knowledge of Cost Engineering (section by Gary Cokins on quality management), 5th ed. Morgantown, WV: AACE International, 2004.
Cokins, Gary. Performance Management (Finding the Missing Pieces to Close the Intelligence Gap). New York: John Wiley & Sons, 2004.
Klammer, Thomas P. Managing Strategic and Capital Investment Decisions. Burr Ridge, IL: Irwin Professional Publishing, 1993.
Watson, Gregory H. Strategic Benchmarking. New York: John Wiley and Sons, Inc., 1993.
[TOC] [PREVIOUS] [NEXT]
2008 By AACE® International www.aacei.org