Framework: An Integrated Approach to Portfolio, Program and Project
|[AACE HOME] -- [TCM HOME] -- [BUY THE TCM FRAMEWORK] --[DOWNLOAD PDF VERSION]|
[TOC] [PREVIOUS] [NEXT]
II. STRATEGIC ASSET MANAGEMENT PROCESS - CHAPTER 6 - STRATEGIC ASSET PERFORMANCE ASSESSMENT
6.4 Forensic Performance Assessment
The forensic performance assessment process is conducted outside of, or as an extension to, cost and schedule control processes. The forensic study process is usually applied when performance variances, trends, deviations, or changes result in disputes or claims (see Sections 6.2, 9.2, and 10.3), or at project close-out to support lessons learned development. While the adjective forensic is usually associated with work in a legal context, the process and methods are not limited to legal matters.
The primary goal of performance assessment in the control mode (see Sections 6.1 and 10.1) is to assess the need for, identify, and take timely corrective actions to keep performance on plan and avoid damages. In the forensic mode, the perceived damage has occurred, and the primary goal of performance assessment is then to determine causation, damages, and liability (i.e., responsibility and entitlement). Forensic performance assessment is also used to resolve disputes in a legal context and/or to gain knowledge to support long term performance improvement (i.e., reduce risk of future failure) of an asset or project system. It is also used in loss and damages valuation and in analyzing the cost of mitigating and/or remediating the effects of failures or defects.
This process is a branch of forensic science. Related practices include forensic engineering, which involves the analysis of physical and safety failures while forensic accounting focuses on economic crime, fraud, and business valuation. Cost engineering, with forensic specialization (i.e., claims and dispute resolution), generally involves the analysis of asset and project cost and schedule performance failures.
The forensic performance assessment process is usually initiated during the asset or project life cycle when a change order remains unresolved, a claim is made, or some other disputes arises (see Sections 6.2 and 10.3). However, the process may continue for some considerable amount of time after the asset or project life cycle; it is not an asset or project control function. Often, the process becomes a separate project in its own right, with its own business or legal goals and objectives. In some cases, particularly when the situation is highly contentious, those project personnel involved in a claim may not be the best to lead the study effort (e.g., operations or project management) because of potential bias, conflicts of interest, and/or distraction from ongoing management and technical responsibilities.
The following sections highlight the two primary reasons that forensic performance assessments are conducted (i.e., claims and lessons learned), and the key role that good information has in either case.
.1 Claims and Dispute Resolution
The interactions/interfaces of the owner, architect, suppliers, subcontractors, and contractors in change management are addressed in contract documents and change management plans (Section 7.7). However, performance variances, failures, or changes sometimes result in disputes and claims if the parties cannot agree on some aspect of the performance or change in relation to contract agreements (i.e., scope, compensation, time, relief, damages, delay, etc.). Generally, disputes are best resolved by the operations or project team using the established contract mechanisms (i.e., changes and dispute clauses); these mechanisms may include mediation, arbitration, or litigation.
However, if the dispute cannot be resolved, a claim may result. A claim is a written statement by one of the contracting parties seeking adjustment (e.g., additional time and/or money) or interpretation of an existing contract because of acts or omissions during the preparation of or performance of the contract. Unresolved claims are submitted to mediation or arbitration or filed with the courts and from there the legal claims resolution or litigation process (which varies widely depending on the nature of the claim, contract provisions, jurisdiction, etc.) takes over.
Claims also arise in the context of insurance, letters of credit, and performance and payment bonds and other third party financial risk mitigation mechanisms. If a party suffers damages or loss due to a physical or performance failure or failure to pay, a claim for cost recovery may be filed. This claim needs to be assessed by both the claimant and the insurer or surety (the company that issued the bond).
.2 Lessons Learned
It is a good practice for asset and project management to assess and capture how successful its assets or projects are in achieving their objectives, and what root cause factors contributed to success or failure. As discussed in Section 6.1, lessons learned information is typically elicited through the use of subjective surveys, narrative descriptions, interviews, or formal lessons learned workshops to identify all contributing causes, analyze for root cause, and identify new or improved work processes. When performance is successful, subjective approaches work well because the team is generally willing to share information. However, with performance failures, when stakeholders are not so willing to share information, a more in-depth, objective forensic analysis may be the only way to identify root causes and responsibility for contract variances.
.3 The Role of Information
The success of forensic performance assessment in properly relating causation to performance degradation and then determining responsibility generally depends on having or obtaining good information about the planning, performance measurement, and assessment of the work in question. A goal of forensic analysis is to replace supposition ("project myths") with persuasive factual evidence. Poor or unsubstantiated information works against that goal. In a legal situation, the best preparation for or defense against a claim is using best practices and maintaining thorough, accurate, and complete records of all communications, plans, actual cost and schedule performance data, and so on.
A disciplined process like TCM helps avoid failures, claims, and the need for forensic studies, but when needed, helps support a successful effort or resolution. Most experts agree that sharing accurate planning and performance measurement information about a project during its performance helps avoid disputes and facilitates their resolution. The potential for claims and forensic assessment is a key consideration in the procurement process (Section 7.7); that is, the team must include requirements for good information and records as part of the contract, in addition to considerations for dispute resolution.
6.4.2 Process Map for Forensic Performance Assessment
The basic forensic performance assessment process illustrated in Figure 6.4-1 is common to processes that involve the investigation of performance variances or failure (e.g., forensic engineering, accounting, failure analysis, or root cause analysis). The main work of the process begins with information gathering and background research. This step is not found in control assessment, which relies on regular performance measurement data. Much of the work in the process takes place in the analysis step, which takes the information and applies cost engineering skills and knowledge (e.g., scheduling, cost estimating, etc.) to tie causation and responsibility to performance. The results of the analysis are then reported and communicated as needed for legal action (e.g., mediation, trial, etc.) and/or performance improvement or corrective action of an asset or project system.
Figure 6.4-1 Process Map for Forensic Performance Assessment
The following sections briefly describe the steps in the forensic performance assessment process.
.1 Plan the Study
At the start of a forensic assessment, study leadership is established (by the project team, business unit, or legal management as applicable) with responsibility to plan and prepare for the study effort. The general study goals and objectives, study scope, and plan for study activities are established, with a study team whose roles, responsibilities, and milestones are identified and assigned.
It is important to identify a clearly understood objective (e.g., recover or avoid costs, defend a principle, etc.). The team should also understand what is at risk (e.g., long term relationship, performance of other ongoing work, financial loss, etc.). The team may determine early that the potential benefit of the effort does not justify the cost and/or risk.
The study team then begins to define the failure or problem scope (i.e., effect) and further define the bounds of the study (e.g., the scope of the affected asset or project, affected or involved parties or stakeholders, locations, time frames, etc.). Legal requirements, issues, and risks (if any) must be identified to help guide remaining work.
Initial hypotheses as to mode and mechanisms of the variance or failure may be established early so as to identify likely information, resources, models, and tools that may be required for the study.
The study scope and plan are revisited as information and analysis indicate need for revision.
.2 Gather Information
As was mentioned, good information (contemporaneous documentation suitable for evidentiary purposes) almost always drives success of the study. Information is gathered regarding liability, causation, responsibility, performance impacts, and damages as appropriate to the goal of the study. Information gathering usually includes locating and collecting contracts and records related to the failure. It is common to have standard checklists to assist in identifying records. It may also involve questioning and interviewing those involved, site visits, and inspections as needed to provide facts and understanding that hard documents cannot convey. In a legal situation, information gathering usually involves a discovery process, including document production, interrogatories, and depositions.
Background research is also done to obtain information about comparable situations and their failures or successes. The study leadership may also obtain or build models of cost, schedule, and other attributes as appropriate for the study.
Technical and programmatic information about the asset or project is also gathered as needed to support analysis. For example, if the analysis effort required a cost estimate of damages to a building, information about the design and construction of the building and its repair will be required for the estimating analysis process.
The key to analysis and settlement of claims and disputes is proper documentation. Facts of the matter are proved or disproved based on contemporaneous documentation. The law that governs a particular matter is tied to a specific fact pattern and legal precedent. If the facts change, the outcome may change. Therefore, establishing the facts accurately is of paramount importance. To increase the probability of capturing events correctly, procedures for documenting events need to be established prior to project commencement.
As information is gathered, plans are revised for the remainder of the study effort.
.3 Analyze Performance Degradation, Causation, and Responsibility
As mentioned, analysis applies cost engineering skills and knowledge (e.g., scope definition, planning and scheduling, cost estimating, resource planning, etc.) to tie performance degradation to causation and responsibility. Some studies such as financial damage valuation may apply deterministic cost estimating. Other studies such as schedule delay impacts may apply more stochastic modeling and event reconstruction techniques, which are particularly useful to study and demonstrate likely causation and determine responsibility.
Analytic methodologies vary widely depending on the type of study and failure. Forensic cost engineering studies tend to evaluate cost, schedule, and/or productivity issues as follows.
Schedule Impacts (Delay, Acceleration, etc.)
Schedule analysis seeks to evaluate impacts and answer questions such as: Was a schedule extension justified? Was the contractor accelerated and what was the result? Was the contractors original project plan reasonable and capable of implementation? Schedule analysis may apply conventional schedule planning and development processes and methods as covered in Section 7.2. However, the starting point for forensic analysis is often a schedule model that resulted from that process, and/or the statused schedule resulting from the progress measurement process (Section 9.2). The following methods are those most commonly used when working with Critical Path Method (CPM) schedules:
Retrospective Analysis This is analysis of schedule impacts from the date the delaying event is overcome or from the project completion date back to the beginning of the event or the project to determine the impact of events on the overall project. Retrospective analytical techniques include:
Contemporaneous Period Analysis (sometimes referred to as the Observational Method) This is the analysis of each periodic update to determine schedule slippage (if any) and assessment of responsibility and liability for the slippage determined.
As-Built vs. As-Planned AnalysisThis is a comparison of the as-planned schedule with the as-built schedule to determine the total amount of delay encountered and, from this, assessment of the causation and liability for such delay.
Collapsed As-Built or But For AnalysisThis is an analysis of the as-built schedule to determine which activities drove the increased time and who was responsible for those activities. Once liability for the delaying activities is determined, the analyst removes only one partys delays from the as-built schedule to determine when the schedule would have been completed "but for" the actions of the other party.
Impacted As-Planned AnalysisThis is an analysis of the project to determine delays caused by one party to the contract and then addition of these delaying events to the as-planned schedule to show the impact of the events on the overall schedule.
Windows Analysis (also referred to as the Snapshot Technique or the Periodic Analysis Technique)This is an analysis of each periodic update to the schedule to determine which events were delayed in that "schedule window" (or time period) and, based on that analysis, to determine responsibility for each event. It is similar to but somewhat different from the Contemporaneous Period Analysis mentioned above.
Global Impact AnalysisThis method is used when a project changed so much from that which was bid and the contractor never had an opportunity to perform in accordance with its project plan.
Bar Chart AnalysisThis method uses a simple format that shows the sequence and timing of activities. To be effective in a forensic analysis, logical relationships between activities would be required. The result is called a "fenced" bar chart.
Resource Profile Analysis-This method is used to show the effect of changes in sequence and timing of activities on resource requirements. The projects as-planned resource requirements are compared to actual or impacted requirements.
Prospective AnalysisThis is analysis of schedule impacts from the date of the event forward to determine what is the likely impact of an event on the remainder of the project. A key prospective analytical technique is Time Impact Analysis. This technique starts with the latest accepted or approved schedule update and impacts this schedule by the addition of events. Once the events are input to the schedule, the schedule is recalculated to determine whether a delay has occurred. If a delay is demonstrated then the event is, by definition, the cause of the delay and responsibility is assigned to the party causing the event.
Productivity Impacts (Disruption, Inefficiency, etc.)
Productivity analysis may apply the conventional assessment methods as covered in Section 10.1. However, the starting point for forensic analysis is often progress measurement (e.g., time card records and physical progress measures) and performance assessment reports and forecasts (Sections 9.1, 10.1, and 10.2). These records are analyzed to link the productivity losses to causes such as schedule changes, delays, or acceleration.
Cost Impacts (Loss, Damage, or Defect Quantification and/or Valuation)
Cost analysis may apply the conventional cost estimating process and methods as covered in Section 7.3. However, the starting point for forensic analysis is often the statused cost report resulting from the cost accounting process (Section 9.1) rather than historical costs or costs from an estimating database.
Monsey explains the difference between conventional and forensic claims estimating as follows: "A cost estimate for a new project has as an objective the formulation and quantification of projected financial considerations for the purposes of establishing a reasonable budget and/or profit. A computation of costs associated with a claim has the general objective of determining the scope of financial damage perceived by one party, recovery costs believed to be unpaid by a party; or an amount of money that will make a party whole from actions, non-actions; or wrong actions undertaken by another party."
Forensic cost estimates also must deal with categories of costs that may not be significant or considered at all in a conventional estimate such as the following:
penalty or liquidated damage costs;
time related and ripple-effect costs;
cost to repair defective or non-conforming work;
attorney fees and claim preparations;
consultants and expert witness costs;
attorney fees and claim preparations.
It is common to use special models to estimate or simulate cost impacts; however, it is reported that there are "no universally accepted methods." Most models require a large dose of judgment regarding input values, calculation methods, and result interpretation.
In forensic estimating, once costs are estimated, there remains the issue of what portion of the cost the claimant is entitled to recover. Much of this depends on what risk the contract and other documentation indicates each party agreed to bear (entitlement), and then relating the causes of the failure (risk event) to their impact. There may also be the need to allocate the costs among responsible parties such as design professionals, contractors, and the various subcontractors based on their contribution or participation in the failure. This allocation may take the form of a subrogation action if insurers or sureties are involved in the claim.
Root Cause Analysis
The study may apply conventional root cause methods as covered in Section 6.1. The team performing the analysis must drill down below the surface of the situation to determine the true root cause of the situation being examined. This is especially true in claims situations because the purpose of root cause analysis is to determine responsibility and liability for each situation under the terms of the contract or controlling case law where the contract is silent or in conflict. The analyst must establish the root cause of any costs that the claimant expects to recover and demonstrate not only that the claimant is not responsible for the cost variance but also that the added cost can be linked to the actions or inactions of the responsible party.
For example, labor disruptions often occur when a project schedule is re-sequenced due to delays or changes. This situation may call for incorporating the contractors labor cost distribution system into its schedule to determine the labor impact caused by the schedule changes. Protocols such as measured mile, industry studies, expert evaluation, modified total cost, and jury verdict method may be applicable, depending on the factual circumstances and available data.
.4 Report Findings
The results of analysis and findings are reported as appropriate to the intended purpose of the study. The report may include or be supplemented by various illustrations, models, displays, and so on. Records management, with careful consideration of confidentiality issues, is particular important in legal situations. Lessons learned are captured in a database.
If not too late for the asset or project in question, initiate corrective actions through the change management process (6.2 and 10.3) to improve performance as appropriate. Often, corrective actions are taken to improve the asset or project system management strategies, policies, organization, processes, or practices as needed to avoid or mitigate the risks of future failures.
6.4.3 Inputs to Forensic Performance Assessment
.1 Management Goals, Objectives, and Policy. At the start of the process, management must communicate its objectives from the forensic performance assessment (e.g., recover or avoid costs, defend a principle, preserve a relationship, and so on).
.2 Requirements. At the start of the process, any management requirements (or constraints) applicable to planning of the forensic assessment (legal or otherwise) must be identified and communicated to the assessment team (e.g., adhere to certain procedures or rules, etc.).
.3 Claims. Contractors submit affirmative claims to recover damages incurred during performance of the work. Damages may include cost of labor losses caused by delay or disruption, additional labor costs for changed work, costs of repair, remission of improperly held liquidated damages, acceleration costs, and improper back charges. A project owner is obligated to evaluate and resolve the issues using a dispute resolution mechanism, or prepare to defend its position in litigation. Owners may assess liquidated damages and back charges. Contractors, if they want to recover those funds, will evaluate the matter and submit proof of entitlement. Sureties who receive a notice of default and claim for payment upon the payment or performance bond are obliged to investigate the matter and, in accordance with the terms of the bond, render performance, payment, or assert various defenses.
.4 Requests for Assessment. In some cases, a forensic assessment may not be initiated by a claim but by management.
.5 Scope Specific Information (including technical and programmatic information). Information such as progress reports, drawings, estimates, schedules, contract documents, material or process submittals, and applicable building and safety codes are gathered as needed to support the analysis. Technical expertise is required to identify and evaluate issues such as document completeness and accuracy.
.6 Background Information. In addition to information specific to the scope and contract, general information about the economy, markets, and other factors that may affect the analysis is gathered from appropriate sources.
6.4.4 Outputs from Forensic Performance Assessment
.1 Report (including basis documentation). Reports need to communicate findings in a clear and understandable format. Reports should document the process followed during the analysis, assumptions relied upon, and the results of the analysis. If the report is to support civil litigation, it must conform to the applicable jurisdictional rules. Most importantly, the forensic report developed in support of litigation is not a tool of advocacy. It is an objective expert analysis, following ethical principles, reasoned in its approach and conforming to engineering and forensic principles customarily followed by the industry.
.2 Lessons Learned. Contractors or owners may elect to take the information gathered, which is an evaluation of the causes of performance variances, and investigate what management practices require modification, implementation, or additional training. As a result, long term performance improvement (i.e., reduced risk of future failure) of an asset will be gained and future variances will thereby be reduced as the new procedure and training become routine management practices.
.3 Information for Change Management (Initiate Correction Actions). See 220.127.116.11.
.4 Information for Procurement (Contract Management). Lessons learned can be used to improve processes or procedures. For example, procedures for contracting may include provisions for bilateral liquidated damages, incorporating labor inefficiency formulae in change order pricing, or incorporating appropriate key intermediate milestones into project schedule requirements. An example for contractors includes implementing labor production monitoring and commodity installation tracking systems or schedule update mechanisms to capture accurate actual project progress as well as to track events that disrupt or delay progress.
6.4.5 Key Concepts for Forensic Performance Assessment
The following concepts and terminology described in this and other chapters are particularly important to understanding the forensic performance assessment process of TCM:
.1 Forensic Performance Assessment.
.4 Dispute Resolution
.5 Cause and Causation.
.7 Supposition vs. Fact.
.8 Root Cause Analysis.
.9 Responsibility and Entitlement.
Further Readings and Sources
The forensic performance measurement and assessment process is covered in a variety of sources. The following references provide basic information and will lead to more detailed treatments.
AACE International. Recommended Practice No. 25R-03: Estimating Lost Labor Productivity in Construction Claims. Morgantown, WV: AACE International, 2004.
Zack, James G., Editor. Professional Practice Guide No. 1: Contracts and Claims, 3rd ed. CD ROM. Morgantown, WV: AACE International, 2004.
[TOC] [PREVIOUS] [NEXT]
2008 By AACE® International www.aacei.org