TCMFramework_SM.gif (5258 bytes)TCM Framework: An Integrated Approach to Portfolio, Program and Project Management
(Rev. 2012-01-09)
[AACE HOME] -- [TCM HOME] -- [BUY THE TCM FRAMEWORK] --[DOWNLOAD PDF VERSION]

[TOC] [PREVIOUS] [NEXT]

III. PROJECT CONTROL PROCESS - CHAPTER 10 - PROJECT CONTROL PERFORMANCE ASSESSMENT

10.1 Project Performance Assessment

10.1.1 Description

    Project performance assessment is the process of comparing actual project performance against planned performance and identifying variances from planned performance. It also includes general methods of identifying opportunities for performance improvement and risk factors to be addressed. After the variances, opportunities, and risks are identified, actions to address them, and the potential effects of the actions on project outcomes, are further assessed and managed through the forecasting and change management processes (Sections 10.2 and 10.3, respectively). Corrective or change actions are then implemented as appropriate through updated project control planning (Chapter 7), which closes the project control cycle. At project closeout, the final assessments of project performance are captured in the project historical database (Section 10.4) for use in future project scope development and planning.

    Performance against each aspect of the project plan must be assessed. Project cost accounting measures of the commitment and expenditure of money (Section 9.1) are compared to the cost plan or budget. Resource tracking measures (e.g., the receipt of materials or consumption of labor hours) are compared to the resource plans. Schedule status (as reflected in the statused network schedule) is compared to the baseline or target schedule. Also, the project status is assessed to determine if any risk factors, identified in the risk management plan or otherwise, are imminent or occurring.

    During the early planning phases of a project life cycle, work performance is assessed against the project control plan established specifically for that phase (per Section 8.1) and for which partial funds and resources have been authorized (per Section 5.1). At the same time, the effects of scope development on the overall capital budget and schedule targets for the project are assessed In other words, day-to-day performance of planning activities is assessed against the project control budget for that phase, while the scope being planned is assessed against the capital budget.

    The level of understanding of project performance is increased when the assessment integrates the evaluation of each aspect of the project plan. One method of integrating schedule and budget assessment is called earned value management. Some branches of the United States government require that this project control technique be used on their capital projects (sometimes referred to as Cost/Schedule Control systems Criteria or C/SCSC). The method is objective, quantitative, and effectively identifies variances from planned schedule and budget performance.

    For complete performance assessment, earned value techniques must be augmented with practices that identify opportunities and risks, not just variances. For materials, resource tracking based on material management system data and expediting and inspection observations provides a wealth of information for assessing performance that may be affected by materials issues. Likewise for labor, work sampling measures, work inspection observations, and other surveillance inputs provide information for assessing labor productivity issues (and to some extent material issues as well).

10.1.2 Process Map for Project Performance Assessment

    The project performance assessment process illustrated in Figure 10.1-1 is the initial problem and opportunity "identification" stage of the project control assessment processes. The primary inputs are performance measurements (see Sections 9.1 and 9.2). The initial performance assessments of these measurements are the outputs for further analysis in the forecasting and change management processes (Sections 10.2 and 10.3, respectively).

Figure10.1-1.jpg (110537 bytes)

Figure 10.1-1 Process Map for Project Performance Assessment
[LARGER IMAGE]

    The following sections briefly describe the steps in the project performance assessment process.

.1 Plan for Performance Assessment

    The process for performance assessment starts with planning for the assessments. Initial planning starts with development of the execution strategy and work packages during scope development (Section 7.1) and continues through development of control accounts during project control plan implementation (Section 8.1). A requirement of work package and control account development is that the resources, activities, and deliverables for each work package and control account be measurable and therefore suitable for quantitative assessment. Planning for measurement (Sections 9.1 and 9.2) and assessment must be done together. Planning ensures that the quantitative measurement and assessment practices used (e.g., earned value) are consistent with the control and cost accounts.

    The use of each assessment method (e.g., earned value, work sampling, etc.) should be planned in a way that optimizes the identification of variances, opportunities, and risks for all aspects of the project plan. Also, planning assigns specific roles and responsibilities for assessing project performance. Unlike the measurement process for which much of the day-to-day responsibility of measuring performance (e.g., units completed) lies with those responsible for the work packages, assessment is generally a function performed by cost engineers in a project control role supporting the project manager. However, those responsible for the work packages should make their own subjective assessments of their performance, and provide other observations that help project management understand their performance.

    Another aspect to plan is the interface (i.e., data transfer) between measurement systems (e.g., payroll systems, material management systems, etc.) and systems used for assessment (e.g., scheduling software). In addition, the interaction/interface of owner, supplier, and contractor reporting systems, if any, must be considered and addressed in performance measurement and assessment plans.

    After project performance measurement and assessment has been planned, and measurement processes or systems have been initiated, the use of assessment methods begins.

.2 Assess Cost Performance

    Cost performance is generally assessed using the earned value method. That method starts with the premise that control accounts have a resource value (typically cost or labor hours) that can be summed to the total value for the project (or some branch of the cost account structure such as labor costs). The percent progress of any sub-component of the work breakdown structure (WBS) or cost account structure branch, at any point in time, is then the sum of the value of each control account in that branch that has been "earned" at that point, divided by the total value of that account structure branch. The earned value for each control account is the value of the control account multiplied by the percent "physical progress" measurement (see Section 9.2) of the scope included in that control account. For example, if the work package was construction of a building’s concrete foundation, with a budgeted cost of $10,000, and the construction physical progress at a certain time was measured as 40 percent complete, the earned value for that foundation control account at that point of time would be $4,000 (i.e., 0.4 x $10,000).

    The earned value is then compared to the expended costs to determine if there is a cost or budget variance for that item (or for that branch of the cost account structure). Using the previous example, assume that $5,000 had been expended on the concrete foundation, and the earned value was $4,000. In this case, the foundation’s work package is $1,000 or 25 percent over budget. This variance would be further evaluated by reviewing status reports by and/or speaking to the foreman responsible for this work package (or their supervisor), as well as by reviewing material management reports, work sampling reports, and so on. All of this information would be used in forecasting the foundation work and cost remaining (see Section 10.2). The cost performance variance (i.e., earned minus expended) of all work packages can then be summed to determine overall project cost variance.

    The earned value method is typically applied at the cost account level. For example, a work package may include installation of a piece of equipment for which there are cost accounts for the equipment purchase (i.e., material cost account) and labor to install the piece of equipment (i.e., labor cost account). The labor and material progress will be measured separately. For example, assume the material cost budget for the equipment item purchase was $8,000, and the labor cost budget for its installation was $2,000. Further, the equipment, which actually cost $10,000, had been shipped to the site but installation, which had cost $1,000 to date, was only 50 percent complete. The weighted work package percent progress would be the total earned value/total budget or [(100 percent x $8,000)+(50 percent x $2,000)]/($8,000 + $2,000) = $9,000/$10,000 = 90 percent. The work package is also $2,000 over budget (i.e., $11,000 expended minus $9,000 earned). In this way, you can view the cost from a work package perspective (i.e., it is $2,000 over budget), or a cost account perspective (i.e., the material is $2,000 over budget, but the labor is on budget).

    The cost assessment is usually reported using both tables and charts. A typical cost report table or chart might include the budgeted, expended, and earned costs tabulated against the project date. The charts are typically cumulative distribution function diagrams (often called "S" curves because the typical plot is shaped like a flattened letter S). To support such a table or chart, the budget must be time phased. The time phasing is determined by the resource allocation methods as described in Section 7.2.

    These observations—along with schedule, resource, productivity, and work process and performance assessments—will be used in the forecasting process (Section 10.2) to evaluate the budget for remaining work, and to address trends and changes (Section 10.3).

.3 Assess Schedule Performance

    Schedule performance assessment starts by entering the actual start, finish, and milestone achievement dates for activities in the project schedule (usually using scheduling software). If earned value techniques are used, the percent complete information is also documented for each activity. However, schedule performance is based on the remaining duration needed to complete the work of the activity, which is then used to evaluate the percent of schedule completion. Performance is assessed by calculating the revised durations and completion dates of remaining activities and of the project as a whole using scheduling algorithms (again, usually using scheduling software). This does not change the planned schedule but is only used to evaluate progress and determine if some corrective action is needed to maintain the planned schedule. Corrective action is usually applied to the activities in progress or the immediately following activities.

    Schedule performance variance may be expressed as either the amount of time or the percent of the planned duration that the schedule is ahead or behind. For example, a project may be 2 months behind schedule (i.e., schedule slippage) or 10 percent behind the planned project duration of 20 months. The variance can be reported for any activity, a group of activities, or the project as a whole.

    If earned value techniques are used, schedule performance variance can also be expressed in terms of cost, hours, or other resource value used. Using earned value, schedule variance is the earned value less the value of the work that was planned or scheduled to be complete. If the earned is greater than the planned value, then the project is ahead of schedule (see Section 10.1.2.5). Earned value and scheduling (i.e., critical path analysis) must both be used because earned value will only indicate if sufficient resources are being applied. Critical path scheduling will tell if the correct activities are being worked on.

    The schedule will also be assessed to determine if the logic for the remaining work is still valid (e.g., is the actual performance now out of sequence). This observation, along with cost, resource, productivity, and work process and performance assessments (i.e., trends) will be used in the forecasting process (Section 10.2) to evaluate the plan and schedule for remaining work, and to address trends and changes (Section 10.3).

    The schedule assessment is usually reported using a schedule plot (e.g., a bar chart) showing the planned (i.e., target) and actual schedule activity status. The assessment can also be displayed in a table showing a percentage or factor that expresses the extent that the schedule is ahead or behind at given points in time. To help assess management priorities, additional schedule status reports may include lists of activities sorted by pending planned start dates (activities requiring immediate attention), pending planned finish dates, or by total float (activities with most immediate potential impact to project completion).

.4 Assess Resource Performance

    In addition to cost and schedule assessment, the status of resource procurements (i.e., materials) and resource expenditures (i.e., labor hours) against resource plans needs to be assessed.

Labor Hours

    Industry data indicate that for most well-defined projects the labor cost account experiences the most variance. For that reason, there is a common saying: "control the hours and control the project." Hour assessments can be used because labor costs are usually proportional to labor hours, at least on the average for a particular discipline, trade, or craft.

    Another reason that labor hour assessment is considered particularly valuable is that labor hour measurements are usually performed more frequently than cost measurements. The cost accounting system (Section 9.1) for many project systems reports costs on a monthly basis. If the project’s major activities or phases are measured in days (e.g., a process plant shutdown) or weeks, cost performance assessment may be inadequate to support effective, timely project control.

    Labor hour performance is usually evaluated the same way as cost performance using basic earned value methods. The methods previously described for cost assessment all apply, with the exception that labor hours are used as the resource "value" measure, rather than cost. Another exception is that total project cost performance cannot be based on using labor hours because not all cost accounts are labor accounts (and, if they were, the cost of labor hours for different people will vary).

    As with cost, the labor hour assessment is usually reported using both tables and charts. A typical labor hour report table or chart might include the budgeted, expended, and earned hours tabulated or plotted against the project date (after forecasting, the report may also include forecasted hours to complete or totals at completion). To support such a table or chart, the hours must be time phased. The time phasing is determined by the resource allocation step described in Section 7.2.

    Variances in labor hour performance often result from labor productivity differing from the estimate used in the plan. The assessment of labor productivity is discussed later in this section.

    The labor hour observations, along with cost, material resource, productivity, and work process and performance assessments, will be used in the forecasting process (Section 10.3) to evaluate resource plans for work remaining, and to address trends and changes (Section 10.4).

Material Management and Fabrication

    Often, material supply and fabrication activities in project schedules start with preparation of the bid packages, and finish with the receipt or acceptance of the material item(s) at the project location. These basic milestones are usually adequate to support project activity logic (e.g., an item cannot be installed until it is received) and earned value assessments (e.g., an item’s cost value is earned when it is received). Because the material supply chain is usually complex, all the supply chain incremental activities (e.g., order placement, intermediate materials shipment, fabrication, product shipments, shipping status, inventories, and so on) are usually not included in a master project schedule unless the schedule logic requires them or it is desirable to earn value for the material progress in increments. However, it is a project control responsibility to assess the supply chain status to some degree in order to identify project performance opportunities and risk. This can be facilitated by using a materials tracking database, which may also include the pre-order approval submittals.

    The status of materials is usually reported by an enterprise’s material management system. This system may be integrated with or part of a material or enterprise resource planning (MRP/ERP) system. The reports will include the current status of each material item in its supply chain (e.g., planned start/finish dates of intermediate activities and supporting notations). In essence, the material management system is an extension of the project scheduling and progress reporting system and in some cases may be linked to it. In any case, project control assesses these reports for performance opportunities and risks.

    For example, the project schedule status shows that a vessel was ordered on schedule. However, material management reports show that the vessel fabricator has not achieved an intermediate inspection milestone date. Having observed this, the project control person may forecast and evaluate the effect of a late delivery, and/or may communicate to the procurement department how critical the vessel is.

    The change management process also requires that the status of material procurements be known so that the effect of a change or trend on material costs or schedule can be estimated. Furthermore, shipments that arrive at the project site unreasonably early may look good on a progress report, but may lead to increased inventory and material handling costs, exposure to damage or loss, and so on.

    Material status assessments are usually reported as qualitative findings to be considered in cost and schedule forecasting and change management. The time-phased cumulative planned and actual use or receipts of key materials can be plotted for the plan during the project’s duration versus the periodic status dates. The planned use is obtained from the resource planning process (Section 7.4).

.5 Assess Integrated Earned Value (i.e., Integrated Cost and Schedule Performance Assessment Using an Earned Value Management System [EVMS])

    An earned value management system (EVMS) integrates the assessment of the project budget and schedule. A full description of such systems is not provided here, but the basis assessment methods are described.

    EVMS cost/schedule integration starts with establishment of a WBS and control accounts (Section 8.1). Each control account has a budget and is represented by at least one activity in the project schedule. EVMS integrates cost and schedule by time phasing the cost and resource budget plans. The budgeted cost or resource for any time period (typically by month) is the budget for all work packages planned to be performed in full within that time period, plus, for any work package planned to be in-progress during that time period, the percent of the work package budget planned to be performed during that time period. The budget for that time period is called the planned value of work scheduled.

    Next, the planned value of work performed or earned value is determined for the time period. This is the budgeted cost or resource for all work packages actually performed in full within that time period, plus, for any work package actually in-progress during that time period, the percent of the work package budget (based on actual percent progress) performed during that time period.

    Next, the actual cost or resource usage for work performed is determined for the time period. This is the actual amount incurred on the work packages completed or in-progress during the time period and is called the actual value.

    Using the above time-phased values, cost, resource, and schedule "variances" for the project (or some portion of the WBS) can be assessed as follows:

    A negative variance (i.e., the earned value is less than that expended) may reflect an unfavorable trend (i.e., an overrun or schedule slip). The variances can also be reported as performance indices or factors as follows:

    An index less than 1.00 reflects unfavorable performance (i.e., earned value is less than that expended). These performance factors can be used in forecasting (Section 10.2). However, the SPI will approach 1.00 as the project nears completion and then loses its utility for forecasting.

    Finally, the percent complete for the project (or some portion of the WBS) can be assessed as follows:

    The strengths of an EVMS are that it integrates cost and schedule performance assessment of variances from plans while being objective and quantitative. However, the method in itself does not explain why the performance is what it is, nor how performance can be improved. It simply waves a red flag when there are problem areas that need further assessment, or when positive performance needs to be assessed for potential reduction in project costs and/or early completion.

    Another challenge is that EVMS relies on cost as its "value" measure. As was mentioned in the labor hour assessment discussion, cost accounting systems often report expended costs monthly. If the project’s major activities or phases are measured in days or weeks, the red flags may be too late for effective project control.

.6 Assess Work Process and Productivity

    In general terms, labor productivity is defined as the ratio of the value that labor produces to the value invested in labor. In an absolute sense, it is a measure of the extent to which labor resources are minimized and wasted effort is eliminated from a work process (i.e., work process efficiency).

    In earned value assessment, productivity is a ratio (i.e., a factor) that compares the labor effort expended to that which was planned (sometimes called the "spent-earned ratio"). In earned value terms, productivity is calculated as follows:

    To be useful, the control budget must reflect the current scope and quantities. This is the same method defined previously in the labor performance assessment description. However, in this case, a factor less than 1.00 is favorable. For example, if work was 60 percent complete on the labor cost account of a work package that had a budget of 200 hours, and 100 hours had been spent, then the productivity factor is 0.83 (i.e., = 100/(0.6 x 200). Care must be taken to understand the nature of any productivity factor that is quoted because alternate sources may use the inverse (i.e., a factor less than one is not favorable).

    Another related way to measure the value of labor input to the work output is to calculate the ratio of labor cost or hours spent to the quantity of work performed. Example ratios are hours per line of code developed, hours per design deliverable, or hours per tonne of steel erected. These types of ratios are often the basis of the control budget labor estimates. A difference between the actual ratio and the ratio used as a basis of estimate indicates whether actual productivity is better or worse than planned.

    These factors and ratios are useful for forecasting (Section 10.2), but they provide no information as to whether wasted effort exists in the work process. In other words, a project’s use of labor may be unproductive in an absolute sense, while still achieving a favorable productivity factor if the plan also reflects unproductive use of labor. If a project objective is to optimize cost competitiveness, then a relative productivity assessment alone may be inadequate.

    To optimize the use of labor, as well as control it, a direct method of measuring and assessing labor productivity is needed; one such method is work sampling. This method provides the specific information needed to eliminate wasted effort. Work sampling provides systematic observations of work activity at the work site to:

  1. determine the proportion of labor hours being spent in non-productive work activity and delays versus productive work activity,

  2. analyze factors that cause non-productive activity or delays, and

  3. identify opportunities to reduce non-productive activity and delays.

    Assessment of work sampling data allows for prompt removal or reduction of roadblocks, optimizing the work process by making the work convenient for the workers (i.e., they have all necessary tools, materials, supplies, information, and supervisory support readily available at all times). Optimizing the convenience of the work is a strong motivating factor and motivation is a key driver of work productivity.

    A full description of the work sampling process is not provided here, but the basic method involves recording a sample of worker activities according to established, pre-defined activity category classifications. The percentages of observations in each productive and non-productive activity category are computed and applied to the total labor-hours available to determine the time spent on each category. The resulting data show the overall crew utilization for the period of work sampled. The goal is to optimize the direct time by analyzing the utilization roadblocks and constraints that have been observed and recorded.

    Short of a full work sampling effort, a project can use limited or informal sampling, manpower surveys, time card notations, and other approaches to capture information about the roadblocks and constraints that workers are experiencing. Time lapse photography of critical work areas is also used. In all cases, the methods must focus on assessing the process, not the worker (i.e., planning and management is the problem).

    Another direct method of assessing and improving absolute productivity is quality work process improvement. In this case, the workers, in a planned quality improvement program, identify, assess, and implement work process improvements as a part of their job responsibility. In this case, project control has a role in helping the workers assess the effect of their improvement ideas on project performance.

    Finally, expediting and inspection reports may indicate that the supplier or contractor is experiencing some sort of process problem (e.g., difficulty in obtaining a critical raw material, difficulty in hiring a critical labor skill, etc.). Project control has a role in helping identify and assess improvements in their work process as they affect project performance.

    If changes in the work process that might improve performance are identified in the performance assessment process, then change requests should be initiated in the change management process (Section 10.3).

.7 Assess Risk Factors

    Each of the assessment methods described above may identify imminent or occurring risk factors, as anticipated in risk management planning (Section 7.6) or otherwise. The risk management plan includes plans for monitoring anticipated risk factors. Risk assessments are considered in the forecasting and change management process.

.8 Report Project Performance Assessment

    The results of assessments (i.e., variances, opportunities, and risks) are reported to the forecasting and change management processes (Sections 10.2 and 10.3, respectively). At project closeout, the final project performance assessment is captured in the project historical database (Section 10.4) for use in future project scope development and planning.

    It is also a project control responsibility to give those responsible for work packages immediate feedback on performance assessments. This feedback may be accomplished by having frequent project team performance review meetings in addition to the progress status meetings. While objective measurements and assessments are required for effective project control, subjective assessments by of those responsible for and with expertise in project control are invaluable.

    Quantitative assessment reports tend to consist of tables and charts (e.g., "S" curves) as were described with each of the assessment methods above. Schedule reports tend to be activity plots (e.g., bar charts, etc.). In each case, the report must clearly show the variance from the plan that must be addressed and controlled.

    Graphical representations are particularly useful when there is no bottom line variance (e.g., cost is on budget), but the measure is unpredictable over time or between accounts indicating a potential risk to address (i.e., your good performance is the result of luck, not good control). Also, cost and schedule performance assessments are most useful when reported in an integrated manner.

    Quantitative assessments must be supported by observations of the probable cause of any significant variances identified. In addition to probable causes, opportunities and risks should be noted as identified by methods such as work sampling, expediting, inspection, work process improvement, and so on.

    At the close of the process, historical project information, including experiences and lessons learned about assessment practices, is used to improve assessment tools such as software, forms, checklists, procedures, and so on.

10.1.3 Inputs to Project Performance Assessment

.1 Project Implementation Basis (Objectives, Constraints, and Assumptions). The enterprise may establish requirements for performance assessment, such as the use of an EVMS system and so on (see Section 4.1).

.2 Project Control Plan. The project control plan (Section 8.1) describes specific systems and approaches to be used in project control including interface with measurement and assessment (cost/schedule control) systems.

.3 Performance Measurement Plans. Plans for cost and resource commitment, incurrence, and expenditure measurement (Section 9.1) and progress and performance measurement (9.2) must be planned with the assessment process.

.4 Risk Management Plan. Project status is monitored for the occurrence of risk factors. Risk management plans (Section 7.6) provide the basis for risk monitoring and assessment.

.5 Project Control Basis. The control accounts (Section 8.1) are work packages from the WBS for which responsibility has been assigned in accordance with the OBS and the procurement plan, and for which cost budgets and resources have been assigned, and activities scheduled. These plans are the baselines against which performance measures are assessed.

.6 Performance Measures and Observations. Cost accounting (Section 9.1), and schedule status, resource tracking, and physical progress measures (Section 9.2) are captured and reported for assessment against baseline plans. Also, direct observations of work are used to identify work process and productivity improvements.

.7 Changes. During project execution, changes to the baseline scope definition and plans are identified in the change management process (Sections 10.3). Change may result in changes to project performance assessment.

.8 Historical Project Information. Successful past project progress and performance measurement and assessment approaches are used as future planning references and to help improve assessment tools.

10.1.4 Outputs from Project Performance Assessment

.1 Project Control Plans. Performance assessment plan considerations are considered in overall project control planning.

.2 Performance Measurement Plans. Plans for cost commitment and expenditure measurement (Section 9.1) and progress and performance measurement (Section 9.2) must be planned with the assessment process.

.3 Risk Management Plan. Project status is monitored for the occurrence of risk factors. New risk factors identified during project execution may require updated risk management planning (Section 7.6).

.4 Project Control Basis. Performance measures are assessed against the control plan baselines. The assessment may raise issues about the appropriateness of the baseline.

.5 Information for Forecasting. Assessments of project progress, performance variances, and past productivity are used in forecasting (Section 10.2) the performance of the remaining work scope.

.6 Information for Project Change Management. Performance variances are evaluated as to the need for and scope of corrective actions as determined in the change management process (Section 10.3). Also, a change request may be originated for any work process change that might improve performance.

.7 Historical Project Information. The project’s performance measurement and assessment approaches are used as future planning references and to help improve assessment tools.

10.1.5 Key Concepts for Project Performance Assessment

    The following concepts and terminology described in this and other chapters are particularly important to understanding the project performance assessment process of TCM:

.1 Earned Value. (See Section 10.1.2.2 and 10.1.2.5).

.2 EVMS. (See Section 10.1.2.5).

.3 Productivity. (See Section 10.1.2.6 and 11.2).

.4 Variance: An empirical difference between actual and planned performance for any aspect of the project control plan.

.5 Trend: Non-random variance of actual asset or project performance from that which was planned. Analysis of performance measurements is required to determine if an observed performance variance is a trend (i.e., predictable) or a random outcome (i.e., unpredictable), and that determination will influence subsequent control actions and forecasts.

.6 Work Process Improvement. (See Section 11.4).

Further Readings and Sources

    The project performance measurement and assessment process is covered in a variety of sources. Most project management and control texts address basic earned value and earned value management systems and resource tracking to some degree. Direct measurement and assessment of supplier performance, work processes, and productivity are more thoroughly covered by procurement and material management, quality management, and industrial engineering texts. The following references provide basic information and will lead to more detailed treatments.

[TOC] [PREVIOUS] [NEXT]

Copyright © 2008 By AACE® International www.aacei.org
Comments/more information on the TCM Framework: An Integrated Approach to Portfolio, Program and Project Management may be directed to tcm@aacei.org