GOLDRATT ’ S LOGICAL ANALYSIS OF POOR DELIVERY IN A MULTI-PROJECT ENVIRONMENT

The objective of this study is to confirm Goldratt’s logical analysis of poor delivery in a multi-project environment. Two hundred and ten experienced managers were invited to participate in a multi-project management game that simulates reality. A statistical analysis of the experimental data of this study indicates that the mode of project planning and execution (unrealistic project planning, a lack of clear working priorities, misuse of safety time, and bad multi-tasking), not uncertainty, is the root cause of poor delivery, and should therefore be improved first. The result confirmed Goldratt’s logical analysis of poor delivery in project management.


INTRODUCTION
When the delivery of projects is poor, and late delivery has major consequences for all concerned, companies involved in a multi-project management environment need to achieve a higher level of reliable on-time delivery (OTD).Despite the publication of numerous academic research papers on improving OTD [1][2][3][4][5][6][7][8][9][10], poor OTD remains a major issue in multi-project management [11].In addition to academic research efforts, businesses adopt numerous approaches developed by industrial practitioners, including Lean/Six Sigma [12], product data management (PDM) systems, and the theory of constraints (TOC) [13].These methods are all aimed at improving OTD in multi-project management.
However, our interviews with local project managers revealed that few were confident about their ability to achieve highly reliable OTD with CCPM.The interviews were conducted in three-hour public workshops1 attended by more than three hundred people.The majority of the participants were project managers, resources managers, and engineers.The first polling question was: why is it difficult to achieve high OTD in multi-project management?We asked them to write down not just their own reasons, but also those that they think others would give.Ninety percent of their responses can be summarised as excessive task time variability (or uncertainty), including technique failures, rework, changes in specifications, delayed customer sample proving, unreliable suppliers, and so on.In the light of these results, it is not surprising that reducing uncertainty has become the focus of efforts to improve, with programmes such as PDM and Six Sigma becoming the norm.Unfortunately, the second polling question (if they have adopted PDM and Six Sigma programmes, did OTD improve significantly?) found that, for eighty percent of the participants, OTD remained a major issue.Only twenty percent of the participants indicated that their OTD improved, and only through long-term effort.
In theory it is not difficult to achieve highly reliable OTD in multi-project management.First, an accepted Project Evaluation and Review Technique (PERT) network and its estimated project lead time (PLT) should be determined for each project.Since uncertainty exists, this estimated PLT should have sufficient safety time to handle uncertainty; if not, it will be difficult to meet the deadline [15].The greater the uncertainty, the bigger the safety embedded in the task's time estimates.
Second, the starting and ending times of each project should be scheduled according to the required completion date and resource limitations.If the required completion date can be achieved, then the project is confirmed.If the required completion date cannot be met due to capacity loading, the project will be given a new completion date.If the new completion date is accepted, planning is complete.If not, negotiation is initiated or the project is simply lost.When planning is complete, project execution begins.
In most multi-project environments, to make better use of human resources, most employees are not dedicated to a single project but must multi-task.They are organised in resource groups according to their skills, and each group performs certain types of tasks for several projects.The responsibility of these teams is to turn task time estimates into commitments.Apart from the resource managers, project managers are also in charge of the project.Their responsibility is to make sure that the project is completed according to the original commitments.In a multi-project environment, projects are usually managed in a matrix structure.The progress of each project is reported periodically, and task priorities are shuffled according to urgency.Recovery plans for projects falling behind schedule are discussed and executed as necessary.
As stated above, the mode of planning and controlling multiple projects to achieve high OTD is obvious.If excessive uncertainty is the main challenge in OTD, as claimed by the managers interviewed in this study, and improvement programmes for reducing uncertainty are also initiated, OTD should be significantly improved.However, in reality it is not improved, or it improves only slowly (Standish Group [11]).So what is the true root cause of poor OTD in multi-project management?Goldratt used logical analysis to prove that the root cause of poor OTD in multi-project management is not uncertainty, but the mode of managing multi-projects (15).However, we realised that, unless managers experienced it for themselves, we could not convince them that the mode of managing multi-projects is the root cause of poor OTD.Since it is hard to identify the true root cause of poor OTD by collecting and analysing data directly from the field, the objective of this study was to design a multi-project management game that simulates reality, and to invite experienced project managers, resource managers, and engineers to participate in the game.We then analysed the game data to identify the true root cause of poor OTD in multi-project management.
Because this experiment offers a valuable educational opportunity, we distributed an invitation to local manufacturing companies and invited them to organise one or more teams to participate in the experiment.The letter explained the purpose of the experiment, the time required, who should be team members, and the value they could gain.The team members should be project managers, task managers, and resource managers in their current organisational positions.The response was extremely good: thirty teams from twenty-five companies were soon selected.The length of work experience for each participant ranged from three to twenty-five years, with an average of seven years.

Experimental group design
The multi-project management game used in this study was originally developed by Goldratt [15], and was slightly modified to meet the needs of this research.The modified multi-project management game involves three similar projects (A, B, and C), as Figure 1 shows.Each project consists of seven paths and 20 tasks, and involves ten types of resources (engineers), most of whom must perform more than one task in each project.All the tasks have the same estimated task duration, and are subject to the same variability.Though this setup is far from realistic, it still allows us to draw realistic conclusions while making it considerably easier to track the progress of each project.
Figure 2 shows the theoretical probability task time distribution that is the assumed Beta distribution, where the average time is four days, and 90% confidence requires eight days.The Beta distribution can be used to model events that are constrained to take place within a time interval defined by an optimistic time and a pessimistic time.Because both time values may vary in their relationship to the modal value (the most likely time), the unimodal probability distribution may be skewed to the right or to the left.Therefore, the Beta distribution -along with the triangular distribution -is used extensively in project management to describe the time to complete a task.
Each project is laid out so that no resource is scheduled for two different tasks at the same time.In terms of resource management, each project's planning is realistic, and the planned net time required to complete a project is 56 days.Since each type of resource has only one engineer, each engineer must work on all three projects.The client requests the completion of all three projects within 104 days.However, if the projects can be delivered to the market in a shorter time, there is a big opportunity to capture a large share of the market.Therefore, each team was asked to determine due dates for their projects before the game, and these were to be used to evaluate OTD performance of their projects.
Each task is designed as a task card, shown in Figure 3.Each task card is associated with a task name and resource type needed for the task.For example, task "A1-Y" represents task A1 worked by resource type Y.Each task card has a maximum of twelve empty boxes, depending on the actual net task time generated by the computer, which ranges from one to 12 days due to uncertainty.Figure 4 illustrates the layout of the game.The game requires seven players per team: three project managers and four task managers.Each project manager owns a project, and each task manager leads two or three pseudo-engineers (meaning that each task manager plays two or three different resource types).The project priority is project A > project B > project C. Before starting the game, each team must discuss how to manage the multi-project game and determine the completion date of each project.Although Parkinson's Law [15] (early finishes are not reported, i.e. work expands to fill the available capacity), student syndrome (delay the starting time to lengthen the duration time), and bad multi-tasking (working back and forth between projects) are quite natural working behaviours in reality, and because a game is a game, it was hard to ask participants to present these behaviours as they might have done in real life.We therefore designed these behaviours into the game.For bad multi-tasking behaviour, we defined a bad multi-tasking rule to be followed by all engineers.
For each task card, engineers were able to work three days at most before having to switch to another task card, unless only one task card remained in his hand (this would indicate whether or not they knew how to avoid bad multi-tasking).We considered both Parkinson's Law and the student syndrome in generating the actual net task time.Without Parkinson's Law and the student syndrome, 90% of the tasks' generated net task time should be within 8 days.With Parkinson's Law and the student syndrome, however, most actual net task time will change to 8 days or more.Figure 5 illustrates the probability task time duration distribution due to Parkinson's Law and the student syndrome.It is generated by PMSim [15] and assumes that 25% of resources have no bad behaviours, so that few of them (less than 25%) will be within 8 days.
The game runs from day 0 until all teams complete their three projects.For each day, project managers must determine whether their projects have tasks that can be released to the corresponding engineers (i.e., whether prior tasks have been completed).If new tasks are available, the project managers must decide whether they want to release the tasks to the engineers.Although the actual task time is pre-defined (generated by computer), project managers will not know the time required before they release the task.Instead, they only see the back of the task card (shown in Figure 6a), knowing that the theoretical average task time is four days, and eight days at a 90% confidence level.Upon deciding to release a task, they turn over the task card (Figure 6b) and hand it to the corresponding engineer (seeing the generated actual task time in the process).Each engineer takes one task card from his queue (unless his queue is empty), and writes the day (which the instructor calls out) in the first available empty box.He repeats the process until all the boxes on a task card are full (Figure 6c); the task is complete, and the task card is returned to the project manager.This process continues until all three projects have been completed.
The experimental process proceeded as follows: (1) explain the purpose of the experiment; (2) explain the game and perform a 20-day (game days) trial run for process familiarisation; (3) hold a thirty-minute discussion among the game players on how to play the game to achieve better results; during this period, each team had to determine due dates for their projects; (4) play the game; (5) analyse the game results.The experiment took about six hours to complete.

ANALYSIS OF THE EXPERIMENTAL GAME RESULTS
Thirty teams participated in the experiment.Table 2 lists the experimental results of each team.Columns one to three show the game performance.Each column consists of three sub-columns: planned completion date, actual starting date, and actual completion date.
Column four shows the OTD performance of the three projects.If the actual completion day is equal to or less than the planned completion date (determined by each team), the project is on time.Column five shows the average project lead time for all three projects.Each project lead time is computed by taking its actual completion date minus the actual starting date.Column six shows all the data related to project execution.This column consists of six sub-columns: the average days of releasing the project too early (compared with the project plan of the control group shown in Figure 7), the increase in total task days caused by bad multi-tasking (elapsed-task time [the time it takes from starting a task until it is finished] minus generated actual task time), the total number of times working on the wrong priority (task is not executed following the priority), the total interruption time in the critical path (critical path tasks were not run in a relay fashion), the total amount of time of late start caused by the cascading effect, and the average task time.

Table 2: Experimental results
The average OTD for all thirty teams was 30%, and the average project lead time was 84 days.
Compared with the results of the control group (88% and 58.9 days respectively), the OTD and average project lead time of the 30 teams are quite poor.Table 3 shows that only three teams (#4, #11, and #26 -high OTD teams) completed all three projects on time, four teams (#10, #13, #19, and #29 -medium OTD teams) completed two projects on time, and the remaining 23 teams (poor OTD teams) completed either one project (10 teams) or no project (13 teams) on time.
Table 3: Experimental results of high, medium, and low OTD teams

Analysis of project plans' reliability
To determine the reliability of the planned project completion day for the thirty teams, we simulated these three projects 1,000 times with the theoretical task time distribution shown in Figure 2, using PMsim [15].Table 4 shows the results of this simulation.For example, for project A, if the planned completion day is at 60 days, this means the project can be completed within 60 days with 99% reliability.The reliability data of Table 4 shows that the projects of the high and medium OTD teams (Table 3), except for project B for teams #26 and #29, have high reliability.In other words, their projects were completed within the planned completion date, and their project plans were realistic (no over-promise on the delivery day).
However, for the poor OTD teams, the reliability of completing projects B and C by the planned completion date was low.Their project plans were unrealistic (over-promise on the delivery day).We also found that when the project plan is unrealistic (poor OTD teams), even for the control group, OTD is only 60% (see Table 3).However, for the high and medium OTD teams, OTD is 100% for the control group.The same result appears in the average project lead time.For the poor OTD teams, the average project lead time (89.6 days) is significantly longer than that for the high and medium OTD teams (about 65 days).

Table 4: Reliability of the planned completion day with simulation
The relationship between the planned completion date (project plan reliability) and OTD (experimental and control groups) can be expressed as the following regression model.Based on this model, hypothesis tests can be performed to further support the analysis results above.
Here, Y 1 denotes the OTD of the experimental group, Y 2 denotes the OTD of the control group, and X is the project plan reliability.β 0 is the intercept and β 1 is the regression coefficient.If the hypothesis test rejects 0 : 1 0 = β H , then the independent variable X represents a significant effect for the dependent variable Y.This means that project plan reliability (or unrealistic project plan) affects OTD performance.Table 5a (the control group) shows that the planned completion date has a significant linear relationship with OTD.This implies that a more realistic project plan leads to a higher OTD.However, Table 5b (the experimental group) shows that there is no significant linear relationship between planned completion date and OTD.With the same project plan reliability, why do these groups produce different test results?The key to this question is project execution.

Analysis of data related to projects execution
An analysis of project execution data (Table 6) shows that the data value (columns 1-5) of the high OTD teams is smaller (or less serious) than the data value of the medium OTD teams.This means that, even with the same project plan reliability, OTD deteriorates if the project execution data value is increased.The data value of the poor OTD teams is much higher (or more serious) than the data value of high and medium OTD teams.
However, six independent variables are related to project execution: • the average days of releasing the project too early; • the increase in total task days caused by bad multi-tasking; • the total number of times working on the wrong priority; • the total interruption time in the critical chain path; • total amounts of time of late start caused by the cascading effect; and • the average net task time.
These variables are correlated among themselves.That is, multicollinearity in the data makes it difficult to determine the relationship between these variables and project performance.It would be extremely helpful, therefore, to create new variables that are linear combinations of the original variables.These new variables would be uncorrelated, and could then be used to develop a regression model.This regression model, in turn, could clarify the relationship between the new variables and project performance.Principal component analysis is an appropriate technique for achieving this purpose: it is a method of forming new variables that are linear composites of the original variables.The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible.
Since this study analyses the collected data related to project execution, it excludes the planning reliability factor and uses the same planned project completion days (realistic project plan) for all three projects in all thirty teams.According to the project plan of the control group (Figure 7), we derived the planned project completion day for project A at day 56, project B at day 88, and project C at day 104.The OTD of each team was then re-computed according to these planned project completion days, as Table 7 shows.In this case, the average OTD was about 10% for the experimental group, while the control group remained at 90%.The average lead times for the experimental and control groups remained the same: 84 days and 58.9 days respectively.
Table 8a shows the output eigenvalues of the data in Table 7.The eigenvalues report the variance accounted for by each new variable in the output (i.e., the principal component in Table 8a).The total variance of the new variables is 6 [26].For example, the first principal component's eigenvalues is 3.559068, and its percentage of total variance is 59.3%.This means that the first principal component accounts for 59.3% of the variability in the data.The first three principal components (Prin1, Prin2, and Prin3) account for 94.4% of the variability in the data.The eigenvectors shown in Table 8b give the weights used to form the equation (i.e., the principal component) that was used to compute the new variables.The eigenvector for the principal component is derived from the same analytical procedure used to estimate the weight.Therefore, the three new variables are expressed as follows: where Prin1, Prin2, and Prin3 are the new variables or linear combinations, and X 1 to X 6 are the original mean-corrected variables.The simple correlations between the original variables and the new variables, also called loadings, provide an indication of the extent to which the original variables are influential in forming the new variables.The higher the loading, the more influential the X variable is in forming the principal components score, and vice versa.Therefore, the loading can be used to interpret the meaning of the principal components or the new variables.Table 8c shows the loadings.

Table 6: Data related to project execution
The first principal component equation and the loadings of the six variables show that variables X 3 , X 4 , and X 5 dominate the formation of Prin1.These variables, X 3 (working on the wrong priority), X 4 (critical chain interruption), and X 5 (cascading effect), show the project control problems caused by a lack of clear priorities in the multi-project environment, which results in working on the wrong priorities, which in turn will cause a critical chain interruption (a delay in critical chain tasks).The delay of any critical chain task will have a cascading effect on other tasks, ultimately leading to missed commitments.We call the principal component, Prin1, a "lack of clear priorities index".Because its factor-variable correlations are negative, the higher index means less of a "lack of clear priorities problem".
The second principal component equation and the loadings of the six variables show that Variables X 1 and X 2 dominate the formation of Prin2.These variables, X 1 (releasing the project too early) and X 2 (bad multi-tasking), show the problems caused by bad multi-tasking.
In multi-project environments, project managers will release a project as soon as possible if they fear that the projects will not finish on time.Releasing projects too early means that too many projects will be executed simultaneously, which means that many resources will suffer from bad multi-tasking.Prolific bad multi-tasking drastically increases the lead time (or elapsed time) for tasks and projects, which further leads to missing commitments.Bad multi-tasking also causes, in the down-stream departments, overloads followed by under-loads, which create a tendency to release more work into the system so that people will always have something to work on -which increases bad multi-tasking, producing a vicious cycle.We call principal component, Prin2, a "bad multi-tasking problem index".Because its factor-variable correlations are positive, the higher index means a higher "bad multi-tasking problem".
Table 7: New OTD according to the planned project completion days affect OTD and project lead time, Table 9 gives the standardised factor scores of the thirty teams.These scores can be obtained by dividing the principal component scores by the respective standard deviations.
Based on these scores, two regression models can be formed as follows:

Table 10a: OTD performance hypothesis test results
The regression model is expressed as follows: This regression model shows that OTD is positively correlated with the principal component (Prin1), and negatively correlated with the second principal component (Prin2).This means that the higher the value of the "lack of clear priorities problem index" (meaning less working on the wrong priority), the higher the OTD will be; and the higher the value of the "bad multi-tasking problem index" (meaning that releasing more work is much more serious), the lower the OTD will be.This shows that the "lack of clear priorities problem" and the "bad multi-tasking problem" significantly affect the project OTD.Contrary to popular belief, net task time variability does not significantly affect project OTD.
Table 10b shows the average project lead time hypothesis test results.The first and second principal components are significant, but the third principal component is insignificant.These test results also signify that the average project lead time is significantly correlated with a "lack of clear priorities problem" and "bad multi-tasking problem", but is not correlated with the third principal component, "net task time variability".

Table 10b: Average project lead time hypothesis test results
The regression model is expressed as follows: This regression model shows that the average project lead time is negatively correlated with the first principal component, but positively correlated with the second principal component.This means that a higher value in the "lack of clear priorities problem index" (meaning less working on the wrong priority) leads to a shorter project lead time; and a higher value in the "bad multi-tasking problem index" (meaning that releasing more work is much more serious) leads to a longer project lead time.Furthermore, the "lack of clear priorities problem" and "bad multi-tasking problem" significantly affect the project lead time.Contrary to popular belief, net task time variability does not significantly affect project lead time.

CONCLUSION
This study uses a game and statistical analysis to confirm Goldratt's logical analysis of poor delivery in a multi-project environment.Thirty teams, involving a total of 210 people, participated in the experiment.A statistical analysis of the experimental data of this study indicates that task time variability (or uncertainty) is not the true root cause of poor OTD or long project lead times.Instead, the mode of project planning and execution is the true root cause.Specifically, four major causes related to the mode of project planning and execution will significantly affect OTD and project lead time, which are: (1) unrealistic planning (over-promise), meaning that most key resources work across projects in a multi-project management, but poor planning fails to consider resource contentions across projects; this makes the plan unrealistic and leads to missed commitments and long project lead times; (2) a lack of clear working priorities, meaning that engineers will work on the wrong priority project in a multi-project management due to a lack of clear priorities; working on the wrong priorities causes an interruption in the critical chain, which in turn has a cascading effect on other tasks, and ultimately leads to missed commitments and long project lead times; (3) bad multi-tasking, meaning that project managers in multi-project environments will release a project as soon as possible because they fear that the projects will not finish on time; releasing projects too early causes too many projects to be executed simultaneously (resources competition), which means that many resources will suffer from bad multi-tasking; extensive bad multi-tasking drastically increases the lead time of both tasks and projects, which further leads to missed commitments and long project lead times; bad multi-tasking also causes overloads in the down-stream departments, followed by under-loads, which create a tendency to release more work into the system so that people will always have something to work on, which in turn increases bad multi-tasking -a vicious cycle; (4) masking and misusing the safety time; people who do the tasks used to add safety time by inflating the time estimate for individual tasks; however, inflating the time estimates, in turn, leads to Parkinson's Law (not reporting on early finishes, and work expands to fill the available capacity) and student syndrome.These effects cause the safety to be misused and masked.Misusing (or wasting) the safety time leads to missed commitments.Consequently, OTD improvement programmes should first focus on improving the mode of project planning and execution instead of reducing task time variability.

Table 1 : Results of the control group
* 0.33: One project completed on time; 0.67: Two projects completed on time 1: Three projects completed on time

Table 5b : Hypothesis tests for the experimental group
Y

Table 9 : Standardised factor scores of the thirty teams where
Y 1 is project OTD and Y 2 is the average project lead time.The variables X 1 , X 2 , and X 3 represent the first principal component (Prin1), second principal component (Prin2), and third principal component (Prin3) respectively.β 0 is the intercept and β 1 , β 2 , and β 3 are regression coefficients.If the hypothesis test rejects Table10ashows the OTD performance hypothesis test results, indicating that the first principal component (Prin1) and the second principal component (Prin2) are significant, but the third principal component (Prin3) is insignificant.These results show that OTD performance is significantly correlated with the first and second principle components, but not with the third principle component.