A MAINTENANCE PERFORMANCE MEASUREMENT FRAMEWORK THAT INCLUDES MAINTENANCE HUMAN FACTORS : A CASE STUDY FROM THE ELECTRICITY TRANSMISSION INDUSTRY

Over the past two to three decades, maintenance management has undergone a paradigm shift; it is no longer seen as a necessary evil, but as an integral part of the business process that creates value for the organisation. The next step in the evolution of maintenance management is a maintenance performance measurement that includes human factors. The human factors in maintenance are wellknown in the aviation industry, as it gained momentum in the early 1990s after a series of serious aviation accidents. Other industries, however, have been slow to integrate the human factor in their maintenance performance measurements. This paper discusses the results of a research project that investigated the use and importance of maintenance management performance measurements that focus specifically on human factors as part of the overall performance management system. From the research presented in this paper, ‘motivation’ and ‘competence’ were identified as the most important human performance factors in the maintenance of electricity transmission systems.


Background
Over the past two to three decades, human factors in the maintenance environment of the aviation industry have been well-researched.The investigation and analysis of human factors in maintenance began in the early 1990s after a series of serious and fatal aviation accidents that were caused by maintenance errors: the DC10 crash in 1979 that killed 273 passengers and crew, the Aloha Flight 243 in 1988 that killed 94 people, and the Fokker F28 crash in 1989 that killed 24 people [1].Other industries, however, have been slow to include human factor awareness, procedures, and measurements in maintenance, irrespective of their applicability.
Knowledge, skills, abilities, and personal characteristics (KSAPs) are known as the 'elements of competence', and appear in many different definitions of competence.This is also in line with the PEAR model, which defines competence as a combination of psychological factors (e.g., experience, knowledge, and training).Lucia and Lepsinger [2] define competence as "a cluster of related knowledge, skills, and attitudes that affects a major part of one's job (a role or responsibility), that correlates with performance on the job, that can be measured against well-accepted standards, and that can be improved via training and development".
Motivation can be linked to performance using Vroom's expectancy theory [3].This theory is based on three variables: expectancy, instrumentality, and valence.Expectancy is the perceived probability -or a person's belief -that their effort will lead to a desired outcome.Instrumentality is the perceived probability -or a person's belief -that performance will be met with a reward.
Valence is the value the person places on the expected outcome or reward.Valence is influenced by the person's values, needs, goals, and preferences.
OverIn the last three decades, performance measurement has progressed from being financiallyfocused and short-term (from the late 1880s to the 1980s) to adopting a balanced scorecard approach (early 1980s) that includes financial and non-financial measurements.Multi-criteria hierarchical frameworks for maintenance performance measurement have been the focus of researchers since the early 2000s.These multi-criteria maintenance performance measurements integrate performance measurements from the strategic level down to the operational level, taking into account different stakeholders' views [4].This shift in maintenance performance measurement has also been fuelled by the broader paradigm shift within maintenance management [5].
Maintenance human factors, maintenance performance, and maintenance performance measurements are uniquely linked.The maintenance performance literature does acknowledge maintenance human factors; however, very few maintenance performance frameworks incorporate these human factors as measureable indicators.Measuring maintenance human factors can be seen as a leading indicator that can predict the quality of maintenance tasks, compliance with maintenance and safety procedures and policies, and the desire to meet performance targets.Kumar et al. [6] also stated that by adding additional categories for measuring human factors to traditional maintenance performance measurements, the uniqueness of maintenance performance measurements will be increased.
This paper focuses on maintenance human factors that influence the maintenance function's performance within the electricity transmission environment.For the purposes of this research, maintenance performance is defined as the ability of the maintenance function to control the cost of maintenance, extend equipment life, and increase safety.A maintenance performance measurement framework was developed to include maintenance human factors.A survey within Company 1 was used to evaluate the importance of the identified maintenance human factors.

Objectives
The primary objective of this research was to determine whether motivation and competence are the most important maintenance human factors influencing the maintenance function's performance within the electricity transmission industry.
The following secondary research questions were also posed:


What influence does the competence level of the maintenance staff have on the maintenance function?


What is the level of staff motivation while performing maintenance tasks? What is the maintenance staff's general perception of certain maintenance performance measurements?

Maintenance human factors
The quality, efficiency, and effectiveness of maintenance work are solely dependent on the maintenance worker executing the maintenance tasks.Acknowledging that there are factors that can influence the maintenance worker's state of mind is critical to all activities relating to maintenance planning and execution.Improving and predicting the maintenance worker's performance allows improvements and predictions to be made to the overall maintenance department's performance.Maintenance human factors can therefore be seen as a leading indicator for maintenance performance.
As presented in the Federal Aviation Administration's (FAA) PEAR model, the key focus points of a maintenance human factor programme within the aviation industry are the people who do the job, the environment in which they work, the actions they perform, and the resources necessary to complete the job [7].These four focus points have various subcategories, as shown in Table 1.

Competence
The Oxford English Dictionary [20] defines competence as "a basic or minimal ability to do something".McClelland (in Hoge et al. [9]) defines competence as "the knowledge, skills, traits, attitudes, self-concepts, values, or motives directly related to job performance".Rodriguez et al.
[10] define competence as "a measurable pattern of knowledge, skill, abilities, behaviour, and other characteristics that an individual needs to perform work roles or occupational functions successfully".Hongli [11] has a similar definition that relates competence to performance by defining competence as "a combination of knowledge, skills, behavior and other traits to improve performance".Lucia and Lepsinger [2] define competence as "a cluster of related knowledge, skills, and attitudes that affects a major part of one's job (a role or responsibility), that correlates with performance on the job, that can be measured against well-accepted standards, and that can be improved via training and development".
Although these definitions are based on similar principles, the definitions differ over the purpose of competence.This adds to the criticism that the word 'competence' is commonly used, but that its definition can be unclear or described as a fuzzy concept because different meanings of the word are experienced within different organisations [12,13].Le Deist and Winterton [13] add to this critique by comparing the historical background, definitions, and concepts of competence in the USA, UK, France, and Germany.
Competence management can identify organisational and employee knowledge, and can be used to determine strategies to bridge the gaps in the knowledge that the organisation and employees should have.This can empower the employees, promote innovation and effectiveness, and lead to increased competitive advantage [14].Competence models and frameworks are some of the main tools used in competence management.
A competence model, which is defined as a set of competencies that are required for performance [2,9], can be used for workforce planning, recruitment management, learning management, performance management, career development, and succession planning [14].
Measurement and recognition of maintenance staffs' competence can promote further competency development, and is regarded as a critical component of maintenance resources management [15,16].Components of competence can be recognised through qualifications, certification, training courses, and practical experience [15].Practical experience should not be disregarded when recognising competence.A survey of Swedish industries showed that 38 per cent of their maintenance workers did not have secondary school education; but they had gained practical experience through work-related activities and industrial courses [16].Record-keeping of the staff's competence can be used to identify scarce skills, competency gaps, knowledge transfer strategies, and resource allocation to maintenance tasks.
Managing maintenance staffs' competence can improve performance, efficiency, and service reactivity by reducing human error in maintenance operations, reducing maintenance rework, and reducing maintenance task duration [17,18].It is essential to manage staff's competence, as this will contribute to the total effectiveness of the maintenance department.
Some of the literature suggests a difference between 'competence' and 'competency', but the Oxford English Dictionary [20], Brown [19], and Le Deist and Winterton [13] suggest that the two are synonymous; and that is how they were understood in this research.Furthermore, for the purpose of this research, 'maintenance staff' refers to artisans, technicians, and engineers who are responsible for the maintenance tasks associated with high voltage (HV) or the secondary / control plant equipment commonly found in the asset base of an electricity transmission organisation.

Motivation
Robbins et al. [21] define motivation as "the processes that account for an individual's intensity, direction, and persistence of effort toward attaining a goal.Intensity has to do with how hard a person tries.Direction defines to what the effort is applied.Persistence is a measure of how long a person can maintain the effort".Motivation can be either intrinsic or extrinsic; intrinsic motivation is self-initiated, and extrinsic motivation comes from external factors such as financial benefits or praise.
Motivation can be fostered by employee involvement, extrinsic rewards (performance-related pay, bonuses, skill-based pay, profit-sharing, and cash alternatives), job satisfaction through job design, management communication and performance feedback, recognition, flexi-time, and telecommuting [3,22,23].Care should be taken when focusing only on the financial motivations method: the rewards might not form part of the employee's valence, and might therefore not address the person's intrinsic motivational needs [24].
Motivation within maintenance management is essential, as motivation can be used to improve the commitment of maintenance staff to maintenance actions, and increase their desire to achieve performance goals.

Maintenance performance measurements
Dwight [25] defined performance as "the level to which a goal is attained".He added that the problem with this definition is mainly that these goals need to be defined, and that they can be subjective.Performance measurements can be defined as "a measure equipped with baselines and realistic targets to facilitate prognostic and/or diagnostic processes and justify associated decisions and subsequent actions at appropriate levels in the organisation to create value in the business process" [26].
Specific drawbacks of maintenance performance measurements are maintenance objectives that are not linked to business strategies and maintenance performance measurements focusing on the operational view; and neglecting the influence of the organisation's maintenance policies and influences from other departments [6].Maintenance performance measurements can also focus on a variety of aspects, such as equipment performance, cost performance, process performance, the maintenance function, the maintenance work management cycle, and others [27][28][29].
Selecting maintenance performance measurements is mostly industry-specific, and care should be taken not to select unnecessary measurements: this could cause wasteful effort in data acquisition and analyses, and could hinder actual work from being done [6,30].Woodhouse [31] suggests that a maximum of six measurements should be used per supervisor/manager, and Kumar et al. [6] suggest that the measurements chosen should be the measurements that will have the biggest impact [6].
Thirty-two transmission maintenance departments from various countries were surveyed by Bodrogi et al. [32] and their key performance indicators were evaluated.The findings of the survey were that the most common KPIs were maintenance work-related -for example, maintenance completion -and that the most important measure was the overall result (reliability of the grid and the number of equipment faults).Other measures of concern were measures of cost effectiveness and the ratio between preventive and corrective maintenance [32].
As seen from the literature, there are several key performance measurements to measure either the maintenance function or maintenance performance in relation to production and manufacturing.The most comprehensive list and discussion of key performance measurements in maintenance is provided by Wireman [33].Some studies have also been done to determine the most commonly-used key performance measurements within the transmission sector [34].However, these frameworks lack the acknowledgement and measurability of maintenance human factors.

CONCEPTUAL MODEL
Various models have been proposed that incorporate maintenance performance measurements.Tsang et al. [35] developed a general maintenance model that takes into consideration different factors that influence maintenance performance.However, a holistic view of all the factors and their relationships is still lacking in this model.A conceptual model was therefore developed in this research to address this knowledge gap relating to human factors in maintenance performance.
Maintenance performance measurements give a quantitative value to maintenance performance.These quantitative values are used to determine whether the maintenance performance is adequate.A feedback loop from maintenance performance measurements to maintenance performance is created through maintenance resource management and maintenance human factors.
Maintenance performance measurements influence maintenance human factors through motivation and the expectancy theory.Positive performance results could be rewarded through either performance bonuses or intrinsic rewards such as job satisfaction, achievements, and the possibility of career advancement.Negative results could influence maintenance human factors if the maintenance staff perceive the maintenance performance measurements to be unattainable or unrealistic.
Maintenance resource management plays a crucial role between maintenance human factors and maintenance performance measurements through maintenance performance.Maintenance resource management manages the maintenance human factors in a positive way to improve maintenance performance, either through sound managerial principles or through procedures and policies such as high performance work systems or talent management.The improved maintenance performance is then seen in the improvement of maintenance performance measurements.Maintenance resource management principles are used to implement corrective actions that address maintenance human factors, should the performance not be adequate.
Figure 1 illustrates the conceptual model used in this research.Motivation and competence were chosen as the main focus areas.The first criterion for choosing the performance measurements was to identify leading performance measurements with the greatest impact on lagging performance measurements.The second criterion was that the supervisor or middle manager should have control over the factors influencing the performance measurement.The third criterion was that the information needed for the performance measure should be available either on an electronic management system or through the departmental manager.Muchiri et al.'s [29] framework was chosen for this research, since it related best to the lead author's model and to KPI trends in her current work place.Maintenance performance measurements meeting the above mentioned criteria were chosen from Muchiri's [29] leading performance indicators and lagging indicators that are illustrated in Table 2.

RESEARCH METHODOLOGY
The importance of maintenance human factors such as competence and motivation levels was evaluated by gathering empirical data.Empirical research is the predominant research method used in the social sciences, especially in the disciplines of organisational behaviour, psychology, and sociology [36].It is also gaining popularity in some engineering fields such as engineering management and industrial engineering.The research methodology used in this study is based on the systematic approach to empirical research, as developed by Flynn et al. [36] for operations management.
In theory, verification hypotheses are formulated and tested through data collection.For this research, the following hypothesis was formulated:  Competence and motivation are the most important maintenance human factors that influence the maintenance function's performance within the electricity transmission industry, compared with supervision, workload, and performance feedback.
This hypothesis was tested through analysis of data collected via a survey questionnaire.The survey respondents were homogeneous in the sense that all of them were responsible for maintenance work within the electricity transmission industry in South Africa.The types of maintenance for which these respondents are responsible are either HV plant or secondary plant maintenance.A survey questionnaire was compiled and sent to all HV plant and secondary plant maintenance workers within Eskom Transmission.The survey questions were categorised as: 1) general information, 2) competence, 3) motivation, and 4) performance measurements.
A pilot survey questionnaire was sent to selected maintenance staff.The respondents were asked to confirm the clarity of each question and to provide feedback on how much time was needed to complete the questionnaire.Feedback from the pilot questionnaire resulted in the questionnaire being revised to reduce the number of questions.
The questionnaire comprised 17 questions with 60 data fields.Ninety-eight respondents from the staff completed the questionnaire.A questionnaire was treated as incomplete if less than 21 of the 60 data fields were completed.The results of 21 respondents were subsequently removed from the survey, and the results of the remaining 77 completed questionnaires were used for data analysis.
The first group of questions on the survey established each respondent's age, gender, work experience within their present position, educational background, educational activities, certification status, and exposure to on-the-job training.A second group of questions aimed to determine the present motivation levels of the respondents, as well as their supervisor's contributions to their motivation levels.The last group of questions aimed to determine which factors are important to staff motivation, as well as which values are connected to different reward incentives.These questions were adapted from work by Robbins [21].Question 16 was used to determine the perceived importance that skill levels, motivation, supervision, workload, and feedback have on improving the respondents' work performance.The last question was used to evaluate the respondents' perceived importance of each of the proposed maintenance performance measurements.
All information gathered through the survey was exported from 'Kwiksurveys' to Microsoft Excel.The raw data file was modified by removing the responses of the participants who did not complete the survey.The modified Microsoft Excel file was then analysed by the University of Pretoria's Department of Statistics.

RESULTS
Using the KSAPs elements of competence (knowledge, skills, abilities, and personal characteristics), the workforce's educational levels and years in their present position were used to evaluate their knowledge elements.Certification levels were used to evaluate the ability elements.On-the-job training refers to both knowledge and ability elements of the maintenance staff's competence.

Competence
A slight majority of personnel have a moderate level of knowledge from work experience: 37.6 per cent of the workforce have been in their present position for between four and seven years, while 32.5 per cent are still relatively new in their present positions (i.e., between one and three years).
The majority of the workforce (55.8 per cent) have some form of an N-level qualification, which is typical for maintenance staff within the company.
For HV plant certification, 27.3 per cent of the staff have no certification and 39 per cent have basic certification (e.g., theory introduction and preventive maintenance).Less than 20 per cent of the staff are certified to perform major overhauls, and less than 7 per cent are certified as subject matter experts.Certification for secondary plant staff follows a similar trend.
The majority of respondents (76.6 per cent) received on-the-job training, with 57.1 per cent of respondents rating their training as 'good', 'very good', or 'excellent'.The minority of respondents (24 per cent) rated their on-the-job training as 'poor' or 'fair'.

Motivation
No definite conclusion can be made about the present motivational levels of the maintenance workers.About 57 per cent of respondents rated their present motivation levels as 'good', 'very good', or 'excellent', and about 42 per cent rated them as 'poor' or 'fair'.Even though the results indicate a slight majority of the employees feeling motivated, the percentage of non-motivated employees is significantly large.
A positive finding is that 70 per cent of the respondents stated that their direct supervisor plays a positive role in improving their motivation.Figure 2 ranks these motivational improvement strategies in terms of the percentage of respondents stating that the motivational improvement strategy was very important.
It is interesting to note that remuneration-related issues rank only number 4 and number 12 on this distribution.Opportunities for personal growth and development and for developing new skills and knowledge rank number 1 and 2 respectively as motivational factors.This strengthens the importance of competence as an important maintenance human factor that influences the maintenance function's performance.

Importance ranking of motivation and competence human factors
In the survey, the word 'skill' was used instead of 'competence'.The reason for this is to respond to political sensitivity within the organisation.The word 'competence' has a negative connotation for most staff members because of the negative use of the word 'incompetent' in the workplace.Table 3 illustrates the five maintenance human factors compared in the survey, and the order of importance these factors have for improving work performance, as stated by the survey respondents.

Performance measurements
The respondents were asked to rate the importance of the 20 maintenance performance measurements.Table 4 indicates the five performance measurements regarded as the most important by the respondents.The number of equipment failures was identified as the most important performance measure; together with the third and fourth most important performance measurements, this indicates a reactive maintenance culture, as immediate breakdowns (lagging indicator to equipment performance) take preference over work identification, work planning, and work scheduling, which are needed for a preventive maintenance culture.The third and fourth most important performance measurements relate to performance measurements used currently.Even though the only official performance measurement for maintenance is 'maintenance completion' (number of work orders completed / number of work orders issued), much focus is placed on backlog because supervisors use these backlog reports to focus their attention on identifying work orders that need to be completed.
The second most important performance measurement relates to skills (number of training [skill improvement] interventions / number of maintenance staff), which refers to the competence levels of the maintenance staff.This correlates with the workforce identifying 'skill level' as the most important factor to improve their work performance (refer to Table 3).This performance measurement, which is a maintenance human factor measurement, together with organisational safety culture (the fifth most important performance measure), indicates the importance of including maintenance human factors as part of maintenance performance measurements.
The fifth most important performance measure correlates with the strong safety culture of the workforce.This culture is due to the danger of working with electricity, as well as working at heights.All levels of management are committed to safety; they make safety part of every meeting, host safety forums, and arrange safety meetings at the beginning of each work day.
Table 5 indicates the ranking for work planning and work scheduling performance measurements, and Table 6 indicates the ranking of maintenance human factor performance measurements.3 per cent and 28.6 per cent of the maintenance staff do not have HV plant or secondary plant certifications, respectively.This could be attributed to the relatively new and inexperienced workforce; 32.5 per cent of the workforce have been in their current positions for only one to three years.
The survey results indicate that 39.0 per cent and 36.4 per cent of the respondents have only basic HV plant and secondary plant certification, respectively.The question can be asked: "Is this percentage high enough to counter the new and inexperienced workforce with only a small amount of expert knowledge?" The survey results also indicated that competence (skill level) was perceived to be the most important maintenance human factor that influences the maintenance function's performance within the electricity transmission industry, compared with motivation, supervision, workload, and feedback.Because of this perception, it is recommended that certification awareness be driven from top management downwards, and that incentive packages and career path advancement possibilities be given to maintenance staff who achieve higher levels of certification.An example of this would be to promote a technician to senior technician should they achieve major overhaul or advanced certification.It is also recommended that these strategies be followed in stages to increase the number of certified personnel to the appropriate levels.
The survey results indicated that motivation was perceived to be the second most important maintenance human factor that influences the maintenance function's performance within the electricity transmission industry, compared with competence (skill level), supervision, workload, and feedback.
The survey also indicated that 41.6 per cent of the respondents had poor or fair motivation levels.Non-motivated employees can sabotage plant equipment, increase the safety risks, and lower the morale of other employees.It is recommended that when the overall staff motivation levels are being tracked, the reasons for poor or fair motivation levels be investigated, in order to address these issues via the maintenance resource management system.
The most significant factors identified by the maintenance staff as contributing positively to their motivational levels were opportunities for personal growth and development, and developing new skills and knowledge at work.
These factors relate significantly to competence (skill level), which was the most important maintenance human factor perceived to influence the maintenance function's performance.Opportunities for personal growth and development that are reinforced by opportunities to advance were the second most important reward strategy identified by the maintenance staff.Maintenance resource management strategies to improve the maintenance staff's competence will create opportunities for personal growth and development, as well as opportunities to develop new skills and knowledge at work.
Alignment between the organisational human resource strategy and maintenance competency strategy should be done to ensure that the efforts of the maintenance resource management strategies are not lost.Assisting maintenance staff to obtain their BTech qualifications speaks of an organisation that provides opportunities for personal growth and development, and assists with developing new skills and knowledge at work.However, a higher qualification enables the maintenance worker to apply for higher positions within the organisation that are normally not within the maintenance department.By aligning the organisational human resource strategy and maintenance competency strategy, job grading and remuneration benefits can be restructured to allow for opportunities of advancement for BTech qualifications within the maintenance department.

Performance measurements
Training opportunities and certification levels ranked second and seventh respectively, when comparing the most important performance measurements indicated by the maintenance staff.This echoes the significance of competence as a maintenance human factor, as well as the most significant factors identified by the maintenance staff as contributing positively to their motivational levels: opportunities for personal growth and development, and developing new skills and knowledge at work.
The top five maintenance performance measurements are a reflection of the top management's present priorities.Work identification, work planning, and work scheduling performance measurement ranked 9 th , 10 th , 12 th , 19 th (second last), and 20 th (last) respectively.This indicates a lack of focus on these activities, which are crucial to moving from a reactive maintenance culture towards a preventative maintenance culture.Maintenance staff, supervisors, and middle management can focus on these activities and can create an awareness of the importance of these activities; but without support and strategies driven by the top management, the chances of a successful maintenance culture change are small.

Implications for and/or contributions to theory and practice
Maintenance human factors were ranked as the second most important maintenance performance measurement out of the 20 factors mentioned in the survey.This illustrates the importance of maintenance human factors in the electricity transmission industry.The results also confirm that competence (skill levels) and motivation are the most important maintenance human factors that influence the maintenance function's performance within the electricity transmission industry, compared with supervision, workload, and feedback.
The results from the survey indicated the present certification and motivation levels and the general perception of maintenance performance measurements by the maintenance staff.The insight gained into the psyche of the maintenance staff can be used to create effective strategies to increase the maintenance staff's level of certification and to identify factors that could be used to increase the maintenance staff's overall motivation levels.

Recommendations
1) It is recommended that the motivation and competence levels (qualifications and certifications) be tracked on a bi-annual cycle, together with the maintenance performance results.The recorded values can be used to create corrective maintenance resource management strategies should the staff's motivation levels decline, and to provide information on whether the maintenance competency improvement strategies are increasing the staff competence levels.This information will also allow the following hypotheses to be tested:  H0: There is no correlation between the maintenance staff's competence and motivation levels within the electricity transmission industry and the associated maintenance performance. H1: There is a positive correlation between the maintenance staff's competence and motivation levels within the electricity transmission industry and the associated maintenance performance.
2) It is also recommended that a comparison be done of the importance of competence and motivation to maintenance human factors from other categories, such as the number of people involved in maintenance task (action), and test equipment and tools available for maintenance tasks (resources).
3) It is also recommended that minimum (baseline) specific measurable standards be set for motivation and competence levels within the electricity transmission industry.

Figure 1 :
Figure 1: Relationship between maintenance human factors, maintenance performance measurements, and maintenance performance Muchiri et al. [29] developed a framework for the maintenance function aligned to manufacturing objectives.The framework comprises three categories: maintenance strategy formulation, maintenance effort/process, and maintenance results.Muchiri et al. [29] also provided a list of 17 leading performance indicators and 14 lagging performance indicators.In order to adapt these performance indicators, maintenance performance measurements that meet the three criteria mentioned in the next paragraph were chosen.

Figure 2 :
Figure 2: Importance of motivation improvement strategies