INTEGRATING QUALITATIVE AND QUANTITATIVE FACTORS IN SUPPLIER SELECTION AND PERFORMANCE EVALUATION

Companies need to cooperate with reliable and high-performing suppliers of goods and services, and build mutual long-term relationships in order to sustain their existence and increase their market share. An engineering manager can determine the most effective strategy in supplier selection by the proposed model, which allows an integration of objective values and the uncertainty of human judgements. A two-stage integrated fuzzy analytic network process (FANP) and super-efficiency data envelopment analysis (DEA) model is proposed and applied to a wind turbine project of a company operating in the renewable energy sector. At the first stage of the methodology, the weights of the qualitative criteria were determined by using FANP. At the second stage, the qualitative supplier selection criteria that were converted into quantitative criteria with FANP were used as outputs; and cost (which was already a quantitative criterion) was used as an input in the DEA, by way of which the performances of the suppliers were assessed. This study, which includes decision-making, performance management, and supply chain management issues, introduces a new approach, both in terms of the sector referred to and the methodology that enables companies to select the best suppliers by assessing both qualitative and quantitative factors.


INTRODUCTION
Companies should determine the best and most effective strategy in choosing reliable and highperforming suppliers with whom to establish long-term relations, since this relationship ensures that they have a competitive advantage.An effective supplier selection not only reduces the operational costs of a company, but also improves its operational quality.In addition, product development times and lead times can be decreased, thereby contributing to the competitiveness of the company [1].Therefore, supplier selection is no longer a matter of making a 'simple' decision based on price alone: it now requires the assessment of several additional and more complicated qualitative and quantitative factors [2].Companies should have an effective performance evaluation system in order to create efficient supplier networks.While establishing this system, it should be ensured that the performance of existing and potential suppliers is continuously evaluated by combining the human factors and the basic management functions with engineering, and modelling the random and variable structure of the system.
Recent studies confirm that more comprehensive supplier selection multi-criteria decision-making and performance assessment techniques, in which several qualitative and quantitative factors are integrated, are more commonly used.This paper proposes a two-stage integrated method for effective supplier selection, which is a highly complicated process.A fuzzy analytic network process (FANP) was used at the first stage of this method, while a data envelopment analysis (DEA) model was applied at the second stage.The relevant supplier selection criteria were identified initially.
With FANP, qualitative supplier selection criteria were converted into quantitative criteria, and these constituted the data used as the outputs of the DEA method.On the other hand, cost, which is an intrinsically quantitative item, constituted the input of the DEA, through which efficient and inefficient suppliers were identified.Finally, super-efficiency scores were calculated and the efficient suppliers were ranked.
FANP was used at the first stage, when the weights of the criteria were determined; thus the network structure that was created could reflect the interaction both between and within the criteria and their feedbacks in the model.In this model, where fuzzy logic was applied, the ambiguous and risky structure of the criteria-setting process relying on human judgement was defined and determined more accurately.At the second stage of the method, the performances of the suppliers were assessed using a super-efficiency DEA model; hence, those criteria with different measurement units could be used in combination.The super-efficiency model also enabled the ranking of suppliers according to efficiency, and the most efficient supplier was presented to the company.
The proposed method was applied to the supplier selection problem for a wind turbine project of a company operating in the renewable energy sector.Studies on supplier selection reveal that the focal point is mainly mass production.Nevertheless, it is equally important for project-oriented companies to identify the most efficient suppliers.This paper not only offers a new approach -in both the model it proposes and the sector to which it is applied -but it also enables projectoriented companies to evaluate qualitative and quantitative factors together.As such, this paper provides valuable insights for engineering managers that would help them to improve their supply chain management performance by evaluating the performance of suppliers.
The rest of the paper is organised as follows.In the second part, a review of the relevant literature is presented.It is followed by the methodology section, in which the procedure of the proposed FANP-DEA model and the steps of the process are described first.Then the FANP and DEA models are introduced.In the fourth section the steps of the proposed model are applied to the energy company.The paper ends with the fifth section, where the conclusions and suggestions for future studies are presented.
The review of the relevant literature shows that the one-stage methods have gradually been abandoned and that two-stage integrated methods, in which both qualitative and quantitative factors are assessed, have become more common.Authors such as De Boer, Eva and Morlacchi [31], Ho, Xu and Dey [2], Jain, Wadhwa and Deshmukh [32], and Chai, Liu and Ngai [33] conducted detailed studies on supplier selection.Between 2000 and 2008, 17.95 per cent of the papers published were about supplier selection using the integrated model [2], while this rate rose to 39.84 per cent between 2008 and 2012 [33].In subsequent years there were hardly any studies that used a single decision-making (DM) model.
In this study, the supplier selection was carried out with FANP at the first stage and then DEA methods at the second stage.Ho et al. [2] analysed the papers published between 2000 and 2008 that were conducted on supplier selection, and found that 14 papers out of 78 used DEA, while three papers used ANP.Chai et al. [33] also analysed papers on the application of DM in the selection of suppliers between 2008 and 2012, and found that 15 papers out of 123 used ANP, whereas 13 papers used DEA.Although there are not as many studies in which ANP and DEA are integrated for supplier selection, the number of such studies has increased in recent years.Some of these are as follows: ANP and DEA methods were integrated for the selection of supply chain partners at small and medium-sized companies [34], for supplier selection at a manufacturing firm [35], and for green supplier selection [36].In Abdollahi, Arvan and Razmi's [37] study, the authors categorised the suppliers as 'lean' and 'agile', and proposed some suggestions for supplier relationship management (SRM).They used the fuzzy decision-making trial and evaluation laboratory (fuzzy DEMATEL) to define the interdependence between the criteria, ANP to find the weights of sub-criteria, and DEA to rank the suppliers.Che and Chang [38] developed a supplier selection method that they named 'hyADMOPSO', in which ANP, DEA, and multiple objective particle swarm optimisation methods were integrated.
This study proposes a model in which FANP and DEA methods were integrated for supplier selection in a wind turbine project.To the best of our knowledge, no other studies have integrated FANP and DEA to model supplier selection; nor are there any studies that have applied supplier selection methods in the renewable energy sector.However, various DM methods have been used for different purposes in the energy sector.For instance, Kaya and Kahraman [39] selected the best renewable energy source for an Istanbul production site by integrating the 'vise kriterijumska optimizacija i kompromisno resenje' (VIKOR) and AHP methods.Cristobal [40] determined the best renewable energy project for Spain through the VIKOR method.Biomass was found to be the best source of energy, which was followed by wind and solar thermo-electric power.Chen, Kang and Lee [41] found the best solar-wind power generation project by applying FAHP.Lee, Chen and Kang [42] proposed a novel AHP-based approach to find the optimum wind farm project.Büyüközkan and Güleryüz [43] integrated DEMATEL and ANP models to determine the optimal renewable energy source; while Yazdani-Chamzini, Fouladgar, Zavadskas and Moini [44] found the optimal source of renewable energy through an integrated COPRAS-AHP model.Pan, Zhang and Zhang [45] measured the industrial energy efficiency of China through a two-stage method: they assessed regional energy efficiency by applying DEA at the first stage; while at the second stage they analysed the energy efficiency variables through panel data analysis.Akbari, Irawan, Jones and Menachof [46] integrated AHP and FANP methods and determined the most appropriate location for off-shore wind farms.Ghosh et al., [2016] [47] found the most suitable location to generate energy from ocean waves by integrating AHP and artificial neural networks (ANN) methods.

Proposed procedure for supplier selection and evaluation
This article suggests a two-stage method using qualitative and quantitative criteria simultaneously in the determination, selection, and evaluation of suppliers.In the first stage of this method, FANP is used to determine the relative weights of qualitative supplier selection criteria.In the second stage, these qualitative factors are converted into quantitative ones, which are then used as the output in the DEA model.By taking these qualitative factors as outputs and a quantitative factor as an input, the super-efficiency scores of suppliers are obtained by DEA, which allows efficient and inefficient suppliers to be ranked.Figure 1 illustrates the proposed method for supplier performance evaluation.The evaluation process is described as follows: Step 1: Decision-maker's wind turbine supplier selection criteria are identified.After determining both the quantitative and the qualitative supplier criteria with the help of the decision committee, the process is continued with the steps that follow.
Step 2: The FANP model is structured according to its goal, factors, and sub-factors determined in the previous step.
Step 3: Pair-wise comparison matrices are formed by the decision committee using a linguistic scale for importance (given in Table I, as defined by Kahraman, Ertay and Buyukozkan [48]).Step 4: The local weights of factors and sub-factors are calculated by Chang's extent analysis method [49,50].
Step 5: The inner dependence matrices and relative importance weights of the criteria are determined.The impact of each criterion on each of the others is assessed with paired comparison matrices, by way of which the interdependence of the criteria is described.
Step 6: The relative importance weights of the criteria derived from the inner-dependency matrices and the local weights are multiplied together, and interdependent weights are obtained.
Step 7: The normalised interdependent weights are multiplied by the local weights of the subcriteria to calculate their global weights.
Step 8: The criteria weights of the suppliers are determined for the sub-criteria of each criterion by comparing the suppliers.While creating the pair-wise comparison matrices of the suppliers, the linguistic scale given in Table 1 is also used to determine their relative priorities.
The paired comparison values of the suppliers are normalised and their relative weights are calculated [51].Then a consistency ration (CR) that measures the consistency of the comparisons made by the decision-maker is calculated.If CR is smaller than 0.10, it means that it is an indication of consistency.The consistency index (CI) and consistency ratio (CR) are defined as [52]: where λ max is the greatest eigenvalue of the matrix, n is the number of items compared in the matrix, and RI is the random index.The values of RI [52] are given in Table 2. Step 9: The final weights are calculated by multiplying the criterion weights of the suppliers by the global weights of the sub-criteria.The criteria obtained are used as outputs in the DEA analysis.
Step 10: The quality, risk, and service criteria obtained from FANP are used as outputs; and the cost criterion is used as an input for DEA analysis.The performances of the suppliers achieving superefficiency scores are assessed and ranked.
The next two sub-sections present the FANP method that was used at the first stage of the study, and the DEA method that was applied at the second stage.

Fuzzy analytic network process (FANP)
The analytic network process (ANP) method that was developed by Saaty [53] is the generalised version of the analytic hierarchy process (AHP), which was also developed by Saaty [52].Both methods are multi-criteria decision-making methods that allow the assessment of several alternatives by taking account of many criteria.However, AHP establishes a one-dimensional hierarchy between the criteria, sub-criteria, and alternatives, whereas ANP describes a multidimensional dynamic relationship.Just like many other decision-making problems, supplier selection also requires a model in which different needs and desires are assessed in an integrated manner and there is a mutual interaction between the criteria.ANP provides the possibility to reflect the outcome of the dependence and feedback within and between clusters of elements.
Since human perception and decisions always contain uncertainty and vagueness, FANP is more appropriate instead of conventional ANP.Fuzzy logic was first used by Zadeh [54] to define and measure uncertainty.Fuzzy set theory is suitable for transforming uncertainty and knowledge-inpieces that relies on personal judgements obtained from decision-makers to several parameters [55].
In cases where it is difficult to define the perceptions using crisp numbers, as in the case of supplier selection, fuzzy numbers and linguistic variables allow more reliable results to be obtained [56,57].
In this study, fuzzy pair-wise comparisons are calculated using Chang's extent analysis method [49,50].
The criteria set is x = {x 1 , x 2 , … ., x n } and the goal set is G = {g 1 , g 2 , … .g n }.The extent analysis is applied to each criterion for each goal and g i values are created.Thus, m extent analysis values for each object can be obtained with the following symbols: where all the M gi 1 (j = 1,2, … m) are triangular fuzzy numbers (TFNs).The steps of Chang's extent analysis [49,50] can be given as below [12,48,58,59]: Step 1: The value of fuzzy synthetic extent with respect to the i.th object is defined as: To obtain ∑ M gi j m j=1 , perform the fuzzy adition operation of m extent analysis values for a particular matrix, such that: , perform the fuzzy additional operation of M gi j (j = 1,2, … , m) values, such that: After that, compute the inverse of the vector in Equation 6as Step 2: The degree of possibility of M 2 = (l 2 , m 2 , u 2 ) ≥ M 1 = (l 1 , m 1 , u 1 ) is defined as: , ℎ where d is the ordinate of the highest intersection point D between μ M 1 and μ M 2 , and hgt(M 1 ∩ M 2 ) is a separation index for two fuzzy numbers.To compare M 1 and M 2 , both V(M 1 ≥ M 2 ) and V(M 2 ≥ M 1 ) values are needed.The intersection between M 1 and M 2 is seen in Figure 2. Step 3: The degree of possibility for a convex fuzzy number to be greater than  convex fuzzy numbers, M i (i = 1,2, . ., k) can be defined as for k = 1,2, ⋯ , n; k ≠ i.Then, the weight vector is given by where A i (i = 1,2, … , k) are  elements.
Step 4: Using normalisation, the normalised weight vectors are where W is a non-fuzzy number.

Data envelopment analysis (DEA) and super-efficiency model
Data envelopment analysis (DEA) is a mathematical programming method that measures the relative efficiency of decision-making units (DMUs) using multiple inputs and outputs that have different measurement units.DEA was first introduced by Farrell [60].Charnes, Cooper and Rhodes [61] developed the CCR-DEA model based on the constant returns to scale (CRS) assumption, and Banker, Charnes and Cooper [62] developed the BCC-DEA model based on the variable returns to scale (VRS) assumption.In DEA, which is not a parametrical method, those DMUs that are on the efficiency frontier are considered to be relatively efficient, and are assigned a '1'.The CCR-DEA model is based on the CRS assumption, and assumes that all DMUs operate at the optimal scale.The input-oriented CCR-DEA model used in this study is formulated by equation ( 13).
The dual of the linear programming model in the equation ( 13) is as follows [64]:

r θ o unconstrainted
The super-efficiency (SE) DEA model used in this paper was proposed by Andersen and Petersen [65], and aims at ranking the DMUs that are found to be efficient with a value of '1', using the CCR-DEA, BCC-DEA, or other DEA models [66].In the super-efficiency model, every efficient DMU is removed from the efficient production frontier, and the distance of the removed DMUs to the efficiency frontier that is redetermined is measured.
These scores are the super-efficiency scores of the efficient DMUs, and have a value greater than 100 per cent [67].This allows the ranking of the efficient DMUs among themselves

RESULTS
The case study for the application of this two-stage model was conducted in an engineering company in Ankara, Turkey that specialises in renewable energy projects, such as wind power plants, geothermal energy, and solar power.In this study, supplier selection through the evaluation of their performances was carried out for wind power plant projects.In wind power plant projects, turbine suppliers are liable not only for the procurement of turbines, but also their transportation, installation, and operation.There are nine main firms that supply wind turbines in Turkish market.
In the study, these nine suppliers' performances were evaluated by FANP and DEA.All of this process was carried out with the decision committee of the wind power plant department of the company, consisting of three individuals.
Step 1: An extensive list of supplier selection criteria was submitted to the decision-making team in the wind power plant department to determine the criteria for selecting the best suppliers.The team agreed on four main criteria as quantitative and qualitative variables, with their sub-criteria, which can be seen in Table 3.
Step 2: At this stage, the FANP model was created by using the factors that were defined in the previous steps.The FANP model that was devised to weight the selection criteria of suppliers is seen in Figure 3.The model comprised three steps.The purpose was described in the first step, while the criteria and sub-criteria were identified in the second and third steps respectively.The curves in the second step show the inner-dependence between quality, service, and risk.In this way, the model allowed for the analysis of the impact of the factors on each another.Quality standards that the company expects from the supplier.Quality assurance (C11) Quality issues committed to, followed, and supervised and controlled by the supplier Step 3: The pair-wise comparison matrices (see Table 4) were obtained from the opinion of the decision-making committee.The matrices were obtained by using a linguistic scale for importance (given in Table 1), as defined by Kahraman et al. [48].

Figure 3: FANP model of supplier selection criteria
Step 4: The local weights of the main criteria and their sub-criteria, which were calculated using Chang's extent analysis method [49,50], are given in Table 4. Step 5: In this step, pair-wise comparison matrices were obtained, based on the dependencies represented in the second part of Figure 3.The inner dependence matrices and the computed relative importance weights of the criteria are given in Table 5. Step 6: The interdependent weights were calculated by multiplying the relative importance weights of the criteria by the local weights, and then interdependent weights were normalised.The values obtained were 0.44 for quality, 0.26 for service, and 0.31 for risk.

Table 4: Local weights and pair-wise comparison matrices of criteria and sub-criteria
Step 7: The global weights of the sub-criteria were obtained by multiplying the weights of the criteria by the local weights of the sub-criteria (Table 6).The results revealed that 'quality' was the most important criterion in the selection of wind turbine suppliers, followed by 'risk'.The 'service' criterion, on the other hand, had the lowest weight.The most important sub-criteria of quality were 'product quality' and 'quality assurance'.'Financial stability' was the sub-criterion of risk with the highest weight, and 'on-time delivery' was the subcriterion of service with the highest weight.
Step 8: Pair-wise comparisons between the suppliers were performed for each sub-criterion, using the linguistic scale (Table 1).The criterion weights and CR are presented in Table 7.The consistency rates were lower than 0.10, and the comparisons made by the decision-maker were consistent.

S: Supplier
Step 9: The final weights were calculated by multiplying the suppliers' criterion weights by the normalised values of the global weights of the sub-criteria.The final weights are presented in Table 8.In this way, the qualitative data to be used as output in DEA were converted into quantitative data.Step 10: The quality, risk, and service factors obtained in FANP were used as outputs, and cost was used as input in DEA analysis.The risk factor was subtracted from "1" and converted into the risklessness variable.The data of the DEA model are presented in Table 9.In the study, the performances of the suppliers were assessed with the input-oriented CCR-DEA analysis.The EMS Version 1.3 computer program developed by Holger Scheel was used to solve the model and calculate the super-efficiency scores.
Table 10 shows the super-efficiency scores obtained from DEA.In this study nine suppliers were assessed; S2, S4, S5, S6, and S8 were the efficient suppliers.These suppliers were chosen by the company because they produced the maximum amount of output at the lowest cost.It was possible to rank the efficient suppliers using the super-efficiency scores.S8 had the highest performance with 110.38 per cent, followed by S6 with 109.62 per cent and S4 with 104.55 per cent.Suppliers that had an efficiency score greater than '1' could continue to be efficient if they increased their inputs.Therefore, even if S8 increased the cost by 10.385 per cent, it would still be an efficient supplier.
The other suppliers did not operate efficiently.The least-efficient supplier, S9, had an efficiency score of 75.80 per cent.This meant that this supplier could obtain the same output even if its cost were reduced by 24.2 per cent.

CONCLUSION
In today's fiercely competitive markets, companies can achieve a substantial competitive advantage if they work with reliable and high-performing suppliers that offer high-quality products and services.A decision-maker should establish a performance evaluation system for the selection of efficient suppliers, and should determine the most effective strategy in choosing suppliers through the proposed model, which allows for the integration of objective values and the uncertainty of human judgements.This study has proposed a model that engineering managers can use to select suppliers efficiently using qualitative and quantitative criteria simultaneously.Additionally, the study offers engineering managers an opportunity to improve the performance of supply chain in respect of supplier selection by interpreting the results of the model.
In this study, a two-stage integrated FANP-super-efficiency DEA was proposed for the effective selection of suppliers.Qualitative and quantitative factors were assessed in an integrated way in this method, which was used to select the most appropriate supplier for a wind turbine project of a company that operates in the renewable energy sector.First, the decision-makers of the company identified the main criteria and sub-criteria for the selection of suppliers for its wind turbine project.At the first stage of the model, the non-linear network structure of the qualitative criteria was created in combination with the sub-criteria, and their weights were calculated through FANP.Due to this network structure, the interactions and feedbacks both between and within the criteria were represented to the model.The uncertainties associated with these qualitative factors, which were derived solely from human judgement, were modelled with fuzzy logic; in this way they were weighted more accurately.By modelling the uncertainty, risk, and variability with the help of FANP in the proposed model, it is possible effectively to convert qualitative factors to quantitative factors.
A manager can combine these acquired factors by the quantitative values to choose the best supplier.Thus it is possible to establish an efficient performance system that includes a set of variables instead of only the cost factor.
At the second stage of the model, the efficient and inefficient suppliers were identified through DEA analysis.Service, quality, and absence of risk -the qualitative supplier selection criteria that were transformed into quantitative data using FANP -constituted the outputs of DEA.Cost was taken as the input of DEA as a quantitative datum.In this study, in which nine suppliers were assessed, S2, S4, S5, S6, and S8 were found to be efficient.These suppliers were chosen by the company because they produced the maximum amount of output at the lowest cost.The performances of these suppliers were ranked using their super-efficiency scores.S8 had the highest performance with 110.38 per cent, followed by S6 with 109.62 per cent and S4 with 104.55 per cent.
It is thought that decision-makers who implement this two-stage integrated model in the supplier selection process will choose more efficient suppliers, and the model will allow suppliers to enhance their performances.Engineering managers who are aware of the importance of managing the supply chain effectively should evaluate the performance of their current suppliers, and incorporate the most appropriate ones into their processes.Thus this study presents decision-makers with a road map.
The paper provides valuable insights for decision-makers in that the performance of a supply chain depends on the selection of the most appropriate supplier for the company.This study, which includes decision-making, performance management, and supply chain management issues within the scope of engineering management, introduces a new approach both in terms of the sector referred to and the methodology, and enables companies to select the best suppliers by assessing both qualitative and quantitative factors.It presents a significant contribution to the literature for managers to implement the proposed model effectively.
The model is applied to a wind turbine project of a company operating in the renewable energy sector.It would be useful also to examine different companies in order to make a generalisation about the sector.It is suggested that supplier selection efforts should be undertaken at projectoriented companies in future studies.Moreover, new models and practices that integrate qualitative and quantitative data in supplier selection in all sectors will also have great importance.In addition, this study can be extended by including other sectors as well.To the best of our knowledge, this is the first study that applies the integrated FANP-super-efficiency DEA model to the supplier selection problem.

Figure 1 :
Figure 1: An integrated FANP-DEA procedure for supplier evaluation

Figure 2 :
Figure 2: The intersection between   and . The values assigned to the inefficient DMUs in the standard model remain unchanged.The DEA model is equivalent to the dual CCR-DEA model, except for the fact that DMUs are excluded from the reference set.The super-efficiency model is presented below [65]: min θ o ..∑      +   − =     ,  = 1, … ,  (15) ∑      −   + =   ,  = 1, … ,    ,   − ,   + ≥ 0, ∀, , to solve quality problems Product quality (C13) Turbine production efficiency, material type, and design Rejection rate (C14) Percentage of damaged or non-functional components of the turbines Service (C2) Quality of the service provided by the supplier during installation and operation On-time delivery (C21) Delivery of the components on due dates Technical support (C22) Technical support provided by the supplier on time and effectively in case of a failure Response to changes (C23) Capability of the supplier to respond to change based on firm's demand, price structure, order frequency, and current business scenario Warranties (C24) Length of warranty period and its scope Risk (C3) Risk factors that may arise for various reasons throughout the whole process.Companies expect suppliers to minimise the risks and maximise the total utility Concept conflict (C31) Possible disagreements with the supplier Geographical location (C32) Location of the wind plant and the possibility of natural calamities affecting the supply of the components Political stability (C33) Political stability of the supplier's country and its economic policies Financial stability (C34) Financial status of the supplier based on its history Foreign exchange rate (C35) Currency fluctuations during the payment period that may affect the customer

Table 3 : Quantitative and qualitative variables
Local weights and pair-wise comparison matrix of main criteria