Data envelopment analysis in performance measurement : a critical analysis of the literature ”

This study examines the benefits of data envelopment analysis (DEA) in evaluating the performance of decision making units (DMUs). DEA is a mathematical programming tool applied in performance measurement. The problem identified is establishing business support units as value adding business units. A case is made for applying DEA when evaluating the performance of such business support units. To this end, a literature review of the results of applications of DEA to the evaluation of information technology and purchasing supply chain management functions was conducted. The findings indicate the benefits of DEA are that the method identifies efficient performers in a given population and, therefore, allows for benchmarking against the ’best in class’ performer. This as opposed to more commonly used parametric methods, such as regression analysis, which result in a comparator that represents the average performance for a given population, therefore, allowing only for measurement against the average. In addition, the findings indicate that in respect of business support units, the DEA methodology allows for the incorporation of intermediate outcomes, which facilitates the measurement of the contribution of these units to overall company performance. Although the DEA methodology has been widely applied, it is still not as well known or generally applied as the more common approaches. The recommendations made in this paper will be beneficial in bringing DEA to the attention of decision-makers. The recommendations will also raise awareness of the potential benefits to be realised when applying the method in developing performance measurement frameworks for business support units.


Introduction 
There is an old management saying that you 'can't manage what you don't measure'.Neely (2004) indicated that having appropriate performance measures in place facilitates the communication of a well-defined structure for moving towards achieving an organization's goals and targets.Saranga and Moser (2010) indicate that increasing global competitiveness is forcing companies to cut costs and develop operational excellence.They state that to achieve this it is necessary to structure, develop and manage organizational activities in line with organizational objectives.However, a challenge in respect of business units designated as support functions, is establishing their direct value added to overall corporate financial performance.This is cited as a key objective of senior management, and therefore a necessary focus in developing a performance management system for such units (Saranga and Moser, 2010).
In order to utilize performance measurement to establish business units as value adding, it is necessary to go beyond merely measuring performance to analyze that performance in such a way as to be able to demonstrate that the function adds value.In this respect, Neely (2004) has identified a key challenge in managing through measurement as being to shift the focus to the targets.He indicated that where managers are presented with large amounts of raw performance data, they tend to focus on justifying individual figures, as opposed to learning from the current situation and applying this to identifying how the targets can be achieved.Neely proposed that managers need to be educated in how to present the data in such a way as to promote such discussion.This paper examines the potential for utilizing data envelopment analysis (DEA) for this purpose.The benefits of applying DEA in evaluating and analyzing the performance of decision making units (DMUs) are outlined.In particular, a case is made for applying DEA when evaluating the performance of business support units, by critically examining examples of research into its application to such units.The research methodology is, thus, archival, using secondary sources of information.

Development of data envelopment analysis.
DEA is a mathematical programming tool that can be applied in performance measurement and analysis.Cooper Seiford and Zhu (2004) define DEA as 'a relatively new "data orientated" approach for evaluating the performance of a set of peer entities called Decision Making Units (DMUs) which convert multiple inputs into multiple outputs' (p. 1).They report that DEA in its current form was first introduced in 1978, and has since been recognized as an excellent methodology for performance evaluations.As such, DEA has been used in evaluating the performance of many different types of business units and activities in the ensuing years.The term DMU was used to allow for the model's application to a wide variety of activities, including governmental, not-for-profit and business units and sub-units.
The original Charnes, Cooper, and Rhodes (CCR) DEA model (Charnes, Cooper and Rhodes, 1978) utilizes linear programming to produce an efficiency measure for a DMU, requiring only that the DMUs convert similar inputs to similar outputs and that these can be quantified.Elkins (2003) outlined the logic of the model as follows, first, defining the underlying premise that efficiency is the sum of weighted outputs over the sum of weighted inputs.He, then, outlines the DEA model formulation for the k th DMU as follows:

m, where:
Objective function E k = the efficiency index of the k th DMU; Parameters y rj = the amount of the r th output for the j th DMU; x ij = the amount of the i th input for the j th DMU; t = the number of outputs; m = the number of inputs; and n = the number of DMUs Decision variables u r = the weight assigned to the r th output; and v i = the weight assigned to the i th input.
With this model, the weights that maximize the efficiency of DMU k are objectively determined.The three constraints ensure, firstly, that the efficiencies of all the DMUs do not exceed one and, then, that the weights assigned are all non-negative.Elkins (2003) reports that in terms of the model any DMU measured as having an efficiency score of one is considered relatively efficient, and any with a score less than one is relatively inefficient.This means that either its outputs could be increased without increasing inputs or inputs could be decreased without decreasing outputs.Cooper et al. (2004) reported that since the original CCR DEA model was first introduced, over two thousand articles had appeared in the literature, outlining its application in numerous studies in the non-profit sector, in the public sector and in the private sector.Tavares (2002), in his bibliography of DEA literature 1978-2001, referenced 3 203 publications.Cooper et al. (2004) also make reference to a number of enhancements to the original model that have appeared in the literature.These are said to include the facility to include non-discretionary inputs and outputs in the model, and an extension to allow for the investigation of efficiency changes over time.Cooper et al. (2004) cited the most significant extension as having been to allow for judgement or prior knowledge to be incorporated into the model, for example, to incorporate strong preferences that managements may have regarding the relative importance of different factors.
1.2.The advantages of DEA in performance measurement.Cooper et al. (2004) indicate that the advantages of DEA that have led to its wide use, include its empirical orientation and the lack of a need for prior assumptions that are inherent in other approaches, such as statistical regression analysis.Furthermore, they report that studies of benchmarking practices using DEA have shown inefficiencies in some of the most profitable firms, and it has, therefore, been found to provide a better vehicle for establishing benchmarks than using profitability as a criterion.Charnes Cooper Lewin and Seiford (1994) describe DEA as an alternative to parametric approaches to eliciting information about a population of observations.They indicate that as opposed to parametric approaches, whose aim is to regress a single optimum plane through the data, the objective of DEA is to optimize each observation in order to calculate a 'discrete piecewise frontier determined by the set of Pareto-efficient decision making units (DMUs)' (p.4).This is achieved through calculating a maximum performance score for each DMU in relation to all other DMUs in the population, with the stipulation that each DMU lies on or beneath the external frontier.Charnes et al. (1994), then, assert that the result is, instead of arriving at a single optimized regression equation representing an average DMU, DEA results in an understanding of the relative performance of each DMU.This is illustrated by Charnes et al. (1994) in the figure reproduced below (Figure 1).Charnes et al. (1994), DEA uses the actual observed inputs and outputs of each DMU to calculate the efficiency of each DMU relative to all other DMUs in the population.The DEA calculations then result in a relative efficiency score for each DMU.Furthermore, DEA produces a piecewise efficient frontier, which represents the frontier of best practice for the observed population.This, in turn, represents the maximum output that can be expected from any DMU in the population given the level of its inputs.Chen and Zhu (2004) describe the efficient frontier as representing the most efficient trade-off between the multiple input and output performance measures.They indicate that the frontier, in turn, allows for any current inefficient performance to be identified and improvements suggested that would move DMUs currently below the frontier onto the efficient frontier.Furthermore, they indicate that DEA does not require there to be a direct match between the inputs and outputs measured, that is for specific inputs to be directly related to particular outputs.They explain that the DEA model effectively treats the production process as a 'black box', simply considering the resources available to a business unit (the inputs) and measuring effectiveness of their conversion into desired outputs.

As indicated by
DEA has also been compared to the use of simple ratio analysis in evaluating efficiency.In one such study which concerned the efficiency of Greek commercial banks, Halkos and Salamouris (2004) noted that the advantage of using DEA as compared to the financial ratios was that DEA provided a single objective score, a ranking and the potential targets for improvements for each of the business units determined as inefficient.Particularly, DEA allowed them to compare efficiency whilst accommodating multiple criteria, by combining a range of elements of efficiency into a single performance measure.They concluded that ratio analysis and DEA should be used as complementary measures and be used in conjunction with each other.
Criticisms of traditional financial ratio measures of bank performance noted by Halkos and Salamouris include that they are based on accounting data, which does not necessarily reflect the current market value of the bank, also they do not consider that the price of inputs relative to the mix of outputs and that the weights ascribed to the different financial ratios is subjective.However, it is noted that in respect of the reliance on accounting data, this criticism can also be levelled at DEA (Halkos and Salamouris, 2004).
1.3.DEA in performance measurement of twostage processes.DEA as originally developed was designed to measure the efficiency of business systems as a whole, without considering the internal structure of the business, often referred to as a 'black box' approach.This approach suggested that within the system, inputs were provided to produce outputs, with generally a positive correlation between such inputs and outputs.However, some evidence arose to suggest that this was not always the case, and that in order to understand the efficiency of a DMU, it was necessary to study the efficiency of its component processes (Kao, 2014).One of the first papers to address this issue was that of Wang Gopal and Zionts (1997) who used DEA to assess the marginal benefits of information technology as pertains a two-stage process, on corporate performance in the banking industry.They took the simple approach of separating the business into two processes: evaluating efficiency based on inputs into the first stage, and outputs from the final stage.However, many more complex cases have been studied where the business system is separated into more processes, either with a series or parallel structure, or some mix of these.These structures are referred to in the literature as network structures and the DEA techniques developed to measure efficiency in such systems is referred to as network DEA (Kao, 2014).
As indicated, the two-stage process adopted by Wang et al. (1997) measured the efficiency of the conversion of the inputs into the first stage of the process into outputs from the second stage of the process, but did not directly include the intermediate measures.As such, the Wang et al. (1997) two-stage model could not guarantee that a firm measured as efficient was efficient in both stages of the process.Liu and Lu (2012) state that this approach calculates technical efficiencies for the two sub-processes as if they are not related.They submit that this results in a potential conflict, in that if the first sub-process is determined as inefficient, in order to become efficient, it needs to increase its outputs without increasing inputs.However, in increasing its outputs it is increasing the inputs into the second subprocess, therefore decreasing the efficiency of this process.A study by Chen and Zhu (2004)   The original Chen and Zhu model has also been evaluated and modifications suggested.An enhancement to the original model takes into account centrality measures for ranking (Liu and Lu, 2012).Also, a modification of the model has been applied in evaluating efficiencies of the Chinese transit systems and their impact on regional economic efficiencies across 30 regions in China (Chiu Huang and Ma, 2011).This modification allowed for outputs from the first stage characterized as undesirable to be excluded from the inputs into the second stage.Also for intermediate inputs to the second stage not arising out of the first stage to be included.The Chen and Zhu (2004) two-stage model is described in Saranga and Moser (2010) as follows: ∑ μ j = 1, and μ j ≥ 0, j = 1,2,….n,where:  x ij is the i th input and γ kj the k th output of DMU j; and  i = 1,2,….m,k = 1,2,….s,j = 1,2,….n;and  'o' is the DMU under evaluation; and  θ and ø are efficiency scores of stage 1 and stage 2, respectively; and  ω 1 and ω 2 are the weights assigned to the efficiency scores in stage 1 and stage 2, respectively (when both stages are of equal importance the weights will be equal); and  z lj are intermediary outputs of stage 1 that become inputs in stage 2; and  ż lo are the unknown decision variables.Much of the subsequent research on the two stage network DEA models has focussed on developments to allow for the decomposition of the efficiency of the respective stages and the determination of overall efficiency.In this respect, two key approaches have been identified, firstly, the multiplier approach, where overall efficiency is determined as the product of the efficiencies of the two stages (Kao and Hwang, 2008).The second is termed the additive approach, where overall efficiency is determined as being the weighted sum of the efficiencies of the two stages (Chen Cook Li and Zhu, 2009).The most recent studies with regard to net-work DEA have reference to this additive model, in that they have focussed on the stage weights applied in the model and their impact in determining stage and overall efficiencies (Ang and Chen, 2016; Despotis, 2016; Guo Shureshjani Foroughi and Zhu, 2016).This reveals an emphasis on refinements in the theory, whereas the purpose of this paper is on the critical review of existing network DEA applications with the purpose of determining their usefulness in performance measurement systems for support units within businesses.

Application of DEA to business support units
There is evidence to suggest that two-stage DEA models have been found to be particularly useful in measuring the performance of business units designated as support functions.Saranga and Moser (2010) refer to the enhanced two-stage DEA model as developed by Chen and Zhu (2004) as a value chain DEA (VC DEA) approach.They applied this approach to evaluate the performance of the purchasing and supply chain management (PSM) function.They highlight that the role of PSM is designated as a support function, a business support unit, which makes measurement of the added value of this function to overall company performance difficult to ascertain.They argue that by using the value chain DEA methodology to evaluate PSM performance, they are able to incorporate intermediate outcomes, which facilitates the measurement of the contribution of this function to overall performance.Simultaneously they are able to measure the performance of the PSM function against the PSM functions of other organizations, which, in turn, facilitates benchmarking.Saranga and Moser (2010) argued for the appropriateness of this two-stage DEA approach, in that they indicated that this model was most suited to their objective, which was to incorporate the indirect nature of the PSM function on overall company profits into their evaluation of PSM performance.They stated that the value chain DEA model of Chen and Zhu (2004) was the most appropriate for this as it was capable of measuring the efficiency of conversion of inputs to the PSM function into PSM outputs, as well as the subsequent conversion of these outputs into ultimate corporate financial performance.
Saranga and Moser (2010) compare the value chain DEA approach to PSM performance evaluation to the approaches of Das and Narasimhan (2000), Narasimhan Jayaram and Carter (2001), and Ellram Zsidisin Siferd and Stanly ( 2002), who all used multiple performance measures at either stage one, stage two or in the conversion of inputs at stage one into outputs at stage two (Figure 3).However, none of these alternative approaches was found to directly link the performance of the PSM function to overall improvement in the performance of the company, which is seen as a key goal from the perspective of PSM senior management.Saranga and Moser (2010) also make reference to an earlier study by Easton Murphy and Pearson (2002) who sought to condense the multiple PSM performance measures into a composite index of overarching corporate performance by applying the Variable Returns to Scale (VRS) DEA approach.The VRS DEA approach is an early adaptation of the CCR model, which allows for returns to scale evaluations (2004).Saranga and Moser (2010) submit that this study by Easton et al. (2002) demonstrated the advantages of the DEA methodology with respect to benchmarking, as compared to the various measures involving a single input and a single output.Saranga and Moser (2010) argue that the advantages of using the VC DEA methodology include its ability to evaluate the performance of each DMU against that of its peers, the ability of the enhanced model to incorporate intermediary inputs, such as the PSM outputs, into the performance evaluation and the ability of DEA to include both financial and perception-based inputs and outputs as long as they can be quantified.They further submit that the most compelling reason for researchers to adopt the DEA approach, as regards performance measurement of PSM functions, is the ability of DEA to aggregate the multiple PSM performance measures into a single measure of overall performance.In contrast, they submit that the disadvantage of applying the VRS DEA models is that they cannot be accurately used in a support function environment where intermediary outcomes are created, which, in turn, help to create final outcomes for the company overall.2004) model to accommodate a second-stage, independent of but parallel to stage one, also producing outputs, which too become inputs into a final third stage.The aim is to allow for further discrimination of efficiency at the final stage and to better understand the causes of inefficient performance.It is acknowledged, therefore, that the Chen and Zhu (2004) model has limitations that are addressed through the many further studies outlined in Kao (2014).However, although Kao (2014) acknowledges areas for further research into DEA models, such as the type of data used, for example incorporating imprecise data, qualitative data or probabilistic data, he states that the most valuable research direction would be to apply existing or new models to solve real world problems.
As is demonstrated through the application of the model by Saranga and Moser (2010) to the purchasing and supply chain management, and by Chen and Zhu (2004) themselves to information technology, it is submitted that the Chen and Zhu (2004) model is ideally suited to incorporation into performance measurement frameworks for business support units.Although the model does not allow for the identification of the impact of other exogenous inputs on the final outcomes, it does allow for the relative efficiency of the support DMU to be directly and simultaneously linked to the relative efficiency of the business as a whole.Saranga and Moser (2010) contend that a key challenge in measuring performance of business support units such as PSM, is that they are not seen as directly adding value to products and services, making their value add difficult to measure.If the aim is to measure the performance of the business support unit and to identify whether it adds value to the business as a whole, and not to unpack all the factors contributing to overall business efficiency, then it is contended that the Chen and Zhu (2004) model is fit for this purpose.Neely (2004) states that one of the 'great unanswered questions' regarding performance measurement is 'is this worth it?'.This, because there are so many factors influencing overall business performance that it is difficult to isolate the role performance measurement has on performance.It is submitted that the application of the Chen and Zhu (2004) model facilitates the establishment of this link between performance of the business support unit and overall company performance.At the same time, as indicated by Kao (2014), the general two-stage network DEA structures have the simplest structure making empirical applications easier to carry out.If the aim is to adopt the model as part of a performance measurement framework, then, ease of application must be a consideration.

Determining the validity of DEA efficiency scores
In respect of the validity of the DEA results, Banker (1993) indicates that the DEA efficient production frontier has the required properties to allow for a wide range of statistical tests.Firstly, tests can be conducted to validate that the DEA estimators of best practice can also be shown to be estimators of maximum likelihood.He argues that while the estimated best practice frontier will be biased below the theoretical frontier for a restricted sample, the bias will approach zero for large samples.In addition, specific tests that address issues, such as the comparison of efficiency of different groupings of DMUs, the existence of economies of scale, and tests to determine whether the impact of each of the inputs on the output is independent of other inputs (Cooper et al., 2004), could be relevant in further analysis of DEA results.
Chen and Zhu (2004) note that users for the model do need to be cognisant of the sensitivity of DEA to certain issues.These include situations where all units are found to be efficient, normally arising, because the number of observations (DMUs) relative to the number of inputs and outputs is too small.This difficulty was evident in the work of Wong Leung and Gilleard (2013), who used DEA to examine the efficiency of Facilities Management of Buildings in Hong Kong.The study was limited to only nine buildings (DMUs), but included three input and nine output measures.The results of the DEA analysis showed six of the nine buildings to lie on the efficiency frontier.In analyzing the limitations of their application, Wong et al. (2013) acknowledge that a general rule in applying DEA is that the number of DMUs should be considerably larger than the number of inputs and outputs.Chen and Zhu (2004) submit that the number of units should be minimally three times the sum of the number of inputs and outputs.
Other studies, such as Golany and Roll (1989), suggest the minimum number of units should be equal to two times the sum of the number of inputs and outputs.
The possibility that the number of inputs and outputs included in the DEA may be restricted, to account for the size of the population of DMU's being examined, has another potential implication for the analysis.A further possible limitation of any application of DEA, is the potential that there are drivers of performance other than those included in the analysis, which are impacting on performance.Saranga and Moser (2010) identified this as a limitation in their study, and indicated that there was scope, particularly in a single industry study where there was access to data from a larger sample, to increase the number of performance drivers and outcomes.Chen and Zhu (2004) had also identified data set limitations as restricting the managerial insights to be gained from their empirical analysis.However, where the size of the population allows, they suggest further dissecting the drivers as an approach to better explain the differences in performance identified.
Therefore, although it is necessary to ensure that the number of DMU's is considerably larger than the number of input and output measures included in the analysis, it is also important to ensure that the analysis is comprehensive enough to adequately differentiate and describe the performance of the DMU's.In this regard Saranga and Moser (2010) note that DEA models are capable of accommodating maximum information about the system being evaluated through including multiple performance dimensions.
A further consideration in applying DEA is that the DEA result will be sensitive to errors in the data set arising from inaccuracy or measurement errors.However, this can be accounted for through the application of sensitivity tests, an example of which are those proposed by Zhu (2001), which allow for a 'stability area' to be determined.

Summary and conclusion
This paper examines the DEA methodology and, more specifically, the value added DEA model of Chen and Zhu (2004) as a potential means of both linking business support unit performance to overall corporate performance, and allowing for the benchmarking of such performance against a set of peer units such that targets for improvement of performance can be established.The problem identified is the incorporation of the measurement of value added by business support units to overall company performance, into the performance measurement framework of such support units.It is acknowledged that such business units do not directly add value to products and services and so identifying their impact on overall corporate performance is difficult (Saranga and Moser, 2010).Also identified is the problem in measurement frameworks of being able to focus management attention on targets and means toward achieving them, as opposed to a preoccupation with justifying specific numbers across a wide range of measures in a performance measurement framework (Neely, 2004).
DEA was defined as an approach for evaluating the performance of a set of peer business units, which are referred to as DMUs, which convert multiple inputs into multiple outputs.The original CCR DEA model (Charnes et al., 1978) was described as utilizing linear programming to produce an efficiency measure for each DMU in the peer group, requiring only that the DMUs convert similar inputs to similar outputs and that these can be quantified.Furthermore, the advantages of DEA over other measures of performance such as regression, profitability and ratio analysis, were examined.When compared with ratio analysis, Halkos and Salamouris (2004) indicated that the advantage of using DEA was that DEA provided a single objective score, a ranking and the potential targets for improvements for each of the business units determined as inefficient.Other advantages identified include its empirical orientation and the lack of a need for prior assumptions, which are inherent in other approaches, such as statistical regression analysis.Also, in comparison with regression analysis it was noted by Charnes et al. (1994) that instead of arriving at a single optimised regression equation representing an average DMU, DEA results in an understanding of the relative performance of each DMU.Based on this evidence, it is concluded that utilizing DEA as part of a performance measurement framework allows for the establishment of targets for performance improvement, and shifts the focus from the individual measures to an overall measure of relative efficiency.
As regards business support unit performance, an enhanced DEA model advanced by Chen and Zhu (2004) is identified.This is reported as being one of many enhancements to the basic DEA model that have been studied since the model was first introduced.However, it is submitted that to the degree that the Chen and Zhu (2004) model links efficiency over two stages, it enables the simultaneous measurement of efficiency of the support unit and the overall business.
It, therefore, establishes a link between the performance of the support unit and the overall company performance.Chen and Zhu (2004) empirically tested their model by analyzing the impact of IT on firm performance in the banking industry, and the model was applied by Saranga and Moser (2010) in evaluating PSM performance.This Chen and Zhu (2004) DEA model, in turn, has its limitations, including that it does not allow for the identification of the impact of other exogenous factors on overall company performance.Many further studies have been undertaken (Kao, 2014) in part to address these limitations.However, to the degree that the purpose in a support unit measurement framework is not necessarily to quantify the impact that the unit has on overall performance, but simply to identify that value is added to overall company performance by improvements in efficiency at the business support unit level, it is submitted that the model is fit for purpose.Also, it is argued that the general two-stage models are easier to carry out (Kao, 2014) and, therefore, are more suited to practical applications.
In summary, although Kao (2014) indicated that there is still much room for further study on the DEA models themselves, he noted that the most valuable research direction would be in the application of the models to real world problems.The further future application of the Chen and Zhu (2004) model to problems of performance measurement for business support units, is submitted as just one area where significant value could be derived through the practical application of the existing DEA theory.
addressed this conflict in that they advanced an enhanced network DEA model, which they applied to the data from the Wang et al. (1997) study.Chen and Zhu (2004) developed a DEA-based methodology, which allows for the identification of the efficient frontier in two-stage processes where there are intermediate measures of performance.The two-stage process considered by Wang et al. (1997) is illustrated by Chen and Zhu (2004) in the figure reproduced below (Figure 2).

Fig. 2 .
Fig. 2. Intermediate measures Source:Saranga and Moser (2010).This enhanced model was applied byChen and Zhu (2004) to the data from theWang et al. (1997) study and a comparison made between the results of the two studies.Chen and Zhu (2004) applied the original model as used byWang et al. (1997) to measure the efficiency of the conversion at stage one and at stage two.In addition, they applied their enhanced model to arrive at an overall efficiency score that incorporates the intermediate measures.The results show that banks reflecting as efficient overall in terms of the enhanced model are not necessarily efficient at each of the two stages.The original DEA score only measures efficiency at each of the two stages, whereas the enhanced model measures overall efficiency in the context of the two-stage process.

Fig. 3 .
Fig. 3.The PSM performance evaluation framework using DEA Source: Saranga and Moser (2010, p. 199).In a further adoption of the Chen and Zhu model to a business support unit, Ebrahimnejad Tavana Lotfi Shahverdi and Yousefpour (2014) highlight how they utilized the model to demonstrate the indirect impact of information technology on the performance of a firm.In their application, the model was used to identify the efficient performance frontier of two value added stages, being information technology and profit generation, and to highlight those firms that could be further analyzed as a benchmarking exercise.In summary, they indicate that the two-stage model ofChen and Zhu (2004) aims to minimize the inputs to stage 1 and maximize the final output from stage 2 simultaneously.Additionally, the model calculates the intermediary variables in the most optimistic case.Kao (2014) summarizes a wide range of network DEA studies based on problems identified and the models that have been developed and applied to address the problems.They include general twostage structures and advance to more complex parallel and hierarchical structures.For example,Ebrahimnejad et al. (2014) adapt the Chen and Zhu (2004) model to accommodate a second-stage, independent of but parallel to stage one, also producing outputs, which too become inputs into a final third stage.The aim is to allow for further discrimination of efficiency at the final stage and to better understand the causes of inefficient performance.It is acknowledged, therefore, that the Chen and Zhu (2004) model has limitations that are addressed through the many further studies outlined inKao (2014).However, althoughKao (2014) acknowledges areas for further research into DEA models, such as the type of data used, for example incorporating imprecise data, qualitative data or probabilistic data, he states that the most valuable