A MULTI-OBJECTIVE COAL INVENTORY MANAGEMENT MODEL USING MONTE CARLO COMPUTER SIMULATION

The ability of a coal-fired power station to meet its generation targets is influenced by periods of coal shortage. In this article, we propose a multi-objective inventory model to assist with the management of coal stockpiles. The model is applicable to power utilities with a network of two or more coal-fired power stations. The aim is to determine the near-optimal amount of coal inventory to stockpile at each station in the network. A Monte Carlo coal stockpile simulator is used to incorporate stochastic uncertainty into the stockpile levels, while a metaheuristic uses the simulator as an estimator of two objective functions. The metaheuristic finds good values for the coal stockpile level at each power station in the network. The algorithm for multi-objective optimisation using the cross-entropy method is proposed as a suitable metaheuristic. A hypothetical case study is used to validate the inventory model and to showcase the optimisation results.


INTRODUCTION
The continued evolution of civilisations is highly dependent on a secure and accessible supply of energy and, as the human population continues to grow, global energy demand will continue to increase.Power utilities are at the forefront of global energy supply.Currently, more than 40 per cent of the world's electricity is generated at coal-fired power stations [1].This figure does not seem that high, but many countries -such as South Africa (93%), Poland (92%), China (79%), India (69%), and the USA (49%) -rely primarily on coal for electricity generation [2].The unpredictable demand for electricity, along with the uncertainties related to inventory replenishment and coal quality, places great emphasis on accurate and effective inventory planning at coal-fired power stations.Like most industries, the inventory problem encountered by power producers is typically of a multi-objective nature.
In this article, we propose a multi-objective inventory model to assist with the management of coal stockpiles.The model is applicable to power utilities with a network of two or more coal-fired power stations, and assumes that some of the stations in the network are capable of burning coal with similar properties.This assumption allows for inventory to be moved between the stations, or for coal deliveries to be redirected from the source in emergency situations.A Monte Carlo coal stockpile simulator coded in the open source R programming language is used to emulate stochastic depletion and replenishment of the stockpile levels.The simulator requires an electricity production plan as input.This production plan is determined using an independently-developed production planning module that allows for the input of different electricity demand profiles.The simulator allows for the input of such profiles that consider weather (cold, hot), sectors (agriculture, industry, mining, households), and different geographical areas.The basic simulator, called the energy flow simulator (EFS), was obtained from Enerweb [3].
The simulated coal stockpiles are used as a basis for the decision variables and objective functions that we propose.The algorithm for multi-objective optimisation using the cross-entropy method (MOO CEM) of Bekker & Aldrich [4] is proposed as a suitable metaheuristic to solve the inventory model approximately.
The remainder of this article is organised as follows: A short discussion on inventory management in the electricity supply sector is provided in Section 2, followed by important fundamentals of multiobjective optimisation in Section 3. The proposed coal stockpile simulator is described in Section 4.
In Section 5, we explain how the output of the simulator is used to formulate the multi-objective coal inventory model.In Section 6, we use a hypothetical case study to illustrate the capability of the inventory model while also testing the effectiveness of the MOO CEM algorithm in finding approximate solutions for the model.Concluding remarks are provided in Section 7.

INVENTORY MANAGEMENT IN THE ELECTRICITY SUPPLY SECTOR
Logistics plays a significant role in the operation of thermal power stations.Several simulation and optimisation studies have been conducted on the logistics systems and supply chains of coal-fired power stations in particular.In their paper, Li and Li [5] propose a simulation and optimisation model for the logistics system of a coal-fired power station using WITNESS software.The model, which was applied to a power station in China, included the coal suppliers, the transportation system, and the power station itself.These researchers regarded these as the three major components of a power station's logistics system.The selection of suppliers is critical to guaranteeing the quantity and quality of the coal supply [6], [7].The transportation system has many requirements and constraints that must be considered, while at the power station the coal storage systems, the coal conveying systems, and human resources must be managed.
In a typical coal supply chain, the coal suppliers and transportation companies are responsible for managing inventory levels at mines and throughout the transportation network, while the electricity suppliers are responsible for managing the coal stockpiles at their power stations.According to Zhanwu, Xiao, Zhonglei, and Xiongbiao [8], the survival and development of coal-fired power stations is seriously affected by the bottleneck that is created during periods of coal shortage.Compared with other enterprises, inventory management at coal-fired power stations has its own features and requirements due to the nature of the goods being stored.Zhanwu et al. [8] mention the following as the four main features of coal inventory management: 1.The uncertainty of coal demand: The uncertain demand for coal at power stations is triggered by the uncertain demand for electricity.Many factors contribute to variation in demand, such as temperature, humidity, and wind speed [9].2. The uncertainty of coal suppliers: The price of coal varies over time, especially where shortterm contracts with suppliers are in place.Similar to any other market, high demand means high prices, and vice versa.3. The uncertainty of inventory replenishment: Due to weather conditions and other uncertainties within coal transportation networks, the inventory replenishment of coal is not constant.4. The requirement of safety stock: Each of the above uncertainties emphasises the need to stockpile coal because coal shortages will result in an electricity shortage.Safety stock is an effective management tool for protecting electricity suppliers against uncertainty.However, there is a trade-off between having the ability always to provide customers with the promised service level and the costs involved in storing large amounts of coal at a power station.
These four features of coal inventory management must be considered when developing a coal stockpile inventory management model.It also follows from the four features mentioned above that at least two conflicting objectives prevail in coal stockpile inventory management: carrying sufficient inventory while trying to minimise the capital invested in inventory.
Next, multi-objective optimisation principles and solution methods are discussed.

MULTI-OBJECTIVE OPTIMISATION PRINCIPLES AND SOLUTION METHODS
Multi-objective optimisation (MOO) problems differ from single objective problems, in that they have at least two conflicting objectives.This means that the meaning of 'optimum' changes.There are many acceptable solutions for a given problem; the aim is to find good 'trade-offs' between the objective functions [10].
The MOO problem for  objectives,  decision variables, and  unequal and  equal constraints, is defined by: where  = [ 1 ,  2 , … ,   ]  is a  dimensional vector of decision variables.Each   ( = 1,2, … , ) can be real-valued, integer-valued, or boolean-valued, and is constrained within some feasible set Ω, which consists of  equality constraints   and  inequality constraints ℎ  .The degrees of freedom are given by  −  and, to avoid an over-constrained problem, it is required that  < .
Approaches to solving MOO problems can be broadly organised into two categories: scalarisation approaches and Pareto approaches.In scalarisation approaches, the MOO problem is solved by translating it back into a single-objective problem (or a series of single-objective problems).The preferences of the decision-maker must thus be incorporated into the optimisation model before solutions are found.A few examples are the weighted sum approach, lexicographic ordering, and goal programming.Pareto approaches differ from scalarisation approaches in that they do not admit a unique solution but a set of solutions, based on the concept of Pareto-optimality.In these approaches, the decision-maker's preferences are incorporated after the optimisation model has been solved.MOO metaheuristics belong to this category ( [11], [12]).
Metaheuristics are global search algorithms that are widely-recognised for their ability to obtain nearoptimal solutions within reasonable time periods.These algorithms are generally designed for combinatorial optimisation problems in the deterministic context, but have recently been successfully applied in simulation optimisation.A major advantage of metaheuristics lies in their ability to approximately solve problems for which no satisfactory problem-specific algorithm is available.Also, metaheuristics perform well in the case of a high-dimensional, discontinuous, and non-differentiable response surface.This means that they are capable of overcoming the trap of local optimality ( [13], [14], [15]).
Metaheuristics used in MOO are mostly population-based.This means that the algorithm searches by iteratively improving a population of solutions.The aim of an MOO metaheuristic is to find a set of decision-variable vectors, known as the Pareto optimal set.(For a comprehensive discussion of these algorithms and some of their variants, see Chapter 2 of Coello, Lamont, and Van Veldhuizen [10].)Multi-objective optimisation using the cross-entropy method (MOO CEM) was selected as a suitable approach to solving the MOO inventory model described in Section 5.The MOO CEM algorithm is a population-based metaheuristic inspired by statistical principles.The cross-entropy method (CEM) for optimisation is a versatile Monte Carlo method, developed by Rubinstein and Kroese [16].The approach was motivated by the work of Rubinstein [17] on variance minimisation methods for rareevent probability estimation, and modified in Rubinstein [18] to solve continuous and combinatorial optimisation problems ( [19], [20]).In contrast to a random search algorithm that searches for an optimal solution by sampling decision variable values from the same probability density function (pdf), the CEM assigns a pdf to each decision variable.The search works on the principle that the parameters of each of these pdfs are iteratively adjusted to increase the probability of drawing decision variables that result in good objective function values.The aim of the CEM is to estimate the parameters of each decision variable's ideal pdf in order for them to converge to an optimal (or near-optimal) solution.The algorithm does this by using the cross-entropy (or Kullback-Leibler distance).As a measure of the distance between two pdfs, a minimum cross-entropy is desired between the sampling distribution associated with each decision variable and the optimal pdf from which to sample ( [12], [16]).
The CEM has been proven to converge quickly when applied to optimisation problems with one objective [16].It is thus an ideal approach for the computationally-expensive, time-dependent problems often encountered in simulation optimisation (SO).This was the motivational factor for expanding the CEM to solve multi-objective problems.The MOO CEM was introduced by Bekker and Aldrich [4] and Bekker [21], and has since been the topic of multiple research studies.

THE PROPOSED COAL STOCKPILE SIMULATOR
The Monte Carlo simulation model that we propose is an extension of the coal stockpile simulator of Micali and Heunis [22].We have added a multi-objective optimisation capability to their coal stockpile simulator.For each replication (or sample path), the simulator computes the daily power production and the end-of-day coal stockpile level for each power station  ∈  over a user-specified study horizon.This is done by stochastically adding uncertainty (or noise) to the simulator's two main inputs: the planned power production for power station  on day  (  ), and the planned coal deliveries to power station  on day  (  ).Further inputs are the heat rate of power station  (  ), the estimated calorific value (CV) of the coal burnt at power station  (CV  ), and the energy availability factor (EAF) of power station  on day  (EAF  ).Each power station has a specific heat rate, and is designed to burn coal within a specific CV range.The energy availability factor is a percentage value that represents a power station's generation availability with respect to its total capacity ( [3]).It is denoted by where PCLF  refers to the power generation capacity lost due to planned maintenance at power station  on day .UCLF  refers to the power generation capacity lost as a result of unplanned maintenance at power station  on day , while OCLF  refers to the power generation capacity lost due to extraordinary events at power station  on day , such as employee strikes or the theft of transmission cables.The PCLF percentages are extracted from the power utility's maintenance plan, while the UCLF and OCLF percentages are forecast based on historical events.
The following uncertainties are considered in the simulator: 1.A power station's actual UCLF on a given day may differ from the expected UCLF.
2. The actual CV of the coal burnt at a power station on a given day may differ from the estimated CV. 3. A power station's actual coal deliveries on a given day may differ from the planned coal deliveries.The three uncertainties are incorporated daily for each power station, as shown by ( 2) to (4).Stochastic noise is added to the UCLF at each power station  on a given day  by adding a random value sampled from a normal probability density function (pdf) with a mean of zero and a standard deviation that is obtained from historical unplanned maintenance data.The actual power production at power station  on day  (  ) is subsequently computed by multiplying   with the ratio of actual EAF over planned EAF ([3]).This is mathematically denoted by where   UCLF is the UCLF standard deviation for power station .
The actual CV of the coal burnt at each station is computed in a similar manner.The CV is adjusted with a standard deviation that is obtained from historical coal quality data, and the actual coal burnt at power station  on day (  ) is subsequently computed by where   CV is the CV standard deviation for power station .The original expressions in (1), (2), and (3) were based on monthly time periods, but the resolution has been adjusted to daily time periods in this study.This is to allow for potential daily spikes in demand that are otherwise averaged out when a monthly resolution is used.
A triangular distribution is assumed for the actual coal deliveries because its parameters can be specified by subject-matter experts.A triangular distribution's pdf is defined by three values: the minimum value, the most likely value, and the maximum value.We propose that the actual coal deliveries to power station  on day (  ) be sampled from a triangular pdf with the most likely value set as   .The minimum and maximum value of each is set as   and   respectively, where  (0 <  < 1) and  (1 <  < 2).The values of  and  are user-specified values that define the distribution's range.This is mathematically denoted by After incorporating the uncertainties for power on day , the coal stockpile level for power station  at the end of day  is computed by Negative stockpile levels are not allowed.Thus, if  , is negative,  , is set equal to zero and the actual coal burnt and actual power production are recalculated by And respectively, where (0,   CV ) is the same as in (3).The expression in (7) has been adapted from Enerweb [3].
A power station's stockpile level at the end of a given day can be expressed in kilotonnes (ktonnes) or in stockpile days.To obtain the number of stockpile days on hand at a given station, the number of ktonnes is divided by the station's standard daily burn (SDB).The SDB of a power station, measured in ktonnes per day, is the amount of coal that the station could burn if it were to operate for one full day without any outages (i.e., EAF = 100%).The units of measurement for all the parameters in the coal stockpile simulator are provided in Table 1.

MULTI-OBJECTIVE MODEL FORMULATION
The multi-objective coal inventory model that we propose is applicable to power utilities with a network of two or more coal-fired power stations.A network such as this represents a continuous system: a certain load is constantly produced and fed into the transmission network.The model is dependent on the assumption that certain stations in the network are capable of burning coal with similar properties.This assumption allows for coal to be moved between certain stations in emergency situations.A power station with some reserve thus supplies coal to another coalcompatible power station that is in need, if it is practically possible.
The aim of the model is to determine the optimal amount of coal inventory to stockpile -referred to as the 'target stockpile levels' -at each station in the network.We denote the target stockpile level for power station  as   .The coal stockpile level at each power station  is simulated by the coal stockpile simulator described in Section 4. Recall that the coal stockpile simulator is a Monte Carlo simulation model that adds noise to the planned power production and planned coal deliveries.
The noise is added in the form of uncertainty about unplanned maintenance (UCLF), coal quality (CV), and delivery reliability.The unplanned maintenance and the coal quality at a particular power station affect the amount of coal being burnt by the station, while the delivery reliability affects inventory replenishment.Ideally, coal delivery and burn should balance each other out so that the target stockpile level can be maintained.However, this is almost never the case.
Two hypothetical scenarios for extreme variation between coal delivery and burn at a power station are shown in Figure 1.The effective decrease in stockpile level shown in scenario 1 may be as a result of low delivery reliability, poor coal quality, or less unplanned maintenance than was expected.Scenario 2 illustrates the opposite, where the amount of coal delivered is significantly more than the coal burnt.A typical case would involve a combination of the two scenarios, as shown in Figure 2, where the inventory level fluctuates around the target level.
The inventory model we propose is to allow coal transfers between appropriate power stations in the network to try to maintain each station's coal inventory level as close as possible to its target level.To achieve this, two transfer levels, called the lower warning limit (  ) and the upper warning limit (  ), are required for each power station  to trigger the movement of coal.
At the end of every day , each station's stockpile level ( , ) is evaluated to determine whether transfers are required.When a given station's stockpile is depleted below   , inventory is replenished by transferring coal from one or more of the other stations.Similarly, when a given station's stockpile becomes more than   , coal is transferred to some of the other stations to reduce the inventory level.Both   and   are decision variables for the inventory model, and since the main goal is to determine each station's   , it also has to be included as a decision variable.To trigger the target stockpile levels, we formulated the inventory model by specifying that a transfer is only allowed when, for at least one of the power stations involved in a transfer transaction, the stockpile level can be forced towards   .Furthermore, coal can only be moved between power stations capable of burning similar coal.All transfers are expressed in terms of the station from which the coal is moved.The coal transferred from power station  on day  is denoted by .The basic constraints on the decision variables are:

variation between coal delivery and burn
To simplify the complex inventory system of large power utilities, the following assumptions are made: 1.   ,   , and   must be positive integers when expressed in terms of stockpile days.2. There is no limit on the maximum stockpile level at any of the power stations.3.By default, no constraint is placed on the amount of coal that can be moved between two power stations on a given day.Also, transfers are allowed on any day throughout the course of the study period, and there is no limit on the number of transfers that may be made.4. All lead times are one day.Thus, if a coal transfer is triggered on a given day , the changes on the stockpiles are visible on day  + 1.
The user has the option to discard assumption 3 by setting a cap for the amount of coal that can be moved between certain stations.Furthermore, to limit the total number of transfers that may be made, the user can specify that the stockpiles may only be evaluated on certain days.This allows the model to comply with practical considerations, as the amount of coal that can be moved between any two power stations on a given day is limited.The available infrastructure and throughput rates dictate transfer amounts; infrastructure includes trains and trucks for movement of coal by road.
For a better description of the coal transfer functions, consider the following: two coal-fired power stations A and B are capable of burning coal with similar properties.If  , <   becomes true on day , a transfer is required from station B. However, coal may only be moved if  , >   .If this condition is true, the amount of coal transferred from B to A on day  + 1 is calculated by where  is the coal transfer cap between the two stations.By default,  = ∞.
After incorporating both the actual coal delivery and burn on day  + 1,  ,+1 and  ,+1 are adjusted according to respectively.
The power stations at which  , <   receive priority (i.e., coal transfers for them are sought first); and the stations are served in descending order based on the difference between  , and   .Two policies are proposed to select the power stations involved in a transfer transaction: The first objective proposed for the model is to minimise the total average coal stockpile level ( ̅ ), measured in ktonnes.This is a typical objective for inventory management models.Assuming a study period of  = 365 days,  ̅ is given by where  is the number of power stations included in the system.As a second objective, it is proposed that the total coal transfers () throughout the course of the study period be minimised.This is given by  = ∑ ∑  ,    =  = (15) where   is the distance (in km) that coal is transferred from power station  on day .The unit of measurement for  is thus ktonnes.km.It is clear that (14) aims to obtain low values for all   , while (15) tries to space  ,   , and   so that coal is transferred optimally, since there is a trade-off between large coal transfers that occur occasionally and frequent smaller transfers.The two objectives are conflicting, and the decision space has a combinatorial nature that requires an MOO metaheuristic.The reader is reminded that coal transfers are expensive and must be avoided as far as possible; in this study they are allowed as a way of penalising the optimisation metaheuristic for 'bad choices' of stockpile levels.In this way, the metaheuristic 'learns' and avoids bad values chosen previously.

HYPOTHETICAL CASE STUDY
In this section, we use a hypothetical case study to test the proposed inventory model.
We use a large hypothetical power utility with 14 coal-fired power stations.Each of these stations receives coal daily, and in quantities that vary from one delivery to the next.We denote the 14 stations 'A' to 'N'.The coal transfer matrix indicating which of the stations can burn coal with similar properties is given in Table 2.A '0' indicates that a transfer may not take place between two stations, while a '1' indicates that a transfer is allowed.The distance matrix showing the distances (in km) between the 14 stations is given in Table 3.Also, recall that the decision variable values must be positive integers when expressed in terms of stockpile days.No maximum stockpile levels are specified, and we set each station's initial stockpile level equal to   .
The approximate Pareto set achieved for the closest-first transfer policy and for the most urgent transfer policy are shown graphically in Figure 3.The expected value is used as the output statistic.
A good distribution of solutions was obtained for both transfer policies, and the non-dominated solutions cover a wide range of values for each objective function.One can clearly see that there is not much difference between the two transfer policies when only the non-dominated sets are considered.

Figure 3: Approximate Pareto set achieved for the two coal transfer policies
To evaluate the quality of the solutions, we compare the approximate Pareto set achieved for each transfer policy with sets that were achieved with 50 000 objective function evaluations in the MOO CEM.These plots are given in Figure 4(a) for the closest-first transfer policy, and in Figure 4(b) for the most urgent transfer policy.One can see that slightly better solutions were found with 50 000 evaluations -i.e., the solutions are closer to the origins of the plots due to minimisation.
When solving an MOO problem, the process does not stop when the Pareto set (which is approximate) is achieved.A single best solution must still be selected by the decision-maker.This process is not trivial, because the members of the approximate Pareto set are the estimations of a stochastic process.A few methodologies for multi-objective decision-making under uncertainty have been developed over the years.The two-phase selection procedure by Chen and Lee [23] may be used, while the technique proposed by Lee, Chew, Teng and Goldsman [24] is fairly recent.We do not elaborate on these techniques here, as they are not part of the research scope.We again used 10 000 objective function evaluations in the MOO CEM.One can see from Figure 5 that there is very little difference between the solutions achieved when using the expected value and the 80 th percentile value as output statistic.The results produced by the model are thus slightly skewed towards the lower percentiles.We conclude that the expected value is sufficient, due to the small differences we observed between the approximate Pareto sets.

CONCLUSION
In this article, we proposed a multi-objective inventory model to assist with the management of coal stockpiles.The model is applicable to power utilities with a network of two or more coal-fired power stations, and is dependent on the assumption that some of the stations in the network are capable of burning coal with similar properties.This assumption allows for inventory to be moved between the stations in emergency situations.We used a Monte Carlo coal stockpile simulator to incorporate stochastic behaviour into the power generation process.The uncertainties considered are the unplanned outages at the power stations, the reliability of coal deliveries, and the quality of the coal.
The aim of the multi-objective inventory model is to determine the optimal (or near-optimal) amount of coal to stockpile at each station in the network.As objectives for the model, we proposed that both the total average coal stockpile level and total coal transfers be minimised.The algorithm for multi-objective optimisation using the cross-entropy method (MOO CEM) was found to be a suitable metaheuristic for creating approximate solutions.In future work, the optimisation model will be evaluated with a real-world case.

Figure 1 :Figure 2 :
Figure 1: Two hypothetical scenarios for extreme variation between coal delivery and burn

Figure 4 :Figure 5 :
Figure 4: A comparison of the approximate Pareto sets achieved for 10 000 vs 50 000 MOO CEM objective function evaluations for (a) the closest-first transfer policy, and (b) the most urgent transfer policy

1 .
Closest first: If a given power station requires coal ( , <   ), the stations from which the transfer will be made are selected by considering the closest one first.Similarly, if a given power station has too much coal ( , >   ), the stations to which the transfer is made are selected by considering the closest first.2. Most urgent: If a given power station requires coal, the stations from which a transfer is made are selected by first considering the one at which  , is furthest above   .Similarly, if a given station has too much coal, the stations to which the transfer is made are selected by first considering the one at which  , is furthest below   .