FLEXIBLE SOFTWARE RELIABILITY GROWTH MODELS

Numerous Software Reliability Growth Models (SRGMs) have been discussed in the literature. These models are used to predict fault content and reliability of software. It has been observed that the relationship between testing time and the corresponding number of faults removed is either exponential or S-shaped, or a mix of the two. Another important class of SRGMs, known as flexible SRGMs, can depict both exponential and S-shaped growth curves. The paper introduces a new concept of power logistic learning function that proves to be very flexible, in the sense that it represents various curve types – exponential, Rayleigh, Weibull or simple logistic. The flexible nature of the power logistic function gives the flexible SRGM a higher degree of accuracy and wider applicability.


INTRODUCTION
Due to the incredible growth in information technology and science, computers and computer-based systems have invaded every sphere of human activity -for example, nuclear research, telecommunications, space research programmes, aviation, transportation, banking, education, and health-care. With the world of information technology changing at a staggering rate, not to mention the proliferation of the Internet, more and more systems are being automated, resulting in an increase in people's dependence on computers.
Although this technology revolution has made our lives better, concern for safety and security has never been greater. There are numerous instances where failures of computer-controlled systems have led to colossal loss of human lives and money. With the increased complexity of product design, short development cycles, higher customer demands for quality, and the highly destructive consequences of software failures, a major responsibility lies with those who are active in the areas of software debugging, testing and verification. Moreover, competition in the software market is intense, and as a result of the applications involved, purchasers look for quality and reliability in software. The nature and complexity of software requirements has drastically changed over the last few decades, and users worldwide have become much more demanding in terms of cost and quality.
Computer-based systems typically consist of hardware and software. Quality hardware can now be produced at a reasonable cost, but the same cannot be said of software. Software development consists of a sequence of activities where perfection is yet to be achieved. Thus there is every possibility that faults are introduced and remain in software. These faults can lead to failures that have catastrophic results. So great emphasis is placed on avoiding the introduction of faults during software development and removing latent faults before the product is released for use. The only way to verify and validate the software is by testing. Software testing involves running the software and checking for unexpected behavior in the software output. During the testing phase, test cases that simulate the user environment are run on the software. Any departure from specifications or requirements is called a failure, and immediately an effort is made to remove the cause of that failure (the fault). A successful test can be considered to be one that reveals the presence of latent faults. Therefore, software should be thoroughly tested to expose as many software faults as possible. The testing phase is an extremely important feature of the software development life cycle (SDLC), in which about 50% of the developmental resources are consumed. Therefore, testing software by executing all its statements and paths (even just once) is not practically possible for a large-scale software system. A compromise testing approach is used, which involves dividing the software into blocks and executing each block at least once. Potentially troublesome combinations of blocks are executed many times. However, testing techniques can be categorized into two main groups: 1. Top-down testing involves starting the test at the sub-system level. Then, the modules that comprise the sub-system are tested. The procedure is recursively repeated until the test reaches the lowest level software component (function, object).
2. Bottom-up testing involves reversing the previous process.
Each test technique has its merits and demerits. Choosing the testing approach mainly depends on the software development technique. Software testing is a destructive process, since it aims at forcing the software to behave abnormally under some conditions. For this reason, software programmers subconsciously avoid bringing their product into this stage. Therefore, it is preferable that an independent team tests the software. This test method is called independent validation and verification (IVV), which means that the testing team is functionally independent of the development team, and tests the software from the user's point of view. This method is also called the black box method. The process of locating faults and designing the procedures to remove them is called the debugging process. The process of fault removal (repair) involves rewriting the code if the fault is due to coding and design error, or changing the requirements (which requires major repair). The chronology of failure occurrence and fault removals can be utilized to provide an estimate of the software reliability and the level of fault content. In the light of this, there is a need to develop a tool that can utilize this information to help software engineers and managers to monitor the process of testing. The software reliability model (SRM) is a tool that can be used to evaluate software quantitatively, develop test status and schedule status, and monitor the changes in reliability performance.
Numerous Software Reliability Growth Models (SRGMs), which report the number of failures (faults identified/removed), have been discussed in the literature [7,10,13]. These models are used to predict the fault content and reliability of the software. It has been observed that the relationship between the testing time and the corresponding number of faults removed is either exponential [4] or S-shaped [17], or a mix of the two. Another important class of SRGMs, known as flexible SRGMs, exists: depending upon parameter values, these depict both exponential and S-shaped growth curves [7]. In these models the role of the learning process during the testing phase is taken into account, which comes from the experience gained in software testing.
In this paper, section 2 discusses three existing flexible SRGMs developed by Obha [12], Bittanti et al [2], and Kapur and Garg [6]; section 3 discusses their discrete equivalents. Two other discrete equivalents of the Kapur and Garg [6] model are also included: these exist in the literature due to Innoue [19] and Satoh [15]. They are subsequently shown to be equivalent to a discrete equivalent of Kapur and Garg [6] that is mathematically and computationally much simpler. The derivation of continuous SRGMs from the equivalent discrete model has also been shown. The interesting part of the paper begins in section 4, with the introduction of a new concept of a power logistic learning function, which proves to be very flexible in the sense that it represents various curve types -exponential, Rayleigh, Weibull or simple logistic. In the same section the concept of generalized SRGMs is taken further by defining the software fault detection rate as the convex combination of the power logistic and Weibull functions. From the numerical illustrations, it is seen that the newly defined class of generalized SRGMs yields considerably improved results, with a better predictability resulting from a lower MSE and higher coefficient of variation.

Inflection S-shaped SRGM Ohba [12]
This model has been developed under the assumption that the more that errors are detected, the more undetected errors become detectable.
( ) The solution of the above differential equation with the initial condition m(t = 0) = 0 is: where r, ( ) , is a proportion of independent errors and is the inflection parameter. The model becomes exponential when r = 1 and takes the logistic form when r = 0. It implies that initially r is almost zero, but as faults are detected and removed it approaches unity. Here ) (t ϕ reflects the gradual increase in testing effort and the skill gained by the testers as the testing progresses.

Flexible SRGM Bittanti et al [2]
This model is based on the following differential equation: Here k i and k f are initial and final values of the fault exposure coefficient. If ki = k f , then it reduces to the exponential model. If k f >> k i ; the failure growth curve takes S-shape. If k f is very small compared to k i and is almost equal to zero, the failure growth curve becomes flat at the end.
The solution of the equation with initial condition m(t = 0) = 0 is : For different values of k f and k i , it describes different growth curves. [6] This model is based upon the assumption that the detection of errors also results in detection of some of the remaining errors without these errors causing any failure.

SRGM for an error removal phenomenon Kapur and Garg
The differential equation for this model is given by: The solution is: : Failure occurrence rate. q : Fault removal rate of additional removed faults.
This model was developed to account for some additional faults being detected without their causing failure. Here it may be observed that if q = 0, then on each failure, only the error causing the failure is removed and it corresponds to the exponential model. The failure growth curve defined by the above model is S-shaped whose nature depends on q/p.

One-stage equivalent model for (1), (2) & (3)
All three models -Ohba [12], Bittanti [2], and Kapur and Garg [6] -can be written in general form (one-stage process): Upon solving we get: By giving different forms to b and β, the above three models can be derived.
Here, in the alternate formulation, it may be observed that the fault detection rate for the above specified models is a logistic function, and b(t) → b as t→ ∞ . Here β is the learning factor, and it represents the skills and experience gained by the testers

DISCRETE SRGMS
NHPP-based SRGMs described above are continuous time models that use the execution time (i.e., CPU time) or calendar time. The other group contains models, that use the test cases as a unit of fault removal period. Such models are called discrete time models, since the unit of software fault removal period is countable. A test case can be a single computer test run executed in an hour, day, week or even month. Therefore, it includes the computer test run and length of time spent to inspect the software source code visually. A large number of models have been developed in the first group, while fewer are in the second group owing to the difficulties caused by their mathematical complexity.
Nevertheless, the utility of discrete SRGMs cannot be underestimated. As the software failure data sets are discrete, these models often provide a better fit than their continuous time counterparts. Therefore, in spite of the difficulties caused by mathematical complexity, discrete models are proposed regularly [7,18].
Therefore we develop flexible discrete SRGMs on the lines of the abovementioned continuous time SRGMs, using a probability generating function (PGF). Two other forms of the model are also given, and it can be shown that the three forms are equivalent. It is further shown how a continuous time SRGM can be derived from the discrete model. Since most of the software failure data sets are discrete, therefore, it is suggested that discrete SRGMs should be used. [6] Most of the software reliability growth models assume that the fault removal phenomenon also describes the failure phenomenon. This may not always be true. The test team can remove some faults in the software without these faults causing any failure, although this may involve some additional effort. A fault that is removed consequent to a failure is known as a leading fault. While removing the leading faults, some other faults are removed that might have caused future failures. These are known as dependent faults.

Discrete time flexible SRGM for error removal
The difference equation under the above assumptions for the SRGM is given by: The solution can be obtained as follows (using PGF): Now by multiplying both sides n z and sum over n from 0 to ∞ , we get:

Other discrete SRGMs
Various other discrete SRGMs have been reported in the literature corresponding to the continuous model. One is developed by Yamada and Inoue [19]. It is based on the discrete Riccati equation.
It is given by The exact solution of the above difference equation is given by: Similarly, using the Satoh [15] approach, another discrete equivalent of a continuous model (3) can be obtained.  Again, it can be shown that as 0 → δ , m(n) given by (6) and (7) converge to the corresponding continuous SRGM given by (3).   It is the same as equation (5).

Generalized Model-1
An interesting variation of a continuous time SRGM developed by Kapur and Garg [6] is now presented: it adds extra flexibility and wider scope to the already flexible model.
Consider the case when the failure growth phenomenon also depends upon the testing time, in addition to the number of faults remaining in the software, as well as the number of faults already identified [6]. Based on these assumptions, the differential equation for fault identification / removal can be written as: Here 0 ≥ k . It can be easily seen that if k = 0 it is same as the differential equation for SRGM developed by Kapur and Garg [6].

Alternate formulation -1
The number of failures during testing is dependent upon the number of instructions executed. Here the number of instructions executed is a function of testing time.
Using these assumptions, the failure removal phenomenon can be expressed as:

Alternate formulation-2
Another alternate formulation for SRGM given by (10) is described, which will allow definition of a new type of logistic function.  (11) Solving it with the initial condition ) 0 ( m = 0, we get the same result as in (9).

Generalized model-2
The basic levels of testing are 'white-box' testing and 'black-box' testing. White-box testing is performed to find faults that lie within the internal structure of the software. It requires the tester to have a detailed knowledge of the internal structure. The main objective of white-box testing is to ensure that test cases execute every path throughout the code. The importance of white-box testing is expressed in terms of test or code coverage metrics, which measure the fraction of code executed by test cases.
Black-box testing is carried out to judge how well software meets the user's requirements.
Testing coverage is defined as the extent or degree to which software is executed by the test cases. During testing, if few bugs are encountered, it does not necessarily indicate that the coding is of a high quality; on the contrary, it implies the poor design of test cases. To ensure better testing, testing coverage analysis is used to assess the quality of test cases. Testing coverage is actually a structural testing technique in which software performance is judged with respect to the specification of the source code. The basic testing coverage measures are: [1] Statement coverage: It is defined as the proportion of lines executed in the program. If it is assumed that the faults are uniformly distributed throughout the code, then the percentage of executable statements covered shows the percentage of faults discovered. [2] Decision / condition coverage: This measure indicates whether Boolean expressions tested in control structures evaluated to both true and false. [3] Path coverage: This measure shows the percentage of all possible paths existing in the code exercised by the test cases. [4] Function coverage: This measure indicates the proportion of functions/ procedures influenced by the testing.
Testing coverage provides an important criterion for the optimal release policy, based on available testing resources and the importance of risk-free and safe operations of the software on implementation. Therefore, safety-critical systems have a high coverage objective.
A software reliability growth model that incorporates the concept of testing coverage in the model building is now presented. The concept of convex combination of two different types of functions to represent the testing coverage function is used. This approach helps to capture various types of coverage function simultaneously, depending on the various possible values of the parameters. The goodness of the fit has been tried on two real software failure datasets. The results obtained are fairly accurate, and show considerable improvement and a better fit.
Let c(t) define the proportion of the software executed by the test cases. So, 1c(t) defines the proportion of the code which is yet to be covered by the test cases. Then, the first order derivative of c(t) , denoted by c'(t), represents the testing coverage rate. The ratio of the two -i.e.
-will be taken as measure of the fault detection rate.
where u and b are the shape parameters γ λ , are the scale parameters.
Consider the model having the fault detection rates as defined above in (13) respectively: It can be seen that failure intensity not only depends on the number of remaining faults in the software, but also on the ratio of the rate at which the remaining faults are covered and the present proportion of uncovered faults.
Using the fault detection rate as defined in (13), we get: Using the initial condition (16) where m(t) → a as t → ∞.
If k=1 it is the same as the Weibull model developed by Yamada. [20].

MODEL VALIDATION
To check the accuracy of the generalized SRGM-1 given by (10) with power logistic and generalized SRGM-2 given by (16) with convex combination of two different types of function with respect to flexible SRGM of type (4), parameter estimation on real software failure datasets has been done.

Data Set-I (DS-I):
The data was obtained from Musa et al [9]. The software was a real time command and control system which was tested for 92 days (21 weeks). The delivered object instructions were 21,700 involving 9 programmers; 136 faults were removed during testing.
The estimation results are given in Table 1.

Data Set -II (DS-II):
The data are cited from Brooks and Motley [3]. The fault data set is for a radar system of size 124 KLOC (kilo lines of code) tested for 35 months, in which 1301 faults were identified.
The estimation results are given in Table-  It has been observed that the generalized SRGM-1 and 2 given by (10) and (16) has always provided results that are the same as or better than the flexible SRGM of type (4). This is due to the flexibility given to the model by the presence of the 'k' factor. The increased accuracy achieved shows the ability of the model to capture different types of failure datasets -e.g. exponential, S-shaped Rayleigh or Weibull types, etc.

CONCLUSION
Software reliability engineering is rapidly emerging as an important field of study in the area of information technology. Mathematical models play a significant role in its growth. These models provide quantitative tools to assess the reliability of the developers' software. In this paper, an important class of SRGMs, known as 'flexible SRGMs', has been assessed. New dimensions have been added to the flexible modelling by introducing the concept of the 'k' factor and convex combination, which proves to be more flexible and yields a better predictability and higher degree of accuracy.
The usability of this newly introduced concept is not limited to these SRGMs. It can be extended to improve the results of any SRGM, whether exponential, S-shaped or of any other type -for example, multi-stage models or SRGMs related to distributed environment.

ACKNOWLEDGEMENT
This research work was initiated while Prof. P.K. Kapur was visiting researcher at the Department of Industrial and Systems Engineering, University of Pretoria, South Africa.