Information and Software Technology 1994 36 (2) 107-117
AIDA: a dynamic analyser for Ada programs F E Eassa Al-Azhar University, Cairo, Egypt
L J Osterweil University of California, Irvine, USA
M Z Abdel-mageed AI-Aztmr University, Cairo, Egypt
In this paper, we have developed a dynamic analyser for Ada programs, called AIDA. In software engineering, dynamic analysers that have been built previously have often incorporated first-order-logic assertion languages. For dynamic testing of both sequential and concurrent programs, however, temporal logic may be advantageous since it deals with the development of situations over time. AIDA investigates the applicability of temporal logic in building a dynamic analyser for Ada programs. AIDA is designed to test, debug and specify programs written in the Ada language. It affects the instrumentation of programs as well as the collecting, organizing and reporting of results of the executions of the instrumented program. The instrumentation approach is based on the idea that the intended function of a program can often be specified in terms of assertions or values that must be assumed by variables, at certain strategic paints in the program. This paper describes the design, implementation and experimental evaluation of AIDA. The goal of this work is to apply AIDA as a comprehensive dynamic analyser for Ada programs. AIDA can handle sequential processes, and concurrent tasks as well as it can understand fully all Ada statements. Keywords: AIDA, Ada programs, dynamic analysers, temporal logic, testing, experimental evaluation
Testing is an important and costly component of any program's development cycle. There are many techniques which are used in program testing; one of these techniques is dynamic testing and analysis. This technique is supported by a variety of tools and has been the subject of many research projects. A dynamic testing tool must be able to monitor, trace and record the dynamic changes in values of tested variables. A historical review of dynamic analysers indicates that one example of these analysers was PET I. It was a testing tool for FORTRAN programs. It exploits counters to count the number of executable statements and the number of executable branches. Depending on the values of these counters one can decide whether an error exists. Also, it provides an option for determining the first and last values for each underlid variable. A series of dynamic testing tools have followed PET. While the PET instruments are only counters, such tools 0950-5849/94/020107-11 © Butterworth-Heinemann Ltd
have provided the programmer with the ability to insert assertions at key points within the tested program. An example of these tools is NEWTON system 2. It is a dynamic tester for FORTRAN 77 programs. NEWTON assertions are translated into dynamically executable embedded probes. Outputs from these probes are kept in a structured database. Anna 3 is a specification language that utilizes first-order logic for testing A d a 4-7. It is an extension to Ada that includes facilities for specifying formally the intended program behaviour. Unfortunately Anna does not, however, allow a facility for testing concurrency in Ada programs. TSL-18 is a language for specifying sequences of tasking events in the execution of a concurrent Ada program. However, TSL-1 is limited for testing sequence errors and it cannot detect other types of run-time errors. Our dynamic analyser AIDA investigates for the first time the applicability of temporal logic for building a 107
AIDA: a dynamic analyser for Ada programs: F E Eassa et al.
comprehensive testing too19-15. AIDA can be used to test sequential tasks (i.e. it covers Anna) and concurrent processes (i.e. it covers TSL-1). Moreover, it can be relied on to specify the precedence between tasks. It carries out the required testing by instrumenting the user program that contains the assertion statements. These statements are based on temporal logic and are used to direct the program instrumentation. Accordingly, it controls the testing process during program execution, that is, during run-time.
Dynamic testing Dynamic testing entails executing a program and studying the progress of the execution ~6'~7. Dynamic testing can test program functions and specific detailed issues of program logic5. In what follows we present some properties of dynamic testing by making use of a set of illustrative examples. A functional test is derived from program specifications and is intended to indicate that the program computes correct results under normal operating conditions, i.e., correct output values from nominal input values. The following stack operation example illustrates this. If a stack manager has a function POP(S), which returns and deletes the top element of a stack S, and PUSH(S,X), which returns S with X pushed on to S, then the functional test PUSH(S,POP(S)) could be used to ensure that S is returned unchanged when S is not empty, and functional test POP(PUSH(S,X)) could be used to ensure that X is returned (and S is effectively unchanged), when S is not full. Logical tests are concerned with the manner in which the code performs its computation. There are a number of classes of errors which can be studied by performing logical tests. These include arithmetic, error handling, initialization, and interfaces. Arithmetic is tested to check the precision of calculations (namely 'overflow' and 'undertow' due to round-off errors) and to determine sensitivity to inaccuracies in the input data. Error handling is tested to assure the proper processing of illegal input data (e.g., attempted square root of a negative number). Initialization testing ensures that items are initialized to the correct values for all possible execution and reexecution sequences that can occur, starting from the initial computational state of each module. In interface tests, data and control interfaces are candidates for dynamic testing. Data interfaces are manifested in the parameters passed between routines and in the protocols for access to common data areas and databases. Errors can exist in the numbers and types of parameters; alignment mismatches of variables in common data areas can occur; and the access protocol to data bases can be incorrect. Control interfaces exist in the invocations and the entry and exit points of modules and routines. Dynamic testing sometimes uses the instrumentation technique to increase the error detection capability of a test. The instrumentation approach is based on the idea that the intended function of a program can often be specified in 108
terms of assertions or values that must be assumed by variables, at certain strategic points in the program. Thus, additional information for error detection can be automatically produced using software probes to monitor the values of variables or to detect violations of an assertion. In order to do this, a dynamic tester automatically inserts probes into a code then provides aids for capturing, organizing and analysing probe output. During testing, an assertion violation will typically cause the recording of some information, may or may not transfer control to the user, and may or may not cause termination of the invoking program or program unit. Sequential programs specify a sequence of actions that are to be performed one after the other. Concurrent programs, on the other hand, specify two or more sequences of actions that may be performed concurrently 19'2°. Each sequence of actions is performed by a task. Sequential programs consist of a single task that executes the main procedure. In concurrent programs, each task performs actions in strict sequence, but several tasks may be in progress at the same time 21. There are many problems and properties in concurrent programs that dynamic testing can help detect and analyse. These are mutual exclusion (shared variables), deadlock, liveness properties ~2 and precedence of tasks events. Illustrative examples for detecting and analysing the problem and properties are included later in the paper.
The implementation of AIDA This section introduces the reader to AIDA. It discusses the following topics: (1) AIDA as dynamic instrumentation system. (2) The design of the AIDA instrumentor. (3) Some examples of how AIDA can be used to analyse some important properties of programs.
AIDA as dynamic instrumentation system
Figure 1 illustrates how AIDA works. The user first writes a program and inserts temporal assert statements in it. The syntax of the assert statement is given by the following Backus-Naur Form (BNF): Assert__statement: AIDA__start__token AIDA__label "ASSERT" temporaL___expression AIDA__statL_token:-- ] AIDA.._label : [0-9] [.0-9] + [a-zA-Z] temporal~expression: [TL__operator] [ " ( " ] (logicS.___ expression) [ " ) " ] : [ " ) " ] (logicaL_expression) [TL__operator] (logic__expression) [ " ) " ] TL operator: [ ] always operator :'~ eventually operator :@ next operator :U until operator :P precede operator Information and Software Technology 1994 Volume 36 Number 2
AIDA: a dynamic analyser for Ada programs: F E Eassa et al.
control the flow of execution to interrupt execution, to continue it at defined points in the source program, and to watch the values of Ada program variables and the history of these variables. The commands can be used either when an assertion violation occurs or when the user interrupts the program execution. If there is an error during the execution of the program, a message is generated, giving the name of the program unit which has the error, the statement number which has the error and the label of the assert statement which is violated. The execution of the program is then interrupted and the user can execute AIDA commands.
u s e r ~
H o w AIDA analyses program properties
The behaviour of a concurrent program can be characterized by the set of its legal execution sequences. We will now show how to utilize temporal logic to state properties of the execution sequences of a given program, thus describing the desired properties of the dynamics of the program.
A r.un-time H
Invariance (safety) properties. We begin by examining some examples of program properties which hold continuously throughout a computation. They can be effectively specified by temporal expressions of the form: [ ]W. A temporal expression of this form, which specifies that W is always true, expresses an invariance property, indicating that they specify that something will always (invariably) hold true. A sample of important properties falling under this category is given below.
AuserusesAIDA~ , ~ ' ~ commands ~
Figure 1 AIDA system logical_._expression : (variables ] constant) relational.__ connector (constant I variable) :(logical__expression) logical___connector (logical__expression) relational._connector: > I < J < = J > = J< > logical___connector: : AND : OR : NOT end-assert-statement: AIDA-start-token AIDA-label "ASSERT . . . . END" The AIDA instrumentor instruments the user program by changing the temporal assert statements into Ada statements and by inserting calls to routines in the AIDA runtime library. The instrumented program is compiled by the Ada compiler, and the loader then loads the AIDA run-time routines with the object code. The AIDA assertion language incorporates five temporal assert statements; "always", "eventually . . . . next . . . . until" and "precede". All of these assert statements (except " n e x t " ) are coupled with end-assert-statements, enabling the user to control the scope of the assert statement. During execution users can use AIDA run-time system commands to debug and analyse the program to watch and Information and Software Technology 1994 Volume 36 Number 2
Mutual exclusion. Consider two processes, P1 and P2, being executed in parallel. Assume that each process contains a critical section (C1 and C2 respectively) which includes some task, T, critical to the cooperation of the two processes. For example, the critical section might access a shared resource. If the nature of T is such that it must be done by both of them simultaneously, we call C1 and C2 critical sections. The property stating that the processes will never execute their respective critical sections simultaneously is called mutual exclusion. If we set a flag to true at the start of each critical section, such as "flag__Cl: = true;", and "flag__C2: = true;" and reset these flags to false at the end of the critical section, then the property of mutual exclusion for C 1 and C2 can be described by [ ] (NOTC2))flag__C2))flag.__C2)). The flags flag__C 1 and flag__C2 must be global to both tasks and to a main unit. Therefore their declaration must be in the specification part of the main unit. Figure 2 shows actual Ada code using the always assert capability to specify and enforce critical sections. The program in Figure 2 computes the binomial coefficient (~) for input o _ k _< n. The computation of this program is based on the formula:
n! k! ( n - k ) !
n.(n-1 ... (n-k+l) k. ( k - 1) . . . . 1
The program consists of two tasks; BC1 and BC2. Task BC1 is used to compute the value of numerator of the formula. Task BC1 is used to compute the value of denominator of the formula. 109
AIDA: a dynamic analyser for Ada programs: F E Eassa et al. begin yl: = n;y2: = 0;y3:y4: = 1; Task body BC1 is begin BCl_flag:
= false; -- all critical flags --are initialized by false.
accept BCfir do loop exit when yl = (n-k);
--critical region of BC1 tl : = y3*yl ; BCl__flag; = true;--critical region flag -- I 1.3A ASSERT  NOT ( B C l _ f l a g = true AND B C 2 _ f l a g = true); --The 1.3A is an example of AIDA labels. y3: = tl; end of critical region B C l _ f l a g : = false; yl: = y1-1 ; end loop; end BCfir; end BC1 ; task body BC2 is begin BC2._._flag: = false:--all critical flags --are initialized by false. accept BSsec do
loop exit when y2 = k;
y2: = y2 + 1; --critical region of BC2 will start. t2: = y3/y2; BC2_flag: = true;--BC2 critical region flag. y3: = t2; --end of BC2 critical region BC2._flag: = false; end loop; end BCsec; end BC2; --I 1.3A END ASSERT; end;
Figure 2 Binomial coefficient program for testing mutual exclusion Deadlock freedom. A concurrent program consisting of m processes is said to be deadlocked if no process is able to run because each is waiting for at least one of the other. This leaves the idling step as the only possible choice for the schedular. The rest of the computation will therefore consist of an endless (indefinite) repetition of the current deadlocked state. Clearly, in a deadlock situation, each process must be blocked at a certain location whose exit condition is false. We refer to such blocked locations as waiting locations. Figure 3 illustrates how we test for deadlock freedom in a specific example program. Example We will use the above example of the binomial coefficient and the same two flags BCl__flag and BC2__flag to test for deadlock freedom. Eventuality (liveness) properties. A second category of properties are those which are effectively specified by 110
begin yl: = n;y2: = 0;y3: = 1;y4: = 1; task body BC1 is begin B C l _ f l a g : = false; --flag is initialized by false. accept BCfir do loop exit when yl = (n-k); tl: = y3*yl ; B C l _ f l a g : -- true; --critical region flag -- [ 1.3A ASSERT  ((BCl-flag = true AND BC2__flag = false) OR B C l _ f l a g = false AND B C 2 _ f l a g = true)); --The 1.3A and 1.4A are two different -- labels. y3: = tl; --I 1.3A END ASSERT; B C l _ f l a g : = false; --I 1.4A ASSERT  ( B C l _ f l a g = false); yl: --- y1-1 ; --[ 1.4A END ASSERT; end loop; end BCfir; end BC1; task body BC2 is begin BC2_flag: = false; --all critical flags --are initialized by false. accept BSsec do loop exit when y2 = k; y2: = y2 + 1; t2: = y3/y2; --critical region of BC2 will start. BC2_flag: = true;- BC2 critical region flag. --I 1.3A ASSERT  ( ( B C l _ f l a g = true AND BC2 flag = false) OR ( B C l _ f l a g = false AND B C L f l a g = true)); y3: = t2; --end of BC2 critical region --I 1.3A END ASSERT; BC2_flag: = false; end loop; end BCsec; end BC2; end;
Figure 3 Binomial coefficient program for testing deadlock freedom 'eventuality'. These properties depend on some formula, W l , being initially true, then another formula, W2, being eventually true. The eventually-operator ("~) is used to test that a specific condition is satisfied at least once during the scope of eventually-assert-statement. Figure 4 illustrates how we test for eventuality. Precedence properties. Precedence properties can be expressed by using the until (U) or precede (P) temporal operators. We can use the until operator when we need to test if task T2 will be true in the future and that task T1 is always true until T2 becomes true. The precede operator is used when we need to test that task T2 will be true in the future and task T1 happens first. The tasks execution order is not affected by the AIDA assertions. This is because the number of probes that are added to each task due to the preceding AIDA statement is the same. In Ada, a task T~ can communicate with another task T 2 by calling T2's entry or by accepting a call on one of its own entries. When T~ calls an entry in T2 and T2 accepts that call, a rendezvous takes place. This is called a tasking event. Information and Software Technology 1994 Volume 36 Number 2
AIDA: a dynamic analyser for Ada programs: F E Eassa et ai.
end BCfir; --I 1.2 END ASSERT; end BC1 ; task body BC2 is begin --I 1.3C ASSERT ,~ (y2 = k); accept BSsec do loop exit when y2 = k; y2: = y2 + 1; t2: = y3/y2; y3: -- t2; end loop; end BCsec; end BC2; --I 1.3C END ASSERT;
begin Get(n); Get(k); yl: = n;y2:0;y3: = 1;y4: = 1; --[[email protected]
(yl> =0ANDy2=0AND y 3 = 1 A N D y 4 = 1); -- precondition task body BC1 is begin --[ 1.2C ASSERT ,~, (yl = n-k); --- ~ is the eventually operator in AIDA accept BCfir do loop exit when yl -- (n-k); tl: -- y3*yl ; y3: = tl; yl : = y1-1; end loop;
Figure 4 Binomial coefficient program used to show the use of eventuality
Example procedure Gas_Station is task Operator is entry p r e _ p a y ( . . . ) ; entry Charge(...); end Operator; ---here AIDA will insert a flag statement for each --entry for example Operator__pre~pay_B: -= Boolean: = false; task type Customer is entry Get__Customer_ld(...); entry C h a n g e ( . . . ) ; end Customer; ---here AIDA will insert a flag statement for each -- entry task type pump is entry Get__pump_ld(...) entry StarL_pumping; entry Finish_pumping(...); end pump; ---here AIDA will insert a flag statement for each entry Task body Operator is begin loop select --The operator accepts pre-pay from a customer. accept p r e _ p a y ( . . . ) d o --here AIDA will insert a flag statement. --for example, operator__pre_pay B: -- true; --here AIDA will insert a flag -- o p e r a t o r _ p r e _ p a y _ B : = false. end pre_pay OR accept charge(...)do --AIDA will insert a flag statement. --AIDA will insert a flag statement. end charge; OR terminate; end select; end loop; end operator; Figure 5.
task body Customer is begin accept Get~Customer__ld(...) do end Get_Customer__ld; --I 1 .t .2A ASSERT (operator_per__pay__B P Pump_start_pumping_B); --This assert statement test if p r L p a y _ _ B occurs before start Pumping_B. Operator. p r L p a y ( . . . ) ; --I 1 1.2B ASSERT (pump_start_pumping-B P pump_Finish_pumping_B); --This assert statement test if start_pumping_B occurs before Finch_Pumping_B. pump.start~pumping; -- I 1.2A ASSERT END; pump. Finish_pumping(...); - I 1.2B ASSERT END; accept change(...); end Customer; task body pump is begin accept G e t ~ p u m p _ l d ( . . . )
end G e t _ p u m p _ l d ; loop select accept Activate(...) do end Activate; accept Start_pumping; end Start_pumping; accept Finish_pumping(...) do; end Finish_pumping; OR terminate; end select; end loop; end pump;
Gas-station program to test the precedence of task events
Information and Software Technology 1994 Volume 36 Number 2
AIDA: a dynamic analyser for Ada programs: F E Eassa et al. With Math_Library; Use Math Library; With Text_.lO; Use Text_lO; Procedure S q u a r e _ R o o t ~ M is X1 ,X2:Float; a,b,c,e: Float; package NUMBERF_IO is new FLOAT_IO(FLOAT); Use NUMBERF_IO; Procedure Under-Root is separate; begin - - Square__Root__M Get(a); Get(b); Get(e); --[ 1.1.1A ASSERT (a > 0.0); -- I 1.1.2B ASSERT [ ] (b*b > = 4.0*a'c); -- I 1.1.3B ASSERT ,~ (e > = 0.0); UNDER ROOT;---called separate procedure XI: = - b + Math_Library.Square__root(e); - - T h e first root X2: = - b - Math_Library.square_Root(e); --- The second root; - - I 1.1.3B ASSERT END; put (xl); Put (x2); - - [ 1.1.2B ASSERT END; --I 1.1.1A ASSERT END; end square_Root__M;
package Math_Library is function power (A__power, Number:Integer)return Integer; function Square_Root(Number:Float)return Float; function Cube_Root(Number:Float)return Float; end Math_Library;
Figure 6(c) Package specification of M a ~ i b r a r y package body Math~Library is function power(A__power,Number:lntege0return Integer is begin -- power return Number* *A__Power; end Power; function Square_Root(Number:Float)return Float is begin --- Square_Root
end Square_Root; function Cube_Root(Number:Float)return Float is begin -- Cube_Root
Figure 6(a) The source program (root__m.a) separate (Square_Root~M) procedure Under_Root is J,K: Float; begin -- Under__Root J: = b'b; K: = 4.0*a'c; e: -- J-K; end Under__Root;
Figure 6(I)) The called separate procedure (under__roo0 AIDA can be used to test precedence of tasks events, and also to test the soundness of a rendezvous. To do this AIDA inserts a flag after each accept statement in the program during the instrumentation of the program. The name of the flag is the name of the task and is following by " _ _ " and the name of the identifier which comes after the "accept" keyword; the name of the identifier is followed by "___B". The value of each flag is set to false before the accept statement and to true after the accept statement. Example of such use of these assert statements is illustrated in the following Ada program (Figure 5). This program uses temporal logic to specify the precedence of tasks in an Ada program, first introduced by Luckham, to simulate an automated gas stations. The gas station contains tasks representing the station operator, the customers, and the pumps. A customer arrives at the station, prepays the operator, pumps gas, receives change from the operator, and leaves. The operator accepts prepayments, activates pumps, computes charges and gives customers their change. The pumps are used by the customers, and they report the amount pumped to the operator.
end Cube_Root; end Math_Library;
Figure 6(d) Package body of Math__Library programs in the AIDA-library. All these instrumentstatements are Ada statements. The value of program instrumentation in software testing is widely recognized L17. It can be used to facilitate testing in an effective manner and to obtain additional information for error detection. Furthermore, it is easy for programmers to use. To illustrate how AIDA instruments an Ada program, we take the user program in Figure 6(a), (b), (c) and (d), into which the user has inserted three temporal assert statements. This example shows a program for finding the roots of a second order equation. Figure 6(a) is source code containing the temporal assert statements. Figure 6(b) is the code for a called separate procedure called by the source code in Figure 6(a). Figure 6(c) is the specification of a package which is called Math-Library. Figure 6(d) is the body of the package Math-Library. The AIDA instrumentor consists of three stages (see Figure 7) that are executed sequentially. The three stages are: (1) Insertion of the assert-statements from the main program into the subprogram called by this main program. (2) Generation of tables used during the insertion of probes (instrument--statements) into the program and into the program's called subprograms. (3) Actual insertion of the instrumentation statements into the program.
AIDA instrumentor design
The following is the explanation of each stage.
AIDA has an instrumentor which inserts instrumentstatements into a user program. Some of these instrumentstatements correspond to the temporal assert statements that the user wrote. The others are calling statements to sub-
The insertion of assert statements into called subprograms (first stage). If the user has a main program and many called subprograms, as shown in Figures 6(a)-(d), he or she
Information and Software Technology 1994 Volume 36 Number 2
AIDA: a dynamic analyser for Ada programs: F E Eassa et al.
User Program -IAssert Statement
package body Math_Library_Square_Root._M is function Power(A__Power,Number:lnteger)return Integer is begin -- power return Number**A__power;
end power; function square_Root(kfath:lnteger;Number:FIoat)return Float is
AIDA Stage I
begin -- Square_Root --- I 1.1.3B ASSERT'~(Number > = 0.0 AND (kfath = 1 OR kfath = 2));
---I 1.1.3B ASSERT END;
end square_Root; begin --- Cube_Root main unit
"-.. / i .,os,ooe. I
end cube_Root; end M a t h _ L i b r a r y _ S q u a r e _ R o o t _ M ; Figure 80)) The called package after insertion of assert statements
separate (Square__Root__M) Procedure Under__Root is J,K:Float; begin -- Under_Root ---I 1.1.1A ASSERT (a > 0.0 AND(kathy-I)); ---I 1.1.2B ASSERT (b*b > 4.0*a*c AND (kathy = 1)); --- I 1.1.3B ASSERT [ ] '~,(e > = 0.0 AND (kathy = 1)); J: = b'b; K: = 4.0*a'c;
e: = J-K; AIDA Stage 3
i nstr umented 1 program j
Figure 7 Stages of instrumentation can then write assert statements which are global to all subprogram in the main program. It is also possible to write assert statements that are local in a subprogram. Testing specified by assert statements in calling programs must be carried out in called subprograms as well. Therefore, in the first stage, the AIDA-instrumentor translates and processes assert statements in calling programs into necessary called subprograms and repeats this operation for all nested called subprograms. Figures 8(a), (b), (c), (d) show the corresponding
package Math_Library_Square_Root__M is function power (A_Power,Number:lnteger)return Integer; function Square_Root(kfath:lnteger;Number:Float)return Float; end Math_Library_Square_Root_M; Figure $(a) Package specification of Math__Library Information and Software Technology 1994 Volume 36 Number 2
---I 1.1.1A ASSERT END; ---I 1.1.2B ASSERT END; ---I 1.1.3B ASSERT END; end Under__Root; Figure 8(c) The called separate procedure after insertion of assert statements specification of the called package, the called package body, the called separated subprogram and the main unit after processing of stage 1. Note that: (a) Assert statements in the scope of the called subprogram are inserted after the first begin statement of the called separate subprogram. (b) Endwassert statements are inserted before the last end statement of the called subprogram. If we look at Figures 8(a) and 8(b) we find: (1) The name of the called package has been changed in both the package specification and package body by adding the calling subprogram name into the package name. This change in the name of the called package is to make this copy of the package special for this calling subprogram, and to enable concurrent invocation of the original package from different calling subprograms. (2) The argument list of the called subprogram is lengthened by one, namely "kfath: Integer", where kfath is a counter that counts the sites of subprograms 113
AIDA: a dynamic analyser for Ada programs: F E Eassa et al.
calls from inside a calling program. This counter is used to indicate when and which of the assert statements in called procedures should be executed. (3) The assert statements in whose scope the called subprogram exists are inserted into the called subprogram immediately after the first begin statement. End assert statements which correspond to the assert statements are inserted before the last end statement in the called subprogram. (4) The value of the counter kfath is added to the assert statement to control the execution of assert statements in corresponding called subprograms. The counter kfath is added because AIDA creates a new version for each assert statement that encompasses a called subprogram and inserts it at the beginning of the executable statements of the called subprogram. Different assert statements encompass different subprogram calls at different positions within the main unit. Therefore, not all of the assert-statements should be tested during an invocation for the called subprogram. The counter kfath is added to the assert statement to determine which assert statements inside the called subprogram must be tested during each invocation of the called subprogram. (5) The variables of the assert statements are translated into correct local names.
With Math_Libra~._Square_Root__M; Use Math_Library_Square_Root~M; --- changed With Text__lO;Use Text._lO; Procedure Square_Root_M is kfath: Integer;---Added x l ,x2:Float; a,b,c,e:Float; package NUMBERF_IO is new FLOAT_IO(FLOAT); Use NUMBERF_IO; procedure Under__Root is separate; begin --- Square_Root_.M
We find the following changes in the copied file: (1) The name of the called package which will be instrumented is changed by adding the calling procedure name to it. (2) The argument list of the called procedure is lengthened by appending the argument kfath. (3) The statement kfath: = kfath + 1 is inserted before every called subprogram in the calling program. Stage two of the instrumentor. In this stage, the AIDA instrumentor takes the program. AIDA scans the program and it generates three tables; watch Table (Table 1), information.._Table (Table 2) and variable Table (Table 3). These three tables are used in the third stage. The watch__Table consists of three fields. These fields are the variable name, the label of the assert statement, and the kind of assert. The character A is used to represent an 'always-assert-statement', and E is used to represent an 'eventually-assert-statement'. The information__Table consists of two fields. These two fields are the label of the assert statement and the logical expression. The variablewTable (Table 3) is used in the third stage in order for AIDA to determine the value of any variable during the running of the program. The table consists of three fields: the variable___name, the type of variable and the unit name. The instrumentation o f the program (the third stage). In this stage, AIDA takes the program which should be instrumented (output of stage one) and generates the instrumented program, i.e., AIDA inserts the probes (the instrument-statements) that are the equivalent to, and derives from assert statements. It also inserts a variety of other instrument-statements which are helpful in debugging of the program.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
kfath: = 0; .... Added Get(a);
Get(b); Get(c); ---[ 1.1.1A ASSERT (a > 0.0); ---I 1.1.2B ASSERT [ ] ( b * b > = 4.0*a'c); - - I 1. 1.3B ASSERT '~ (e > = 0.0); kfath: = kfath + I;---Added Under__Root;--called separate procedure kfath: = kfath + 1---Added x l : = b + Math_Library_Square_Root__M. Square_Root(kathy,e); --- The first root -- changed kfath: = kfath + I;---Added x2: = b-Math_Library_Square_Root__M. Square_Root(kathy,e); --- The second root; --- Changed - - I 1.1.3B ASSERT END; put(x1); put(x2); - - I 1.1.2B ASSERT END; ---I 1.1.1A ASSERT END; end Square_Root__M;
Figure 8(d) The changed source program 114
Table 1. W a t c h ~ T a b l e variable__name
a a b
I.I.1A 1.1.2B 1.1.2B
A A A
Table 2. Information_Table
(b*b - 4.0*a'C) (e > = 0;
Table 3. Variable_Table variable__name
a b x1
Float Float Float
Square__Root__M Square__Root.__M Square__Root__.M
Information and Software Technology 1994 Volume36 Number 2
AIDA: a dynamic analyser for Ada programs: F E Eassa et al.
Experience in using AIDA Effectiveness o f AIDA. AIDA has been implemented in 8800 Ada language statements, where stage one of AIDA was implemented in about 3800 statements, stage two and three in about 3000 statements and the run-time system in about 2000 statements. AIDA has been applied to a number of Ada programs, some were sequential, and others were concurrent. AIDA has been used successfully for testing many programs at University of Colorado, Boulder, and University of California, Irvine. During our testing, we used AIDA to test program for invariance properties (where we used the always-assertstatement to test mutual exclusion), eventuality properties (where we used the eventually-assert-statement to test the termination of the program), and precedence of tasks events (where we used the precede-assert-statement and until-assertstatement). We also used it for such other logical tests as arithmetic precision, error handling, ensuring that variables are initialized to the correct values, and that the range of values of variables is correct. During this testing we measured the instrumentation time, the compilation time of the instrumented program and the execution time. We also measured compilation time and execution time for the programs without instrumentation. From the measurements, we found that the time taken for instrumentation was 1.5 to 3 times larger than the time taken for compilation of the uninstrumented version. Also, the time required to compile the instrumented version was larger than the time required to compile the uninstrumented version by a factor of 1.5 to 2. The time taken to execute the instrumented version was larger than the time taken to execute uninstrumented programs by a factor of 1.4 to 4. The times taken for instrumentation depend on the number of assert statements in the program and on the number of times the variables of the assert statement are repeated inside the progam. The timings show that instrumentation of the programs is slower than compilation. The reasons for this are: (a) the Ada compiler used during development of AIDA had a significant effect on running times due to the relatively poor quality of the object code generated; (b) the Ada object code is much slower than the object generated for the equivalent FORTRAN or C programs. We found also that the number of lines added to the instrumented versions of the programs often exceeded the number of lines in the original programs. The number of statements in the instrumented programs increases by factors of about 2 to 2.5 times the original programs. Also, the size of the increase in execution time for the instrumented programs depends on the number of assert statements in the program. AIDA implementation limits. The following limits are actually enforced by the Ada compiler and by the virtual memory space available during the development of AIDA:
(1) Number of tokens for any statement cannot be more than 100. (2) Number of accept statements per task cannot be greater than 10.
Information and Software Technology 1994 Volume 36 Number 2
(3) Number of tasks inside any unit (procedure or function) cannot be greater than 30. (4) Number of until-assert-statements or precede-assertstatements cannot increase beyond 10 per program unit. Also, the limits of AIDA show that: AIDA does not cover the faults that can occur due to the code modification. This is because AIDA does not support what is called regression testing, where regression testing is only the testing of the modified code.
A comparison with previous work Before AIDA there were well-known systems which have the capabilities to test and specify Ada program. These software systems are Anna 3 and TSL-18. Anna specifies the properties of the program using annotations and is designed as a language extension of Ada to provide additional facilities for formal specification and explanation. It specifies and permits information about various aspects of a program to be expressed in a precise machine processable form. Therefore, we will compare the temporal assertion language with the annotations of Anna. Annotations are built up from Boolean-valued expressions and reserved words indicating their meanings. Some of these annotations must be written inside the declarative part of the Ada program and the others must be written inside the body of the program. The annotations of Anna do not support the testing and debugging of Ada tasks. That is, the annotations support the testing and debugging of the sequential program properties only. The temporal assert statement supports AIDA for testing and debugging the sequential and concurrent properties of Ada programs. Therefore, it is considered as a general system for testing and debugging all kinds of Ada programs. Every annotation of Anna has a region of Anna text over which it applies, called its scope. Its scope is determined by its position in the Anna according to its Anna scope rules. For example, if the annotation occurs in the position of a declaration, its scope extends from its position to the end of that declarative region. The scope of temporal assert statement starts from the assert statement until its coupled end-assert-statement. Therefore, the user is able to control the scope of some of the temporal assert statements and this gives more power than Anna annotations. Example 1 (Figure 9 (a,b)) indicates this power. This example is simply one in which there are arithmetic operations and there are many assignment statements to the variable I. We need the value of I to be always more than 20 in the first part of the scope of I and it must always be less than 5 in the next part of the scope of I. As shown in Figure 9(a), the user inserted an annotation (called statement annotation) after each assignment statement to I. But as shown in Figure 9(b), the user inserted only two always assert statements inside the scope of the variable I. If the number of assignment statements to I is increased, we need only the two assert statements. With Anna, if there is a called subprogram between two assign115
AIDA: a dynamic analyser for Ada programs: F E Eassa et al. PROCEDURE arith Op is I: Integer; begin I: = 25; --/i>20 I: . . . . . .
PROCEDURE arith = Op IS I: INTEGER; begin
I: = 25; --I 1.1.1A A S S E R T
"11>20 I: . . . . . .
-11>20 I: . . . . . .
I: . . . . . .
I: . . . . . .
--I 1.1.1A A S S E R T END; I:=
I: . . . . . . ; - I 1.1.1B A S S E R T I: . . . . . . ;
I:= ..... ;
I: . . . . . .
I: . . . . . .
I: . . . . . . ; "11.1.1B ASSERT END; ;
end; ~re 9(a) program
Figure 9(b). Using temporal assertion language to test the arithmetic program
Using Anna annotations to test the arithmetic
ment statements to I and I is a global variable for this subprogram, then the user must insert an annotation after each statement that has the variable I in the called subprogram. But with temporal assert statement of AIDA, the user does not need to insert assert statements in the called subprogram, because AIDA tests the called subprogram automatically. This indicates one way in which temporal assert statement seem more powerful than annotations of Anna and easier to use. Example 1 In this example a mathematical Ada program is proposed. Figure 9 shows the use of AIDA temporal assertions versus Anna annotation for testing a sequential Ada program. Another system is used for the testing and debugging of Ada programs; it is called TSL-1 s. The formal comments of TSL-1 will now be compared with temporal assert statement use with AIDA. TSL-1 formal comments are relied on to specify sequences of tasking events occurring in the execution of concurrent Ada programs. They express constraints to be satisfied by the sequences of actual tasking events. TSL-1 formal comments are not able to detect other tasking error in Ada program, for example, deadlock. TSL-1 is used to specify the intended order and synchronization of interactions between tasks. Therefore, we can say that TSL-1 formal comments, like Anna annotation, do not support TSL-1 to test and debug all Ada programs, while temporal assert statements support AIDA to test and debug all Ada programs. Also, AIDA is used to test and specify the intended order (precedence) and synchronization of interactions between tasks in an easier fashion than TSL-1 formal comments. The number of formal comments which the user must use to specify the order of tasks in TSL-1 is many times larger than the number of assert statements used by AIDA. Figure 10 indicates how TSL-1 specifies the order of tasks. 116
This figure is the same as that shown in Figure 5 (Gas station example). The underlying example (Figures 5, 10) indicates that the AIDA assert statement is easier than the TSL-1 formal comments in specifying the order of tasks. For example, the syntax of the assert statement of AIDA is easier. The number of specifications used with TSL-I is larger than the number of assert statements used with AIDA. Therefore, the size of the user program using TSL-1, is larger than the size of the user program using AIDA. The TSL-1 specifications where execution body is excluded are stated at a global level so that all threads of control are constrained by them (Figure 10). The specification numbered (1) constrains customers. The first specification requires that a customer task is always prepay for a pump before pumping 8. The specification allows a customer to pump more than once. Therefore, the property, HAS. PAID is defined, which is true every time the customer pays, and reverts to false the moment the customer finishes pumping gas. The customer protocol specification defines the order in which a customer will interact with other tasks during a transaction. Specification (2) is a similar protocol defining the order in which a pump carries out its interactions with other tasks. Specification (3) requires the operator to give customers change promptly. It does not, however, specify that the change is given to the correct customer. Specification (4) is the constraint against races for a pump. It requires that whenever a customer prepays, that task will get to start pumping before customer makes a later task. Testing and debugging of Ada in the earlier work takes an entirely different approach than AIDA. Earlier work used formal languages; these formal languages are based on Boolean-valued expressions. Some of the formal language must be inside the specification part of the program and their scopes must agree with the scopes of Ada. The other part of the formal languages must be written inside the body Information and Software Technology 1994 Volume 36 Number 2
AIDA: a dynamic analyser f o r Ada programs: F E Eassa et al.
TSL-1. PROCEDURE GAS STATION is task OPERATOR is end OPERATOR; task type CUSTOMER is
end CUSTOMER; task type PUMP is end PUMP; --1. specification of constraints on customers. --+ property HAS PAID(CUSTOMER, PUMP): BOOLEAN : = FALSE --+ is else ?c from PRE - PAY (PUMP ID = > ? P) -- + when OPERATOR --+ then set HAS_PAID(?C,?P) : = TRUE; - w h e n ?c calls?p at FINISH_PUMPING --+ the set HAS_PAID(?C,?P): = FALSE; -- + end HAS_PAID; --+ not ?C calls ?P at START_PUMPING where not HAS__PAID(?C,?P); -- a customer's protocol --+ when ?C calls OPERATOR at PER__DAY --+ then?C calls?P at START_PUMPING = > -+ ?C calls?P at FINISH_PUMPING --- > --+ ?C accepting CHANGE - + before?C accepts OPERATOR at CHANGE; --2. specification of a pump's protocol. --+ when?P accepts OPERATOR at ACTIVATE --+ then?P accepts?C at START_PUMPING = > --+ ?p accepts?C at FINISH_PUMPING = > --+ ?p calls OPERATOR at CHARGE -+ before?P accepts OPERATOR at ACTIVATE; --3. constraint on the operator. - + when OPERATOR accepts?P at CHARGE --+ THEN OPERATOR calls?C at CHANGE --+ before OPERATOR accepting; -- + specification guarding against race for a pump. --+ when OPERATOR accepts?C1 at PRE-PAY - + then not OPERATOR accepts?C2 at PER_DAY = > -- + ?P1 accepts?C2 at START_PUMPING = > -+ ?P1 accepts?C1 at START_PUMPING - - + until?P2 accepts?C1 at START_PUMPING; begin body of program
end Gas Station; Figure 10. Using TSL-I to specify the order of tasks of the Gasstation problem
o f the program. Also the earlier systems do not concentrate on testing and specifying most o f the sequential and concurrent properties o f the A d a language, but are used to test and specify specific properties. However, the approach o f A I D A is based on using assertion that inserted inside the body o f a program to facilitate testing and debugging o f both sequential and concurrent tasks (processes).
Conclusion A I D A is a dynamic analyser for testing and analysing sequential and concurrent program, namely A d a programs. A I D A uses an assertion language for specifying the properties of the programs under test. The assert statements o f such language exploit the notion o f temporal logic. Program Information and Software Technology 1994 Volume 36 Number 2
testing by A I D A consists of three stages. The first stage includes insertion o f the assert-statements from the calling main program into the subprograms called by it. The second stage is the generation o f tables used during the insertion o f probes (instrument-statements) into the called subprograms. The last stage compresses the actual insertion o f the instrumentation statements. Thus testing by A I D A is an easy and friendly task since the user inserts the assert-statements in the main program only with no regard to any subprograms, even when they are nested. Some o f the available dynamic analysers can test sequential programs only while others are dedicated only for testing specific concurrent properties. However, A I D A is a comprehensive tester, which proves that temporal logic can be relied successfully for testing both sequential and concurrent programs. It has some implementation limits but its advantages are explicit relative to other dynamic analysers.
References 1 Stucki, L G and Foshee, G L 'New assertion concepts in self-metric software' Proc. 1975 International Conf. on Reliable Software IEEE Cat. 75-CHO940-7-CSR, (1975) pp 59-71 2 Feiber, J D, Taylor, R N and Osterweil, L J 'NEWTON--A dynamic program analysis tool capabilities specification' Tech. Rpt. CU-CS200-81, University of Colorado, Dept. ofComp. Sci. (February 1981) 3 Luckham, D and Henke, F W 'An overview of Anna, a specification language for Ada' IEEE Soft. (March 1985) pp 9-22 4 Barnes, J G P Programming in Ada Addison-Wesley (1982) 5 Barringer, H and Means, I 'Axioms and proof rules for Ada tasks' 1EEE Proc., V129 It. E, No 2 (March 1982) 6 Helmbold, D and Luckham, D 'Debugging Ada tasking programs' IEEE Soft. (March 1985) pp 47-57 7 Luckham, D, Neff, R, Rosenblum, D 'An environment for Ada software development based on formal specification' Technical Rpt. CLS TR 86-305 Stanford University, 1986 8 Luckham, D C, Helmbold D P, Medal, S, Bran, D L and Harberler, M A 'Task sequencing language for specifying distributed Ada system: TSL-I' Tech Rpt. CLS-TR-87-334 Stanford Univ. Computer Systems Laboratory (July 1987) 9 Clark, E M, Emerson, E A and Sista, A P 'Automatic verification of finite-state concurrent systems using temporal logic specification' ACM Trans. on Programming Languages and Systems V8, 2 (April 1986) pp 244-263 10 Hailpern, B R Verifying concurrent processes using temporal logic Springer-Verlag (1982) 11 Lamport, L ' "SOMETIME" is sometimes "NOT NEVER" in the temporal logic of programs' ACM Trans. on Programming, V11, 7, (1980) pp 174-185 12 Manna, Z and Pnueli, A 'Verification of concurrent programs, Part 1: the temporal framework' Tech. Rpt. STAN-CS-81-836 Stanford Univ. Dept. of Comp. Sci. (June 1981) 13 Manna, Z and Pnueli, A 'Verification of concurrent programs: a temporal proof system' Tech. Rpt. STAN-CS-83-867 Stanford Univ. Dept. of Comp. Sci. (June 1983) 14 Moszkowski, B. Executing temporal logic programs Press Syndicate of The University of Cambridge (1986) 15 Tolba, H, Charpiilet, F and Haton, P 'Representing and propagating constraints in temporal reasoning' AI Communication Vol 4 No 4 (December 1991) 16 Osterwell, L J 'A strategy for integrating program testing and analysis' in Chandrasekaran, B and Radiechi, S (eds), Computer program testing North-Holland (1981) pp 187-229 17 Osterwell, L J 'Integrating the testing analysis and debugging of programs' in Hausen, H L (ed) Software validation Elsevier (1984) pp 73-102 18 Fairley, R E 'Tutorial: static analysis and dynamic testing of computer software' IEEE Trans. on Computer (April 1978) pp 14-23 19 Lamport, L 'Specifying concurrent program modules' ACM Trans. on Programming Languages and Systems V5, 2 (April 1983) pp 190-122 20 Pnueli, A 'The temporal semantics of concurrent programs' Concurrent Computation (1979) pp 1-20 21 Cohen, N H ADA as a second language McGraw-Hill (1986) 117