White Papers and Support Videos
- Introductory white paper
- Automatic White Box Testing of Ada Code white paper
Technical Description Contents
- Introduction to Mika;
- A Small Example;
- A Larger Example;
- A Real Example;
- New Testing Technology for Ada;
- Applicability;
- Ada;
- The GUI;
- Conclusions.
Introduction to Mika
Mika is a tool that generates test inputs automatically from your Ada source code. Mika does not make wild guesses; it understands your code and every test input increases the code coverage achieved. The test inputs are targeted at portions of the code that have not yet been executed by the set of test inputs generated so far. Thus, Mika does not generate 1 000s of unnecessary tests.
Mika does not need to be manually guided in its search for test inputs: there is no need to give extra information to Mika; all that Mika requires is the code under test. Mika is easy to setup and get started with; it is a completely separate tool and will not disturb your existing processes nor working environment. Because Mika is not a bloated, feature-laden tool, it is quick and easy to evaluate and master.
You do not need to change your existing Ada code in any way: there are no annotations to write, no specific subprograms to call at strategic points in your code, your code does not need to be structured in a special way (indeed it may even be messy, legacy code). Furthermore, Mika treats your code, and even your folders, as read-only assets that should be preserved; Mika will not write into your existing folder structure nor change your code automatically: no damage can occur, as every file is considered read-only by Mika.
Mika can be downloaded for free for evaluation purposes. Upgrading to the full license is a small investment given that hours of tedious work can be performed automatically in minutes. Think about it, how long does it take to generate test inputs to achieve a suitable level of code coverage for 1 000s of lines of new code? What about regression testing? Legacy code that is only partially covered by an existing test suite (Mika can complete your original test suite while preserving it)? What about integration testing?
Mika is the first tool of its kind: no other tool can generate targeted test inputs automatically on arbitrary code. Existing, traditional, testing tools execute tests, record tests outcomes, monitor tests executions etc. These activities are essential and Mika does not replace these traditional testing tools. Simply, Mika automatises, in a straightforward manner, the last manual task of software testing: the test data generation process.
Mika is specially targeted at Ada code developed for the embedded system, real time system, high integrity system and safety critical system market.
Mika is exceptionally suitable when dealing with long, complex, code, algorithms and data structures: this is exactly when manual test inputs generation becomes a nightmare. On straightforward code Mika will just save you time.
Mika is extremely powerful, it can generate targeted unit test inputs automatically of course, but also integration tests. You can point Mika at a fairly high level Ada subprogram and it will generate automatically test inputs for that subprogram but these tests, will also exercise all the reachable constructs of all the lower level subprograms directly or indirectly called by your chosen top level subprogram. That is, Mika can perform automatic integration testing.
Mika also helps in the process of obtaining test cases, that is test inputs along side expected code behaviour.
A Small Example
To start with we will look at a small example to establish Mika's purpose. Here is the contents of credit_card.ads
package credit_card is
type CreditCardLimit is (Refused, Basic, Intermediate, Full);
function decide(over18, existingCustomer, badCredit, houseAsCondition : Boolean) return CreditCardLimit;
end credit_card;
and below is the contents of credit_card.adb:
package body credit_card is
function decide(over18, existingCustomer, badCredit, houseAsCondition : Boolean) return CreditCardLimit is
begin
if not over18 then return Refused;
else if over18 and not existingCustomer and not badCredit and not houseAsCondition then return Basic;
else if over18 and existingCustomer and badCredit and not houseAsCondition then return Basic;
else if over18 and existingCustomer and not badCredit and not houseAsCondition then return Refused;
else if over18 and not existingCustomer and not badCredit and not houseAsCondition then return Refused;
else if over18 and existingCustomer and badCredit and houseAsCondition then return Full;
else if over18 and not existingCustomer and not badCredit and houseAsCondition then return Full;
end if;
end if;
end if;
end if;
end if;
end if;
end if;
if over18 and existingCustomer and not badCredit and not houseAsCondition then return Intermediate;
end if;
return Refused;
end decide;
end credit_card;
The code is not very readable and is not correct but it is a very simple example. Mika produces the following test inputs in its attempt to achieve 100% branch coverage of the decide subprogram:
----------------------------------------------------------
-- MIKA TEST DATA GENERATOR --
-- Copyright Midoan Software Engineering Solutions Ltd. --
-- http://www.midoan.com/ --
----------------------------------------------------------
-- Original Directory: F:/bck_Mika/examples/to distribute/
-- Package: credit_card
-- Subprogram: decide
-- Strategy: branch
-- Context: not ignored
-- Coverage Depth: Subprogram Only
----------------------------------------------------------
TEST NUMBER 1
PREDICTED COVERED BRANCHES :
Number 6, outcome true, in file credit_card.adb, on line 10 and column 32
Number 5, outcome false, in file credit_card.adb, on line 9 and column 27
Number 4, outcome false, in file credit_card.adb, on line 8 and column 22
Number 3, outcome false, in file credit_card.adb, on line 7 and column 17
Number 2, outcome false, in file credit_card.adb, on line 6 and column 12
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 6, outcome true, in file credit_card.adb, on line 10 and column 32
Number 5, outcome false, in file credit_card.adb, on line 9 and column 27
Number 4, outcome false, in file credit_card.adb, on line 8 and column 22
Number 3, outcome false, in file credit_card.adb, on line 7 and column 17
Number 2, outcome false, in file credit_card.adb, on line 6 and column 12
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
CONSTRUCTED TEST INPUT
over18 = true
existingcustomer = true
badcredit = true
houseascondition = true
CODE BEHAVIOUR
decide_return = full
----------------------------------------------------------
TEST NUMBER 2
PREDICTED COVERED BRANCHES :
Number 3, outcome true, in file credit_card.adb, on line 7 and column 17
Number 2, outcome false, in file credit_card.adb, on line 6 and column 12
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 3, outcome true, in file credit_card.adb, on line 7 and column 17
CONSTRUCTED TEST INPUT
over18 = true
existingcustomer = true
badcredit = true
houseascondition = false
CODE BEHAVIOUR
decide_return = basic
----------------------------------------------------------
TEST NUMBER 3
PREDICTED COVERED BRANCHES :
Number 4, outcome true, in file credit_card.adb, on line 8 and column 22
Number 3, outcome false, in file credit_card.adb, on line 7 and column 17
Number 2, outcome false, in file credit_card.adb, on line 6 and column 12
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 4, outcome true, in file credit_card.adb, on line 8 and column 22
CONSTRUCTED TEST INPUT
over18 = true
existingcustomer = true
badcredit = false
houseascondition = false
CODE BEHAVIOUR
decide_return = refused
----------------------------------------------------------
TEST NUMBER 4
PREDICTED COVERED BRANCHES :
Number 8, outcome false, in file credit_card.adb, on line 19 and column 7
Number 7, outcome false, in file credit_card.adb, on line 11 and column 37
Number 6, outcome false, in file credit_card.adb, on line 10 and column 32
Number 5, outcome false, in file credit_card.adb, on line 9 and column 27
Number 4, outcome false, in file credit_card.adb, on line 8 and column 22
Number 3, outcome false, in file credit_card.adb, on line 7 and column 17
Number 2, outcome false, in file credit_card.adb, on line 6 and column 12
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 8, outcome false, in file credit_card.adb, on line 19 and column 7
Number 7, outcome false, in file credit_card.adb, on line 11 and column 37
Number 6, outcome false, in file credit_card.adb, on line 10 and column 32
CONSTRUCTED TEST INPUT
over18 = true
existingcustomer = false
badcredit = true
houseascondition = true
CODE BEHAVIOUR
decide_return = refused
----------------------------------------------------------
TEST NUMBER 5
PREDICTED COVERED BRANCHES :
Number 7, outcome true, in file credit_card.adb, on line 11 and column 37
Number 6, outcome false, in file credit_card.adb, on line 10 and column 32
Number 5, outcome false, in file credit_card.adb, on line 9 and column 27
Number 4, outcome false, in file credit_card.adb, on line 8 and column 22
Number 3, outcome false, in file credit_card.adb, on line 7 and column 17
Number 2, outcome false, in file credit_card.adb, on line 6 and column 12
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 7, outcome true, in file credit_card.adb, on line 11 and column 37
CONSTRUCTED TEST INPUT
over18 = true
existingcustomer = false
badcredit = available in full version only, please buy a license
houseascondition = true
CODE BEHAVIOUR
decide_return = full
----------------------------------------------------------
TEST NUMBER 6
PREDICTED COVERED BRANCHES :
Number 2, outcome true, in file credit_card.adb, on line 6 and column 12
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 2, outcome true, in file credit_card.adb, on line 6 and column 12
CONSTRUCTED TEST INPUT
over18 = available in full version only, please buy a license
existingcustomer = false
badcredit = false
houseascondition = false
CODE BEHAVIOUR
decide_return = basic
----------------------------------------------------------
TEST NUMBER 7
PREDICTED COVERED BRANCHES :
Number 1, outcome true, in file credit_card.adb, on line 5 and column 7
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 1, outcome true, in file credit_card.adb, on line 5 and column 7
CONSTRUCTED TEST INPUT
over18 = false
existingcustomer = true
badcredit = false
houseascondition = false
CODE BEHAVIOUR
decide_return = refused
----------------------------------------------------------
FINAL REPORT
87% branch OVERALL COVERAGE PREDICTED.
14 BRANCHES PREDICTED COVERED:
Number 1, outcome false, in file credit_card.adb, on line 5 and column 7
Number 1, outcome true, in file credit_card.adb, on line 5 and column 7
Number 2, outcome false, in file credit_card.adb, on line 6 and column 12
Number 2, outcome true, in file credit_card.adb, on line 6 and column 12
Number 3, outcome false, in file credit_card.adb, on line 7 and column 17
Number 3, outcome true, in file credit_card.adb, on line 7 and column 17
Number 4, outcome false, in file credit_card.adb, on line 8 and column 22
Number 4, outcome true, in file credit_card.adb, on line 8 and column 22
Number 5, outcome false, in file credit_card.adb, on line 9 and column 27
Number 6, outcome false, in file credit_card.adb, on line 10 and column 32
Number 6, outcome true, in file credit_card.adb, on line 10 and column 32
Number 7, outcome false, in file credit_card.adb, on line 11 and column 37
Number 7, outcome true, in file credit_card.adb, on line 11 and column 37
Number 8, outcome false, in file credit_card.adb, on line 19 and column 7
2 BRANCHES PREDICTED REMAINING TO BE COVERED:
Number 5, outcome true, in file credit_card.adb, on line 9 and column 27
Number 8, outcome true, in file credit_card.adb, on line 19 and column 7
7 test inputs generated
0 paths fully attempted but for which test inputs could not be generated
----------------------END OF REPORT-----------------------
A few remarks are necessary at this stage:
- The test inputs generation speed: these test inputs took 3 seconds to generate on a Centrino Duo run laptop;
- Mika gives the expected output of the test run, given the code, for each test input: these predictions should not be blindly relied upon in any meaningful way as Mika has not actually executed the test inputs and may be wrong. We just think that during the test data generation process these predictions may be useful to some users: you can validate the behaviour of your code without having to actually execute it if you wish: this information if correct may lead to test cases;
- Every generated test input increases the coverage of the code under consideration;
- Only 87% branch coverage is achieved: this is not due to Mika's limitations but rather because 2 of the indicated branches are unreachable and the subprogram contains unreachable code.
Even on this simple example Mika clearly shows its usefulness.
A Larger Example
Mika handles enumeration types, integers, floating point numbers organised in arrays, records or any combinations of these. It also handles conditional statements, loops etc. For example the following package specification can be easily handled by Mika:
package array_date is
type name_t is (mon, tue, wed, thu, fri, sat, sun);
type year_t is range 1900 .. 3000;
type day_t is range 1..31;
type month_t is (jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec);
type date_t is
record
name : name_t;
day : day_t;
month : month_t;
year : year_t;
end record;
type index is range 1..50;
type list is array(index) of date_t;
procedure InsertionSort(L: in out list);
end array_date;
and the algorithms may be complex:
package body array_date is
function is_leap(y : year_t) return boolean is
begin
return (y mod 4 = 0 and y mod 100 /= 0) or y mod 400 = 0;
end is_leap;
function preceeds(d1, d2 : date_t) return boolean
is
p : boolean;
begin
if d1.year < d2.year then
p := true;
elsif d1.year > d2.year then
p := false;
elsif d1.month < d2.month then
p := true;
elsif d1.month > d2.month then
p := false;
elsif d1.day < d2.day then
p := true;
elsif d1.day > d2.day then
p := false;
else
p := false;
end if;
return p;
end preceeds;
procedure InsertionSort(L: in out list) is
place : index;
current : date_t;
found : boolean;
begin
for firstunsorted in index range index'succ(index'first) .. index'last loop
if preceeds(L(firstunsorted), L(index'pred(firstunsorted))) then
place := firstunsorted;
current := L(firstunsorted);
loop
place := index'pred(place);
L(index'succ(place)) := L(place);
if place = index'first then
found := true;
else
found := preceeds(L(index'pred(place)), current);
end if;
exit when found;
end loop;
L(place) := current;
end if;
end loop;
end InsertionSort;
begin
null;
end array_date;
but still, even on this example which contains a loop within a loop, Mika generates integrations test inputs in seconds. For example, here is test number 2 out of 5 that Mika generated in under 6 seconds for the insertion sort procedure above:
TEST NUMBER 2
PREDICTED COVERED BRANCHES :
Number 12, outcome false, in file array_date.adb, on line 63 and column 4
Number 15, outcome true, in file array_date.adb, on line 75 and column 16
Number 6, outcome true, in file array_date.adb, on line 40 and column 5
Number 14, outcome false, in file array_date.adb, on line 70 and column 9
Number 15, outcome false, in file array_date.adb, on line 75 and column 16
Number 9, outcome true, in file array_date.adb, on line 46 and column 8
Number 8, outcome false, in file array_date.adb, on line 44 and column 8
Number 7, outcome false, in file array_date.adb, on line 42 and column 8
Number 6, outcome false, in file array_date.adb, on line 40 and column 5
Number 7, outcome true, in file array_date.adb, on line 42 and column 8
Number 13, outcome true, in file array_date.adb, on line 64 and column 5
Number 12, outcome true, in file array_date.adb, on line 63 and column 4
Number 13, outcome false, in file array_date.adb, on line 64 and column 5
Number 8, outcome true, in file array_date.adb, on line 44 and column 8
Number 14, outcome true, in file array_date.adb, on line 70 and column 9
BRANCHES COVERAGE INCREASED BY BRANCHES:
Number 9, outcome true, in file array_date.adb, on line 46 and column 8
Number 8, outcome false, in file array_date.adb, on line 44 and column 8
CONSTRUCTED TEST INPUT
l = ((wed, 24, mar, 1917),(fri, 14, sep, 1900),(sat, 28, jan, 1902),(wed, 12, jun, 1903),(wed, 6, jun, 1905),(sat, 3, jan, 1917),(fri, 7, jan, 1909),(sat, 24, jan, 1910),(thu, 4, sep, 1982),(wed, 17, oct, 1963),(mon, 4, oct, 1965),(thu, 20, mar, 1966),(thu, 16, feb, 1967),(tue, 24, oct, 1969),(wed, 24, may, 1972),(thu, 2, sep, 1973),(tue, 18, sep, 1974),(mon, 6, apr, 1975),(thu, 1, aug, 1976),(wed, 21, aug, 1977),(sat, 12, aug, 1978),(fri, 19, sep, 1979),(tue, 27, nov, 1980),(wed, 14, aug, 2881),(fri, 23, may, 1983),(fri, 19, oct, 1986),(wed, 7, mar, 1987),(mon, 26, oct, 1991),(thu, 8, jul, 1993),(fri, 5, jul, 2083),(thu, 8, aug, 2042),(wed, 9, oct, 2785),(wed, 20, aug, 2871),(mon, 9, apr, 2878),(tue, 14, may, 2990),(fri, 23, oct, 2929),(thu, 2, may, 2948),(mon, 8, feb, 2973),(wed, 17, mar, 2983),(thu, 28, oct, 2984),(wed, 1, sep, 2985),(tue, 15, oct, 2987),(mon, 14, jun, 2988),(wed, 16, jan, 2999),(thu, 12, feb, 2991),(mon, 28, jul, 2992),(fri, 4, nov, 2993),(wed, 8, oct, 2994),(mon, 5, jul, 2996),(fri, 5, jun, 2994))
CODE BEHAVIOUR
l = ((fri, 14, sep, 1900),(sat, 28, jan, 1902),(wed, 12, jun, 1903),(wed, 6, jun, 1905),(fri, 7, jan, 1909),(sat, 24, jan, 1910),(sat, 3, jan, 1917),(wed, 24, mar, 1917),(wed, 17, oct, 1963),(mon, 4, oct, 1965),(thu, 20, mar, 1966),(thu, 16, feb, 1967),(tue, 24, oct, 1969),(wed, 24, may, 1972),(thu, 2, sep, 1973),(tue, 18, sep, 1974),(mon, 6, apr, 1975),(thu, 1, aug, 1976),(wed, 21, aug, 1977),(sat, 12, aug, 1978),(fri, 19, sep, 1979),(tue, 27, nov, 1980),(thu, 4, sep, 1982),(fri, 23, may, 1983),(fri, 19, oct, 1986),(wed, 7, mar, 1987),(mon, 26, oct, 1991),(thu, 8, jul, 1993),(thu, 8, aug, 2042),(fri, 5, jul, 2083),(wed, 9, oct, 2785),(wed, 20, aug, 2871),(mon, 9, apr, 2878),(wed, 14, aug, 2881),(fri, 23, oct, 2929),(thu, 2, may, 2948),(mon, 8, feb, 2973),(wed, 17, mar, 2983),(thu, 28, oct, 2984),(wed, 1, sep, 2985),(tue, 15, oct, 2987),(mon, 14, jun, 2988),(tue, 14, may, 2990),(thu, 12, feb, 2991),(mon, 28, jul, 2992),(fri, 4, nov, 2993),(fri, 5, jun, 2994),(wed, 8, oct, 2994),(mon, 5, jul, 2996),(wed, 16, jan, 2999))
---------------------------------------------------------
It is worth remarking that the 5 test inputs generated for the insertion sort procedure have been constructed to also achieve 100% branch coverage of the called function preceeds: integration testing can be performed immediately using the thorough test inputs generated by Mika.
Real Example
For confidentiality reasons, as well as space, we cannot provide here details of industrial code that Mika has been used on. But as an example, testing a relatively top level subprogram led to the analysis of 102 372 comments free lines of code. The parsing phase took 30 seconds and the test inputs generation phase 2 minutes and 5 seconds. A typical test is given below (variables names have been changed for confidentiality reasons):
CONSTRUCTED TEST
xxxmap = (0.06617305067123169, 0.12295904277980485, 0.8785759587789579, 0.39659430955693376, 0.14488553006802718, 0.21718711131474588, 0.1800765271681204, 0.7372410229476714, 0.7755867113041441, 0.45786957261984984, 0.4739810056769589, 0.4975957269100171, 0.8827100644338154, 0.7255320749737055, 0.6440613602274969, 0.9093686077387186, 0.551605420156744, 0.904164378158945, 0.31909690801214663, 0.7541334781333321, 0.1088794915657405, 0.06214741498853171, 0.31087054475045317, 0.5940601981711333, 0.12779432033980953, 0.5683564215543053, 0.9467390017672566, 0.02856818062399169, 0.6587920217169372, 0.7807627961070114, 0.02343253299568282, 0.40715488084810314, 0.5328413289976393, 0.5094237517343556, 0.11775805830797559, 0.6981756258743719, 0.32732702894467325, 0.5583751075317254, 0.7211939728689295, 0.4291297077705405, 0.43851500602467275, 0.5540766053668811, 0.5319810688961897, 0.7690359547295909, 0.2620176842371129, 0.6667749283863884, 0.8753277571929208, 0.11476446354700531, 0.6855823982689258, 0.13457393324248157, 0.2856003749225571, 1749.8889814929664, 0.8330609976527367, 0.9733725771068924, 0.8619237433458031, 0.28837495501487376, 0.08217568847809353, 0.5220031239833154, 0.48193049720074344, 0.8017210056115953, 0.8758419701599862, 21286793, 23901991, ((242147, false, 1, 2, 162),(392843, false, 2, 0, 93),(280886, false, 1, 1, 33),(356425, false, 1, 0, 239),(355162, false, 1, 0, 19),(511829, false, 0, 0, 169),(38609, false, 0, 0, 63),(162703, false, 0, 1, 252),(438944, false, 2, 0, 74),(301242, false, 2, 0, 172),(465669, false, 2, 0, 80),(455658, false, 1, 2, 218),(371023, false, 2, 2, 70),(73770, false, 1, 0, 203),(171107, false, 0, 2, 209),(373387, false, 1, 1, 242),(229615, false, 0, 1, 246),(325222, false, 0, 2, 154),(358890, false, 0, 1, 223),(352106, false, 0, 2, 156),(430752, false, 0, 0, 209),(354852, false, 1, 2, 84),(213851, false, 0, 0, 205),(249742, false, 1, 0, 66),(370908, false, 1, 2, 159),(372670, false, 0, 1, 58),(117485, false, 1, 1, 241),(480745, false, 0, 1, 67),(194318, false, 2, 1, 162),(419577, false, 1, 1, 250),(22403, false, 0, 0, 177),(436888, false, 1, 1, 233)), ((160671, false, 1, 0, 156),(317582, false, 1, 1, 113),(435168, false, 0, 0, 168),(23854, false, 1, 2, 127),(166671, false, 0, 2, 171),(24423, false, 0, 0, 143),(76125, false, 1, 0, 110),(83952, false, 2, 0, 42),(74849, false, 2, 1, 56),(270655, false, 1, 0, 143),(235115, false, 1, 2, 102),(73211, false, 1, 0, 97),(274679, false, 0, 2, 125),(410665, false, 2, 0, 120),(489716, false, 2, 2, 120),(417027, false, 0, 2, 120),(125799, false, 2, 0, 8),(466276, false, 1, 0, 77),(350914, false, 1, 2, 23),(271670, false, 1, 1, 130),(349782, false, 2, 0, 196),(230979, false, 2, 0, 190),(418653, false, 1, 1, 112),(451233, false, 2, 1, 17),(250810, false, 0, 1, 111),(127770, false, 2, 1, 180),(218697, false, 0, 1, 24),(513376, false, 0, 2, 68),(370610, false, 0, 0, 240),(37044, false, 0, 0, 243),(231206, false, 1, 1, 98),(390582, false, 2, 1, 44)), ((156693, false, 2, 1, 14),(178797, false, 0, 2, 148),(297497, false, 2, 1, 114),(229236, false, 1, 1, 173),(287680, false, 2, 0, 126),(122296, false, 1, 1, 190),(30921, false, 1, 2, 182),(7507, false, 1, 0, 75),(427346, false, 0, 2, 248),(324476, false, 1, 2, 199),(398320, false, 2, 1, 80),(194904, false, 2, 2, 195),(351259, false, 0, 1, 23),(430672, false, 1, 2, 57),(13022, false, 2, 1, 252),(470952, false, 2, 0, 89),(522034, false, 2, 0, 235),(352713, false, 1, 2, 234),(443771, false, 1, 0, 62),(61324, false, 1, 0, 211),(153456, false, 1, 1, 35),(171423, false, 1, 0, 152),(298793, false, 1, 0, 40),(342502, false, 0, 1, 158),(362682, false, 2, 2, 244),(324593, false, 2, 0, 170),(300010, false, 0, 1, 41),(260067, false, 1, 0, 219),(385143, false, 1, 1, 246),(406910, false, 1, 0, 223),(49082, false, 1, 2, 247),(400337, false, 1, 0, 236)), ((360094, false, 0, 1, 102),(160058, false, 2, 2, 87),(93184, false, 2, 0, 173),(250432, false, 0, 1, 201),(200566, false, 1, 2, 100),(156314, false, 0, 0, 28),(156065, false, 1, 0, 219),(379822, false, 0, 2, 134),(159383, false, 1, 1, 63),(402656, false, 1, 0, 77),(319117, false, 2, 1, 156),(128170, false, 2, 1, 227),(288631, false, 1, 1, 137),(94816, false, 0, 0, 134),(23921, false, 1, 2, 181),(316375, false, 0, 1, 121),(143098, false, 2, 1, 153),(370786, false, 2, 2, 134),(228222, false, 1, 0, 149),(451825, false, 1, 2, 48),(19323, false, 2, 1, 187),(338560, false, 1, 2, 9),(169532, false, 2, 2, 181),(12801, false, 2, 1, 206),(132064, false, 1, 1, 65),(57558, false, 2, 1, 147),(432264, false, 2, 2, 33),(124289, false, 1, 1, 47),(458676, false, 0, 2, 153),(50345, false, 0, 1, 231),(468324, false, 0, 2, 224),(94047, false, 0, 1, 134)), (false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false), (false, false, false, false, false, false, false, false, false), (16477, false, false, false, false), -19376324, 22708901, 32946683, (false, false), (false, false), 39398, 53293, ((false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false)), ((false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false)), ((false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false)), ((false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false),(false, false, false, false)), 49834, (hw_xxxx => false), (false, false, false, false, false, false, false), (false, false, false, false, false, false), (hw_xxxx => false), (false, false), (false, false, false), (false, false), (false, false, false, false, false, false), (false, false, false, false), (false, false, false, false, false, false, false), (false, false, false, false, false, false, false), (false, false, false, false), (false, false, false, false, false, false, false, false, false, false), (false, false, false, false, false), (false, false, false, false, false, false), (false, false, false, false, false, false), 18297, 20088, 14462, 33362, 3597, 51683, 11265, 35762, (hw_xxxx => false), 58064, 3869, 58840, 3553, 39461, 36326, 25207, 29325, 59018, 48582, 30910, 15392, (false, false, false), 15034, (false, false, false), 11417, (false, false, false), 8330, (false, false, false), 46787, (false, false, false), 21202, (false, false, false), 4917, (false, false, false), 39985, (false, false, false), 22654, (false, false, false, false, false, false, false), (false, false, false, false, false, false, false), 15666, 38055, 3145, 36247, 20515, 3468, 14700, 8909, 29071, 15229, 40000, 8700, (false, false, false, false, false, false, false, false), (false, false, false, false, false, false, false, false), (false, false, false, false, false, false, false, false), 43231, 33892, 55145, 33107, 9232, 57526, 62323, 13403, 44771, 9121, 39741, 18259, (7337,19179,51042,34365,25970,47676,27200,31153,42886,16436,34553,60455,32291,13264,50891,20311,64785,13039,39636,62640, 37618,41650,17192,51718,64239,15326,34138,24555,64813,7239,10613,19369,7143,20589,36756,8268,3696,10786,42506,11905,44219, 39125,25834,58873,42001,10873,39161,26090,4036,57037,29313,39316,54915,17843,64433,39569,35790,44410,30071,50817,7182,14720, 1291,26850,64675,23572,20535,33702,2728,33802,63271,14321,42126,4199,50302,7091,6851,64381,41457,20322,51466,23286,59769, 49478,38907,40559,16885,22317,61383,38322,14645,46136,60033,60200,34042,45246,51069,62022,48052,53744,16216,9528,61427,8141, 32813,14133,3338,7869,34155,8604,60137,59540,44514,27930,45578,52071,43533,917,57940,14121,45722,57473), (5526,21935,20216,31280,46818,9491,33417,12150,45493,28362,34346,37640,44363,58687,1704,22309,38887,62136,1521,39096,19598, 12578,45471,61650,39473,56941,7603,58791,21301,40024,57510,25859,29454,45953,59896,56204,37706,22815,10700,18531,19153,15447, 3757,34940,23687,61234,35474,35624,54057,26631,59013,37834,20631,51727,61688,61522,23430,20419,24220,28954,51258,83,5305,7796, 58393,60125,26611,26896,63340,24367,16485,22362,60600,80,23488,14443,29926,34945,43587,50533,57031,47029,64506,40322,25432, 4532,54726,18616,8850,9065,31329,21345,44662,107,37050,63284,50658,39965,3359,47834,28658,6616,62866,50249,28921,18230,20889, 7333,17660,11085,48616,4804,45044,12033,41347,26592,65173,40724,59197,41310,55385,22382), (5224,43101,49840,26804,59477,61362,48391,7590,55665,8316,12751,41649,20179,62535,35054,20300,13252,28706,29799,64001,56166, 17486,18306,31974,36157,28240,6873,37806,33730,6344,29593,25289,23321,48360,25925,18994,48045,12743,34286,62119,36660,8584, 2606,12377,42381,18708,10791,8616,10099,37917,21702,37871,52123,51818,19899,31917,63945,3879,36114,55629,41556,9734,21666, 32955,15188,59746,40224,55225,30503,50667,32731,25483,63372,55999,33688,115,19236,37841,49432,18844,58890,31190,51676,14645, 16440,14318,37875,26149,2155,33362,19865,56715,10225,5539,51545,32575,59292,34287,18926,2939,28417,5034,5213,29685,19426, 22990,38523,25019,4424,20985,17002,36599,61777,21412,6403,22767,45611,27474,4188,42031,17589,51166), (false, false), 59865, 9318, 13089, (hw_xxx => false), (112,180,128,201,227,39,196,110,197,23,45,204,190,83,91,141,111,155,102,105,26,72,143,188,157,179,197,193,143,16,108,240,93, 205,46,236,22,6,160,62,45,52,246,86,246,11,44,123,225,164,80,15,73,50,213,51,110,140,178,183,198,61,58,135), (237,43,17,251,59,236,124,143,251,49,59,63,5,219,56,215,161,76,138,188,146,29,131,225,5,159,40,150,86,214,208,78,5,16,198,197 ,24,45,116,237,230,161,121,110,4,39,150,40,135,179,166,68,42,36,160,73,189,238,219,38,176,11,67,97))
txxxxxxrator = (-29163133, 8389424, -4096926, 277, false, false, false)
txxxxegrator = (18160214, 3440635, -10489686, 162, false, false, false)
tcxxxxx = (-21387016, -25093086, -20733255, 341, false, false, false)
txximer1 = (131072.00268120313, 131075.4866431916, false, false)
txxxpass = true
nxx = 110.03891085218137
cxxx = (4, 235, 121, 0.07029246237696718, 0.9117002200274165, 0.3241691788330361, 0.2556708266732173, 0.958574375707818, 50.9436345215915, 0.6533296436826379, 0.4569205210974012, 0.2387383126297391, 0.8424016090085364, 0.4320727661317829, 0.6120647794539869, 0.44314829041797354, 0.9020592814796486, 0.2819404649078203, 0.3691900248453661, 0.5027896038533632, 0.44586498836912436, 0.9109376329620771, 0.27334696342138753, 0.7323986913761003, 0.7805191277589112, 0.6792931205803194, 0.5312737457133601, 1549.586758961848, 0.45186767019888263, 0.18721976513469674, 0.28749095661033364, 0.6163248154666034, 0.20739865208244934, -25479072, 0.43273609019933845, 0.2993819311242243, 0.6639845302475132, 28450533, -32479900, 12, 196, 219, 101, 88, 198, 164, 193, 55, 225, 29, 225, 101, 224, 214, 142, 248, 112, 58, 82, 244, 249, 148, 2, 98, 37, 26, 81, 51, 45, 52, 165, 5, 150, 196, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false, true, false, false, false, false, false, false, false, false, false, false, false, (true,true,false,false,false,false,false,false,true,true,false,true,false,true,false,false,true,false,false,false,false,true, false,true,true,true,true,true,true,false,false,false,false,true,false,true,true,false,true,true,false,false,false,true,true,true,true,false,false,true,false,true,true,false,true,false,false,false,true,false,true,false,true,true,true,true,true,true,true,false,true,true,true,true,false,true,false,true,true,false,true,false,true,true,false,false,true,true,true,true,true,true,false,true,false,false,true,false,false,false,true,true,false,true,true,false,true,true,true,true,false,true,true,true,true,false,true,true,false,false,true,true,true,false,false,true,true,true,false,true,true,true,true,true,false,true,true,false,true,false,false,true,true, false,true,true,true,false),(true,false,false,true,false,false,true,false,true,false,true,true,true,true,false,true,true,false,false,true,true,true,true, true,true,true,false,false,true,true, true,true,false,false,true,true,true,true,false,false,false,true,true,true,false,false,false,true,false,true,true,true,true,true, false,true,true,true,false,false,false,true,false,true,true,true,true))
hxxxxault = false
cjxx = 50.876315285841535
cxxxat = true
Remember that the test inputs above is not random: it has been constructed to traverse a specific path in the code under test. Also the full behaviour of the code for this test input is generated by Mika; once validation of the code behavior is performed, a test case for a very specific sub-behaviour of the code has been created and can be re-used automatically via Mika regression testing functionality.
New Testing Technology for Ada
Mika uses new algorithms to construct test inputs that target precise areas in your Ada code; not a single random test input is ever generated. Each test input is designed to exercise an hitherto uncovered portion of your code. Using the test inputs generated automatically, the entire unit and integration testing can be performed in seconds.
Of prime interest is the wide applicability of Mika: test inputs for unit testing purposes of course, but more useful, test inputs for inter-procedural testing too that are suitable for integration testing at any level; Mika is able to construct test inputs that will fully exercise called subprograms. Hence subprograms can be tested within their usage environment; the tests are more realistic and bugs are more likely to be detected.
The coverage criteria that Mika can handle are: branch, decision and masked MC/DC.
Applicability
Mika is primarily aimed at the following overlapping domains: embedded systems, real time system, high integrity systems and safety critical systems. Although Mika is not certified it can be used in such domains since it replaces a manual process and the actual testing can be performed using your existing, certified or not, basic testing tool. These traditional tools offer you the possibility to measure the actual code coverage achieved by the tests generated by Mika; no loss of integrity in your testing process occurs by using Mika. Running your tests can still be performed in your chosen environment using your own target compiler.
Mika is just a very fast automatic testing tool for generating targeted tests at your Ada source code; you can use the tests generated for any purpose you wish, and using whatever your current testing environment is. Adopting Mika does not interfere with current setup or processes.
Ada
Ada code targeted at high integrity systems is usually written in a subset of Ada. One such well known subset is Spark Ada. Mika can fully handle Spark Ada.
So if your code is written using a subset of Ada, Mika can probably handle it. To check if your code can be handled by Mika you can try it for free and check for yourself. Mika can be set up in under 2 minutes and requires no training (beyond being familiar with the short user manual).
Examples of constructs that Mika does not handle include:
- Ada task;
- Access types;
- Exceptions;
- Generic Packages.
More details of the subset handled by Mika can be found in the latest documentation. It is important to note that, whilst some constructs are not explicitly handled (e.g. exceptions), you can still generate test inputs from code containing such constructs as Mika will ignore those code portions: the tests may not be as thorough but may well still be useful.
The subset handled by Mika does not contain any restrictions with respect to scope, visibility, renaming or overloading. So if your code is genuinely targeted at the embedded or high integrity area then it is unlikely that Mika's limitations will be a problem on your actual industrial code.
We are obviously open to suggestions regarding what additional constructs Mika's next version should cater for.
The GUI
We have designed an efficient user interface for the purpose of automatic test inputs generation from Ada source code. Mika's interface cannot be used to edit source code; your own, familiar to you, existing development environment should be used for that purpose.
As mentioned, Mika is non intrusive: it works in a separate working directory, it does not change your code, it is impossible to change your code via Mika's interface etc. In the same philosophy, you should still use you own compiler, traditional testing tool etc.
Here is a screenshot of Mika's GUI (you may click on it to get the full picture)
Briefly:
- Area 1 allows you to navigate to your code directories;
- Area 2 is a mixed view of your Ada packages, subprograms and previous test sets;
- Area 3 allows you to customise the parsing phase;
- Area 4 allows you to customised the test inputs generation phase and if you wish actually run the tests generated;
- Area 5 is the error, status and warning area;
- Finally Area 6, displays code files and the tests generated.
Conclusions
As seen, Mika replaces the tedious manual test inputs generation process; days of manual work can now be automatically performed within minutes. Mika can integrate with your existing testing tools and does not displace any of your existing testing apparatus.
Also, the process is entirely automatic. No scripting is necessary: Mika analyses your code automatically and generates test inputs that are targeted at executing your code as thoroughly as possible.
More information can be found in the actual documentation documents. You should also contact us should you have any queries or questions about Mika.
Try Mika for free today, tell us your problems, request special features or extensions and upgrade to a full license.