INTEGRATION RELEASE ALMA 5.1.1
The release notes are available at the link R5_1_1 Release Notes
The user documentation is available at the link R5_1_1 UserDoc
The integration has been performed based on ACS-7.0.1
The software has been tagged with ALMA-5_1_1_32
The report with the results on the tests which has been run can be found at the twiki page R511 .
Short overview on the test cases run in the STEs:
|
Total number of test cases available |
237 |
|
Total number of test cases executed |
175 |
|
PASSED |
150 |
|
FAILED |
25 |
|
Total number of test cases not executed |
62 |
Short overview on the Jira tickets dealt with in the STE (simulation):
|
Total number of Jira tickets about ALMA 5.1.1 |
77 |
|
Newly opened Jira tickets for ALMA 5.1.1 |
52 |
|
Old jira tickets still to be solved for ALMA 5.1.1 |
25 |
We are also collecting metrics on the test cases written and executed by ITS and the Jira tickets opened as a consequence of running the test cases. The metrics will be periodically updated. The available graphs are :
- number of test cases written and executed
- % of test cases passed and failed
- number of jira tickets opened running the test cases
SLOC detailed figures:
Note: Starting from Release R1.1 ITS and SE have used a common approach in calculating SLOC.
Assessment on the sufficiency of Doxygen-like in-line documentation: Graph
|
Modules inline |
Global inline
|
You will have a page with the results per each subsystem (see list below). In this page of results, there is a table with 3 columns:
- the first one lists the modules belonging to the subsystem,
- the second gives the results of the tests (PASSED, FAILED, UNDETERMINED, when it is neither PASSED nor FAILED); sometimes the test directory can be missing, so you will just see the message: "No test directory, nothing to do here".
- the third one gives a resume produced by Purify/Coverage and/or Cobertura (For information about the tool, have a look at the Cobertura official web site)
- Purify/Coverage reports results on:
Functions
Functions and exits
Statement blocks
Implicit blocks
Decisions
Loops
The values reported for every item in the list above give the number of hits for that item.
In the same cell with the resume there is a link to the "Complete Report" produced by Purify. In the Complete Report one has information about the lines where the hit happened. For a loop, one has also the values: 0 (the loop is executed without entering into it), 1 (the loop is entered once), 2+ (the loop is entered and repeated once or more times).
Sometimes, instead of the resume, you will see a message like:
ERROR: No coverage info available
No atlout.spt
This happens when the modular test of that module is a fake one (for example the test is actually just an "echo something"), so there is no code (Java, Python or Cpp) that can be instrumented.
Note: the Test Coverage Analysis is not yet stable. Work is in progress to improve the results. You will see errors like:
This means that the test exists and is executed but Purify is not able to dump the information collected in the result file. This error is under investigation. We had 5 cases in ACS only
still under investigation, we don't know yet the reasons
We hope to clarify the problems under investigation as soon as possible!
