Chapter 21:

Testing Upgraded Applications

Testing the .NET upgraded application is arguably the most important part of the conversion process -and of any software development project as well-, which is why you need to design an adequate test strategy depending on the characteristics of the migrated application and the experience of your testing team. This final chapter provides up-to-date information on the main application test methodologies, the VB.NET testing tools available and the upgrade code review process; having a clear understanding of these matters will make testing more efficient and increase the accuracy of the results.

• How many approaches can I take to test an upgraded application?
• Why should I test my upgraded application?
• What are the basic steps in a testing process?
• Why is the test plan so important?
• What methodology should I follow to test my application?
• What items should be included in a test environment?
• Is it necessary to review the design of the application during the testing process?
• When reviewing the upgraded code, what actions must I take?
• What does ‘unit testing’ mean?
• Why should I perform black box testing on an upgraded application?
• In the context of white box testing, what does ‘profiling’ mean?
• What are the most commonly used test strategies for upgrade projects?
• What tools can be used to automatically test VB.NET applications?

How many approaches can I take to test an upgraded application?

Depending on their level of experience in testing migrated applications, testers can take two approaches. The first approach requires relatively more experience and less test effort than the second option.

In the first approach, it is assumed that the code automatically upgraded by the upgrade wizard works properly except in places where errors, warnings, and issues (EWIs) are present. In this approach, the test effort is focused on testing the fixes for the EWIs. However, even when EWIs are fixed, there can be functional holes whose pattern can be very different from those present in a normal development project. A tester who has considerable experience in testing upgraded applications can identify the functional holes by reviewing the code and the design of the application.

In the second approach, a more traditional test strategy and test process is followed, although this will likely require more test effort than the first approach. This approach is recommended for testers with limited experience testing upgrade projects.

Why should I test my upgraded application?

Because you have to ensure that the upgraded application meets the upgrade objectives. If the upgrade objective is only to achieve functional equivalence, the test objective would be to test for functional equivalence. However, if the application is upgraded for performance and/or security enhancement, testing must include performance and security. Furthermore, if the application is being advanced beyond functional equivalence with new features, the test effort must include functional, performance, and security testing of these new features.

What are the basic steps in a testing process?

Regardless of the test or upgrade strategies used, the testing process includes a basic set of steps that are to be followed:

• Create a test plan and test code
• Create a test environment
• Review the design
• Review the code
• Modifying existing test cases and code, or create additional test cases and code
• Perform unit testing – white box testing
• Perform black box testing
• Perform profiling – white box testing

Why is the test plan so important?

Because it documents the test cases that will be used to test the upgraded application. It also helps in planning and estimating the actual test effort that is required to prepare the test code and to execute the tests. Finally, the test plan helps in tracking the tests and their results, especially when there are multiple iterations or changes in specifications.

What methodology should I follow to test my application?

The methodology for creating test cases depends on the type of testing that will be performed. Test cases for design review, white box testing, and code review are prepared by reviewing the source code. Test cases for black box testing are prepared based on the use cases and the functional requirements. In the case of the upgrade project, if test cases are already available for the Visual Basic 6.0 version of the application, these test cases can be upgraded for the Visual Basic .NET version.

What items should be included in a test environment?

• The upgraded application or component to be tested
• Basic software for execution of the application, which consists of the following items:
• The .NET Framework 1.1 and Visual Studio .NET 2003
• Visual Basic 6.0 (in the case of iteration based test strategy or test driven upgrade strategy)
• Design review and code review tools (such as FxCop) and documents
• Unit testing tools (such as NUnit)
• Profiling tools, which may include the following tools:
• CLR Profiler
• Enterprise Instrumentation Framework (EIF)
• Windows Management Instrumentation (WMI)
• Performance testing tools, which may include the following tools:
• Application Center Test (ACT) for ASP.NET applications
• Performance counters

Is it necessary to review the design of the application during the testing process?

Not precisely. This step should be carried out only if design optimization is an upgrade objective or if the application is being upgraded to take advantage of features that are specific to the Microsoft .NET Framework, such as performance or security enhancements.

When reviewing the upgraded code, what actions must I take?

Review of the upgraded code includes, but is not limited to:

• Validating that the implementation conforms to the design. This validation is performed only for the features that are advancements in the upgraded application or for design changes in the upgraded application. The tester must have the design documents to validate that the implementation adheres to the design.
• Validating that the naming standards and commenting standards have been followed in the code. There are several standard coding conventions recommended for Visual Basic and .NET Framework code that provide various guidelines such as capitalization styles, case sensitivity, indentation, acronyms and abbreviations. Adhering to coding standards makes code easier to understand and maintain.
• Validating that performance and scalability guidelines have been followed in the code. This validation is performed only if performance enhancement is an upgrade objective or if advancements have been applied the upgraded application that might impact performance.
• Validating that the guidelines for writing secure code have been followed. This validation is performed only if security enhancement is an upgrade objective or if the upgrade application has been advanced with .NET Framework technologies that might impact security. Security code reviews help in identifying areas in code that have security vulnerabilities because of failure to adhere to security guidelines.

What does ‘unit testing’ mean?

Unit testing involves testing a single component. The components of the upgraded application are unit tested to detect failure scenarios in loops, conditional constructs, and internal subroutines. Unit testing also focuses on testing if upgrade EWIs have been properly fixed without affecting functionality or introducing new defects. Unit testing is also performed to make sure that the behavior of a method in an upgraded component is the same as that of the corresponding method in the original Visual Basic 6.0 version of the component.

Along with profiling, unit testing is part of what is called “white box testing”.

Why should I perform black box testing on an upgraded application?

In black box testing, the implementation details of the component or application to be tested are unknown. Black box testing simulates the end-user/client experience and tests the response of the upgraded applications to end-user or client actions. The objectives of black box testing are:

• To test for functional equivalence
• To test for functional equivalence of the upgraded application with the original Visual Basic 6.0 version of the application.
• To test that the upgraded application has achieved the upgrade objectives.
• To test that the upgraded application can respond properly to normal and exceptional usage scenarios. For invalid inputs, the upgraded application should provide error messages that are clear and present sufficient information to the user to enable diagnosis of the error.
• To test performance and security features, if required.

In the context of white box testing, what does ‘profiling’ mean?

Profiling is the process of gathering statistical information about an application during run time, such as the time and resource usage of an application and the execution tree exercised. This information can be analyzed for different purposes, such as identifying components that can be optimized to improve speed or resource use.

What are the most commonly used test strategies for upgrade projects?

There are three main strategies based on testing practices followed in different software development methodologies. The test process is fairly consistent across the different strategies, but the test planning and the test effort varies for these strategies, which are:

• Test Strategy Based on the Waterfall Methodology: In this strategy, after a single project of the entire application is upgraded, the upgraded project is unit tested. Then after all projects of the application are upgraded, all the upgraded projects/components are integrated to get a working version of the application, which is then system tested for functional equivalence.
• Test Strategy Based on the Iterative Methodology: This test strategy overcomes the shortcomings of the waterfall-based test strategy and is best used in complete upgrade projects for complex applications and for staged upgrade projects. In a staged upgrade strategy, each project/component is upgraded and is integrated with the system. The system consists of the .NET upgraded projects and the Visual Basic 6.0 projects communicating through interoperability techniques. After each project is upgraded, it is unit tested. After it passes testing, it is integrated with the system and the entire system is tested for functional equivalence.
• Test Strategy Based on the Agile Methodology: If the applications to be upgraded are large and very complex, the test strategies mentioned above become less effective and reliable. This is also true if the Visual Basic 6.0 application has to be advanced with .NET technologies, such as replacing all business COM components with Web services.

What tools can be used to automatically test VB.NET applications?

The current version of Visual Studio .NET does not provide tools specifically for automating the testing of Visual Basic .NET applications. However, other tools do exist for automating tests, for example:

NUnit: A popular tool that provides a unit-testing framework that can be applied to any .NET Framework language, including Visual Basic .NET.

FxCop: The FxCop tool is used to automate code review. It analyzes the application binaries to determine whether the implementation adheres to the predefined rules. These rules are customizable and can be extended

Application Center Test (ACT): Application Center Test is a performance testing tool. It is designed to stress Web servers and to test the performance of ASP.NET applications.

Visual Studio Analyzer: It is a performance analysis tool that is used to analyze and debug distributed applications. This tool analyzes components at a high level and presents information that can be used to identify the components that fail in a distributed environment.

Trace and Debug Classes: The .NET Framework comes with a pair of classes that can be used to print useful debugging messages during execution to allow you to see the flow of your application and to help you spot unusual behavior. These classes are Debug and Trace and both are defined in the System.Diagnostics namespace. Both classes are very similar and contain static methods to print arbitrary text messages, print messages only when tested expressions evaluate to true, pause and ask the user what to do when certain conditions do not evaluate to true, and format the text messages using simple indentation commands.

TraceContext Class: The TraceContext class provides functionality similar to the Trace and Debug classes, but it provides this functionality for ASP.NET applications. This class contains fewer methods but allows you to add conditional text to your HTML pages that are generated by ASP.NET and formats them for viewing in your browser. The page also automatically includes information that is related to your request and that can help debug problems in your Web applications.

CLR Profiler: CLR Profiler is a tool used for profiling memory utilization by an application. CLR Profiler profiles the interaction between the managed applications, the managed heap, and the garbage collector. CLR Profiler is used to identify memory leaks.

Enterprise Instrumentation Framework (EIF): EIF is used to instrument a .NET application so that it can be profiled and traced. After it is profiled and traced, application metrics can be captured. EIF encapsulates the functionality of event logging, Event Tracing for Windows (ETW), and Windows Management Instrumentation (WMI).

Performance Counters: Windows monitors many different aspects of the .NET runtime and the rest of the operating system. You can even define your own performance counters and then monitor those counters from the Performance Monitoring tool (Perfmon.exe) or from within your own applications.