HP Performance Validation SDK: Protocol Add-on Certification Procedure

This document provides a checklist for certifying your protocol add-on for LoadRunner according to Hewlett-Packard (HP) standards for protocol development. After reviewing this document, complete the tests specified. Fill in the accompanying HP Performance Validation SDK: Protocol Add-on Certification Request Form. Submit the request form and the files specified in the form to HP. When HP engineers receive the certification request form and accompanying files, they test the protocol. The results of HPs test and other factors help the HP engineer determine if the protocol add-on is certifiable.

Protocol Certification Procedure

Legal Notices
Warranty
The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. The information contained herein is subject to change without notice.

Restricted Rights Legend

Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Third-Party Web Sites

HP provides links to external third--party Web sites to help you find supplemental information. Site content and availability may change without notice. HP makes no representations or warranties whatsoever as to site content or availability.

Copyright Notices
Copyright 2000 - 2008 Hewlett-Packard Development Company, L.P.

Trademark Notices
Microsoft and Windows Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.

Protocol Certification Procedure

Tests to Perform Before Submitting to HP

Installation
Perform the following installation tests on all operating systems the protocol supports, verifying that all installers run properly and that the installation is correct. Verify that the installation does not overwrite files that belong to other protocols Verify that all supported localization systems are tested (I18N) Verify that the uninstall procedure removes the protocol completely (files, registry entries, start menu entries, and so on) Verify that the upgrade procedure works properly

Product integrity check

Check the integrity of the protocol and its impact on the LoadRunner product. Verify that no original LoadRunner files are corrupted by installing or running your protocol Verify that no conflicts exist between your protocol and the LoadRunner infrastructure Verify that there are no undocumented conflicts between your protocol and existing protocols. If there are known conflicts list them on the Protocol Add-on Certification Request Form and in the protocols User Guide. All files, as specified in the HP Performance Validation SDK Developers Guide, have been installed and are accurate (for example, LRP, CFG)

Short sanity check (for Windows XP with SP2, English)

Perform a basic sanity check on the following features to verify that your protocol runs on Windows XP with SP2 in English. Record a simple business process Replay a simple business process Replay on a remote Load Generator Recording options Run-time settings Regeneration options Multi-protocol tests

Functionality
Test the protocols functionality by performing functionality tests on all operating systems the protocol supports. Verify that the client application with which the protocol is used can be specified in the Recording dialog box in VuGen

Protocol Certification Procedure

Verify that custom recording option tabs display and function correctly Verify that recording options are used during recording sessions Verify that the configuration information entered in the Recording Options dialog box, the RunTime Settings dialog box, and the Regenerate Options dialog box does not overwrite configuration files of another protocol Verify that this protocol does not modify another protocols functionality unless you list that protocol under Protocol collision in the Protocol Add-on Certification Request Form Verify that VuGen can invoke the client when starting a recording session Verify that the client application behaves normally while being recorded Verify that the VuGen recorded events counter increments while executing a business process on the client Verify that all expected events are captured. Use the recorded event log for verification Verify that messages recorded by the protocol match communication as recorded by a sniffer Verify that all protocol API functions and each functions arguments behave as designed Verify that current run-time settings are used during the script run Verify that dynamically-changed run-time setting values are used by the Vuser for which the settings were changed, and that other Vusers are not affected Verify that recorded scripts that emulate typical business processes run correctly Verify that multi-protocol scripts with the SDK-based protocol run correctly. Test all combinations of protocols that you expect the user to use with your protocol Verify that script logging levels are implemented correctly Verify that error and warning messages are logged appropriately Verify that the protocol run is thread-safe if configured to allow running a virtual user as a thread Verify that steps generated by the protocol look exactly like the same steps created in the VuGen script view when inserting a new step using the Insert > New Step command. Verify that non-application events (comments, transactions, C statements, and so on) created during code generation appear in the correct location in the generated script Verify that user delays are reflected accurately by lr_think_time steps in the script Verify that the recorded script can be regenerated Verify that API calls are written to the script correctly and in the correct order in single- and multi-protocol scripts Verify that the code generation log contains all the messages sent by the protocol Verify the code generation log does not contain errors Verify that test steps, operators, and constants are configured for highlighted correctly in VuGen Verify that the Add Step dialog box and IntelliSense perform correctly with the protocols API functions Verify that all log messages are clear and informative Verify that help can be invoked by pressing the F1 key and contains information relevant to the context from which F1 was pressed. For example, if the cursor is on an API function name when F1 is pressed, the API reference should be displayed with focus on the main page for that specific API.

Protocol Certification Procedure

I18N (Internationalization)
If your protocol supports I18N, perform the following tests: Verify that application input and return responses are compatible with the locale handled by protocol Verify that business requirements for error messages or logging that require conversions to or from the locale are handled correctly by protocol

Accuracy
Perform accuracy tests on all operating systems the protocol supports: Verify that the virtual user creates an accurate simulation of the real client, running the same scenarios and sending the same messages, so that network traffic is similar Verify that transaction time is similar to real client transaction time Verify that non-negligible process time on the client side during test run is reported as wasted time in the transaction

Load and Stability

Test the protocols stability and ability to process load: Perform the load and stability tests on all operating systems the protocol supports Run a stability test for at least two hours, and for at least 1000 iterations with at least 50 virtual users. Verify that: CPU usage does not exceed 75%. The maximum increase of memory consumption per one virtual user in one iteration does not exceed 0.5%. The transaction times are the same for a virtual user whether the stability test runs as a single virtual user or in the same process with other virtual users. When virtual users are run as processes, the transaction times are the same for a virtual user regardless of whether other virtual users are running. Verify that the non-negligible process time on the client side during test run is reported as wasted time in the transaction Verify that CPU and memory consumption do not increase when the script is regenerated more than twice. A small increase in system resource use between the first and second regeneration is acceptable, but resource use should not increase after the second regeneration. Verify that code generation does not take an unreasonable length of time Verify that exceptions are captured and handled

Protocol documentation
Verify that the documentation you provide is presentable and accurate. You must provide the following documentation:

Protocol Certification Procedure

User Guide Quick Install and Getting Started Guide, including upgrade information API Function Reference Readme file Sample script In addition to the items listed above, provide a sample script and video for HP engineers use only. This sample script and video are not required for the end user, but help HP engineers certify your protocol.