Challenge Overview
Project Overview
Our long term goal is to get an exhaustive list of test cases/scenarios (unit, functional, load, stress) created and packaged up for easy deployment and use.
But for a first round we want to gather the following deliverables.
Implementation and documentation of bare minimum unit tests (expanded from the tests included in code)
Implementation and documentation of bare minimum functional tests (possibly expanded form our own SoapUI functional tests)
Implementation and documentation of bare minimum load test scenarios
A Jenkins setup running the unit tests every build (with flag to build w/o running the unit tests)
A script for launching the load tests via LoadUI command line, LoadUI GUI, and Jenkins
The message bus project referred to is accessible here - https://github.com/Comcast/cmb
Contest Overview
The purpose of this contest is to design load test scenarios for the CMB web services. These scenarios will later be implemented through the LoadUI Test Tool.
The focus is on scalability of the system under load in all aspects such as number concurrent connections, number of active queues, number of messages pushed into the system, number of messages retrieved, and more. Push the system to it's limits.
The CQS and CNS follow the standard of Amazon Simple Queue Services and Notification Services.
You can find more information here:
Notification Services: http://docs.aws.amazon.com/
Client provide sample SoapUI tests for CQS and CNS services (download from this contest).
You can find the implementation from src/com/comcast/cqs and src/com/comcast/cns (download the code from github)
Note: This contest shouldn't just call the API with simple test data many times. The focus is on scalability under load in all aspects such as number concurrent connections, number of active queues, number of messages pushed into the system, number of messages retrieved, and more.
Submission Deliverables
- Updated QA Plan with any changes necessary (e.g. new tools, new test procedures, etc).
- Completed Test Scenarios listing.
- A Microsoft Excel compatible spreadsheet with the detailed Test Scenarios inventory.
Final Submission Guidelines
LoadUI: http://www.loadui.org/