Parasoft Service Virtualization Supports DevOps and QA
by Bob Aiello
DevOps provides a set of principles and practices which enable development and operations teams to communicate and collaborate more effectively. High performance teams which embrace DevOps have been demonstrating that they can deliver quality systems with amazing speed and agility. Many DevOps teams focus on scripting the application build, package and deployment process, resulting in an automated framework, known as the deployment pipeline. Deployment automation enables the organization to enjoy the benefits of continuous integration and continuous delivery, significantly enhancing productivity and agility.
Nevertheless, improving any process—from software development to manufacturing—inevitably involves an apparent trade-off between speed and quality. Understanding opportunities for acceleration is key for process improvement. Yet, speeding up the process without understanding the impact to quality opens up a distinct risk. In the same vein, organizations that are looking to accelerate their software delivery cycle need to ensure that the DevOps initiative is balanced with achieving the organization's definition of quality.
Software organizations are learning that QA and testing practices must be utilized throughout the entire software development lifecycle. Robust testing practices enable the organization to meet the demands and challenges of today’s fast-paced technology driven world. Without them, your entire team will be mired in firefighting software glitches and trying to patch bad software. The good news is that there are methodologies and tools available today that can empower you and your team to implement effective testing strategies that meet the demands of even the most complex IT systems embracing DevOps.
When an organization is looking at "Continuous Everything," an emerging best practice known as Continuous Testing is a critical component in the overall process. Another emerging best practice—Service Virtualization—enables Continuous Testing by providing anytime, anywhere access to a complete simulated test environment.
Delivering large scale application systems can be a challenging endeavor. The pressures of meeting time-to-market demands and testing ever more complex systems make it increasingly difficult to deliver reliable systems in a timely manner.
The only possible way to meet the demands of today’s business environment is to start testing from the very beginning of the software and systems development process. This approach should include testing requirements and designing systems to be testable through automated procedures.
This is where Development Testing practices are becoming increasingly popular within development organizations. Although many Development Testing techniques are longstanding industry best practices, few teams have been applying them religiously. This could very well change as more and more organizations recognize the escalating risk associated with software failure.
At the same time, QA testing is often limited by access to complete (fully-functional) build of the application under test as well as an environment that includes all of the dependencies required to execute end-to-end tests.
Developing an Effective End-to-End Test Suite
Today, the underlying architecture associated with most enterprise applications transacts primarily on a machine-to-machine basis over myriad communication protocols and structured message payloads. Ultimately, more and more leading organizations are finding that a mature, repeatable process for hardening GUI-less back-end services from the API level is now a "must have" for ensuring the security, reliability, and performance of critical transactions.
With APIs, testing a broad range of conditions and corner cases is critical, so automation comes to the forefront. The creation and execution of simple automated tests with limited or manual validation might have sufficed for internal given web services that were used internally (e.g., via SOA), but more sophisticated and extensive automation is required to be confident that APIs are robust enough to meet business expectations. You need a level of automation that gives you a comprehensive set of functional test cases that can be repeated in a systematic manner.
Recommended capabilities for this goal include an intuitive interface for automating complex scenarios across the messaging layer, ESBs, databases, and mainframes:
Defining automated test scenarios across the broad range of protocols and message types used in APIs: REST, WADL, JSON, MQ, JMS, EDI, fixed-length messages, etc.
Automating rich multilayer validation across multiple endpoints involved in end-to-end test scenarios.
Parameterizingtest messages, validations, and configurations from data sources, values extracted from test scenarios, or variables.
Defining sophisticated test flow logic without requiring scripting.
Visualizing how messages and events flow through distributed architectures as tests execute.
The Test Environment Access Needed to Test Continuously
With the convergent trends of Agile/parallel development and increasing system complexity/interdependency, it has become extremely rare for a team to have ubiquitous access to all of the dependent applications required to execute the type of complete end-to-end tests outlined above. By leveraging Service Virtualization to remove these constraints, an organization can gain full access to (and control over) the test environment—enabling Continuous Testing to occur as early and often as needed.
Service virtualization enables rapid iterative development by providing simulated test environments that can help scale continuous testing. The goal of service virtualization is to simulate interfaces and resources that may not always be available for testing due to cost or other real world constraints. This emerging industry best practice promises to provide a much more robust and comprehensive approach to ensuring that we can continuously deliver defect-free code.
Want to start testing the component you just built even though not much else is completed? Don't have 24/7 access to all the dependencies involved in your testing efforts—with all the configurations you need to feel confident that your test results are truly predictive of real-world behavior? Tired of delaying performance testing because access to a realistic environment is too limited (or too expensive)? Service Virtualization can remove all these constraints.
Service Virtualization: What's Involved?
Software components and system interfaces may be simulated to allow the applications to be tested against virtualized test assets that look and act as if they are the actual real assets, which are often few in number and only available at specific times. Virtualized test assets look and act like the real thing but may be duplicated and available at times when the real assets are not available to the testing team, possibly because they are being used to run production. Virtualizing your test assets enables your organization to create robust test frameworks that can provide comprehensive test coverage while keeping costs low.
One effective way to create virtualized test assets is to record the messaging protocol which you can then use to create additional test cases.
Proxies enable you to discover the messaging that occurs when you initially execute a specific test scenario. By recording this behavior, you can capture messages which can actually be duplicated to create additional test cases. You may begin this effort by exercising the application through a user interface, but once you have the messaging recorded, you can duplicate each session and modify the messages to create additional test scenarios which may not be accessible if you were only able to test via a user interface.
Understanding messaging is essential for testing any complex system. Service virtualization testing provides the framework for developers and testers to collaborate, analyze and understand the messaging protocols between components.
This approach helps the DevOps team to build a comprehensive knowledge management system that implicitly documents how the system actually behaves under test. This information can then be used to build robust and comprehensive automated tools to exercise the application under test using JMS, HTTP, and MQ message protocols. Another goal is to capture database behavior including database connectivity.
Parasoft Service Virtualization
Parasoft Service Virtualization provides the tools and methodology to configure proxy settings to discover messaging protocols that may be recorded and structured as virtualized test assets. In the DevOps environment, there are many tools that need to be used together and Parasoft’s open automated infrastructure enables integration with essential tool-chains including application performance monitoring and continuous delivery. This approach enables you to develop robust testing frameworks that include test scenarios that may not even be accessible if you were limited to the user interface. Parasoft supports over a hundred protocols and message types, including those most prevalent for distributed systems, packaged application, databases, message broker/ESB, and service calls.
Parasoft’s Environment Manager can be utilized to choose between the actual systems, known as real assets, when they are available, or virtualized assets after they have been constructed. This helps you to understand the application being tested and develop robust test scenarios that would otherwise be impossible to achieve.
Figure 1 – Parasoft Environment Manager
Parasoft Service Virtualization also enables you to capture the knowledge of how components interact with each other. The resulting "virtualized assets" enable you to test your systems at a deep middleware level and to scale your testing by using the initial assets created during the recording process to create additional test cases.
Moreover, Parasoft Service Virtualization testing can help you scale your testing more effectively. Test cases can be loaded into external spreadsheets or even their own dedicated databases. Large hierarchical datasets may also be used for test input, enabling the test engineers to develop both positive and negative test scenarios. Using service virtualization in practice helps your organization improve both the quality and productivity of the test engineering effort.
Software and systems reliability is essential for any organization to achieve success. If your organization is embracing continuous integration and continuous delivery then you must scale your testing processes to handle the increased iteration and delivery cycles. Continuous testing is a must-have for any organization to meet competitive pressures and ensure that their customer base trusts the systems being developed to be free of defects while simultaneously achieving the demand for robust features and service reliability and security.
Parasoft Service Virtualization gives you a robust testing framework to understand, capture, modify and playback virtualized test resources. The value of using tools to enhance your DevOps approach to collaboration between development and testing results in more agility, higher quality and more reliable systems, delivering a triple play that your customers will truly come to trust and value.
For more information on Parasoft products:
or visit http://alm.parasoft.com/sv-demo to schedule a demo.
 Ambler, Scott W. and Lanes, Scott, Disciplined Agile Delivery: A Practitioners Guide to Agile Software Development in the Enterprise, Pearson 2012
 Humble, Jez and Farley, David, Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, Addison-Wesley 2010
 Beizer, Boris, Software System Testing and Quality Assurance, International Thomson Publishing Press, 1996
 Aiello, Bob and Leslie Sachs. 2010. Configuration Management Best Practices: Practical Methods that Work in the Real World. Addison-Wesley Professional.
More articles by this author