1.April 19, 2010 Version 1.0 Application Testing ServicesUniversity of Stellenbosch – System Testing Disclaimer This document does not constitute a formal offer, but serves to outline the discussion elements for contractual consideration based on IBM’s standard terms and conditions.2. The current situation3. Ad Hoc Statements John Roodt : Senior Architect “… you guys are brutal.Brutally honest…” Jolene Saayman : Lead Java Developer “… if it weren't for you I would have left this company a long time ago…” Adam Granger : Lead Java Developer “… you are our safety net…” Michael Kiergard : Customer Project Manager “… you must be involved.I trust you guys….I don’ttrust the developers…” Karen Greeves : Project Delivery Executive “… I can rest assured that after it has been through your hands, then the application is pretty solid…” Meike Voight : SAS Developer “… but what are they going to test if I have already tested it?...” Cherryl Lundt : Project Manager “… I now want to involve you on all my projects...” “ As a software tester, the fate of the world rests on your shoulders.This statement is not an exaggeration if you accept the dual premises that computer software runs the modern world and that all software has bugs – then you will quickly reach the inescapable conclusion that unless the most disruptive bugs are removed, the world as we know it will grind to a halt.” Scott Loveland, Geoffrey Miller, Richard Prewitt, Jr., and Michael Shannon have collectively spent more than 70 years in the software testing trenches.They currently hunt bugs in the IBM mainframe development laboratory in Poughkeepsie, New York.Collectively they have spent more than 70 years in the software testing trenches. 4. Agenda Being a tester Test Strategy : Example case study of mature client Test Methodology : Verification and Validation ________________________________________________________________ What every developer needs to know about testing Characteristics of a good tester 5. Being a Tester What people think we are: Testers are negative thinkers, they complain, they like to break things, they take a special thrill in delivering bad news. BUT WE PROTEST !! Who we really are: Tester’s don’t complain, we offer evidence.Tester’s don’t like to break things, we dispel the illusion that things work.Testers don’t enjoy giving bad news, we enjoy freeing our clients from the thrall of false belief. 6. Why Test? To define the role of a testing group is not easy.The following metaphor might best explain this role:"A project is like a road trip.Some projects are simple and routine, like driving to the store in broad daylight.But most projects are more like driving a truck off-road, in the mountains at night.Those projects need headlights.The tester is theheadlight .The tester illuminates the road ahead so the developers and managers, however they struggle with the map, can at least see where they are, what they're about to run over, and how close they are to the cliff." The detailed mission of testing groups might differ from company to company, but behind those details is a common factor.Testing is done to find information. Critical decisions about the project or the product are made on the basis of the information found by the test group. 7. IBM Testing Services- Overview Provide integrated testing services that leverage test best practices and highly skilled test professionals from across IBM Deliver flexible testing services Exploit technology to deliver innovative solutions Leverage leading practices and Global Reach Lead industry through proven processes, methods, and tools Maintain high customer satisfaction through delivery excellence and measurable business value Provide testing infrastructure On-Demand Test Team IBM Provision of experienced test resources to IBM &/or Customer Projects Testing Project and/or Programme Take responsibility for all aspects of testing for a project or programme. Build estimates, provide the team, provide management support Consultancy Services Managed Testing Services “ Testing on Demand”for a subset of testing scope or full Outsourced TestingTest Process AutomationTest Environment Management Test Support (Systems, Integration, OAT, UAT) Technical Testing Performance, Load and Stress Education & Skills Transfer Component Solutions Web Testing Test Leadership Legacy Transformation TestAutomation Test Process Assessment Capacity Planning Complex Architecture Performance Scalability & Interoperability Software Product Testing Full Lifecycle Test Services tailored8. Test Strategy – Example Case study Business Case Feasibility Analysis Specification Design Architectural Design Module Design Construction Module Test Acceptance Test System Test Implementation Integration Test Business Realization BusinessProcess Owners Business IT Vendor or Int IT Business Governance9. Test Strategy – Example Case Study Business Case Feasibility Analysis Specification Design Architectural Design Module Design Construction Module Test Acceptance Test System Test Implementation Integration Test Business Realization BusinessProcess Owners Business IT Vendor or Int IT Business case Requirement Catalogue Change / Decision register Non Functional Requirement Specification Interface Specification Software System Requirement Specification Application Architecture Technical Architecture Definition Test strategy Implementation Strategy Deliverables Catalogue Project Contract/Schedule AsIs and ToBe model Logical Data Model System Context Diagram Project approach Project Management Plan System Design Document Physical Data Model QA Plan Detailed Test Plan Implementation Plan Application Impact Implementation Requirement Specification User Manual Test Data Operational Acceptance Test plan Unit Test report Integration Test report Data Migration Test report Factory Acceptance report Application support documentation Global HelpDesk documentation Operational Management support Operations Handbook User Acceptance Test report Stress and Performance report Application Impact Assessment Operations Acceptance test report Maintenance and Support Contracts Handover Report to Business Owner Close Down report 10. Test Strategy – Example Case Study Test plans AUT, AIT, FAT Test cases AUT, AIT, FAT Execute AUT Execute AIT Execute FAT Test plans SIT, UAT, SPT Test cases SIT, UAT, SPT Execute SIT Execute SPT Execute UAT Promote PROD Warranty Period Design Analysis Construct and Vendor test Cust Test Impl Prod Execute OAT Execute AIA11. Testing Methodology - Simplified “V”-Model of Testing Capture Requirements Design System Implement System Component Test Integration/System Test Acceptance Tests Test team involved on day one of project Requirements, high-level design are primary drivers for behavioral test phases Low-level design, implementation are primary drivers for structural testing Features dropped during development (left side of V) rather than slipping test phase entry dates or violating test entry criteria Application Concept Develop Tests Develop Tests Develop Tests System Tests Integration Tests Unit Tests Verification Validation12. Testing Methodology - Validation ARE WE BUILDING THE RIGHT SYSTEM? IdealReviewers : ReviewGranularity Business Owners Business Representative Business Analyst Conceptual Application Business Owner Architects Test Engineers Prototypes Simulations High Level Designs Developers Deployers Architects Detailed Design Execution Activities For review phases Requirements/Specification reviews Architectural Design reviews Detailed Design reviews Test Planning13. Testing Methodology - Verification ARE WE BUILDING THE SYSTEM RIGHT? IdealReviewers : ReviewGranularity Developers DB / Net / System Administrators Deployers Structural (“White Box”) Test Engineers Test Technicians (Some) Developers Behavioral (“Black Box”) Tech Support / Help Desk Sales / Marketing Business Analyst / User Customer Live (“Alpha/Beta”) Execution Activities For review phases Unit Component Integration System Acceptance Pilot14. Test Process – Main process15. Test Process – Define Mission16. Test Process - Verify Test Approach17. Test Process – Validate Build Stability18. Test Process – Test and Evaluate19. Test Process – Achieve Acceptable Mission20. Test Process – Improve Test Assests21. Simplified Test Process22. What every developer needs to know : 1. The product is more than Software Application Code Framework Documentation Manuals Operator instructions Kit OS Infrastructure23. What every developer needs to know : 2. Quality is more than the lack of bugs (FURPS) Functionality Suitability Correctness Interoperability Compatibility Security Installability Usability Understandability Learnability Operability Performance Reliability Maturity Fault tolerance Integrity Recoverability Safety Maintainability Analyzability Changeability Stability Testability Portability Does it solve the problem when it works Is it practical to use Does it keep on working Is it economical to build and maintain24. What every developer needs to know : 3. Quality Assurance is more than testing Quality Assurance is everything you do to minimize the risk of failure and to promote excellence Risk Management Customer Involvement Skillfull developers Process definition and improvements Inspections and active testing Experience based improvements 25. What every developer needs to know : 4. Testing is hard to do Anticipate your users: Data Skills Actions Expectations Environment … to examine a product that is.. Invisible Volatile Sensitive Complex Unfamiliar ...using a process that is often… Endless Ambiguous Negative Boring Laborious Think of the permutations!! Functions vs Input data vs state of data Products vs platforms External factors Expectations of customers Timeframes Versions of the product 26. What every developer needs to know : 5. You can make testing easier Document your design User internal error checking Test units before Integration Tell the tester what is new, concerns you or is shaky Test the build yourself…….First Evolve product in functional layers..Ie Iterative development with testability in mind Build in testability 27. Characteristics of a good tester Know at least some programming . There’s a popular myth that testing can be staffed with people who have little or no programming knowledge. Since they’re testing software, without know some programming, they can’t have any real insights into the kinds of bugs that come into software and the likeliest place to find them. Know the application . The ideal tester has deep insights into how the users will exploit the program’s features and the kinds of cockpit errors that users are likely to make.Intelligence . The single most important quality for testers (just as for programmers) is raw intelligence, good testers, just as programmers, are smart people.Hyper-sensitivity to little things . Good testers notice little things that others miss or ignore. Testers see symptoms, not bugs.Tolerance for chaos . People react to chaos and uncertainty in different ways. If the tester waits for all issues to be fully resolved before starting test design or testing, he won't get started until after the software has been shipped. Testers have to be flexible and be able to drop things when blocked and move on to another thing that is not blocked. Testers always have many irons in the fire. People skills . You can be an effective programmer even if you are hostile and anti-social; that won’t work for a tester. Testers can take a lot of abuse from outraged programmers. A sense of humor and a thick skin will help the tester survive. Testers may have to be diplomatic when confronting a programmer with a fundamental goof. Diplomacy, tact, a ready smile- all work to the independent tester’s advantage.Tenacity(Merriam-Webster’s dictionary explains it as: persistent in maintaining or adhering to something valued or habitual). An ability to reach compromises and consensus can be at the expense of tenacity. The best testers are both socially adept and tenacious where it matters. The best testers are so skilful at it that the programmer never realizes they’ve been had.Organized . There’s just too much to keep track of to trust to memory. Good testers use files, Databases and all the other accoutrements of an organized mind.Sceptical . That doesn’t mean hostile. I mean skepticism in the sense that nothing is taken for granted and that all is fit to be questioned. Only tangible evidence in documents, specifications, code and test results matter. While they may patiently listen to the reassuring words from the programmers ("Trust me. I know where the bugs are.") - and do it with a smile - they ignore such in-substantive assurances.Self-sufficient and tough . If they need love, they don’t expect to get it on the job. They can’t be looking for interaction between them and programmers as a source of ego-gratification and/ or nurturing. Their ego is gratified by finding bugs, with few misgivings about the pain (in the programmers) that such finding might engender.Cunning . Systematic test techniques such as syntax testing and automatic test generators have reduced the need for much cunning, but the need is still with us and undoubtedly always will be because it will never be possible to systematize all aspects of testing. There will always be room for that off-beat thinking that will lead to a test case that exposes a really bad bug.Technology hungry . They hate dull, repetitive work; they’ll do it for a while if they have to, but not for long. The silliest thing for a human to do, in their mind, is to pound on a keyboard when they’re surrounded by computers.Honest . Testers are fundamentally honest and incorruptible. They’ll compromise if they have to, but they’ll righteously agonize over it. This fundamental honesty extends to a brutally realistic understanding of their own limitations as a human being.28. Thank you
Comments
Report "deeper IBM Global Business Services – Application Services"