#5 Software Testing

 Purpose: "Introduce overview and fundamental concepts of Software testing"


 Part 1: What is testing? Testing concepts
 Part 2: Why testing & Major testing activities

 Part 3: General testing principles?

Misunderstandings about Testing
 Testing is nothing but debugging
 Testing is not the job of a programmer
 If programmers were more careful testing would be unnecessary
 Testing never ends
 Testing activities start only after the coding is complete
 Testing is not a creative task

Part 1: What is test?
 Testing is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements
 Terms:
  Process: sequence of steps performed for a given purpose. (IEEE)
  Software Process: a set of activities, methods, practices,
and transformations that people use to develop and maintain software and associated products.(SEI-CMM)

Testing Objectives
Primary: Execute a program with the intent of finding defects to
 Determine whether system meets specifications
 Determine whether system meets user’s needs
 Secondary:
 Continuously improve the testing process

Testing Theory & Reality
 Theory
 Need sufficient time
 Baseline & freeze requirements
 Automation testing
 Have trained testers: test process, business flow,…
 Reality
 Time Pressures to keep delivery date
 Continuously updated requirements
 Don’t know when to stop: Update & update Requirement continuously (CRs)  Tester updates TC and re-tests
 Manual testing
 Lack of dedicated software test environment: share, steal
 Inexperienced testers without appropriate training

Testing as a part of QA in sw. process
 Testing falls into the category of defect removal
 Testing in whole development processes
 Testing is an important phase of development activities
 Testing belongs to Dynamic Test

Test…
 Types
 Functional Testing
 Non Functional Testing
 Alpha Test
 Beta Test …
 Methods
 Dynamic Testing
 Static Testing
 Levels/Stages
 Unit Test (UT)
 Integration Test (IT)
 System Test (ST)
 User Acceptance Test (UAT)
 Techniques
 Equivalent partitioning
 Value boundary
 Decision Table
 State transition

Part 2: Why is testing necessary?

 Because software is likely to have faults
 To learn about the reliability of the software
 Because failures can be very expensive
 To avoid being sued by customers

 To stay in business

Software Failures
 Software failures can lead to:
 Loss of money
 Loss of time
 Loss of business reputation
 Injury/Death

Major testing activities
 Test planning & Preparation
 Define Test Plan
 Design test
 Prepare test cases or test scripts, test data
 Test execution
 Execute testing basing on test cases
 Report test result
Analysis and follow-up
 Analyze test result
 Ensure removal of detected defects
 Further analysis to provide valuable feedback for processes
(testing, development,…)
 Long-term product quality improvement

Why can't we test our own work ?
 We make assumptions
 We are emotionally attached to the product (it's our baby
and there's nothing wrong with it).
 We are so familiar with the product we cannot easily see
the obvious faults
 We have a vested interest in passing the product as ok and
not finding faults
=> Independent Testing

Independent Testing

 Tests by the person who wrote the item under test;
 Tests by another person within the same team, such as
another programmer
 Tests by a person from a different organizational group,
such as an independent test team;
 Tests designed by a person from a different-organization
or company, such as outsourced testing or certification by

an external body.

Part 3: Testing Principles

 Testing shows presence of defects
 Exhaustive testing is impossible
 Early testing
 Defect clustering
 Pesticide paradox
 Testing is context dependent

 Absence-of-errors fallacy


Principle 1: Testing shows presence of defects
 Testing can show that defects are present, presence of defects but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.
 Ex: We see many white swans, we cannot say 'All swans are white'. Otherwise, we see one black swan, we can say 'Not all swans are white'.

Principle 2: Exhaustive testing is impossible
 Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts
 Ex: A software program designed to count the number of
vowels (a, e, i, o, u) in a string of five letters. To test this program with all possible inputs, require 265
(11,881376)cases
=> How much testing is enough?

How much testing is enough?

 Testing helps us find defects and improve software
quality
 Some choices:
 Test everything
 Test nothing
 Test some of the software
 your immediate response is that “everything must be tested” and We don't want to use software that has not been completely tested

Decide how much testing is enough and when to stop testing
 Define the scope of testing before any testing is actually started
 A priority must be assigned to each test
 Define exit criteria -> testing can stop when exit criteria has been met
 The exit criteria depends on the following:
 The risk to the business process of the project
 The time and budget constraints within the project
 The resource constraints within the project

Principle 3: Early Testing

 Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives
 Early testing such as early test design and review activities , finds defects early on when they are cheap to find and fix.

Principle 4: Defect Clustering

 A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures

Principle 5: Pesticide Paradox
 If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. To overcome this 'pesticide paradox', the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.

Principle 6: Testing is context dependent
 Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site.

Principle 7: Absence-of-errors fallacy
 Finding and fixing defects does not help if the system built is unusable and does not fulfill the users' needs and expectations.