Unit 2: Software metrics and quality assurance
Software Testing and Quality Assurance notes · PTU syllabus (BSIT602)
On this page
Unit summary
What gets measured gets managed: metrics quantify software quality, and quality assurance builds it in. This unit covers the scope and classification of software metrics, measurement basics, product quality metrics — MTTF, defect density, customer satisfaction and function points — quality concepts, cost of quality, SQA activities, reviews, inspections and audits, quality attributes and Ishikawa's seven basic tools.
After this unit you can
- Classify software metrics and explain measurement basics
- Calculate product quality metrics
- Explain quality control, quality assurance and the cost of quality
- Explain SQA activities, reviews and the seven basic tools
PTU syllabus topics
- Scope and classification of software metrics
- measurement basics
- product quality metrics (MTTF, defect density, customer satisfaction, function point)
- quality concepts (quality control, quality assurance, cost of quality)
- SQA activities
- software reviews/inspections/audits
- quality attributes
- Ishikawa's Seven Basic Tools
Cause-and-effect diagram
Fishbone: root causes
Check sheet
Tally of defects
Control chart
Process stability
Histogram
Distribution of data
Pareto chart
Vital few causes
Scatter diagram
Relationship between variables
Stratification
Split data by source
Topic 1
Scope of software metrics
- Cost and effort estimation
- LOC, function points, COCOMO
- Productivity measures
- Output per person-month
- Data collection
- Defects, effort, changes
- Quality models and measures
- Reliability, maintainability
- Reliability models
- Predict failures
- Performance evaluation
- Speed and resource use
- Structural and complexity metrics
- Cyclomatic complexity, coupling
- Management by metrics
- Track and control projects
Topic 2
Classification of software metrics
Product metrics
Size, complexity, quality of the product
Process metrics
Effectiveness of the process — defect removal efficiency
Project metrics
Schedule, cost, staffing of a project
Direct (base) measures
LOC, effort, defects counted directly
Indirect (derived) measures
Defect density, productivity computed from others
Topic 3
Measurement basics
- Measurement: assigning numbers or symbols to attributes of entities according to defined rules.
Nominal
Categories without order
Defect type: logic, interface, data
Ordinal
Ordered categories
Severity: low, medium, high
Interval
Equal intervals, no true zero
Calendar dates
Ratio
Equal intervals with a true zero
LOC, effort, defects
- Good metrics are simple, computable, consistent, objective, language-independent and give useful feedback.
Topic 4
Product quality metrics
Mean time to failure (MTTF)
Average operating time between failures
Mean time between failures
MTBF = MTTF + MTTR (mean time to repair)
Availability
MTTF ÷ (MTTF + MTTR) × 100
Defect density
Defects ÷ size (KLOC or function points)
Defect removal efficiency
Defects found before release ÷ (before + after release)
Customer problems
Problems reported ÷ user-months
Example
45 defects in a 15 KLOC module: defect density = 3 per KLOC. 90 defects found before release and 10 after: DRE = 90 ÷ 100 = 90%. MTTF 980 h and MTTR 20 h: availability = 98%.
- Customer satisfaction metrics: surveys on a five-point scale (very satisfied … very dissatisfied); measures such as percentage satisfied and net satisfaction index; CUPRIMDSO (capability, usability, performance, reliability, installability, maintainability, documentation, service, overall).
- Function points: measure functionality from the user's view — inputs, outputs, inquiries, internal files and external interfaces weighted by complexity, adjusted by 14 factors; useful because they are independent of language. FP = count total × (0.65 + 0.01 × ΣFi).
Topic 5
Quality concepts and cost of quality
Software quality is conformance to explicit requirements, standards and implicit expectations. It has two views: quality of design and quality of conformance. Cost of quality: prevention costs (training, planning), appraisal costs (reviews, testing) and failure costs (internal — before release; external — after release, the most expensive).
Focus
Product — finding defects
Process — preventing defects
Nature
Reactive, detection
Proactive, prevention
Activities
Testing, inspections of products
Standards, process definition, audits, training
Responsibility
Testing team
Everyone, led by the SQA group
- External failure costs
Complaints, returns, patches after release — highest
- Internal failure costs
Rework, retesting before release
- Appraisal costs
Reviews, testing, inspections
- Prevention costs
Training, planning, standards — the best investment
Topic 6
SQA activities
SQA is a planned set of activities to ensure software meets quality standards: setting standards, reviews and audits, testing, error collection and analysis, change management and education. A formal technical review (FTR) is a meeting of 3–5 people who examine a work product (design or code) to find errors early. Guidelines: review the product, not the producer; set an agenda; limit debate; keep it under two hours; record issues. Statistical SQA collects defect data, traces each defect to its cause and uses the Pareto principle (80% of defects come from 20% of causes) to fix the vital few causes first. Six Sigma aims for 3.4 defects per million opportunities (DMAIC: Define, Measure, Analyse, Improve, Control).
- Prepare the SQA plan
- Evaluations, audits, standards, error reporting
- Participate in process description
- Ensure the process follows standards
- Review engineering activities
- Check compliance with the process
- Audit work products
- Verify standards are met
- Track deviations
- Document and handle non-compliance
- Report to management
- Independent status reporting
Topic 7
Reviews, inspections and audits
Walkthrough
Author guides peers through a product to gather feedback
Informal
Technical review
Peers evaluate technical content against specifications
Moderately formal
Inspection (Fagan)
Trained team with moderator, reader, recorder finds defects using checklists
Most formal: planning, overview, preparation, meeting, rework, follow-up
Audit
Independent check of compliance with standards and processes
Formal, often by external auditors
Topic 8
Software quality attributes
- Functional suitability
- Complete, correct, appropriate functions
- Performance efficiency
- Time, resources, capacity
- Compatibility
- Coexistence and interoperability
- Usability
- Learnable, operable, accessible
- Reliability
- Maturity, availability, fault tolerance, recoverability
- Security
- Confidentiality, integrity, accountability
- Maintainability
- Modular, reusable, analysable, modifiable, testable
- Portability
- Adaptable, installable, replaceable
- McCall's model groups factors into product operation (correctness, reliability, efficiency, integrity, usability), product revision (maintainability, flexibility, testability) and product transition (portability, reusability, interoperability).
Topic 9
Ishikawa's seven basic tools
Check sheet
Structured data collection
Histogram
Distribution of a variable
Pareto chart
Vital few causes (80/20)
Cause-and-effect diagram
Fishbone — man, machine, material, method, measurement, environment
Scatter diagram
Relationship between two variables
Flow chart (or stratification)
Process steps or data segregation
Control chart
Process stability over time
- Advantages of SQC: detects problems early, reduces inspection cost, improves quality and productivity, provides objective data, builds quality consciousness.
- Limitations: needs trained staff and accurate data; detects but does not itself remove causes; assumes stable measurement; may be costly for small runs.
Example
Software use: a check sheet tallies defects by module; a Pareto chart shows that 3 of 15 modules cause 70% of defects; a fishbone diagram traces late releases to unclear requirements, untrained testers and missing test environments; a control chart tracks weekly defect arrival.
Key terms
- Defect density
- Defects per unit of size
- MTTF
- Mean time to failure
- Quality assurance
- Process-focused activities preventing defects
- Inspection
- Formal, checklist-based defect-finding review
- Pareto chart
- Bar chart showing the vital few causes
Quick revision
- Scope and classification of metrics; direct and indirect measures; scales.
- MTTF, MTBF, availability, defect density, DRE, customer metrics, function points.
- QC vs QA; prevention, appraisal, internal and external failure costs.
- SQA plan and activities; walkthroughs, reviews, inspections, audits.
- ISO 25010 and McCall attributes; seven basic tools.
Important exam questions
Practice questions written to the PTU exam pattern for this unit's syllabus: short answers (Section A style) and long answers (Sections B and C style).
Short-answer questions
- Q1.Distinguish product and process metrics.
- Q2.Calculate defect density for 24 defects in 8 KLOC.
- Q3.Define availability.
- Q4.Distinguish QC and QA.
- Q5.Distinguish a walkthrough and an inspection.
- Q6.Name Ishikawa's seven basic tools.
Long-answer questions
- Q1.Explain the scope and classification of software metrics.
- Q2.Explain product quality metrics with examples.
- Q3.Explain the cost of quality and SQA activities.
- Q4.Explain reviews, inspections, audits and the seven basic tools.
Stuck on this unit?
Message SBS on WhatsApp for help with Software Testing and Quality Assurance, or to ask about studying B.Sc IT at Synetic.
