No history yet

Quantifying Quality Attributes

From Vague to Verifiable

Saying a system must be 'fast', 'scalable', or 'reliable' is like telling a chef to cook 'something tasty'. It’s a nice sentiment, but it isn't a recipe. In system design, vague goals lead to ambiguous outcomes and project failure. We need precision.

Not all requirements are created equal. While the colour of a button is a requirement, it doesn’t fundamentally shape the system’s blueprint. The requirements that do are called Architecturally Significant Requirements (ASRs). These are the high-stakes quality attributes that dictate major design decisions. Getting an ASR wrong often means a costly, ground-up redesign.

An ASR is a requirement that has a measurable effect on the system's architecture. If changing the requirement forces a change in the technical design, it's architecturally significant.

Scenarios Tell the Story

To make ASRs concrete, we use quality attribute scenarios. These are short stories that describe a specific situation and the system's expected reaction. They transform a fuzzy goal into a testable contract. Each scenario has three parts:

  1. Stimulus: What is happening to the system? (e.g., a surge in users, a hardware failure, a security attack).
  2. Environment: Under what conditions is this happening? (e.g., during normal operation, while running a maintenance task, at peak load).
  3. Response: How should the system react, and how do we measure it? (e.g., response time remains below a threshold, the system fails over to a backup, the intrusion is logged and blocked).

So, 'the system must be high-performance' becomes:

  • Stimulus: 10,000 users simultaneously request their news feed.
  • Environment: The system is operating normally.
  • Response: The 95th percentile response time for generating the feed is less than 200 milliseconds.

This is a specific, measurable, and testable requirement that an engineer can actually build against.

Measuring What Matters

Once we have scenarios, we need the right metrics to define the response. Choosing the wrong metric can be misleading. Let's look at a few key areas.

Latency

noun

The time it takes for a system to process a single request. It’s the delay between a cause and its effect.

When measuring performance, averages can lie. A system with an average response time of 100ms sounds great. But what if most requests are super fast (20ms) and a few are painfully slow (2000ms)? The average hides the bad user experience of that minority.

This is why we use percentiles. A p95 latency of 200ms means 95% of requests were faster than 200ms, and 5% were slower. A p99 latency of 400ms means 99% of requests were faster than 400ms. These percentiles give us a much clearer picture of the user's actual experience, especially the worst-case scenarios, which are often the ones that matter most.

For scalability, simply counting 'visitors' is too vague. Are they all hitting the homepage, or are some performing complex database queries? A better metric is throughput, often measured in Transactions Per Second (TPS) or Requests Per Second (RPS). This quantifies the actual load on your system, helping you design for specific processing capacity.

QualityVague MetricBetter Metric
PerformanceFastp99 Latency < 400ms
Scalability1 million daily users5,000 RPS at peak
AvailabilityAlways on99.99% uptime

Availability is often expressed in 'nines'. A system with 99.9% availability can be down for about 8.77 hours per year, while a system with 'five nines' (99.999%) can only be down for about 5.26 minutes per year. To understand how to achieve this, we use two key reliability metrics:

  • (MTBF): The average time the system runs correctly before it fails. A higher MTBF means the system is more reliable.
  • Mean Time To Repair (MTTR): The average time it takes to fix a failure once it occurs. A lower MTTR means you can recover faster.

Availability is a function of both. You can increase overall availability by making the system fail less often (increasing MTBF) or by fixing it faster when it does fail (decreasing MTTR).

Availability=MTBFMTBF+MTTR\text{Availability} = \frac{\text{MTBF}}{\text{MTBF} + \text{MTTR}}
Quiz Questions 1/5

What is the primary characteristic of an Architecturally Significant Requirement (ASR)?

Quiz Questions 2/5

A quality attribute scenario transforms a vague goal into a testable requirement by defining a ______, the environment in which it occurs, and the expected system ______.

By moving from vague adjectives to specific, measurable scenarios, you create a solid foundation for designing and evaluating system architecture. These numbers become your north star, guiding your decisions and ensuring the final product meets the actual business needs.