18 August 2025 · Journal
Sampling strategies when you cannot record every session
Full capture is rare, expensive, and often the wrong privacy posture. Quality work still needs a sample frame that is not the vendor’s “frustrated” filter.
Replay libraries grow faster than attention. Teams then watch whatever the product surfaces: rage clicks, long sessions, error tags. That is not sampling. It is a highlight reel with a UI. Cohort Replay Analysis exists because this pattern is now the default in Southeast Asian product orgs that bought a recorder and never wrote a log.
Stratify on purpose
Pick strata that match how the product actually fails: device class, network corridor, time of day, and — in Thailand — payday versus mid-month. Draw a fixed number from each cell, even if some cells look “fine” in aggregate. Quiet cells are where silent failures hide.
Keep a sampling log
Date, frame, inclusions, exclusions, who watched, what was out of scope. If you cannot reconstruct why a clip entered the review, it is anecdote. Cloudpacketbase grades the log as seriously as the findings. It is unglamorous paperwork. It is also how you survive an internal audit.
Purposive clips are allowed — labelled
Incident reviews need purposive clips. Label them as such. Never average them into the weekly quality sample. Mixing incident theatre with the stratified draw is how a single viral failure becomes “the number moved.”
When recording is off the table
Some partners cannot export replay at all. Event-only Session Quality Analytics still works if the contract includes start, fatal, complete, cancel, and a few task markers. You lose gesture evidence. You keep a score. Say the loss out loud in the charter.