Auditing Organisations That Change a Thousand Times a Day (Short Version)

A 50 item audit sample covers 0.02% of a company doing 250,000 deployments a year. That’s where I started this talk, at a lunch and learn hosted by IIA Luxembourg and ISACA Luxembourg Chapter.

Sampling made sense when systems changed a few times a year. Today a control can change with every deployment. So the question for audit changes too: did the control work for the whole period, or only in the sample?

In the talk I went through four signals that tell you sampling alone is no longer enough:

• The size of the population.
• Controls that change during the period.
• Teams implementing the same control in different ways.
• And evidence spread across systems.

For each one, I showed what you find when you test the full population. A static analysis gate left in advisory mode on one pipeline. An in house approval workflow that let people approve their own changes.

Sampling still has its place. But in organisations that deploy all day, it needs data analytics and continuous monitoring next to it.

I later gave an extended version at GSDC, built around a story. The recording is here.