<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Assurance on Brewed in the Cloud by Chris Hailes</title><link>https://blog.brewedinthecloud.com/tags/assurance/</link><description>Recent content in Assurance on Brewed in the Cloud by Chris Hailes</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><lastBuildDate>Thu, 19 Nov 2026 00:00:00 +1100</lastBuildDate><atom:link href="https://blog.brewedinthecloud.com/tags/assurance/rss.xml" rel="self" type="application/rss+xml"/><item><title>Measure Exposure Reduction, Not Just Control Deployment</title><link>https://blog.brewedinthecloud.com/p/exposure-reduction-evidence/</link><pubDate>Thu, 19 Nov 2026 00:00:00 +1100</pubDate><guid>https://blog.brewedinthecloud.com/p/exposure-reduction-evidence/</guid><description>&lt;p&gt;Consider the report that says: one unintended operation stopped, the approved dependency still works, two cases remain unknown, and a recovery path needs further validation.&lt;/p&gt;
&lt;p&gt;That is more defensible than &amp;ldquo;six controls deployed&amp;rdquo; or a percentage derived from a changing inventory. The question is not whether the configuration changed. It is whether the reachable consequence changed under comparable conditions.&lt;/p&gt;
&lt;p&gt;That is the report I would rather review. It tells me where to look next instead of asking me to trust a headline percentage. The evidence-pack template and query below make that judgment inspectable: compare the same operations, keep the unknowns visible, and show what the change cost in service and recovery capability.&lt;/p&gt;
&lt;h2 id="the-mental-model"&gt;The Mental Model
&lt;/h2&gt;&lt;blockquote&gt;
&lt;p&gt;Exposure reduction is a bounded comparison, not an organization-wide security score.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Each path ID represents a declared source, principal, destination and operation within a defined topology.&lt;/p&gt;
&lt;p&gt;Separate unwanted paths from required dependencies. Removing an unwanted operation is useful; removing DNS, responder access or an approved data dependency is an outage. Reporting both as fewer reachable paths would reward the wrong behavior.&lt;/p&gt;
&lt;div class="mermaid"&gt;flowchart LR
B[Baseline tests and scope] --&gt; C[Comparable case set]
A[After-change tests and scope] --&gt; C
C --&gt; D[Removed, retained and new paths]
C --&gt; U[Unknown and missing evidence]
D --&gt; E[Evidence pack]
U --&gt; E
R[Availability, recovery and exceptions] --&gt; E
E --&gt; J[Bounded reviewer judgment]
&lt;/div&gt;
&lt;p&gt;Could another engineer challenge the result without having attended every meeting? That is the standard I would set for the pack.&lt;/p&gt;
&lt;h2 id="how-it-really-works"&gt;How It Really Works
&lt;/h2&gt;&lt;h3 id="freeze-the-comparison-contract"&gt;Freeze the comparison contract
&lt;/h3&gt;&lt;p&gt;Agree on what &amp;ldquo;before and after&amp;rdquo; means before making the change. Record scope version, path IDs, source/principal/operation, intended disposition, baseline and change versions, observation windows, and expected results. If the identity or destination changes, settle whether it is still the same case before calculating a difference.&lt;/p&gt;
&lt;p&gt;Use three outcome states for an operation: reachable, blocked and unknown. Keep &amp;ldquo;test failed&amp;rdquo; separate from &amp;ldquo;operation blocked.&amp;rdquo; A missing credential, stopped service or unavailable query can make the test unknown.&lt;/p&gt;
&lt;p&gt;The &lt;a class="link" href="https://blog.brewedinthecloud.com/azure-exposure-reduction-toolkit/#6-evidence-comparison" &gt;draft toolkit&amp;rsquo;s evidence-comparison module&lt;/a&gt; includes a local fixture example. Its purpose is to show bookkeeping behavior, not to infer Azure security from synthetic data. Reject duplicate IDs, mismatched scope, missing evidence and invalid states rather than silently remove them.&lt;/p&gt;
&lt;h3 id="keep-configuration-and-runtime-evidence-distinct"&gt;Keep configuration and runtime evidence distinct
&lt;/h3&gt;&lt;p&gt;A deployment succeeded. A route is effective. A service rejected the operation. Each is useful evidence, but each answers a different question. Correlated logs can help explain the relationship; no single source supplies the whole answer.&lt;/p&gt;
&lt;p&gt;For a Firewall network-rule case, Microsoft&amp;rsquo;s &lt;code&gt;AZFWNetworkRule&lt;/code&gt; schema documents fields including source/destination IP, destination port, action, policy and rule. A draft query is:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;div class="chroma"&gt;
&lt;table class="lntable"&gt;&lt;tr&gt;&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code&gt;&lt;span class="lnt"&gt; 1
&lt;/span&gt;&lt;span class="lnt"&gt; 2
&lt;/span&gt;&lt;span class="lnt"&gt; 3
&lt;/span&gt;&lt;span class="lnt"&gt; 4
&lt;/span&gt;&lt;span class="lnt"&gt; 5
&lt;/span&gt;&lt;span class="lnt"&gt; 6
&lt;/span&gt;&lt;span class="lnt"&gt; 7
&lt;/span&gt;&lt;span class="lnt"&gt; 8
&lt;/span&gt;&lt;span class="lnt"&gt; 9
&lt;/span&gt;&lt;span class="lnt"&gt;10
&lt;/span&gt;&lt;span class="lnt"&gt;11
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;
&lt;td class="lntd"&gt;
&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-fallback" data-lang="fallback"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;let windowStart = datetime(2026-11-10T00:00:00Z);
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;let windowEnd = datetime(2026-11-10T00:05:00Z);
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;let sourceIp = &amp;#34;10.20.1.4&amp;#34;;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;let destinationIp = &amp;#34;10.30.1.4&amp;#34;;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;AZFWNetworkRule
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;| where TimeGenerated between (windowStart .. windowEnd)
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;| where SourceIp == sourceIp and DestinationIp == destinationIp
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;| project TimeGenerated, _ResourceId, Protocol, SourceIp, SourcePort,
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; DestinationIp, DestinationPort, Action, ActionReason,
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; Policy, RuleCollectionGroup, RuleCollection, Rule
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;| order by TimeGenerated asc
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
&lt;/div&gt;
&lt;/div&gt;&lt;p&gt;Use the declared test window and actual source context instead of the illustrative values shown. The query needs an authorized Log Analytics workspace with resource-specific network-rule logging enabled and the table available.&lt;/p&gt;
&lt;p&gt;The query does not contain a workload principal or Blob request ID. Correlation by time and network tuple can be ambiguous, especially with address translation or concurrent traffic. Retain the source-side operation record and service evidence separately; do not invent a deterministic join that the sources cannot support.&lt;/p&gt;
&lt;h3 id="assemble-the-evidence-pack"&gt;Assemble the evidence pack
&lt;/h3&gt;&lt;p&gt;Use this review structure:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Pack component&lt;/th&gt;
&lt;th&gt;Minimum evidence&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Scope and baseline&lt;/td&gt;
&lt;td&gt;Stable cases, source/principal, topology, versions, successful baseline operation where removal is claimed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Change&lt;/td&gt;
&lt;td&gt;Reviewed diff, deployment actor/scope, actual configuration and convergence context&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Runtime comparison&lt;/td&gt;
&lt;td&gt;Expected/actual operation results, positive controls, request/time references and relevant raw logs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Availability and recovery&lt;/td&gt;
&lt;td&gt;Approved dependency, telemetry continuity and responder/recovery checks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exceptions and residuals&lt;/td&gt;
&lt;td&gt;Owner, expiry, compensating boundary, unknowns and reopening authority&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Decision and retest&lt;/td&gt;
&lt;td&gt;Reviewer conclusion, confidence limits, triggers and accountable owner&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Store sanitized raw evidence alongside summaries. A green workbook tile is a navigation aid, not the original evidence.&lt;/p&gt;
&lt;h2 id="real-world-impact"&gt;Real-World Impact
&lt;/h2&gt;&lt;p&gt;The pack gives security, engineering and architecture reviewers the same evidence to inspect. It also makes an unfinished result useful: the next engineer can see what is known and what still needs work.&lt;/p&gt;
&lt;h3 id="show-both-reduction-and-new-information"&gt;Show both reduction and new information
&lt;/h3&gt;&lt;p&gt;Better discovery can increase the number of known reachable paths after a useful change. Distinguish newly discovered baseline gaps from newly introduced reachability where evidence permits; otherwise state that the comparison cannot attribute the increase.&lt;/p&gt;
&lt;p&gt;Suppose the new collection finds a destination the baseline missed. Putting it straight into the denominator can make a useful fix look worse, or make a narrower collection look better. Compare the stable set, then report scope changes separately. Do not ask one percentage to explain both remediation and discovery.&lt;/p&gt;
&lt;h3 id="preserve-the-decisions-operational-cost"&gt;Preserve the decision&amp;rsquo;s operational cost
&lt;/h3&gt;&lt;p&gt;Include actual availability effects, exception age, telemetry cost and maintenance burden when observed. None should be invented to make a case study feel complete.&lt;/p&gt;
&lt;p&gt;I would retain a necessary recovery path with bounded authority rather than remove it to improve the chart. Explain that decision in the pack. Consequence weighting is still practitioner judgment; graph counts do not turn it into a probability estimate.&lt;/p&gt;
&lt;h2 id="gotchas-and-edge-cases"&gt;Gotchas and Edge Cases
&lt;/h2&gt;&lt;p&gt;A reassuring report does not need a dishonest author. Missing context is often enough. These checks help a well-intentioned team avoid measuring the wrong thing.&lt;/p&gt;
&lt;h3 id="silence-is-not-a-deny"&gt;Silence is not a deny
&lt;/h3&gt;&lt;p&gt;No Firewall record may mean no transit, missing logging, ingestion delay, inaccessible data or an empty window. Verify collection and run a known positive control before interpreting absence.&lt;/p&gt;
&lt;h3 id="query-schemas-can-change-the-conclusion"&gt;Query schemas can change the conclusion
&lt;/h3&gt;&lt;p&gt;Resource-specific tables, legacy diagnostic records and ASIM parsers are not interchangeable sources. Verify actual fields and parser behavior against known events. A parser returning rows is not proof it preserved the relevant action and entities.&lt;/p&gt;
&lt;h3 id="counting-every-graph-path-rewards-arbitrary-modeling"&gt;Counting every graph path rewards arbitrary modeling
&lt;/h3&gt;&lt;p&gt;Different graph granularity can produce different path counts for the same capability. Count stable test cases and report their consequence; do not equate combinatorial path counts with independent risks.&lt;/p&gt;
&lt;h3 id="an-outage-can-resemble-successful-containment"&gt;An outage can resemble successful containment
&lt;/h3&gt;&lt;p&gt;If the service is unavailable, both unwanted and approved operations may fail. The required positive control makes that distinction visible. A rollback can restore service while reopening the exposure, and both facts belong in the result.&lt;/p&gt;
&lt;h2 id="best-practices"&gt;Best Practices
&lt;/h2&gt;&lt;p&gt;Define expected outcomes before changing configuration. Keep the principal, source context, destination operation and observation method comparable. Record deliberate differences and invalidate comparisons that lack enough evidence.&lt;/p&gt;
&lt;h3 id="assumptions-and-evidence-boundary"&gt;Assumptions and evidence boundary
&lt;/h3&gt;&lt;p&gt;The query and comparison method use documented schemas and illustrative inputs; no live Azure test or tenant outcome is claimed. Record actual diagnostic categories and table schema, workspace/table RBAC, log ingestion and retention, Firewall tier/configuration, service operation audit coverage, ASIM parser version if used, and Azure cloud/region. ARM/API versions describe the configuration snapshot; they do not substitute for runtime results.&lt;/p&gt;
&lt;p&gt;Use read-only collectors and sanitized exports for local comparison. Query errors, incomplete runs and missing evidence must remain visible. Collect only what is needed and avoid retaining tokens, object contents or personal identifiers in published examples.&lt;/p&gt;
&lt;p&gt;I would approve a reduction claim only with a successful relevant baseline, an attributable post-change boundary, preserved required service and an honest residual/unknown record. If that evidence is incomplete, report configuration progress or partial assurance. There is no need to turn useful progress into a stronger claim than it supports.&lt;/p&gt;
&lt;p&gt;The pack becomes a measured result when authorized tests supply actual observations. Until then, it is a test design worth reviewing, not a result worth celebrating.&lt;/p&gt;
&lt;div class="insight"&gt;
&lt;div class="insight-icon"&gt;🍺&lt;/div&gt;
&lt;div class="insight-content"&gt;
&lt;strong&gt;Brewed Insight:&lt;/strong&gt; The strongest exposure report is not the one with the largest reduction percentage. It is the one whose scope, evidence, unknowns and operational consequences another engineer can inspect and challenge.
&lt;/div&gt;
&lt;/div&gt;
&lt;style&gt;
.insight {
display: flex;
align-items: center;
background-color: #0089e41c;
border-left: 10px solid #D69A2D;
padding: 10px;
margin: 20px 0;
border-radius: 4px;
}
.insight-icon {
font-size: 24px;
margin-right: 10px;
}
.insight-content {
flex: 1;
}
&lt;/style&gt;&lt;h2 id="learn-more"&gt;Learn More
&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;&lt;a class="link" href="https://learn.microsoft.com/azure/azure-monitor/reference/tables/azfwnetworkrule" target="_blank" rel="noopener"
&gt;AZFWNetworkRule table reference&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a class="link" href="https://learn.microsoft.com/azure/azure-monitor/essentials/diagnostic-settings" target="_blank" rel="noopener"
&gt;Azure Monitor diagnostic settings&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a class="link" href="https://learn.microsoft.com/azure/sentinel/normalization" target="_blank" rel="noopener"
&gt;Microsoft Sentinel normalization&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a class="link" href="https://blog.brewedinthecloud.com/azure-security-telemetry-toolkit/" &gt;Azure Security Telemetry Toolkit&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;</description></item></channel></rss>