Catching CIDR Overlaps Before Peering Fails

A VPC peering request fails silently in one of two ways: either the cloud provider’s API rejects it outright because of an obvious address collision, or worse, it succeeds because the overlap is partial and non-obvious — a /24 buried inside a /16 that nobody audited before the peering connection was accepted. The second case is the one that takes down production traffic three months later when someone adds a new subnet to the smaller block. This is the exact failure class that a properly engineered CIDR overlap detection utility is built to eliminate, and it is precisely the kind of diagnostic tool that belongs in an internal IT toolkit rather than in a spreadsheet someone updates manually.

Most organisations running multi-account AWS, Azure, and GCP environments rely on IPAM spreadsheets or, at best, a loosely governed IPAM tool that gets checked before provisioning. Neither approach scales past a few dozen VPCs, and neither catches transitive overlap introduced by route propagation across peering meshes or Transit Gateway attachments. What follows is the architecture for a purpose-built CIDR overlap detection engine capable of running as a pre-flight check in a provisioning pipeline, not just a manual lookup tool.

#The Core Bottleneck: Naive String and Range Comparison Doesn’t Scale

The instinctive implementation treats CIDR blocks as string ranges and does pairwise comparison — O(n²) complexity across every VPC in the estate. At 50 VPCs this is tolerable. At 5,000 VPCs across three cloud providers and a dozen peering meshes, pairwise comparison becomes a multi-minute batch job that nobody runs before provisioning because it’s too slow to be useful in a CI/CD gate.

The second common mistake is comparing CIDRs as decimal integer ranges without respecting bit-boundary alignment. This produces false negatives on non-contiguous, non-power-of-two subnet arrangements that are increasingly common when organisations use VLSM (Variable Length Subnet Masking) to conserve RFC 1918 space across hundreds of accounts.

#Architectural Breakdown: Trie-Based CIDR Overlap Detection

The correct data structure for this problem is a binary trie (also called a radix trie or PATRICIA trie) keyed on the bit representation of the network address, with each node annotated by prefix length. This converts the overlap check from O(n²) pairwise comparison into O(n log n) insertion with O(log n) lookup, because overlap between two CIDRs is fundamentally a question of whether one is an ancestor or descendant of the other within the bit tree — not a range comparison problem at all.

Two CIDR blocks overlap if and only if one prefix is a bitwise prefix of the other, up to the length of the shorter mask. A /24 overlaps a /16 if the first 16 bits match. This is trivially checked with a masked AND operation:

1import ipaddress
2
3def cidrs_overlap(a: str, b: str) -> bool:
4    net_a = ipaddress.ip_network(a, strict=False)
5    net_b = ipaddress.ip_network(b, strict=False)
6    if net_a.version != net_b.version:
7        return False  # IPv4/IPv6 never collide directly
8    return net_a.overlaps(net_b)
9
10# Trie insertion node structure
11class TrieNode:
12    __slots__ = ('children', 'cidr', 'is_terminal')
13    def __init__(self):
14        self.children = {}
15        self.cidr = None
16        self.is_terminal = False

For production-scale CIDR overlap detection, the trie is built once per scan cycle and held in memory (or in Redis as a serialized structure for horizontal scaling across API instances). Every insertion walks the bit path of the network address; if a terminal node is hit before the walk completes, or if the walk completes and children already exist beneath it, an overlap has been detected and both offending CIDRs are recorded with their owning account/VPC metadata.

#Multi-Cloud Normalisation Layer

AWS, Azure, and GCP each expose VPC/VNet CIDR data through structurally different APIs. Before any trie logic runs, a normalisation layer must flatten these into a common schema. This is the layer most implementations get wrong — they either hardcode a single provider or bolt on providers as an afterthought, which breaks the overlap detection guarantees when secondary CIDR blocks (common in AWS VPCs and GCP subnetworks) aren’t captured.

1{
2  "resource_id": "vpc-0a1b2c3d4e5f6g7h8",
3  "provider": "aws",
4  "account_id": "482910337421",
5  "region": "eu-west-2",
6  "primary_cidr": "10.42.0.0/16",
7  "secondary_cidrs": ["100.64.0.0/20"],
8  "peering_connections": ["pcx-0d9e8f7g6h5i4j3k2"],
9  "route_propagation": true
10}

The route_propagation flag matters more than most engineers assume — see the transitive overlap failure mode below.

#Implementation Logic: Building the Pre-Flight Gate

The utility is most useful when it runs as a gate inside the provisioning pipeline rather than as an ad-hoc dashboard query. The implementation sequence is:

• Ingest: Pull current CIDR allocations from AWS EC2 DescribeVpcs, Azure Resource Graph, and GCP Compute Engine APIs on a scheduled basis (5–15 minute intervals is typical for a mid-sized estate).
• Normalise: Map all responses into the common schema above, resolving both primary and secondary/additional CIDR ranges.
• Build the trie: Insert every CIDR, IPv4 and IPv6 trees kept separate, tagging insertions with resource metadata.
• Scope the check: On a new peering or Transit Gateway attachment request, only walk the trie for CIDRs reachable within the proposed routing domain — not the entire estate, which would produce excessive false positives for VPCs that will never route to one another.
• Reject or warn: Return a structured verdict object consumable by Terraform, a CI pipeline, or a self-service portal.

This kind of gated validation logic follows the same principle used across other architectural patterns for infrastructure guardrails: fail fast, fail with context, and never let a human manually re-derive the reason for rejection.

#CLI Wrapper for Local Pre-Flight Checks

Engineers requesting peering ad hoc need a fast local check before submitting a Terraform PR. A thin CLI wrapping the same trie logic via an API call keeps the source of truth centralised while giving instant feedback:

1$ cidrcheck overlap --source vpc-0a1b2c3d4e5f6g7h8 --target vpc-9z8y7x6w5v4u3t2s1
2
3[FAIL] CIDR overlap detected
4  source: 10.42.0.0/16 (vpc-0a1b2c3d4e5f6g7h8, eu-west-2)
5  target: 10.42.32.0/24 (vpc-9z8y7x6w5v4u3t2s1, us-east-1)
6  overlap_type: subset
7  recommendation: reallocate target VPC to a non-conflicting /24 within 10.90.0.0/16
8
9exit code: 1

The exit code matters as much as the message — this is what makes the tool usable as a hard gate in a CI job rather than an advisory step that gets ignored under delivery pressure.

#Failure Modes and Edge Cases

A naive CIDR overlap detection implementation passes unit tests and still fails in production for several recurring reasons.

#Transitive Overlap via Route Propagation

Two VPCs, A and B, have non-overlapping CIDRs and peer successfully. VPC A also peers with VPC C. If route propagation is enabled across a Transit Gateway and C’s CIDR overlaps B’s, traffic destined for B from C’s network can silently blackhole or, worse, get misrouted to A instead. Direct pairwise checking between B and C never happens because they never peer directly — the overlap only manifests through A’s routing table. The detection engine must walk the full transitive closure of the peering/TGW graph, not just direct connections, which is why the route_propagation and peering_connections fields in the normalisation schema are non-optional.

#IPv6 and Dual-Stack Ambiguity

IPv4 and IPv6 CIDRs never overlap in address space, but dual-stack VPCs frequently have IPv6 blocks auto-assigned from provider-managed GUA pools (e.g. AWS Amazon-provided IPv6 CIDRs). Overlap detection logic that only checks IPv4 gives a false sense of coverage once IPv6-only peering paths or NAT64 gateways enter the topology. Maintain fully separate tries per address family — never attempt to coerce IPv6 into the same bit-length comparison logic as IPv4.

#RFC 1918 Exhaustion Across Shared Services Accounts

Large estates frequently reuse RFC 1918 space across isolated accounts that were never expected to peer — until a shared-services VPC (logging, DNS, identity) needs connectivity to all of them simultaneously. This is the single most common trigger for large-scale renumbering projects, and it is exactly the scenario where automated CIDR overlap detection run continuously (not just at peering time) surfaces latent conflicts months before they become a hard blocker for a shared-services rollout.

#Non-Byte-Aligned Prefixes

A /27 comparison against a /29 nested three levels deep inside it is handled correctly by the trie approach because comparison operates at the bit level, not the octet level. Range-based comparison implementations frequently mishandle these because engineers round mentally to octet boundaries when writing test fixtures, leaving off-by-a-few-bits overlaps unvalidated.

#Scaling and Security Trade-offs

Running this as a shared internal service rather than a per-team script introduces trade-offs worth deciding on deliberately rather than by default.

• In-memory trie per instance vs. shared Redis-backed trie: In-memory is faster (sub-millisecond lookups) but requires cache invalidation logic on every CIDR change across three cloud providers; Redis-backed sharing avoids stale state across replicas at the cost of network round-trip latency (typically 1–3ms per lookup).
• Read access scope: The engine needs read-only, cross-account IAM roles (AWS Organizations, Azure Lighthouse, GCP Shared VPC host project access) spanning the entire estate. This is a high-value target — scope the role to DescribeVpcs, DescribeSubnets, and equivalent read-only actions only, never write permissions.
• Blast radius of a false negative: A missed overlap doesn’t fail loudly — it manifests as intermittent packet loss or asymmetric routing weeks later, which is substantially harder to diagnose than a rejected peering request. This asymmetry justifies erring toward stricter detection even at the cost of occasional false positives on genuinely non-routable overlapping CIDRs.
• Audit logging: Every overlap verdict, pass or fail, should be logged with the requesting identity and full CIDR metadata. This becomes the evidentiary trail during a renumbering project when reconstructing why certain VPCs were historically allowed to keep conflicting ranges.
• Polling interval vs. API rate limits: Aggressive polling (sub-minute) against cloud provider description APIs risks throttling on large estates; a 5–15 minute ingestion cycle balances freshness against AWS VPC peering API quota consumption.

Treat the overlap detector as infrastructure in its own right — versioned, monitored, and subject to the same change control as the networks it protects. A tool that silently drifts out of sync with the real estate is worse than no tool at all, because it manufactures false confidence exactly where confidence is most dangerous.