North-South Is Not East-West: Putting API Gateways and Service Meshes Back in Their Lanes
Photo: MSDN.WhiteKnight, CC0, via Wikimedia Commons
Few infrastructure debates waste more money than the one between API gateways and service meshes. The two categories are often presented as alternatives, or as overlapping layers that should be consolidated. In practice, they govern different traffic, serve different audiences, and fail in different ways. The confusion is understandable, but it is not harmless. Teams that deploy a mesh as an edge gateway, or a gateway as an internal service control plane, end up with complexity they cannot explain and bills they cannot defend.
The useful distinction is direction. API gateways primarily manage north-south traffic: requests entering the platform from clients, partners, or public networks. Service meshes primarily manage east-west traffic: service-to-service communication inside the platform. That boundary is not absolute, but it is the right starting point for design.
Two Layers, Two Audiences
An API gateway is an edge product. It handles authentication, rate limiting, request routing, API keys, quotas, and often developer-facing concerns such as versioning and documentation. Its consumers are external or semi-external clients. Its job is to make the platform safe and legible to the outside world.
A service mesh is an internal operations product. It handles mutual TLS, service identity, retries, timeouts, traffic shifting, and telemetry between workloads. Its consumers are platform teams and service owners. Its job is to make internal communication observable and policy-controlled without forcing every application to reimplement those concerns.
When those roles are clear, the architecture becomes easier to reason about. The gateway answers who may enter and under what contract. The mesh answers how services talk once they are inside.
How the Swap Happens
Teams usually swap the layers for one of three reasons. First, they already own one tool and want to stretch it. A mesh with ingress capabilities looks like a gateway. A gateway with sidecar support looks like a mesh. Second, platform consolidation pressure encourages a single control plane for all traffic. Third, cloud providers bundle edge and internal networking features in ways that blur the categories.
Each reason is understandable, but each carries a hidden cost. Edge policy is not the same as internal policy. A rate limit for a public API client is a business contract; a retry budget between internal services is a reliability control. When one layer absorbs both, teams lose the ability to tune them independently and often pay for data plane capacity they do not need.
The Bill and the Blast Radius
Cost is the most visible symptom. Meshes add sidecars or node-level proxies to every workload, which increases compute and memory overhead. Gateways concentrate traffic at the edge, where capacity can be scaled more deliberately. Using a mesh as an edge gateway can multiply proxy instances across the fleet for traffic that only needs to enter at a few points.
Blast radius is the quieter symptom. A gateway outage affects external clients and partner integrations. A mesh misconfiguration affects internal service calls. Those are different incident profiles with different rollback strategies. Combining them into one control plane can make a routine edge policy change riskier than it needs to be.
A Decision Framework
Start with the traffic path. If a request originates outside the platform and needs authentication, quotas, or a public contract, it belongs at the gateway. If a request originates inside the platform and needs service identity, traffic policy, or internal telemetry, it belongs in the mesh.
Next, ask who owns the policy. Edge policies are often owned by API product teams, security, and developer relations. Internal traffic policies are owned by platform engineering and service owners. If a single tool forces those groups into one workflow, the organizational friction will eventually become technical debt.
Finally, ask what failure looks like. If the edge layer fails, can internal services keep communicating? If the mesh fails, can external clients still reach the platform through a degraded path? The answers should shape the separation, not the vendor's feature matrix.
Where the Layers Should Meet
The layers should meet at a clear handoff: the gateway authenticates and routes the request to an internal entry service, and the mesh takes over from there. Identity should be propagated in a way that both layers understand, and telemetry should share correlation identifiers so an edge request can be traced through internal calls.
That handoff is also where policy duplication should be eliminated. The gateway should not reimplement service-level retries, and the mesh should not own public API quotas. A small amount of overlap is acceptable for defense in depth, but broad duplication creates two sources of truth.
What to Do This Quarter
Inventory the traffic paths that cross the edge and those that stay inside. Map each policy to the layer that owns it. Identify any mesh features used for external traffic and any gateway features used for service-to-service calls. Then decide whether those uses are deliberate exceptions or accidental drift.
API gateways and service meshes are not rivals. They are adjacent layers with different jobs. Keeping them in their lanes is not architectural purity; it is how teams keep policy clear, incidents bounded, and cloud spend defensible in 2026.
