An API gateway is a server that sits between clients and backend services, acting as the single entry point for all API traffic. It accepts incoming requests, applies policies such as authentication, rate limiting, and transformation, then routes each request to the appropriate upstream service and returns the response to the caller.
In practice, an API gateway consolidates cross-cutting concerns that would otherwise be duplicated across every microservice: access control, traffic shaping, observability, and protocol translation. Instead of embedding this logic in each service, teams centralize it at the gateway layer, reducing code duplication, simplifying deployments, and giving platform teams a single control plane for governing API behavior at scale.
How Does an API Gateway Work?#
The request lifecycle through an API gateway follows a well-defined pipeline:
Client sends a request. A mobile app, browser, or upstream service issues an HTTP/HTTPS request to the gateway's public endpoint. The client never communicates directly with individual backend services.
Gateway evaluates policies. The gateway inspects the incoming request and runs it through a chain of plugins or middleware. This typically includes validating authentication tokens (JWT, OAuth 2.0, API keys), enforcing rate limits, checking IP allowlists, and applying request transformations such as header injection or body rewriting.
Gateway routes to the upstream. Based on the request path, host header, or other matching criteria, the gateway selects a target upstream service. If multiple instances are registered, the gateway applies a load-balancing algorithm (round-robin, least connections, consistent hashing) to pick a specific node.
Backend processes the request. The upstream service handles the business logic and returns a response to the gateway.
Gateway processes the response. Before forwarding the response to the client, the gateway can apply response transformations, inject CORS headers, compress the payload, or cache the result for subsequent identical requests.
Gateway returns the response. The final response reaches the client with appropriate status codes, headers, and payload. Throughout this entire cycle, the gateway emits metrics, access logs, and traces that feed into observability systems.
This pipeline executes in milliseconds. High-performance gateways like Apache APISIX complete it in under 1ms of added latency, making the overhead negligible even for latency-sensitive workloads.
Key Features of an API Gateway#
A production-grade API gateway provides a broad surface of capabilities. The following features represent the core functionality that distinguishes a gateway from a simple reverse proxy.
Request Routing#
The gateway matches incoming requests to upstream services based on URI paths, HTTP methods, headers, query parameters, or custom expressions. Advanced gateways support regex-based matching, wildcard routes, and priority-weighted rules. Apache APISIX supports radixtree-based routing that scales efficiently even with thousands of route entries.
Load Balancing#
Distributing traffic across service instances prevents hotspots and improves availability. Gateways typically support round-robin, weighted round-robin, least connections, consistent hashing, and EWMA (exponentially weighted moving average) algorithms. Health checks --- both active probes and passive failure detection --- automatically remove unhealthy nodes from the upstream pool.
Authentication and Authorization#
Centralizing identity verification at the gateway eliminates the need for each service to implement its own auth stack. Common mechanisms include JWT validation, OAuth 2.0 token introspection, HMAC signatures, LDAP, and API key authentication. Some gateways also integrate with external identity providers through OpenID Connect.
Rate Limiting#
Rate limiting protects backend services from traffic spikes, abusive clients, and cascading failures. Gateways enforce limits at multiple granularities: per consumer, per route, per IP, or globally. Apache APISIX provides configurable rate limiting plugins that support both fixed-window and leaky-bucket algorithms, with shared counters across gateway nodes via Redis.
Caching#
Response caching at the gateway layer reduces backend load and improves latency for read-heavy endpoints. Gateways cache responses based on configurable TTLs, cache keys (URI, headers, query strings), and invalidation rules. For APIs serving relatively static data --- product catalogs, configuration endpoints, reference data --- caching can reduce upstream requests by 80% or more.
Request and Response Transformation#
Gateways can rewrite requests before they reach the backend and transform responses before they reach the client. This includes header manipulation, body rewriting, protocol translation (HTTP to gRPC, REST to GraphQL), and payload format conversion. Transformation eliminates the need for adapter services and simplifies API versioning.
Monitoring and Observability#
A gateway sees every request, making it the natural instrumentation point for API metrics. Production gateways export access logs, request/response latencies (P50, P95, P99), error rates, and throughput to systems like Prometheus, Datadog, and OpenTelemetry collectors. Apache APISIX ships with built-in integrations for Prometheus, Grafana, SkyWalking, and Zipkin.
SSL/TLS Termination#
The gateway handles TLS handshakes, certificate management, and encryption offloading so that backend services can communicate over plain HTTP internally. This simplifies certificate rotation, centralizes security policy, and reduces CPU overhead on upstream services. Modern gateways also support mTLS for service-to-service authentication.
Circuit Breaking#
When a backend service becomes degraded or unresponsive, a circuit breaker at the gateway stops forwarding requests to it, preventing cascading failures across the system. After a configurable cooldown, the gateway sends probe requests to test recovery. This pattern is critical in microservices architectures where a single failing service can take down an entire request chain.
API Versioning and Canary Releases#
Gateways can route a percentage of traffic to new service versions, enabling canary deployments and blue-green releases without infrastructure changes. Traffic-splitting rules let teams gradually shift load from v1 to v2, monitor error rates, and roll back instantly if metrics degrade.
API Gateway vs Load Balancer vs Reverse Proxy#
These three components overlap in functionality but serve different primary purposes:
| Capability | Reverse Proxy | Load Balancer | API Gateway |
|---|---|---|---|
| Request forwarding | Yes | Yes | Yes |
| SSL termination | Yes | Sometimes | Yes |
| Load balancing | Basic | Advanced | Advanced |
| Health checks | Limited | Yes | Yes |
| Authentication | No | No | Yes |
| Rate limiting | No | No | Yes |
| Request transformation | No | No | Yes |
| API-aware routing | No | No | Yes |
| Response caching | Yes | No | Yes |
| Observability/metrics | Basic | Basic | Comprehensive |
| Protocol translation | No | No | Yes |
| Plugin/middleware ecosystem | Limited | No | Extensive |
A reverse proxy (e.g., Nginx, HAProxy in proxy mode) forwards client requests to backend servers, provides SSL termination, and can cache static content. It operates at the HTTP level but lacks API-specific intelligence.
A load balancer (e.g., AWS ALB, HAProxy, Envoy in LB mode) distributes traffic across server instances using health checks and balancing algorithms. Layer 4 load balancers work at the TCP level; Layer 7 load balancers can inspect HTTP headers but still lack API-layer logic like authentication or transformation.
An API gateway builds on reverse proxy and load balancing capabilities but adds an API-aware policy layer: authentication, rate limiting, request/response transformation, observability, and developer portal integration. It is purpose-built for managing API traffic.
In practice, many organizations start with a reverse proxy or load balancer and later adopt an API gateway as their API surface grows. Some gateways, including Apache APISIX, are built on top of proven proxies (APISIX uses Nginx and OpenResty) and inherit their performance characteristics while adding the API management layer.
API Gateway Use Cases#
Microservices Architecture#
In a microservices system with dozens or hundreds of services, an API gateway provides the single entry point that abstracts internal service topology from external consumers. Clients interact with one stable endpoint; the gateway handles service discovery, routing, and cross-cutting concerns. Without a gateway, each client must know the location and protocol of every service, creating tight coupling and operational fragility.
Mobile and IoT Backends#
Mobile clients operate under bandwidth, latency, and battery constraints that differ significantly from desktop browsers. An API gateway can aggregate multiple backend calls into a single response (the Backend-for-Frontend pattern), compress payloads, and adapt protocols. For IoT devices that may use MQTT or CoAP, the gateway translates between device protocols and internal HTTP/gRPC services.
Multi-Cloud and Hybrid Deployments#
Organizations running services across AWS, GCP, Azure, and on-premises data centers use an API gateway as the unified traffic layer. The gateway abstracts the underlying infrastructure, enabling consistent routing, security policies, and observability regardless of where a service is deployed. This is especially valuable during cloud migration, where services move between environments incrementally.
API Monetization#
Companies that expose APIs as products --- payment processors, data providers, communication platforms --- use gateways to enforce usage tiers, track consumption per API key, and generate billing data. Rate limiting by tier, quota enforcement, and detailed usage analytics are all gateway responsibilities in this model.
Zero-Trust Security#
A gateway enforces authentication and authorization at the network edge, ensuring that no unauthenticated request reaches backend services. Combined with mTLS for internal traffic, IP allowlisting, and WAF integration, the gateway becomes a core component of a zero-trust architecture. It can also mask or redact sensitive fields in responses to prevent data leakage.
Legacy System Modernization#
When migrating from monolithic to microservices architectures, an API gateway acts as the facade in the strangler fig pattern. New services are deployed behind the gateway alongside the legacy monolith. The gateway gradually shifts traffic from old endpoints to new ones, allowing incremental migration without disrupting existing clients.
Benefits of Using an API Gateway#
Simplified Client Integration#
Clients interact with a single, well-documented endpoint instead of tracking the addresses and protocols of individual services. This reduces client-side complexity, eliminates service discovery logic in front-end code, and makes API consumption predictable.
Centralized Security#
Authentication, authorization, encryption, and threat detection are enforced at one layer rather than reimplemented in every service. A single policy change at the gateway propagates instantly across all APIs. This consistency eliminates the security gaps that emerge when individual teams implement auth differently.
Operational Visibility#
Because every request passes through the gateway, teams gain comprehensive metrics, access logs, and distributed traces without instrumenting each service individually. Dashboards built on gateway telemetry provide real-time visibility into traffic patterns, error rates, and latency distributions across the entire API surface.
Reduced Backend Load#
Caching, request deduplication, and rate limiting at the gateway layer prevent unnecessary calls from reaching backend services. This directly reduces infrastructure costs and improves system stability during traffic spikes. For read-heavy APIs, gateway caching alone can cut upstream load by an order of magnitude.
Faster Time to Market#
Developers focus on business logic rather than reimplementing cross-cutting concerns. Adding authentication to a new service takes a single plugin configuration at the gateway instead of weeks of development. Teams ship faster because infrastructure concerns are already solved.
Independent Scalability#
The gateway and backend services scale independently. During a traffic surge, teams can horizontally scale the gateway layer without modifying any backend service. Conversely, backend services can be scaled, redeployed, or replaced without any client-facing changes.
How Apache APISIX Works as an API Gateway#
Apache APISIX is a high-performance, cloud-native API gateway built on Nginx and LuaJIT. It is designed for environments where throughput, latency, and extensibility are critical requirements.
Performance at scale. APISIX handles over 18,000 requests per second per CPU core with a median latency of 0.2ms. This performance comes from its non-blocking, event-driven architecture and the efficiency of LuaJIT-compiled plugin execution. For comparison, this throughput exceeds most Java- and Go-based gateways by a significant margin.
Extensive plugin ecosystem. APISIX ships with over 100 built-in plugins covering authentication (JWT, OAuth, LDAP, OpenID Connect), traffic control (rate limiting, circuit breaking, traffic mirroring), observability (Prometheus, SkyWalking, OpenTelemetry), and transformation (gRPC transcoding, request rewriting, response rewriting). Plugins can also be written in Lua, Java, Go, Python, or WebAssembly.
Dynamic configuration. Unlike traditional gateways that require restarts for configuration changes, APISIX reloads routes, upstreams, and plugin configurations in real time through its Admin API. This enables zero-downtime deployments and makes APISIX well-suited for CI/CD pipelines and GitOps workflows.
Proven adoption. APISIX powers over 147,000 deployments across more than 5,200 companies globally, spanning industries from fintech to telecommunications. Its Apache Software Foundation governance ensures vendor-neutral, community-driven development.
To get started with APISIX, see the getting started guide.
Frequently Asked Questions#
What is the difference between an API gateway and a load balancer?#
A load balancer distributes incoming traffic across multiple server instances using algorithms like round-robin or least connections. It operates at the network or transport layer (L4) or HTTP layer (L7) but does not understand API semantics. An API gateway performs load balancing as one of many functions, and adds API-specific capabilities: authentication, rate limiting, request transformation, caching, and observability. If you only need to distribute traffic, a load balancer suffices. If you need to manage, secure, and observe API traffic, you need a gateway.
Do I need an API gateway for a monolithic application?#
An API gateway is not strictly required for a monolith, but it can still add value. If your monolith exposes APIs consumed by external clients, mobile apps, or third-party integrators, a gateway provides centralized authentication, rate limiting, and monitoring without modifying the application. It also positions your architecture for incremental migration to microservices using the strangler fig pattern.
How does an API gateway affect latency?#
A well-implemented gateway adds minimal latency --- typically 0.2ms to 2ms per request depending on the number of active plugins. High-performance gateways like Apache APISIX are optimized for sub-millisecond overhead. The latency tradeoff is almost always worthwhile: the gateway eliminates redundant auth checks, reduces backend calls through caching, and prevents cascading failures through circuit breaking, all of which improve overall system response times.
Can an API gateway replace a service mesh?#
An API gateway and a service mesh serve different layers. The gateway handles north-south traffic (external clients to internal services), while a service mesh manages east-west traffic (service-to-service communication within the cluster). They are complementary, not competing, technologies. Some organizations use APISIX as both a gateway and an ingress controller, bridging the two layers, but a full service mesh (Istio, Linkerd) addresses concerns like mutual TLS between services and fine-grained internal traffic policies that fall outside a gateway's scope.
Is an API gateway the same as an API management platform?#
No. An API gateway is the runtime component that processes API traffic. An API management platform is a broader category that typically includes a gateway, a developer portal, API documentation tools, lifecycle management, and analytics dashboards. The gateway is the engine; the management platform is the full vehicle. Apache APISIX provides the high-performance gateway layer, and organizations often pair it with additional tooling for the complete API management lifecycle.