6 Must-Have Features of an API Gateway

by Adrian Machado

API gateways are constantly evolving - I should know since I work on one at Zuplo. There are many hot topics within the space that are getting a lot of attention, including AsyncAPI support, fully-managed gateways, and special features for AI APIs (aka AI gateway). I think one of the most useful way to cut through the noise and find out what really matters to API developers out there is to just ask them what they are looking for. I chatted with several developers and engineering leaders to get a better understanding of which API gateway features matter most to them or have had the biggest impact. Hopefully, this guide will be helpful in choosing your next API gateway.

Must-Have Features for API Gateways#

Here's the tl;dr in case you're short on time. List is in no particular order.

1. Caching: Customizable caching helps reduce server load and improved API user experience.
2. Rate Limiting: Rate limiting helps ensure fair usage and prevents abuse.
3. Circuit Breaking: Preventing overload of backend services is critical in a high-scale API.
4. Local Development: Build and test the API gateway locally, allowing devs to quickly iterate on new features without having to wait for a lengthy build process.
5. Multi-Protocol Support: Whether its legacy protocols like XML/SOAP or new protocols like gRPC - gateways can't just support REST anymore.
6. Monitoring and Analytics: Detailed analytics (and especially real-time ones) are crucial in debugging issues and learning how to improve the API through usage patterns.

1. Caching#

Caching is one of the most popular features for an API gateway - as it is a win-win. The cached response saves your backend from having to do redundant work, freeing it up to handle other requests. Meanwhile, responses cached in a CDN are served lightning-fast to API users, speeding up their applications. Caching is especially important in AI models where compute is expensive. Here's what one interviewee had to say:

The right caching implementation ensures that frequently accessed data doesn't require a fresh query to the backend every time. In our case, this means quicker response times when dispatching locksmiths, even during peak demand. A gateway with configurable caching policies allows us to fine-tune what gets cached, for how long, and under what conditions, which is incredibly important when dealing with sensitive or time-dependent information.

For example, caching a customer's contact details for a short duration is useful, but you wouldn't want to cache something like an updated service status for too long, as it might lead to outdated information being presented.

Eli Itzhaki, CEO & Founder, Keyzoo

Like Eli mentioned - configurable caching is key to ensuring your data is cached for the right amount of time. Too long and it goes stale, while too short requires unnecessary calls to your backend. It's a bit of a goldilocks problem.

2. Rate Limiting#

Rate limiting is a core feature of most gateways. Some offer it as a separate service, like AWS API gateway. Others bake it into the gateway allowing the gateway to customize the rate limiter's behavior. Here's what a LinkedIn engineer things:

From managing LinkedIn's API infrastructure serving 100B+ monthly requests, I can tell you that a robust API gateway is absolutely critical - ours processes 2.3M requests per second with 99.99% reliability. [...]I've found that three key features consistently deliver the most value. First, advanced rate limiting capabilities that can handle complex rules based on multiple factors beyond just IP address...

Harman Singh, Senior Software Engineer, LinkedIn

Dynamic rate limiting based on properties of the user or request is quickly becoming table-stakes in high volume APIs.

Implementing a rate limit isn't easy, as we discussed in our API rate limiting best practices guide. You need to do some research on your traffic patterns to set the right limits. This company in the fintech space uses a combination of techniques to protect their API:

At the same time, the gateway needs to handle sudden spikes in mortgage applications by prioritizing high-demand services like credit checks and loan approvals. Load balancing, rate limiting, and real-time monitoring help prevent slowdowns and keep everything running efficiently. A well-structured API gateway keeps mortgage transactions moving without unnecessary interruptions.

Shaun Bettman, Chief Executive Officer, Eden Emerald Mortgages

Of course, combining rate limiting with authentication/authorization mechanisms provides a better level of protection for your API.

Features like OAuth, API key management, rate limiting, and IP whitelisting are essential to protect APIs and sensitive data.

Patric Edwards, Founder & Principal Software Architect, Cirrus Bridge

To learn more API rate limiting concepts, check out our advanced guide - the subtle art of rate limiting.

Over 10,000 developers trust Zuplo to secure, document, and monetize their APIs

Learn More

3. Circuit Breaking#

I was surprised to see so many people mention the circuit breaker pattern, as it is not often discussed or prominently featured by API gateways. Circuit breakers essentially prevent a service from becoming overwhelmed when dealing with failures. The circuit breaker can either be:

1. Closed: Requests can flow through normally and the system monitors for failures. If failure rate exceeds some threshold, the circuit breaker opens.
2. Open: Requests will not reach the failing service and are either redirected, or automatically responded to with cached data or an error message.
3. Half-Open: A small number of requests are allowed to hit the service. If they succeed, the circuit breaker transitions back to a closed state.

Here's what our interviewees had to say about circuit breakers:

When choosing an API gateway, I always prioritize features like throttling and circuit breaker patterns. These tools are lifesavers for maintaining system stability, especially during traffic spikes or unexpected surges in demand. Throttling helps control usage by limiting the number of requests, while circuit breakers stop overloaded services from dragging the entire system down.

Together, they create a safety net that keeps APIs running smoothly even when things get chaotic. It's all about delivering reliable performance without compromising the user experience, no matter the load.

Shawn Plummer, CEO, The Annuity Expert

Circuit breaking proved essential after rebuilding our entire API infrastructure. While everyone talks about basic routing and security, we discovered something crucial when one of our core services started failing. Our new gateway automatically detected the failure pattern and started gracefully degrading services instead of letting the entire system crash.

We built our gateway selection criteria after nearly losing a major client when a service cascade failure took down our platform for six hours. Rate limiting alone wasn't enough - we needed smart traffic management that could handle partial system failures. During our evaluation, we found most teams overlook this critical resilience factor.

Tristan Harris, Sr. VP of Marketing, Next Net Media

The most underrated feature we've discovered through extensive production use is sophisticated circuit-breaking capabilities. After analyzing thousands of outage incidents, we found that properly configured circuit breakers at the gateway level prevented 85% of potential cascading failures.

Harman Singh, Senior Software Engineer, LinkedIn

Wow, that's a lot of evidence that circuit breakers are useful. I think I will consider writing a built-in policy for that one.

4. Local Development#

This didn't come up very often - but from personal experience - local development support is definitely an underrated feature. I think most people think of API gateways as being a clunky piece of infrastructure stuck in the cloud, but in reality they can be nimble like the other developer tools you use.

Let me tell you what everyone overlooks in API gateways but actually matters: local development experience. Everyone talks about rate limiting and auth, but if your gateway makes local development a pain in the ass, it doesn't matter how many enterprise features it has.

I want to see request debugging that actually works on my local machine without needing to spin up the entire production infrastructure. If I have to deploy something just to test a basic API change, that's an immediate no from me.

Most teams get caught up in fancy features like traffic shifting and circuit breakers, but spend half their development time fighting their gateway just to test simple changes. Give me good local development tools over enterprise features any day.

Tim Hanson, CCO, Penfriend

5. Multi-Protocol Support#

Although I expected multi-protocol support to be mentioned a few times - I was dead-wrong on which protocols. I totally expected GraphQL, gRPC, or even AsyncAPI to be mentioned - but I am woefully out of touch. SOAP support still dominates this conversation - and its no surprise once you look at the numbers:

Here's why protocol translation matters in the first place:

Data comes in different formats and we need a tool to convert these data into a format that our system can understand and use. For example, a customer may input their information in one format, but our quotation system might need that data in a different format to generate a quote. The API gateway's Protocol Translation feature helps bridge this gap, enabling smooth communication between different systems.

Scott Chesarek, Co-Founder, J&S Transportation

It seems that the most common use-case for protocol translation is that you want to maintain a backend that only handles one format (for consistency and lower maintenance) but customers need to provide data in different (often legacy) formats. The gateway's role is to translate to your format of choice (ex. XML to JSON).

In diverse ecosystems, an API gateway should handle both REST and SOAP APIs as well as more recent protocols such as gRPC or WebSockets, providing flexibility across different platforms and use cases. This flexibility is essential when working with a variety of systems, ensuring that we can support both traditional and modern architectures.

Rodger Desai, CEO, Prove

Protocol translation isn't limited to the intake of data. You might also need to respond in a different format (ex. JSON back to SOAP for legacy customers, or to GraphQL for mobile apps). This is very common in the fintech space:

We use an API gateway, particularly for altering requests (like changing headers or reformatting) or responses (such as converting JSON to XML) to accommodate different clients. Our choice is based on its built-in functionalities, which reduce the need for extensive code.

Arslan Naseem, CEO, Kryptomind

6. Monitoring & Analytics#

Knowledge is power - especially when it comes to complex systems like APIs. Part of an API gateway's role is to centralize all of your API traffic - which enables you to consolidate all of your logs. This allows you to identify patterns in traffic like endpoints with surging traffic or errors. That's exactly how this interviewee uses data from their gateway:

Last but certainly not least, the ability to see logs and monitor activity is critical. Having the ability to view the trends of requests, the time it took to get a response, and in what instances there were errors enables us to react to problems far quicker.

Stephen Dove, Owner, Games Latest News

Your monitoring and analytics data isn't just limited in determining technical thresholds or issues. Sometimes a surge in use can indicate your business should prioritize developing new API features or improving the API developer experience.

Having a granular read on traffic-like which specific features are surging or which endpoints might be malfunctioning-lets us pivot quickly on both marketing and technical strategies. Essentially, our gateway doubles as an early warning system for product priorities.

Derek Pankaew, CEO & Founder, Listening.com

Wrapping Up#

Here's one last story that perfectly summarized why investing early into an API gateway is useful.

I remember brainstorming with our CTO, Elena, about how to make our APIs both powerful and user-friendly. That's when she suggested implementing an API gateway. At first, I was skeptical—it seemed like an extra layer of complexity. But Elena's enthusiasm was contagious, and we decided to give it a shot.

Within six months of launch, we had over 100 companies integrating our forecasting tool into their platforms. Our API gateway made it seamless for them to get started and scale their usage as needed.

One success story stands out. A small startup, founded by a brilliant developer named Marcus, used our API to build a personal finance app. Thanks to the easy integration, Marcus was able to focus on his unique value proposition rather than reinventing the wheel on financial forecasting. His app went viral, and our API usage skyrocketed along with his success.

Looking back, implementing an API gateway from the start was one of the best decisions we made. It allowed us to scale rapidly, foster a thriving ecosystem around our product, and ultimately deliver more value to end-users. In today's interconnected tech landscape, a well-managed API strategy isn't just a technical necessity—it's a powerful driver of business growth.

Ayush Trivedi, CEO, Cyber Chief

Well said! If you’re looking into adopting an API gateway, get in touch! We’re passionate about helping companies bring their APIs to market in a secure, scalable, and user-friendly way.