Apache APISIX

Apache APISIX is a high-performance, open-source API gateway designed for teams running microservices, APIs, and increasingly, AI workloads. Built on NGINX and etcd, it handles roughly 18,000 requests per second per core with around 0.2ms added latency. That's not a marketing claim – it's a product of its radixtree route matching and plugin architecture.

The gateway is fully dynamic. Configuration changes propagate across the cluster via etcd in real time, with no restarts required. Plugins hot-reload, routes update live, and health checks run continuously. For teams that need to iterate quickly without downtime, this matters.

Key capabilities include:

• 100+ plugins covering authentication (JWT, Key Auth, OIDC, Keycloak), rate limiting, CORS, circuit breaking, and observability integrations
• LLM traffic management – proxy and govern requests to 20+ providers including OpenAI, Anthropic, AWS Bedrock, DeepSeek, and Ollama, making it a capable AI gateway alongside tools like LiteLLM or Portkey
• Protocol flexibility – HTTP, gRPC, WebSockets, Dubbo, MQTT, and HTTP-to-gRPC transcoding
• Platform agnostic – runs on bare metal, VMs, Kubernetes, and ARM64 hardware
• Custom extensibility – write your own plugins, load balancing algorithms, and routing logic

Compared to alternatives like Tyk or KrakenD, APISIX leans heavily on its plugin ecosystem and the Apache Software Foundation's governance model. It's a top-level ASF project, Apache 2.0 licensed, with a broad community behind it.

The AI gateway layer isn't a separate product. It's the same routing and plugin infrastructure applied to LLM traffic, which means you get the same observability, security controls, and load balancing for your AI apps that you already run for your REST APIs.