alibaba

higress

DevOps#API Gateway#Kubernetes#Istio#Envoy#Wasm

8,096

// summary

Higress is a cloud-native API gateway built on Istio and Envoy that supports extensibility through Wasm plugins written in Go, Rust, or JavaScript. It provides specialized capabilities for AI applications, including unified management for LLM APIs and hosting for Model Context Protocol (MCP) servers. Originally developed at Alibaba, the platform offers production-grade performance with support for high-concurrency traffic and seamless configuration updates.

// technical analysis

Higress is a cloud-native API gateway built on Istio and Envoy, designed to provide high-performance traffic management with a focus on AI-native capabilities. It addresses the limitations of traditional gateways by eliminating traffic jitter during configuration reloads and offering robust support for gRPC, Dubbo, and AI model integration. By leveraging Wasm plugins, the architecture ensures memory safety and extensibility, allowing developers to implement custom logic in Go, Rust, or JavaScript without disrupting active connections.

// key highlights

Provides an AI-native gateway that supports mainstream LLM providers with features like multi-model load balancing, token rate limiting, and observability.

Enables hosting of Model Context Protocol (MCP) Servers via plugins, allowing AI agents to securely call tools and services with unified authentication.

Utilizes Wasm-based plugin architecture to support hot updates of gateway logic without traffic loss or connection drops.

Offers high-performance streaming processing for request and response bodies, which is optimized for high-bandwidth AI scenarios like Server-Sent Events (SSE).

Functions as a feature-rich ingress controller that is compatible with Nginx ingress annotations and supports the Kubernetes Gateway API.

Integrates seamlessly with various service registries like Nacos, ZooKeeper, and Consul, making it suitable for complex microservice environments.

// use cases

AI Gateway: Provides unified protocol support, observability, multi-model load balancing, and token rate limiting for LLM providers.

MCP Server Hosting: Enables AI Agents to call tools and services by hosting MCP servers with integrated authentication, rate limiting, and observability.

Kubernetes Ingress and Microservice Gateway: Functions as a high-performance ingress controller and microservice gateway with deep integration for service discovery and security protocols.

// getting started

To begin using Higress, you can deploy it locally using a single Docker command, which provides an all-in-one image containing the gateway and the UI console. For Kubernetes environments, you can use Helm to install the project, with options to configure regional mirror registries for faster image pulling. Once running, you can access the UI console on port 8001 to manage your routes, plugins, and AI gateway configurations.