Key takeaway
- Envoy Proxy explained: how listeners, filters and xDS work, edge versus sidecar roles, Istio and Envoy Gateway options, and when to adopt it.
Envoy Proxy is an open source edge and service proxy, designed for cloud-native and AI-native applications, according to Envoy's own site. It sits next to each application or at the edge of a cluster, handles the network on the application's behalf, and gives platform teams one place to apply routing, retries, TLS and telemetry without changing application code.
For enterprise architects the useful question is not whether Envoy is fast, but where it belongs in the stack. It is the data plane in Istio, which we cover in Istio service mesh architecture and service mesh best practices. This guide explains how Envoy works, which tools are built on it, and when a simpler proxy is the better choice.
What is Envoy Proxy?
Envoy is a C++ distributed proxy, originally built at Lyft and now a Cloud Native Computing Foundation graduated project. The CNCF project page records acceptance at the Incubating level on 13 September 2017 and graduation on 28 November 2018 (CNCF Envoy page). It targets cloud-native applications built from many services, where each service needs the same network behaviour.
According to the Envoy documentation, the design goal is that the network should be transparent to applications, and that problems should be easy to trace. Envoy runs as a self-contained process alongside every application server, so the application talks only to localhost and Envoy handles the rest (Envoy docs: what is Envoy).
How does Envoy Proxy work?
Envoy works at two levels at once. At its core it is an L3/L4 network proxy with pluggable filters for TCP and UDP traffic. On top of that, an HTTP connection manager accepts HTTP filters for tasks such as routing, buffering and rate limiting, so the same process can forward raw connections or make per-request decisions.

Which building blocks does a request pass through?
| Building block | What it does | Why an architect cares |
|---|---|---|
| Listener | Accepts inbound connections on an address and port, then runs its filter chain | Defines what traffic enters the proxy and under which TLS settings |
| Filter chain | Network filters (L3/L4) and HTTP filters (L7) applied in order | Where authentication, rate limiting and routing policy is enforced |
| Route | Matches a request to a destination cluster | Controls traffic shifting, retries and request shadowing |
| Cluster and endpoints | A named upstream service and its members, with load balancing and health checks | Where circuit breaking and zone-aware balancing apply |
The Envoy site lists automatic retries, circuit breaking, global rate limiting, request shadowing and zone-aware load balancing among its load-balancing features, plus first-class HTTP/2, HTTP/3 and gRPC support and transparent HTTP/1.1 to HTTP/2 proxying (Envoy's own site).
How is Envoy configured without restarts?
Envoy can consume a layered set of dynamic configuration APIs, known together as xDS, for centralised management. The documentation names the Listener (LDS), Route (RDS), Cluster (CDS), Endpoint (EDS) and Secret (SDS) discovery services, which cover listeners, routes, upstream clusters, cluster members and certificates respectively. Simpler setups can use static configuration or DNS (Envoy docs: xDS dynamic configuration). A control plane such as Istio's istiod, or Envoy Gateway, sits behind these APIs and pushes configuration to a fleet of proxies.
What can Envoy see and secure?
Envoy provides statistics for all subsystems through an admin port, exportable to statsd-compatible sinks, and supports distributed tracing through third-party providers. For security, its site lists TLS termination and origination, mutual TLS, SNI, and filters for JWT authentication, RBAC and external authorisation. Extensions can be written in Wasm, Lua, Go or Rust dynamic modules, or handled through external processing, without forking Envoy (Envoy's own site).
What is the difference between edge Envoy and sidecar Envoy?
Envoy is deployed in two common roles. An edge (front) proxy is a standalone instance that is the single entry point to a cluster. A sidecar proxy runs beside each service instance and handles that instance's inbound and outbound traffic. Meshes use sidecars for service-to-service traffic and a gateway at the edge for north-south traffic.
| Role | Where it runs | Typical use | Trade-off |
|---|---|---|---|
| Edge or gateway | Cluster boundary | TLS termination, routing by host and path, authentication for external callers | One shared entry point to size, secure and monitor |
| Sidecar | Beside every service instance | Mutual TLS, retries, telemetry between services | More proxies to run and upgrade, with resource overhead per pod |
| Per-node or per-namespace proxy (Istio ambient) | Node and optional namespace level | Sidecar-less mesh with L4 on each node and optional L7 | Newer model, confirm feature status on the Istio page |
Istio's architecture page says Istio uses an extended version of Envoy that mediates all inbound and outbound traffic for services in the mesh, with istiod providing service discovery, configuration and certificate management (Istio architecture). Istio's ambient mode removes sidecars from workload pods: a per-node ztunnel handles L4 functions such as mutual TLS, and optional waypoint proxies, which are Envoy deployments, handle L7 features (Istio ambient overview). Confirm the current release stage of ambient mode on the official page before you commit to it.
Which tools are built on Envoy?
| Tool | What it is, according to its site | Fits when | Status to confirm |
|---|---|---|---|
| Istio | Service mesh using an extended Envoy as its data plane | You need mesh-wide mutual TLS, traffic policy and telemetry | Sidecar and ambient modes, confirm on istio.io |
| Envoy Gateway | Manages application and API traffic, makes Envoy easier to adopt, and implements and extends the Kubernetes Gateway API | You want an Envoy-based ingress or API gateway on Kubernetes | Active in 2026 per its site, confirm the version you deploy |
| Emissary-ingress | Open source Kubernetes-native API gateway built on Envoy Proxy | You already run it as an ingress and need to keep it | Confirm project and CNCF status on emissary-ingress.dev |
| Envoy standalone | The proxy itself, configured with static files or your own xDS control plane | You need a custom proxy or gateway and have the platform skills | You own the control plane and upgrades |
Envoy Gateway's own page describes it as a tool for managing application and API traffic that can be deployed as a standalone or Kubernetes-based API gateway (Envoy Gateway). The Envoy site also lists Envoy AI Gateway for AI traffic, including token-aware rate limiting and provider fallback; check the project's own documentation for its current status before planning around it. For a lighter mesh, see Linkerd benefits and best practices, and for the Kubernetes layer underneath, Kubernetes architecture components.
When should you use Envoy Proxy, and when not?
| Situation | Recommendation | Reason |
|---|---|---|
| Many services in several languages need the same retries, mTLS and telemetry | Use Envoy, usually through a mesh or gateway | One out-of-process proxy avoids a per-language library |
| You need Gateway API ingress on Kubernetes | Consider Envoy Gateway | It implements the Kubernetes Gateway API on Envoy |
| A handful of services behind one ingress | A simpler load balancer may be enough | Mesh and xDS add operating cost you may not need |
| No platform team to run a control plane | Prefer a managed mesh or gateway, or wait | Envoy configuration is powerful but not trivial |
| Regulated traffic needing audit of who changed routes | Use Envoy with config in version control and change review | Dynamic configuration changes live traffic, so it needs governance |
What are the governance, security and cost considerations?
A proxy fleet is part of your attack surface and your change process. Treat route, filter and certificate configuration as code: review it, version it and roll it out in stages. Mutual TLS and authorisation filters reduce lateral movement between services, but they only help if certificate issuance and rotation are owned by a named team. Sidecars add CPU and memory per pod, so measure the overhead on your own workloads before a cluster-wide rollout rather than relying on generic figures.
When AI agents call internal services through the same proxy layer, the proxy sees the traffic but not the business reason for it. ElixirData Context OS keeps governed context, versioned policy and decision traces in one decision layer above AI agents, and AgentGRC keeps autonomous agent actions inside policy, with the audit trail risk teams need.
How should you adopt Envoy Proxy?
| Step | Action | Check before moving on |
|---|---|---|
| 1 | Run Envoy as a sidecar or gateway for two or three services in a test namespace | Traffic, latency and error metrics visible |
| 2 | Add retries, timeouts and circuit breaking with conservative limits | No retry storms under failure tests |
| 3 | Turn on mutual TLS between those services | Certificate rotation tested |
| 4 | Shift traffic to new versions in small steps, for example 1 percent, then 10, 50 and 100 | Rollback proven at each step |
| 5 | Extend cluster-wide and add alerts on proxy and service metrics | On-call runbooks updated |
Keep application code free of network logic, keep the proxy configuration in version control, and deploy new versions side by side before they take traffic. Metrics from Prometheus and Grafana, or your existing observability stack, give the evidence at each step.
To map one traffic or access decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.
Frequently asked questions
What is Envoy Proxy used for?
Envoy is used as an edge or service proxy that handles routing, load balancing, TLS, retries and telemetry for applications, so that this logic does not need to be built into each service. It is also the data plane in Istio.
Is Envoy a service mesh?
No. Envoy is a proxy. A service mesh adds a control plane that configures many Envoy proxies. Istio, for example, uses an extended version of Envoy with istiod as its control plane.
What is xDS in Envoy?
xDS is the set of dynamic configuration APIs Envoy can consume, including listener, route, cluster, endpoint and secret discovery services. A control plane uses them to update proxies without restarting them.
What is the difference between Envoy and Envoy Gateway?
Envoy is the proxy. Envoy Gateway, according to its site, manages application and API traffic, makes Envoy easier to adopt and implements and extends the Kubernetes Gateway API.
Is Envoy a CNCF project?
Yes. The CNCF lists Envoy as a graduated project, accepted at the Incubating level on 13 September 2017 and graduated on 28 November 2018.
Do I need sidecars to use Envoy?
Not always. You can run Envoy as an edge gateway only. Istio also offers an ambient mode with a per-node proxy and optional Envoy waypoint proxies, so confirm its current status on the Istio site.
Ready to put AI agents to work? See an ElixirClaw demo: the agentic OS to build, orchestrate and govern AI agents across your enterprise systems.
Explore the XenonStack platform family: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AgentGRC · AgentSRE · AgentAnalyst · AgentInstruct · AgentRAI · AgentScrum · AgentSearch · AgentForce · AI Aviator · SAIF Aviator · CohortIQ · XenonStack AI · XenonStack UAE