Technical Overview

A transparent proxy captures and redirects network traffic without needing changes to the application. Kuma does this using iptables and has experimental support for eBPF.

The diagram below shows a high-level view of how transparent proxying works:

A life without transparent proxying

If you choose to not use transparent proxying, or you are running on a platform where transparent proxying is not available, there are some additional considerations.

You will need to specify inbound and outbound ports to capture traffic on
.mesh addresses are unavailable
You may need to update your application code to use the new capture ports

Without manipulating iptables to redirect traffic you will need to explicitly tell kuma-dp where to listen to capture it. As noted, this can require changes to your application code as seen below:

Here we specify that we will listen on the address 10.119.249.39:15000 (line 7). This in turn creates an envoy listener for the port. When consuming a service over this 15000 it will cause traffic to redirect to 127.0.0.1:5000 (line 8) where our app is running.

type: Dataplane
mesh: default
name: demo-app
networking:
  address: 10.119.249.39
  inbound:
  - port: 15000
    servicePort: 5000
    serviceAddress: 127.0.0.1
    tags:
      kuma.io/service: app
      kuma.io/protocol: http

How it works

Inbound TCP traffic

The inbound port, 15006, is the default for capturing requests to the system. This rule allows us to capture and redirect ALL TCP traffic to port 15006.

iptables \
  --table nat \
  --append KUMA_MESH_INBOUND_REDIRECT \
  --protocol tcp \
  --jump REDIRECT --to-ports 15006

An envoy listener is also created for this port which we can see in the admin interface (:9901/config_dump). In the below example you can see the listener created on all interfaces (line 8) and port 15006 (line 9).

     "name": "inbound:passthrough:ipv4",
     "active_state": {
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "inbound:passthrough:ipv4",
       "address": {
        "socket_address": {
         "address": "0.0.0.0",
         "port_value": 15006
        }
       },
      ...
       "use_original_dst": true,
       "traffic_direction": "INBOUND",

Notice the setting use_original_dst (line 13). This listener will send traffic to a special type of cluster, ORIGINAL_DST. This is important since we are redirecting traffic here based on the iptables rules, which means when this service was requested it was not likely it was requested over this port, 15006, but rather whatever the target application is listening on (i.e. demo-app port 5000).

     "name": "inbound:10.244.0.6:5000",
     "active_state": {
      "version_info": "9dac7d53-3560-4ad4-ba42-c7e563db958e",
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "inbound:10.244.0.6:5000",
       "address": {
        "socket_address": {
         "address": "10.244.0.6",
         "port_value": 5000
        }
       }
      }
     }

Using the Kuma counter demo app as an example, when the client needs to talk to the node app, it does not do so over 15006, but rather the actual application port, 5000. This is the “transparent” part of the proxying as it is not expected that apps will need to be redesigned or changed in any way to utilize mesh.

So, when the request comes into the system, the iptables rule grabs the traffic and sends it to envoy port 15006. Once here, we check where the request was originally intended to go, in this case 5000 and forward it.

A further review of the envoy config will show our Node app listener where the IP address, 10.244.0.6, is that of the demo-app pod. Now that envoy is in control of the traffic we can now (optionally) apply filters/Mesh policies.

     "name": "inbound:10.244.0.6:5000",
     "active_state": {
      "version_info": "9dac7d53-3560-4ad4-ba42-c7e563db958e",
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "inbound:10.244.0.6:5000",
       "address": {
        "socket_address": {
         "address": "10.244.0.6",
         "port_value": 5000
        }
       },
          "filter_chains": [
            {
              "filters": [
                {
                  "name": "envoy.filters.network.http_connection_manager",
                  "typed_config": {
                    "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
                    "stat_prefix": "localhost_5000",
                    "route_config": {
                    "http_filters": [
                      {
                        "name": "envoy.filters.http.fault",
                        "typed_config": {
                          "@type": "type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault",
                          "delay": {
                            "fixed_delay": "5s",
                            "percentage": {
                              "numerator": 50,
                              "denominator": "TEN_THOUSAND"
...

Outbound TCP traffic

The outbound port, 15001, is the default for capturing outbound traffic from the system. That is, traffic leaving the mesh. This rule allow us to capture and redirect all TCP traffic to 15001.

iptables --table nat --append KUMA_MESH_OUTBOUND_REDIRECT --protocol tcp --jump REDIRECT --to-ports 15001

     "name": "outbound:passthrough:ipv4",
     "active_state": {
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "outbound:passthrough:ipv4",
       "address": {
        "socket_address": {
         "address": "0.0.0.0",
         "port_value": 15001
        }
       },
      ...
       "use_original_dst": true,
       "traffic_direction": "OUTBOUND"