Transparent Proxy on Universal

Using the transparent proxy in Universal mode makes setup easier and enables features that wouldn’t be possible otherwise. Key benefits include:

• Simplified Dataplane resources: You can skip the networking.outbound section, so you don’t have to list each service your application connects to manually.

• Simplified service connectivity: Take advantage of Kuma DNS to use .mesh domain names, like https://service-1.mesh, for easy service connections without needing localhost and ports in the Dataplane resource.

• Flexible service naming: With MeshServices and HostnameGenerators, you can:

  • Keep your existing DNS names when moving to the service mesh.
  • Give a service multiple DNS names for easier access.
  • Set up custom routes, like targeting specific StatefulSet Pods or service versions.
  • Expose a service on multiple ports for different uses.

• Simpler security, tracing, and observability: Transparent proxy makes managing these features easier, with no extra setup required.

Installation

Here are the steps to set up a transparent proxy for your service. Once set up, your service will run with a transparent proxy, giving you access to Kuma’s features like traffic control, observability, and security.

Before you start, check the prerequisites below and adjust the settings for your environment, including IP addresses, custom ports, and DNS configurations.

Prerequisites

1. Control plane deployment and network accessibility: Ensure Kuma’s control plane is already deployed and accessible from the service via an IP address or DNS name. Adapt the steps below to reflect the correct address for your setup.

   In a basic setup, the control plane should not be deployed in the same environment as the transparent proxy. This is because the transparent proxy redirects all traffic to kuma-dp (unless excluded), including traffic meant for the control plane.

   For deployment instructions, see Deploy a single-zone control plane.

2. Service accessibility: Ensure the environment with the transparent proxy and service is reachable by other services in the mesh via IP or DNS. Adjust the setup steps to match the appropriate address where the service is accessible.

3. Data plane proxy token: Place the data plane proxy token at /kuma/example-server-token (or adjust the steps if stored elsewhere).

   For instructions on generating this token, refer to the Data plane proxy token section in the Authentication with the data plane proxy documentation.

4. Binary availability: Ensure kuma-dp, envoy, and coredns binaries are accessible in /usr/local/bin/, and kumactl is in the system’s PATH. Adjust any subsequent commands if the binaries are located elsewhere.

   To get the required binaries, see Install Kuma.

Step 1: Create a dedicated user for kuma-dp

For proper functionality, the service must run under a different user than the one designated for running kuma-dp. If both the service and kuma-dp are run by the same user, the transparent proxy will not work correctly, causing service traffic (inbound and outbound) to fail. To create a dedicated user for kuma-dp, use the following command:

useradd -u kuma-dp -U kuma-dp

Step 2: Prepare the Dataplane resource

In transparent proxy mode, configure your Dataplane resource without the networking.outbound section. Instead, use networking.transparentProxying for handling traffic redirection. Here’s an example:

If your /kuma/example-server-dataplane.yaml looks like this:

type: Dataplane
mesh: default
name: {{ name }}
networking:
  address: {{ address }}
  inbound:
    - port: {{ port }}
      tags:
        kuma.io/service: example-server
  outbound:
    - port: 6379
      tags:
        kuma.io/service: redis

After updating it for transparent proxy, it should look like this:

type: Dataplane
mesh: default
name: {{ name }}
networking:
  address: {{ address }}
  inbound:
    - port: {{ port }}
      tags:
        kuma.io/service: example-server
  transparentProxying:
    redirectPortInbound: 15006
    redirectPortOutbound: 15001

Redirect ports

In this example, 15006 and 15001 are the default ports for inbound and outbound traffic redirection. You can change these ports in the transparent proxy configuration during installation (redirect.inbound.port and redirect.outbound.port). For more details, see the Transparent Proxy Configuration Reference.

Using variables in your configuration

The placeholders {{ name }}, {{ address }}, and {{ port }} are Mustache templates, which will be dynamically filled using values passed via --dataplane-var CLI flags in a later step.

In practice, if these values are static, you can simply hard-code them in your configuration. This feature is designed to allow more flexible and reusable resources, making it easier to dynamically adjust values when starting kuma-dp.

More resources

For additional information on setting up the Dataplane resource, refer to the Data Plane on Universal documentation.

Step 3: Start the service

The service can be started in various ways depending on your environment and requirements. To keep this instruction simple, we’ll use Python 3’s built-in HTTP server. This server will listen on port 8080, with both STDOUT and STDERR redirected to a service.log file in the current directory, running in the background.

To start the service:

python3 -m http.server 8080 > service.log 2>&1 &

Step 4: Start the kuma-dp

If you are using systemd to manage processes, you can refer to the Systemd documentation for an example of a systemd resource that can be customized to run kuma-dp in your environment.

Alternatively, if you’re starting kuma-dp using a script or another form of automation, you can use runuser to run the process as the kuma-dp user. Below is an example of how to start kuma-dp with the necessary parameters:

# Set the Control Plane address (IP or DNS) accessible by this environment
export CP_ADDRESS="..."  

# Set the Data Plane address (IP or DNS) for the current environment, accessible by
# other services
export DP_ADDRESS="..."

runuser -u kuma-dp -- \
  /usr/local/bin/kuma-dp run \
    --cp-address="https://$CP_ADDRESS:5678" \
    --dataplane-token-file="/kuma/example-server-token" \
    --dataplane-file="/kuma/example-server-dataplane.yaml" \
    --dataplane-var="name=example-server-dataplane" \
    --dataplane-var="address=$DP_ADDRESS" \
    --dataplane-var="port=8080" \
    --binary-path="/usr/local/bin/envoy" \
    --dns-coredns-path="/usr/local/bin/coredns"

This command runs kuma-dp under the kuma-dp user, connects it to the control plane, and configures it with the necessary settings for the example server. Adjust the paths and values as needed for your specific setup.

Step 5: Install the transparent proxy

Before proceeding with this step, please keep the following in mind:

kumactl install transparent-proxy --redirect-dns --exclude-inbound-ports "22"

To install the transparent proxy in your environment, run the following command:

kumactl install transparent-proxy --redirect-dns

This will result in the following:

• The transparent proxy will be set up to use the kuma-dp user as the designated user for running the kuma-dp process.

  By default, kumactl searches for a user with UID 5678 or the username kuma-dp. If you prefer to use a custom username or UID, you can specify it using the --kuma-dp-user flag. If the specified user cannot be found, the installation process will fail with an error.

• All incoming traffic will be redirected to kuma-dp on port 15006.

  Port 15006 is the default for redirecting incoming traffic. If you used a different port in your Dataplane resource definition, ensure it aligns with the transparent proxy configuration. To update it, modify the redirect.inbound.port setting during installation, for example by using the --redirect-inbound-port flag.

• All outgoing traffic will be redirected to kuma-dp on default port 15001.

  Port 15001 is the default used for redirecting outgoing traffic. If you used a different port in your Dataplane resource definition, ensure it aligns with the transparent proxy configuration. To update it, modify the redirect.outbound.port setting during installation, for example by using the --redirect-outbound-port flag.

• DNS traffic directed to the DNS servers configured in /etc/resolv.conf will be redirected to the coredns managed by kuma-dp on port 15053.

  Port 15053 is the default used for redirecting DNS traffic. If you wish to modify this port, it is crucial to ensure consistency between the transparent proxy and kuma-dp. In this case, you must update the kuma-dp run command to include the --dns-coredns-port flag with the desired custom port. To adjust it for the transparent proxy, modify the redirect.dns.port setting during installation, for example by using the --redirect-dns-port flag.

  /etc/resolv.conf is the file used by default to parse DNS servers for redirecting DNS traffic through the transparent proxy. This file is relevant only if you opt to redirect DNS traffic specifically to the DNS servers listed within it (for example, by using the --redirect-dns flag during installation). If you’re redirecting all DNS traffic (for example, by using the --redirect-all-dns-traffic flag), this path becomes irrelevant.

  We strongly recommend against changing this path. However, if your environment uses a different file to specify DNS servers (which must follow the same format as /etc/resolv.conf), you can adjust it by modifying the redirect.dns.resolvConfigPath setting during installation. This can be done, for example, using the KUMA_TRANSPARENT_PROXY_REDIRECT_DNS_RESOLV_CONFIG_PATH environment variable.

Upgrading

The core iptables rules applied by Kuma’s transparent proxy rarely change, but occasionally new features may require updates. To upgrade the transparent proxy on Universal environments, follow these steps:

Step 1: Cleanup existing iptables rules (conditional)

To manually remove existing iptables rules, you can either restart the host (if the rules were not persisted using system start-up scripts or firewalld), or run the following commands:

iptables --table nat --flush         # Flush all rules in the nat table (IPv4)
ip6tables --table nat --flush        # Flush all rules in the nat table (IPv6)
iptables --table nat --delete-chain  # Delete all custom chains in the nat table (IPv4)
ip6tables --table nat --delete-chain # Delete all custom chains in the nat table (IPv6)

# The raw table contains rules for DNS traffic redirection
iptables --table raw --flush         # Flush all rules in the raw table (IPv4)
ip6tables --table raw --flush        # Flush all rules in the raw table (IPv6)

# The mangle table contains rules to drop invalid packets
iptables --table mangle --flush      # Flush all rules in the mangle table (IPv4)
ip6tables --table mangle --flush     # Flush all rules in the mangle table (IPv6)

Step 2: Install new transparent proxy

After clearing the iptables rules (if necessary), reinstall the transparent proxy by running:

kumactl install transparent-proxy [...]

This command will install the new version of the transparent proxy with the specified configuration. Adjust the flags as needed to suit your environment.

Configuration

The default configuration works well for most scenarios, but there are cases where adjustments are needed.

Kuma uses a unified configuration structure for transparent proxy across all components. For a detailed breakdown of this structure, including examples, expected formats, and variations between configuration methods, refer to the Transparent Proxy Configuration reference.

In Universal mode, Kuma there are three methods to adjust the configuration. Each can be used on its own or combined with others if needed.

YAML / JSON

You can provide the configuration in either YAML or JSON format by using the --config or --config-file flags.

Below are examples of using these flags in different ways:

1. Providing configuration via the --config-file flag

   Assume you have a config.yaml file with the following content:

   kumaDPUser: dataplane
   verbose: true
   

   You can install the transparent proxy using:

   kumactl install transparent-proxy --config-file config.yaml
   

2. Passing configuration directly via the --config flag

   To pass the configuration content directly:

   kumactl install transparent-proxy --config "kumaDPUser: dataplane\nverbose: true"
   

   Alternatively:

   kumactl install transparent-proxy --config "{ kumaDPUser: dataplane, verbose: true }"
   

   Both formats are valid YAML inputs.

3. Passing configuration via STDIN

   If you need to pass the configuration via STDIN, set --config-file to - as shown below:

   echo "
   kumaDPUser: dataplane
   verbose: true
   " | kumactl install transparent-proxy --config-file -
   

Environment Variables

You can customize configuration settings by using environment variables. For example:

KUMA_TRANSPARENT_PROXY_IP_FAMILY_MODE="ipv4" kumactl install transparent-proxy

CLI Flags

Most configuration values can also be specified directly through CLI flags. For example:

kumactl install transparent-proxy --kuma-dp-user dataplane --verbose

Order of Precedence

1. Default Values
2. Values from --config / --config-file flags
3. Environment Variables
4. CLI Flags

To understand how the order of precedence works, consider this scenario:

1. You have a config.yaml file with the following content:

   redirect:
     dns:
       port: 10001
   

2. You install the transparent proxy using this command:

   KUMA_TRANSPARENT_PROXY_REDIRECT_DNS_PORT="10002" \
     kumactl install transparent-proxy \
       --config-file config.yaml \
       --redirect-dns-port 10003
   

3. In this situation, the possible values for redirect.dns.port are:

   • 15053 (Default Value)
   • 10001 (From Config File)
   • 10002 (From Environment Variable)
   • 10003 (From CLI Flag)

4. Since CLI flags have the highest precedence, the final value for redirect.dns.port will be 10003.

firewalld support

The changes made by running kumactl install transparent-proxy will not persist after a reboot. To ensure persistence, you can either add this command to your system’s start-up scripts or leverage firewalld for managing iptables.

If you prefer using firewalld, you can include the --store-firewalld flag when installing the transparent proxy. This will store the iptables rules in /etc/firewalld/direct.xml, ensuring they persist across system reboots. Here’s an example:

kumactl install transparent-proxy --redirect-dns --store-firewalld