Last updated November 21, 2022

Use Mutual TLS to create a secure and mutually authenticated channel between an external resource and an Apache Kafka on Heroku cluster running in a Private Space or a Shield Private Space. External resources can include any mTLS-enabled application or system running in private data centers or public clouds. To use this feature, you must allowlist the external IP used to connect to your cluster.

Overview

All Kafka clusters running on Heroku have mTLS enabled by default.

When provisioning the Kafka add-on, Heroku automatically creates mTLS certificates for its use. Connections to Kafka clusters require authenticating with these certificates. They’re available in the KAFKA_TRUSTED_CERT, KAFKA_CLIENT_CERT and KAFKA_CLIENT_CERT_KEY config vars for your application.

Private and Shield Kafka clusters exist in an isolated data resource network associated with their Private or Shield space. By default, only resources within the Private or Shield space can connect to the cluster. Use the mTLS feature to configure IP addresses for your external resources and allow connections from those sources.

Heroku Prerequisites

Mutual TLS setup requires the following Heroku resources:

• A Private Space or Shield Private Space
• A Heroku app running in the Space with an attached Private or Shield Apache Kafka on Heroku add-on

Step 1: Install the Mutual TLS CLI Plugin

$ heroku plugins:install mtls

Find the different commands available and further details about this plugin at Heroku mTLS Plugin CLI commands.

Step 2: Allowlist external IPs

You can allowlist up to a hard limit of 60 IP blocks per Kafka cluster. Allowlist an IP block to access your Kafka cluster using the following Heroku CLI command:

$ heroku data:mtls:ip-rules:create KAFKA_ADDON_NAME --app APP_NAME \
  --cidr CIDR_BLOCK \
  --description DESCRIPTION

Provide the following parameters:

• KAFKA_ADDON_NAME: The name of your Kafka cluster (for example, kafka-sushi-12345)
• APP_NAME: The name of your application
• CIDR_BLOCK: The CIDR block to add to the allowlist. Define individual IP addresses as /32 blocks.
• DESCRIPTION: The description of the specified CIDR block (for example, "Office IP")

$ heroku data:mtls:ip-rules:create kafka-sushi-12345 --app example-app \
  --cidr "1.2.3.4/32" \
  --description "My Office IP"

Creating IP Rule for kafka-sushi-12345... done
cidr:        1.2.3.4/32
created_at:  2021-10-27T15:44:22.901+00:00
description: My Office IP
id:          38c466b6-dcfb-4869-b5ac-40420b786fb4
status:      Authorizing
updated_at:  2021-10-27T15:44:22.901+00:00

The new IP rule has an id that you can use to check the rule details or to delete it.

It usually takes 10-15 minutes to allowlist an external IP. Track the progress and check the status of the configured IP ranges with the data:mtls:ip-rules:get command. Provide the IP rule id with the --id flag.

$ heroku data:mtls:ip-rules:get KAFKA_ADDON_NAME --id IP_RULE_ID --app APP_NAME

For example:

$ heroku data:mtls:ip-rules:get kafka-sushi-12345 --id 38c466b6-dcfb-4869-b5ac-40420b786fb4 --app example-app

cidr:        1.2.3.4/32
created_at:  2021-10-27T15:44:22.901+00:00
description: My Office IP
id:          38c466b6-dcfb-4869-b5ac-40420b786fb4
status:      Authorized
updated_at:  2021-10-27T15:44:22.901+00:00

mTLS CLI Plugin

See the full documentation of the mTLS CLI plugin.