Skip to content

Latest commit

 

History

History
145 lines (104 loc) · 7.36 KB

README-amq-streams-ocp-configure.adoc

File metadata and controls

145 lines (104 loc) · 7.36 KB

AMQ Streams (KAFKA) on OCP - Configuration Workshop Activities

Table of Contents

Scenario 1: Configure and manage topics

Adding Topic Options

  • Option 1: Directly add KafkaTopic CRD, see: kafka-topic.yaml via Template configuring Topic ConfigMap and directly on zookeeper 2 more option

  • Option 2: Create topic through Kafka cli

    oc exec my-cluster-zookeeper-0 -i -t -- bin/kafka-topics.sh --zookeeper localhost:21810 --create --topic my-topic-2-in-kafka --partitions 3 --replication-factor 1 --config cleanup.policy=compact

  • Option 3: Create topic through Config Map, see topic-template.yaml

    • Add template for strimzi-topic into the project $ cd ./ocp-amq-streams-workshop $ oc create -f install_and_examples_0/examples/templates/topic-operator/topic-template.yaml -n amq-streams-ephemeral template "strimzi-topic" created $ oc get templates NAME DESCRIPTION PARAMETERS OBJECTS strimzi-topic This template creates a "Topic ConfigMap". Used in conjunction with the Strim…​ 5 (all set) 1

      	$ oc process --parameters -n amq-streams-ephemeral strimzi-topic
	NAME                DESCRIPTION    												GENERATOR	VALUE
	CLUSTER_NAME        Specifies the name of the Kafka cluster in which the topic should be created.
	TOPIC_NAME          Specifies the name of the topic in the Kafka cluster. This should be a valid Kubernetes resource name.
	TOPIC_PARTITIONS    The number of partitions in the created topic.										1
	TOPIC_REPLICAS      The number of replicas in the created topic.										1
	TOPIC_CONFIG        The topic config as a JSON map, for example: { "retention.ms":"345600000" }
			    See https://kafka.apache.org/10/documentation/#topicconfigs for config key names and value syntax. 				{}
*** Add topic via template
      $ oc process strimzi-topic -p CLUSTER_NAME=my-cluster -p TOPIC_NAME=my-topic-1-template -p TOPIC_REPLICAS=3  -p TOPIC_CONFIG='{ "retention.ms":"345600000", "segment.bytes" : "1073741824" }'
{
    "kind": "List",
    "apiVersion": "v1",
    "metadata": {},
    "items": [
	{
	    "apiVersion": "kafka.strimzi.io/v1alpha1",
	    "kind": "KafkaTopic",
	    "metadata": {
	        "labels": {
	            "strimzi.io/cluster": "my-cluster"
	        },
	        "name": "my-topic-1-template"
	    },
	    "spec": {
	        "config": {
	            "retention.ms": "345600000",
	            "segment.bytes": "1073741824"
	        },
	        "partitions": 1,
	        "replicas": 3
	    }
	}
    ]
}
      oc process strimzi-topic -p CLUSTER_NAME=my-cluster -p TOPIC_NAME=my-topic-1-template -p TOPIC_REPLICAS=3  -p TOPIC_CONFIG='{ "retention.ms":"345600000", "segment.bytes" : "1073741824" }' | oc create -f - -n amq-streams-ephemeral

  • View Topics

    $ oc exec my-cluster-zookeeper-0 -i -t -- bin/kafka-topics.sh --zookeeper localhost:21810 --describe
Defaulting container name to zookeeper.
Use 'oc describe pod/my-cluster-zookeeper-0 -n amq-streams-ephemeral' to see all of the containers in this pod.
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
    Topic:my-topic	PartitionCount:3	ReplicationFactor:3	Configs:segment.bytes=1073741824,retention.ms=7200000
	Topic: my-topic	Partition: 0	Leader: 2	Replicas: 2,1,0	Isr: 2,1,0
	Topic: my-topic	Partition: 1	Leader: 0	Replicas: 0,2,1	Isr: 0,2,1
	Topic: my-topic	Partition: 2	Leader: 1	Replicas: 1,0,2	Isr: 1,0,2
Topic:my-topic-1-template	PartitionCount:1	ReplicationFactor:3	Configs:segment.bytes=1073741824,retention.ms=345600000
	Topic: my-topic-1-template	Partition: 0	Leader: 2	Replicas: 2,0,1	Isr: 2,0,1
Topic:my-topic-2-in-kafka	PartitionCount:3	ReplicationFactor:1	Configs:cleanup.policy=compact
	Topic: my-topic-2-in-kafka	Partition: 0	Leader: 0	Replicas: 0	Isr: 0
	Topic: my-topic-2-in-kafka	Partition: 1	Leader: 1	Replicas: 1	Isr: 1
	Topic: my-topic-2-in-kafka	Partition: 2	Leader: 2	Replicas: 2	Isr: 2

oc exec my-cluster-zookeeper-0 -i -t — bin/kafka-consumer-groups.sh --zookeeper localhost:21810 --describe

--bootstrap-server localhost:9092 --list

Scenarion 2: Configure Cluster Operator in a central fashion to Watch multiple Namespaces for Topic Creation

Prerequisites

Edit the installation files according to the OpenShift project or Kubernetes namespace the Cluster Operator is going to be installed in.
sed -i 's/namespace: .*/namespace: my-namespace/' install/cluster-operator/*RoleBinding*.yaml

Procedure

Edit the file install/cluster-operator/050-Deployment-strimzi-cluster-operator.yaml and in the environment variable STRIMZI_NAMESPACE list all the OpenShift projects or Kubernetes namespaces where Cluster Operator should watch for resources. For example:
apiVersion: extensions/v1beta1
kind: Deployment
spec:
  template:
    spec:
      serviceAccountName: strimzi-cluster-operator
      containers:
      - name: strimzi-cluster-operator
        image: strimzi/cluster-operator:latest
        imagePullPolicy: IfNotPresent
        env:
        - name: STRIMZI_NAMESPACE
          value: myproject,myproject2,myproject3
For all namespaces or projects which should be watched by the Cluster Operator, install the RoleBindings. Replace the my-namespace or my-project with the OpenShift project or Kubernetes namespace used in the previous step.
On OpenShift this can be done using oc apply:
oc apply -f install/cluster-operator/020-RoleBinding-strimzi-cluster-operator.yaml -n my-project
oc apply -f install/cluster-operator/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml -n my-project
oc apply -f install/cluster-operator/032-RoleBinding-strimzi-cluster-operator-topic-operator-delegation.yaml -n my-project
Deploy the Cluster Operator
On OpenShift this can be done using oc apply:
oc apply -f install/cluster-operator -n my-project