HDP Kafka Vs CDH Kafka

      No Comments on HDP Kafka Vs CDH Kafka

Feature Comparision

Kafka FeatureHDPCDH
Packaging RPM based packgingParcel packaging
VersioningApache versions CDK versions
Sample Package Namekafka_3_1_0_0_78-2.0.0.3.1.0.0-78.noarchKAFKA-2.1.0-1.2.1.0.p0.115-el7.parcel
Components 1. Kafka Brokers1. Kafka Broker
2. MirrorMaker
3. Gateway

Configuration Comparision

Configuration PropertyHDPCDH
auto.create.topics.enabletruetrue
auto.leader.rebalance.enabletruetrue
broker.id20
controlled.shutdown.enabletruetrue
controlled.shutdown.max.retries33
default.replication.factor11
delegation.token.enablefalse
delegation.token.expiry.time.ms86400000
delegation.token.master.key.generator/var/run/cloudera-scm-agent/process/31-kafka-KAFKA_BROKER/altscript.sh
delegation.token.max.lifetime.ms604800000
delete.topic.enabletruetrue
kafka.http.metrics.host0.0.0.0
kafka.http.metrics.port24042
kafka.log4j.dir/var/log/kafka
kerberos.auth.enablefalse
leader.imbalance.check.interval.seconds300300
leader.imbalance.per.broker.percentage1010
log.cleaner.dedupe.buffer.size134217728
log.cleaner.delete.retention.ms604800000
log.cleaner.enabletrue
log.cleaner.min.cleanable.ratio0.5
log.cleaner.threads1
log.dirs/kafka-logs/var/local/kafka/data
log.retention.bytes-1
log.retention.check.interval.ms600000300000
log.retention.ms604800000
log.segment.bytes10737418241073741824
message.max.bytes10000001000000
min.insync.replicas11
num.io.threads88
num.network.threads38
num.partitions11
num.recovery.threads.per.data.dir11
num.replica.fetchers14
offsets.topic.num.partitions50
offsets.topic.replication.factor33
port66679092
replica.fetch.max.bytes1048576
replica.lag.time.max.ms1000010000
sentry.kafka.caching.enabletrue
sentry.kafka.caching.ttl.ms30000
sentry.kafka.caching.update.failures.count3
unclean.leader.election.enablefalse
zookeeper.session.timeout.ms6000
zookeeper.connectc1198-node2.squadron-labs.com:2181,c1198-node3.squadron-labs.com:2181,c1198-node4.squadron-labs.com:2181
kafka.metrics.reportersnl.techop.kafka.KafkaHttpMetricsReporter
security.inter.broker.protocolPLAINTEXPLAINTEXT
listenersPLAINTEXT://c2198-node2.squadron-labs.com:6667PLAINTEXT://c1198-node2.squadron-labs.com:9092
broker.id.generation.enablefalse
sasl.kerberos.service.namekafka
sasl.enabled.mechanismsGSSAPIGSSAPI
compression.typeproducerNot Present
controlled.shutdown.retry.backoff.ms5000Not Present
controller.message.queue.size10Not Present
controller.socket.timeout.ms30000Not Present
external.kafka.metrics.exclude.prefixkafka.network.RequestMetrics,kafka.server.DelayedOperationPurgatory,kafka.server.BrokerTopicMetrics.BytesRejectedPerSecNot Present
external.kafka.metrics.include.prefixkafka.network.RequestMetrics.ResponseQueueTimeMs.request.OffsetCommit.98percentile,kafka.network.RequestMetrics.ResponseQueueTimeMs.request.Offsets.95percentile,kafka.network.RequestMetrics.ResponseSendTimeMs.request.Fetch.95percentile,kafka.network.RequestMetrics.RequestsPerSec.requestNot Present
fetch.purgatory.purge.interval.requests1000Not Present
kafka.ganglia.metrics.groupkafkaNot Present
kafka.ganglia.metrics.hostlocalhostNot Present
kafka.ganglia.metrics.port8671Not Present
kafka.ganglia.metrics.reporter.enabledtrueNot Present
kafka.metrics.reportersorg.apache.hadoop.metrics2.sink.kafka.KafkaTimelineMetricsReporterNot Present
kafka.timeline.metrics.host_in_memory_aggregationfalseNot Present
kafka.timeline.metrics.host_in_memory_aggregation_port61888Not Present
kafka.timeline.metrics.host_in_memory_aggregation_protocolhttpNot Present
kafka.timeline.metrics.hostslocalhostNot Present
kafka.timeline.metrics.maxRowCacheSize10000
kafka.timeline.metrics.port6188
kafka.timeline.metrics.protocolhttp
kafka.timeline.metrics.reporter.enabledtrue
kafka.timeline.metrics.reporter.sendInterval5900
kafka.timeline.metrics.truststore.passwordbigdata
kafka.timeline.metrics.truststore.path/etc/security/clientKeys/all.jks
kafka.timeline.metrics.truststore.typejks
log.cleanup.interval.mins10Not Present
log.index.interval.bytes4096Not Present
log.index.size.max.bytes10485760
log.retention.bytes-1
log.retention.check.interval.ms
log.retention.hours168
log.roll.hours168
offset.metadata.max.bytes4096
offsets.commit.required.acks-1
offsets.commit.timeout.ms5000
offsets.load.buffer.size5242880
offsets.retention.check.interval.ms600000
offsets.retention.minutes86400000
offsets.topic.compression.codec0
offsets.topic.num.partitions50
offsets.topic.segment.bytes104857600
producer.metrics.enablefalse
producer.purgatory.purge.interval.requests10000
queued.max.requests500
replica.fetch.max.bytes1048576
replica.fetch.min.bytes1
replica.fetch.wait.max.ms500
replica.high.watermark.checkpoint.interval.ms5000
replica.lag.max.messages4000
replica.socket.receive.buffer.bytes65536102400
replica.socket.timeout.ms30000
sasl.enabled.mechanisms
socket.request.max.bytes104857600
socket.send.buffer.bytes102400
ssl.client.authnone
ssl.key.passwordnot set not present
ssl.keystore.locationnot set not present
ssl.keystore.passwordnot set not present
ssl.truststore.locationnot setnot present
ssl.truststore.passwordnot setnot present
zookeeper.connection.timeout.ms25000
zookeeper.session.timeout.ms30000
zookeeper.sync.time.ms2000

Leave a Reply