Sitecore Kafka Integration

Sitecore and Kafka Integration using ASP.NET MVC Application.

We can integrate Sitecore with Kafka using .NET Connectors. I will be using Cloudera .NET client library for Apache Kafka called Cloudera.Kafka.

The Cloudera.Kafka assembly can be downloaded and installed through Visual Studio or a command line interface or from nuget.org. Using Cloudera.Kafka, we can Push the data to Kafka system using ProducerBuilder and Pull using ConsumerBuilder.

Getting Started

Download, the solution on your local system and restore the NuGet packages.

Kafka Configurations, to use the Kafka we have to change the Kafka configurations in the following places:

• \Sitecore-Kafka-Integration\KafkaIntegrationSite\KafkaIntegrationSite\Controllers\KafkaController.cs > PublishToKafka Function :-

            var items = this.kafkaRepository.Push(
                new CT.SC.Foundation.Kafka.Models.ProducerModel
                {
                    Topic = topic,
                    Message = message,
                    BootstrapServers = "<Fill details from configuration items>",
                    SecurityProtocol = SecurityProtocolValue.SaslSsl,
                    SaslMechanism = SaslMechanismValue.Plain,
                    SslCaLocation = @"<Fill details from configuration items>",
                    SaslUsername = "<Fill details from configuration items>",
                    SaslPassword = "<Fill details from configuration items>",
                    EnableSslCertificateVerification = false
                }
                ); 

⬆ Back To Top

• \Sitecore-Kafka-Integration\KafkaIntegrationSite\KafkaIntegrationSite\Controllers\KafkaController.cs > PullFromKafka Function :-

            var items = this.kafkaRepository.Pull(
                new CT.SC.Foundation.Kafka.Models.ConsumerModel
                {
                    Topic = topic,
                    GroupId = groupId,
                    AutoOffsetReset = AutoOffsetResetValue.Earliest,
                    EnablePartitionEof = true,
                    BootstrapServers = "<Fill details from configuration items>",
                    SecurityProtocol = SecurityProtocolValue.SaslSsl,
                    SaslMechanism = SaslMechanismValue.Plain,
                    SslCaLocation = @"<Fill details from configuration items>",
                    SaslUsername = "<Fill details from configuration items>",
                    SaslPassword = "<Fill details from configuration items>",
                    EnableSslCertificateVerification = false
                }
                );

⬆ Back To Top

Output

Landing Page

• Once you RUN the application, you will land into the HOME page:

  ⬆ Back To Top

Push message from Sitecore to Kafka

• You can PUSH the message to the Kafka topic:

⬆ Back To Top

Once user click on the Push to Kafka, user will see Please Wait screen:

⬆ Back To Top

After successful Push user will see Scuccess message:

⬆ Back To Top

Pull message from Kafka to Sitecore

• You can PULL the message from Kafka topic:

⬆ Back To Top

Once user click on the Pull from Kafka, user will see Please Wait screen:

⬆ Back To Top

If Messages not present in Kafka Topic user will see Error message:

⬆ Back To Top

If Messages present in Kafka Topic user will see Success message with details:

⬆ Back To Top

License

Copyright (c) 2021.

Licensed under the MIT license.

⬆ Back To Top