The two java processes correspond to zookeeper and the kafka instances. Once the instances are started as daemon processes, grep for kafka in the processes list. Waiting for 10 secs until kafka and zookeeper services are up and running kafka_2.13-2.7.2/config/server.properties echo "Waiting for 10 secs until kafka and zookeeper services are up and running" sleep 10 Using the default configurations (provided by Apache Kafka) for spinning up the instances./kafka_2.13-2.7.2/bin/zookeeper-server-start.sh -daemon. Validate tf and tfio imports print("tensorflow-io version: ".format(tf._version_))ĭownload and setup Kafka and Zookeeper instancesįor demo purposes, the following instances are setup locally:Ĭurl -sSOL tar -xzf kafka_2.13-2.7.2.tgz
Kafka 2.10 download install#
Setup Install the required tensorflow-io and kafka packages pip install tensorflow-io pip install kafka-python Import packages import osįrom sklearn.model_selection import train_test_split Note: A Java runtime environment is required to run this tutorial. Note: A basic understanding of the kafka components will help you in following the tutorial with ease.
It is an essential technical component of a plethora of major enterprises where mission-critical data delivery is a primary requirement. Kafka is primarily a distributed event-streaming platform which provides scalable and fault-tolerant streaming data across data pipelines.
This tutorial focuses on streaming data from a Kafka cluster into a tf.data.Dataset which is then used in conjunction with tf.keras for training and inference.
0 kommentar(er)
