Install Kafka Cluster on Ubuntu


1. Spin up Ubuntu server on AWS
2. Connect using putty.(port 22)
3.Check java version  java -version. if not install than do below:

 sudo apt-get update

sudo add-apt-repository ppa:webupd8team/java

sudo apt-get update

sudo apt-get install oracle-java8-installer

# Managing java
sudo update-alternatives --config java

Setting the JAVA_HOME Environment Variable

sudo update-alternatives --config java

sudo vi /etc/environment

# INSERT
JAVA_HOME="/usr/lib/jvm/java-8-oracle"

source /etc/environment

echo $JAVA_HOME

4. Download Kafka and unzip

# wget http://mirror.vorboss.net/apache/kafka/2.2.0/kafka_2.12-2.2.0.tgz

UNTAR kafka

tar -xvf <kafka_2.12-2.2.0.tgz>

Run below command to see everything works fine

bin/kafka-topics.sh

now add bin directory to PATH

$ pwd
/home/ubuntu/kafka_2.12-2.2.0/bin
sudo vi ~/.bash_profile and put

export PATH="/home/ubuntu/kafka_2.12-2.2.0/bin"

OR change the bashrc file

-------------

make chnages for zookeeper

mkdir data/zookeeper folder in kafka home


go to vi config/zookeeper.properties

make changes as below

dataDir=/home/ubuntu/kafka_2.12-2.2.0/data/zookeeper

save exit

start zookerper

 zookeeper-server-start.sh config/zookeeper.properties


SUCCESS if you get below

[2019-03-28 14:15:36,954] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)

Comments

Post a Comment

Popular posts from this blog

Qlik Sense Important Links

Qlik Sense learning https://learning.qlik.com/ https://www.qlik.com/us/services/training/self-paced-learning Qlik sense pricing   https://www.capterra.com/p/145530/QlikSense/#pricing https://www.qlik.com/us/pricing/qlik-sense-client-managed Qlik Sense Server Installation Document preview of Qlik Sense Server Installation Step By Step Guide.pdf https://community.qlik.com/t5/Documents/Qlik-Sense-Server-Installation-Step-By-Step-Guide/ta-p/1480562 https://community.qlik.com/t5/New-to-Qlik-Sense/how-to-download-Qlik-Sense-server/td-p/869140 Support Portal https://support.qlik.com/QS_Home_Page Qlik FIPS Support FIPS mode is not supported out of the box for Qlik Sense, QlikView, and Qlik NPrinting. Workarounds exist for NPrinting and Sense.  For general information about FIPS and Microsoft's statement around it, see  Microsoft's Security Guidance blog. https://community.qlik.com/t5/Knowledge-Base/Qlik-Software-and-FIPS-mode/ta-p/1717239 https://docs.microsoft.com/en-US/windows/...
Read more

Cloud Architecture Notes

Image
The architectural priorities and needs of every app are different, but the four pillars of architecture are an excellent guidepost you can use to make sure that you have given enough attention to every aspect of your application: Security : Safeguarding access and data integrity and meeting regulatory requirements Performance and scalability : Efficiently meeting demand in every scenario Availability and recoverability : Minimizing downtime and avoiding permanent data loss Efficiency and operations : Maximizing maintainability and ensuring requirements are met with monitoring A layered approach to security Defense in depth is a strategy that employs a series of mechanisms to slow the advance of an attack aimed at acquiring unauthorized access to information The common principles used to define a security posture are confidentiality, integrity, and availability, known collectively as CIA. Confidentiality - Principle of least privilege. Restricts access to informa...
Read more

AWS Rout53 NS records do not match with whois dns records OR Your site NOT working with registered domain name? Check this...

  Log in to your AWS web console Select Route53 service Select Hosted Zones in the left pane Select domain name (but do not click on the domain name, just select the radio button) Notice the 4 name servers in the right pane. Next, go to domain registrar site (in your case stay in Route53), log in and configure name servers for your domain. In Route53: select 'Registered domains' in the left pane, click on your domain name, verify that in the upper right you have name servers from step 5, if they don't exactly match, click 'Add/Edit Name servers' and enter name servers from step 5) After this you will receive email on your registered email id with AWS. Once the sync os done you will get notified. After notification you can try your ULR <some-name>.com or www.<some-name>.com "A" records created during route 53
Read more