Big data

Following topics will be covered:

  1. BigData stack installation
  2. Business case
  3. Development

1. Installation: Best way to go for this step is to have Ubuntu. I found this OS to be very friendly. you can go ahead with 14 Long term license for installation. Use Ubuntu site to download OR
Note: you can try before actually installing. you can also make bootable USBfrom windows and try this USB -OS first before going for actual installation.  

1.1. Which file system ? : HDFS - 

1.2 Which data management Layer ? - YARN

2. Business Use case: It is very important to understand that bid data world or hadoop world is for problems which have 5 V's . volume, variety, velocity, value and veracity.
More technically speaking if you are not going to have more than 5 datanodes no point using hdfs or less than millions/billions of of records thinks twice to use Hbase.
Also check the data generation activity e.g. is it machine or human generated.

3. Development: Eclipse could be used to have your first simple big data project.

Comments

Post a Comment

Popular posts from this blog

Cloud Architecture Notes

Image
The architectural priorities and needs of every app are different, but the four pillars of architecture are an excellent guidepost you can use to make sure that you have given enough attention to every aspect of your application: Security : Safeguarding access and data integrity and meeting regulatory requirements Performance and scalability : Efficiently meeting demand in every scenario Availability and recoverability : Minimizing downtime and avoiding permanent data loss Efficiency and operations : Maximizing maintainability and ensuring requirements are met with monitoring A layered approach to security Defense in depth is a strategy that employs a series of mechanisms to slow the advance of an attack aimed at acquiring unauthorized access to information The common principles used to define a security posture are confidentiality, integrity, and availability, known collectively as CIA. Confidentiality - Principle of least privilege. Restricts access to informa
Read more

Qlik Sense Important Links

Qlik Sense learning https://learning.qlik.com/ https://www.qlik.com/us/services/training/self-paced-learning Qlik sense pricing   https://www.capterra.com/p/145530/QlikSense/#pricing https://www.qlik.com/us/pricing/qlik-sense-client-managed Qlik Sense Server Installation Document preview of Qlik Sense Server Installation Step By Step Guide.pdf https://community.qlik.com/t5/Documents/Qlik-Sense-Server-Installation-Step-By-Step-Guide/ta-p/1480562 https://community.qlik.com/t5/New-to-Qlik-Sense/how-to-download-Qlik-Sense-server/td-p/869140 Support Portal https://support.qlik.com/QS_Home_Page Qlik FIPS Support FIPS mode is not supported out of the box for Qlik Sense, QlikView, and Qlik NPrinting. Workarounds exist for NPrinting and Sense.  For general information about FIPS and Microsoft's statement around it, see  Microsoft's Security Guidance blog. https://community.qlik.com/t5/Knowledge-Base/Qlik-Software-and-FIPS-mode/ta-p/1717239 https://docs.microsoft.com/en-US/windows/securi
Read more