Big data

Following topics will be covered:

  1. BigData stack installation
  2. Business case
  3. Development

1. Installation: Best way to go for this step is to have Ubuntu. I found this OS to be very friendly. you can go ahead with 14 Long term license for installation. Use Ubuntu site to download OR
Note: you can try before actually installing. you can also make bootable USBfrom windows and try this USB -OS first before going for actual installation.  

1.1. Which file system ? : HDFS - 

1.2 Which data management Layer ? - YARN

2. Business Use case: It is very important to understand that bid data world or hadoop world is for problems which have 5 V's . volume, variety, velocity, value and veracity.
More technically speaking if you are not going to have more than 5 datanodes no point using hdfs or less than millions/billions of of records thinks twice to use Hbase.
Also check the data generation activity e.g. is it machine or human generated.

3. Development: Eclipse could be used to have your first simple big data project.

Comments

Post a Comment

Popular posts from this blog

Qlik Sense Important Links

Qlik Sense learning https://learning.qlik.com/ https://www.qlik.com/us/services/training/self-paced-learning Qlik sense pricing   https://www.capterra.com/p/145530/QlikSense/#pricing https://www.qlik.com/us/pricing/qlik-sense-client-managed Qlik Sense Server Installation Document preview of Qlik Sense Server Installation Step By Step Guide.pdf https://community.qlik.com/t5/Documents/Qlik-Sense-Server-Installation-Step-By-Step-Guide/ta-p/1480562 https://community.qlik.com/t5/New-to-Qlik-Sense/how-to-download-Qlik-Sense-server/td-p/869140 Support Portal https://support.qlik.com/QS_Home_Page Qlik FIPS Support FIPS mode is not supported out of the box for Qlik Sense, QlikView, and Qlik NPrinting. Workarounds exist for NPrinting and Sense.  For general information about FIPS and Microsoft's statement around it, see  Microsoft's Security Guidance blog. https://community.qlik.com/t5/Knowledge-Base/Qlik-Software-and-FIPS-mode/ta-p/1717239 https://docs.microsoft.com/en-US/windows/securi
Read more

Cloud Architecture Notes

Image
The architectural priorities and needs of every app are different, but the four pillars of architecture are an excellent guidepost you can use to make sure that you have given enough attention to every aspect of your application: Security : Safeguarding access and data integrity and meeting regulatory requirements Performance and scalability : Efficiently meeting demand in every scenario Availability and recoverability : Minimizing downtime and avoiding permanent data loss Efficiency and operations : Maximizing maintainability and ensuring requirements are met with monitoring A layered approach to security Defense in depth is a strategy that employs a series of mechanisms to slow the advance of an attack aimed at acquiring unauthorized access to information The common principles used to define a security posture are confidentiality, integrity, and availability, known collectively as CIA. Confidentiality - Principle of least privilege. Restricts access to informa
Read more

AWS Rout53 NS records do not match with whois dns records OR Your site NOT working with registered domain name? Check this...

  Log in to your AWS web console Select Route53 service Select Hosted Zones in the left pane Select domain name (but do not click on the domain name, just select the radio button) Notice the 4 name servers in the right pane. Next, go to domain registrar site (in your case stay in Route53), log in and configure name servers for your domain. In Route53: select 'Registered domains' in the left pane, click on your domain name, verify that in the upper right you have name servers from step 5, if they don't exactly match, click 'Add/Edit Name servers' and enter name servers from step 5) After this you will receive email on your registered email id with AWS. Once the sync os done you will get notified. After notification you can try your ULR <some-name>.com or www.<some-name>.com "A" records created during route 53
Read more