Definition of Modern Interface Design 

• User interface design is a subset of a field of study called human-computer 
  interaction (HCI). Human-computer interaction is the study, planning, and design of how people and computers work together so that a person’s needs are satisfied in the most effective way.
• The user interface is the part of a computer and its software that people can see, hear, touch, talk to, or otherwise understand or direct. The user interface has essentially two components: input and output.
• Input is how people communicate his needs to the system using keyboard or any pointing device and output is how the system returns processing result to user 
  through screen or sound.
• The best interface is one which has proper design with combination of effective input and output mechanisms.

1. Fault-efficient scalable, flexible and modular design:

• uses simple and modular programming model.
• The system provides servers at high scalability. The system is scalable by adding new nodes to handle larger data.
• Hadoop proves very helpful in storing, managing, processing and analyzing Big Data.
• Modular functions make the system flexible.
• One can add or replace components at ease.
• Modularity allows replacing its components for a different software tool.

2. Robust design of HDFS:

• Execution of Big Data applications continue even when an individual server or cluster fails.
• This is because of Hadoop provisions for backup (due to replications at least three times for each data block) and a data recovery mechanism.
• HDFS thus has high reliability.

3. Store and process Big Data:

• Processes Big Data of 3V characteristics.

4. Distributed clusters computing model with data locality :

• Processes Big Data at high speed as the application tasks and sub-tasks submit to the DataNodes.
• One can achieve more computing power by increasing the number of computing nodes.
• The processing splits across multiple DataNodes (servers), and thus fast processing and aggregated results.

5. Hardware fault-tolerant:

• A fault does not affect data and application processing. If a node goes down, the other nodes take care of the residue.
• This is due to multiple copies of all data blocks which replicate automatically.
• Default is three copies of data blocks.

6. Open-source framework:

• Open source access and cloud services enable large data store. Hadoop uses a cluster of multiple inexpensive servers or the cloud.

7. Java and Linux based:

• Hadoop uses Java interfaces. Hadoop base is Linux but has its own set of shell commands support.

Characteristics of Big Data, called 3Vs (and 4Vs also used) are:​

Volume:

• - is related to size of the data and hence the characteristic.
• - Size defines the amount or quantity of data, which is generated from an application(s).
• - The size determines the processing considerations needed for handling that data.

Velocity:

• - The term velocity refers to the speed of generation of data.
• - Velocity is a measure of how fast the data generates and processes.
• - To meet the demands and the challenges of processing Big Data, the velocity of generation of data plays a crucial ro​​​le.

Variety:

• -comprises of a variety of data.
• -Data is generated from multiple sources in a system.
• -This introduces variety in data and therefore introduces ‘complexity’.
• -Data is in various forms and formats.
• -The variety is due to the availability of a large number of heterogeneous platforms in the industry.

MongoDB is a document-oriented NoSQL database used for high volume data storage. Instead of using tables and rows as in the traditional relational databases, MongoDB makes use of collections and documents. Documents consist of key-value pairs which are the basic unit of data in MongoDB. Collections contain sets of documents and function which is the equivalent of relational database tables. MongoDB is a database which came into light around the mid-2000s.

MongoDB Features :-

• Each database contains collections which in turn contains documents. Each document can be different with a varying number of fields. The size and content of each document can be different from each other.
• The document structure is more in line with how developers construct their classes and objects in their respective programming languages. Developers will often say that their classes are not rows and columns but have a clear structure with key-value pairs.
• The rows (or documents as called in MongoDB) doesn’t need to have a schema defined beforehand. Instead, the fields can be created on the fly.
• The data model available within MongoDB allows you to represent hierarchical relationships, to store arrays, and other more complex structures more easily.
• Scalability – The MongoDB environments are very scalable. Companies across the world have defined clusters with some of them running 100+ nodes with around millions of documents within the database

Data Structure can be defined as the group of data elements which provides an efficient way of storing and organising data in the computer so that it can be used efficiently. Some examples of Data Structures are arrays, Linked List, Stack, Queue, etc. Data Structures are widely used in almost every aspect of Computer Science i.e. Operating System, Compiler Design, Artifical intelligence, Graphics and many more.

Data Structures are the main part of many computer science algorithms as they enable the programmers to handle the data in an efficient way. It plays a vital role in enhancing the performance of a software or a program as the main function of the software is to store and retrieve the user's data as fast as possible.

Data Structure Classification

• Primitive Data Structure
• Non Primitive Data Structure
  • Linear
    • Static
      • Array
    • Dynamic
      • Linked list
      • Stack
      • Queue
  • Non Linear
    • Tree
    • Graph

Linear Data Structures: A data structure is called linear if all of its elements are arranged in the linear order. In linear data structures, the elements are stored in non-hierarchical way where each element has the successors and predecessors except the first and last element.

• Types of Linear Data Structures are given below:

Arrays: An array is a collection of similar type of data items and each data item is called an element of the array. The data type of the element may be any valid data type like char, int, float or double.

The elements of array share the same variable name but each one carries a different index number known as subscript. The array can be one dimensional, two dimensional or multidimensional.

The individual elements of the array age are:

age[0], age[1], age[2], age[3],......... age[98], age[99].

Linked List: Linked list is a linear data structure which is used to maintain a list in the memory. It can be seen as the collection of nodes stored at non-contiguous memory locations. Each node of the list contains a pointer to its adjacent node.

Stack: Stack is a linear list in which insertion and deletions are allowed only at one end, called top.

A stack is an abstract data type (ADT), can be implemented in most of the programming languages. It is named as stack because it behaves like a real-world stack, for example: - piles of plates or deck of cards etc.

Queue: Queue is a linear list in which elements can be inserted only at one end called rear and deleted only at the other end called front.

It is an abstract data structure, similar to stack. Queue is opened at both end therefore it follows First-In-First-Out (FIFO) methodology for storing the data items.

Non Linear Data Structures : This data structure does not form a sequence i.e. each item or element is connected with two or more other items in a non-linear arrangement. The data elements are not arranged in sequential structure.

• Types of Non Linear Data Structures are given below:

Trees: Trees are multilevel data structures with a hierarchical relationship among its elements known as nodes. The bottommost nodes in the herierchy are called leaf node while the topmost node is called root node. Each node contains pointers to point adjacent nodes.

Tree data structure is based on the parent-child relationship among the nodes. Each node in the tree can have more than one children except the leaf nodes whereas each node can have atmost one parent except the root node. Trees can be classfied into many categories which will be discussed later in this tutorial.

Graphs: Graphs can be defined as the pictorial representation of the set of elements (represented by vertices) connected by the links known as edges. A graph is different from tree in the sense that a graph can have cycle while the tree can not have the one.

Important Human Characteristics in Design  :-

Perception

• Perception is our awareness and understanding of the elements and objects of our environment through the physical sensation of our various senses, including sight, sound, smell, and so forth. Perception is influenced, in part, by experience.
• Other perceptual characteristics include the following: 
  • Proximity. Our eyes and mind see objects as belonging together if they are near each other in space.
  • Similarity. Our eyes and mind see objects as belonging together if they share a common visual property, such as color, size, shape, brightness, or orientation.
  • Matching patterns. We respond similarly to the same shape in different sizes. The letters of the alphabet, for example, possess the same meaning, regardless of physical size.
  • Succinctness. We see an object as having some perfect or simple shape because perfection or simplicity is easier to remember.
  • Closure. Our perception is synthetic; it establishes meaningful wholes. If something does not quite close itself, such as a circle, square, triangle, or word, we see it as closed anyway.
  • Unity. Objects that form closed shapes are perceived as a group.
  • Continuity. Shortened lines may be automatically extended.
  • Balance. We desire stabilization or equilibrium in our viewing environment. Vertical, horizontal, and right angles are the most visually satisfying and easiest to look at.
  • Expectancies. Perception is also influenced by expectancies;  sometimes we perceive not what is there but what we expect to be there. Missing a spelling mistake in proofreading something we write is often an example of a perceptual expectancy error; we see not how a word is spelled, but how we expect to see it spelled.
  • Context. Context, environment, and surroundings also influence individual perception. For example, two drawn lines of the same length may look the same length or different lengths, depending on the angle of adjacent lines or what other people have said about the size of the lines.
  • Signals versus noise. Our sensing mechanisms are bombarded by many stimuli, some of which are important and some of which are not.  Important stimuli are called signals; those that are not important or unwanted are called noise. 

Memory

• Memory is viewed as consisting of two components, long-term and short-term (or working) memory.
• Short-term, or working, memory receives information from either the senses or long-term memory, but usually cannot receive both at once, the senses being processed separately. Within short-term memory a limited amount of information processing takes place. Information stored within it is variously thought to last from 10 to 30 seconds, with the lower number being the most reasonable speculation. Knowledge, experience, and familiarity govern the size and complexity of the information that can be remembered.
• Long-term memory contains the knowledge we possess. Information received in short-term memory is transferred to it and encoded within it, a process we call learning. It is a complex process requiring some effort on our part. 
• The learning process is improved if the information being transferred from short- term memory has structure and is meaningful and familiar.
• Learning is also improved through repetition. Unlike short-term memory, with its distinct limitations, long-term memory capacity is thought to be unlimited. An important memory consideration, with significant implications for interface design, is the difference in ability to recognize or recall words. 

Sensory Storage

• Sensory storage is the buffer where the automatic processing of information collected from our senses takes place. It is an unconscious process, large, attentive to the environment, quick to detect changes, and constantly being replaced by newly gathered stimuli. In a sense, it acts like radar, constantly scanning the environment for things that are important to pass on to higher memory.
• Repeated and excessive stimulation can fatigue the sensory storage mechanism, making it less attentive and unable to distinguish what is important (called habituation). Avoid unnecessarily stressing it.
• Design the interface so that all aspects and elements serve a definite purpose. Eliminating interface noise will ensure that important things will be less likely to be missed. 

Visual Acuity

• The capacity of the eye to resolve details is called visual acuity. It is the phenomenon that results in an object becoming more distinct as we turn our eyes toward it and rapidly losing distinctness as we turn our eyes away—that is, as the visual angle from the point of fixation increases.
• It has been shown that relative visual acuity is approximately halved at a distance of 2.5 degrees from the point of eye fixation.
• The eye’s sensitivity increases for those characters closest to the fixation point (the ―0‖) and decreases for those characters at the extreme edges of the circle (a 50/50 chance exists for getting these characters correctly identified). This may be presumed to be a visual ―chunk‖ of a screen.

Foveal and Peripheral Vision

• Foveal vision is used to focus directly on something; peripheral vision senses anything in the area surrounding the location we are looking at, but what is there cannot be clearly resolved because of the limitations in visual acuity just described.
• Foveal and peripheral vision maintain, at the same time, a cooperative and a competitive relationship. Peripheral vision can aid a visual search, but can also be distracting. 
• In its cooperative nature, peripheral vision is thought to provide clues to where the eye should go next in the visual search of a screen.
• In its competitive nature, peripheral vision can compete with foveal vision for attention. What is sensed in the periphery is passed on to our information- processing system along with what is actively being viewed foveally. 

Information Processing

• The information that our senses collect that is deemed important enough to do something about then has to be processed in some meaningful way.
• There are two levels of information processing going on within us. One level, the highest level, is identified with consciousness and working memory. It is limited, slow, and sequential, and is used for reading and understanding.
• In addition to this higher level, there exists a lower level of information processing, and the limit of its capacity is unknown. This lower level processes familiar information rapidly, in parallel with the higher level, and without conscious effort.
• Both levels function simultaneously, the higher level performing reasoning and problem solving, the lower level perceiving the physical form of information sensed. 

Mental Models 

• A mental model is simply an internal representation of a person’s current understanding of something. Usually a person cannot describe this mental mode and most often is unaware it even exists.
• Mental models are gradually developed in order to understand something, explain things, make decisions, do something, or interact with another person. Mental models also enable a person to predict the actions necessary to do things if the action has been forgotten or has not yet been encountered.
• A person already familiar with one computer system will bring to another system a mental model containing specific visual and usage expectations. If the new system complies with already-established models, it will be much easier to learn and use.
• The key to forming a transferable mental model of a system is design consistency and design standards. 

Movement Control

• Particularly important in screen design is Fitts’ Law (1954). This law states that:
  • The time to acquire a target is a function of the distance to and size of the target.
  • This simply means that the bigger the target is, or the closer the target is, the faster it will be reached.
  • The implications in screen design are:
    • Provide large objects for important functions.
    • Take advantage of the ―pinning‖ actions of the sides, top, bottom, and corners of the screen. 

Learning

• Learning, as has been said, is the process of encoding in long-term memory information.
• A design developed to minimize human learning time can greatly accelerate human performance. People prefer to stick with what they know, and they prefer to jump in and get started that is contained in short-term memory.
• Learning can be enhanced if it:
  • Allows skills acquired in one situation to be used in another somewhat like it. Design consistency accomplishes this.
  • Provides complete and prompt feedback.
  • Is phased, that is, it requires a person to know only the information needed at that stage of the learning process. 

Skill 

• The goal of human performance is to perform skillfully. To do so requires linking inputs and responses into a sequence of action. The essence of skill is performance of actions or movements in the correct time sequence with adequate precision.
• Skills are hierarchical in nature, and many basic skills may be integrated to form increasingly complex ones. Lower-order skills tend to become routine and may drop out of consciousness. 

Individual Differences 

• In reality, there is no average user. A complicating but very advantageous human characteristic is that we all differ—in looks, feelings, motor abilities, intellectual abilities, learning abilities and speed, and so on.
• Individual differences complicate design because the design must permit people with widely varying characteristics to satisfactorily and comfortably learn the task or job, or use the Web site.
• Multiple versions of a system can easily be created. Design must provide for the needs of all potential users. 

Common Usability Problems:

Mandel (1994) lists the 10 most common usability problems in graphical systems as reported by IBM usability specialists. They are:

• Ambiguous menus and icons.
• Languages that permit only single-direction movement through a system.
• Input and direct manipulation limits.
• Highlighting and selection limitations.
• Unclear step sequences.
• More steps to manage the interface than to perform tasks.
• Complex linkage between and within applications.
• Inadequate feedback and confirmation.
• Lack of system anticipation and intelligence.
• Inadequate error messages, help, tutorials, and documentation.

The Web, with its dynamic capabilities and explosive entrance into our lives, has unleashed what seems like more than its own share of usability problems. Many are similar to those outlined above. Web usability characteristics particularly wasteful of people’s time, and often quite irritating, are:

Visual clutter. A lack of “white space,” meaningless graphics, and unnecessary and wasteful decoration often turn pages into jungles of visual noise. Meaningful content lies hidden within the unending forest of vines and trees, forcing the user to waste countless minutes searching for what is relevant. Useless displayed elements are actually a form of visual noise.

Impaired information readability. Page readability is diminished by poor developer choices in typefaces, colors, and graphics. Use of innumerable typefaces and kaleidoscopic colors wrestle meaning from the screen. A person’s attention is directed towards trying to understand why the differences exist, instead of being focused toward identifying and understanding the page’s content. Backgrounds that are brightly colored or contain pictures or patterns greatly diminish the legibility of the overwritten text.

Incomprehensible components. Some design elements give the user no clue as to their function, leaving their purpose not at all obvious. Some icons and graphics, for example, are shrouded in mystery, containing no text to explain what they do. Some buttons don’t look at all like command buttons, forcing the user to “minesweep” the screen with a mouse to locate the objects that can be used to do something. Command buttons or areas that give no visual indication that they are clickable often won’t be clicked. Language is also often confusing, with the developer’s terminology being used, not that of the user.

Annoying distractions. Elements constantly in motion, scrolling marquees or text, blinking text, or looping continually running animations compete with meaningful content for the user’s eye’s and attention—and destroy a page’s readability. Automatically presented music or other sounds interrupt one’s concentration, as do nonrequested pop-up widows, which must be removed, wasting more of the user’s time. A person’s senses are under constant attack, and the benefits afforded by one’s peripheral vision are negated.

Confusing navigation. A site’s structure often resembles a maze of twisting pages into which the user wanders and is quite soon lost. Poor, little, or no organization exists among pages. The size and depth of many Web sites can eventually lead to a “lost in space” feeling as perceived site structure evaporates as one navigates. Embarking on a side trip can lead to a radical change in context or a path with no signposts or landmarks. Navigation links lead to dead-ends from which there is no return, or boomerang you right back to the spot where you are standing without you being aware of it. Some navigation elements are invisible. (See mystery icons and minesweeping above.) Confusing navigation violates expectations and results in disturbing unexpected behavior.

Inefficient navigation. A person must transverse content-free pages to find what is meaningful. One whole screen is used to point to another. Large graphics waste screen space and add to the page count. The path through the navigation maze is often long and tedious. Reams of useless data must be sifted through before a need can be fulfilled. Massive use of short pages with little content often creates the feeling that one is “link drunk.”

Inefficient operations. Time is wasted doing many things. Page download times can be excessive. Pages that contain, for example, large graphics and maps, large chunky headings, or many colors, take longer to download than text. Excessive information fragmentation can require navigation of long chains of links to reach relevant material, also accelerating user disorientation.

Excessive or inefficient page scrolling. Long pages requiring scrolling frequently lead to the user’s losing context as related information’s spatial proximity increases and some information entirely disappears from view and, therefore, from memory. Out of sight is often out of mind. If navigation elements and important content are hidden below the page top, they may be missed entirely. To have to scroll to do something important or complete a task can be very annoying; especially if the scrolling is caused by what the user considers is an irrelevancy or noise.

Information overload. Poorly organized or large amounts of information taxes one’s memory and can be overwhelming. Heavy mental loads can result from making decisions concerning which links to follow and which to abandon, given the large number of choices available. Or from trying to determine what information is important, and what is not. Or from trying to maintain one’s place in a huge forest of information trees. One easily becomes buried in decisions and information. Requiring even minimal amounts of learning to use a Web site adds to the mental load.

Design inconsistency. Design inconsistency has not disappeared with the Web. It has been magnified. The business system user may visit a handful of systems in one day, the Web user may visit dozens, or many more. It is expected that site differences will and must exist because each Web site owner strives for its own identity. For the user’s sake, however, some consistency must exist to permit a seamless flow between sites. Consistency is needed in, for example, navigation element location on a page and the look of navigation buttons (raised). The industry is diligently working on this topic and some “common practices” are already in place. The learning principle of rote memorization, however, is still being required within many sites. For example, the industry practice of using different standard link colors for unvisited sites (blue) and visited sites (purple) is often violated. This forces users to remember different color meanings in different places, and this also causes confusion between links and underlined text. Design guidelines for graphical user interfaces have been available for many years. Too often they are ignored (or the designer is unaware of them). Examples of inappropriate uses abound in design. The use of check boxes instead of radio buttons for mutually exclusive options, for example. Or the use of drop-down list boxes instead of combination boxes when the task mostly requires keyboard form fill-in. The Web is a form of the graphical user interface, and GUI guidelines should be followed.

Outdated information. One important value of a Web site is its “currentness.” Outdated information destroys a site’s credibility in the minds of many users, and therefore its usefulness. A useless site is not very usable.

Stale design caused by emulation of printed documents and past systems. The Web is a new medium with expanded user interaction and information display possibilities. While much of what we have learned in the print world and past information systems interface design can be ported to the Web, all of what we know should not be blindly moved from one to the other. Web sites should be rethought and redesigned using the most appropriate and robust design techniques available.

Some of these usability problems are a result of the Web’s “growing pains.” For other problems developers themselves can only be blamed, for they too often have created a product to please themselves and “look cool,” not to please their users. Symptoms of this approach include overuse of bleeding edge technology, a focus on sparkle, and jumping to implement the latest Internet technique or buzzword. These problems, of course, did not start with the Web. They have existed since designers began building user interfaces.

A menu must communicate to the user information about:

• The nature and purpose of the menu itself.
• The nature and purpose of each presented choice.
• How the proper choice or choices may be selected.

Menu Titles 

• Main menu:—Create a short, simple, clear, and distinctive title, describing the purpose of the entire series of choices.

• Submenus: — Submenu titles must be worded exactly the same as the menu choice previously selected to display them.
• General: —
  • Locate the title at the top of the listing of choices.
  • Spell out the title fully using either an:
    • Uppercase font.
    • Mixed-case font in the headline style.
  • Superfluous titles may be omitted.

Menu Choice Descriptions 

• Create meaningful choice descriptions that are familiar, fully spelled out, concise, and distinctive.
• Descriptions may be single words, compound words, or multiple words or phrases.
  • Exception: Menu bar items should be a single word (if possible).
  • Place the keyword first, usually a verb.
  • Use the headline style, capitalizing the first letter of each significant word in the choice description.
  • Use task-oriented not data-oriented wording.
  • Use parallel construction.
  • A menu choice must never have the same wording as its menu title.
  • Identical choices on different menus should be worded identically.
  • Choices should not be numbered.
  • Exception: If the listing is numeric in nature, graphic, or a list of varying items it may be numbered. 
  •  If menu options will be used in conjunction with a command language, the capitalization and syntax of the choices should be consistent with the command language.
  • Word choices as commands to the computer. 

 Menu Instructions

• For novice or inexperienced users, provide menu completion instructions.
  • Place the instructions in a position just preceding the part, or parts, of the menu to which they apply.
    • Left-justify the instruction and indent the related menu choice descriptions a minimum of three spaces to the right.
    • Leave a space line, if possible, between the instructions and the related menu choice descriptio
  • Present instructions in a mixed-case font in sentence style.
• For expert users, make these instructions easy to ignore by:
  • Presenting them in a consistent location.
  • Displaying them in a unique type style and/or color. 

Intent Indicators

• Cascade indicator:
  • To indicate that selection of an item will lead to a submenu, place a triangle or right-pointing solid arrow following the choice.
  • A cascade indicator must designate every cascaded menu.
• To a window indicator:
  • For choices that result in displaying a window to collect more information, place an ellipsis (. . .) immediately following the choice.
    • Exception do not use when an action:
      • Causes a warning window to be displayed.
      • May or may not lead to a window.
• ​​​​​​​Direct action items: — For choices that directly perform an action, no special indicator should be placed on the menu. 

Keyboard Equivalents 

• To facilitate keyboard selection of a menu choice, each menu item should be assigned a keyboard equivalent mnemonic.
• The mnemonic should be the first character of the menu item’s description.
  • If duplication exists in first characters, use another character in the duplicated item’s description.
  • Preferably choose the first succeeding consonant.
• Designate the mnemonic character by underlining it.
• Use industry-standard keyboard access equivalents when they exist.

Keyboard Accelerators 

• For frequently used items, provide a keyboard accelerator to facilitate keyboard selection.
• The accelerator may be one function key or a combination of keys.
  • Function key shortcuts are easier to learn than modifier plus letter shortcuts.
• Pressing no more than two keys simultaneously is preferred.
  • Do not exceed three simultaneous keystrokes.
• Use a plus (+) sign to indicate that two or more keys must be pressed at the same time.
• Accelerators should have some associative value to the item.
• Identify the keys by their actual key top engraving.
• If keyboard terminology differences exist, use:
  • The most common keyboard terminology.
  • Terminology contained on the newest PCs.
• Separate the accelerator from the item description by three spaces.
• Right-align the key descriptions.
• Do not use accelerators for:
  • Menu items that have cascaded menus.
  • Pop-up menus. Use industry-standard keyboard accelerators 

This is given with respect to Hortonworks Data Platform (HDP). Ambari eliminates the need for the manual tasks that used to watch over Hadoop operations. It gives a simple and secure platform for provisioning, managing, and monitoring HDP deployments. Ambari is an easy to use Hadoop management UI and is solidly backed by REST APIs. The benefits of using Apache Ambari are mentioned below.

Simplified installation, configuration, and management of the Hadoop cluster: Ambari can efficiently create Hadoop clusters at scale. Its wizard-driven approach lets the configuration be automated as per the environment so that the performance is optimal. Master–slave and client components are assigned to configuring services. It is also used to install, start, and test the cluster.

Configuration blueprints give recommendations to those seeking a hands-on approach. The blueprint of an ideal cluster is stored. How it is provisioned is clearly traced. This is then used to automate the creation of successive clusters without any user interaction. Blueprints also preserve and ensure the application of best practices across different environments.

Ambari provides a rolling upgrade feature where running clusters can be updated on the go with maintenance releases and feature-bearing releases, and therefore there is no unnecessary downtime. When there are large clusters involved, rolling updates are simply not possible, in which case express updates are used. Unlike the previous case, here, there is downtime involved but is minimum as when the update is manual. Both rolling and express updates are free of manual updates.

Centralized security and application: The complexity of cluster security configuration and administration is greatly reduced by Ambari which is among the components of the Hadoop ecosystem. The tool also helps the automated setup of the advanced security constructs like Kerberos and Ranger.

Complete visibility to your cluster’s health: Through this tool, you can monitor your cluster’s health and availability. An easily customized web-based dashboard has metrics that give status information for each service in the cluster like HDFS, YARN, and HBase. The tool also helps with garnering and visualizing critical operational metrics for troubleshooting and analysis. Ambari predefines alerts that are integrated with the existing enterprise monitoring tools that monitor cluster components and hosts as per the specified check intervals. Through the browser interface, users can browse alerts for their clusters, search, and filter alerts. They can also view and modify alert properties and alert instances.

Metrics visualization and dashboarding: It provides a scalable low-latency storage system for Hadoop component metrics. Picking the metrics of Hadoop which truly matter requires considerable expertise and understanding of how the components work with each other. Grafana is a leading graph and dashboard builder that simplifies the metrics reviewing process. This is included with Ambari Metrics, along with HDP.

Extensibility and customization: Ambari lets a developer work on Hadoop gracefully in his/her enterprise setup. Ambari leverages the large innovative community which improves upon the tool and it also eliminates vendor lock-in. REST APIs along with Ambari Stacks and Views allows extensive flexibility for customization of HDP implementation.

Ambari Stacks wraps the life cycle control layer used to rationalize operations over a broad set of services. This includes a consistent approach that Ambari uses to manage different types of services like install, start, configure, status, and stop. When provisioning, cluster install experience is rationalized across a set of services by Stacks technology. A natural extension point for operators is provided by Stacks to plug in newly created services that can perform alongside Hadoop.

Third parties can plug in their views through Ambari Views. A view is an application that is deployed into an Ambari container where it offers UI capabilities to be plugged in to give out custom visualization, management, and monitoring features.

• Ambari Server
• Ambari Agent
• Ambari Web UI
• Database

1. Ambari Server

The entry point for all administrative activities on the master server is known as Ambari Server. It is a shell script. Internally, this script uses Python code, ambari-server.py, and routes all requests to it.

Ambari Server consists of several entry points that are available when passed different parameters to the Ambari Server program. They are:

• Daemon management
• Software upgrade
• Software setup
• LDAP (Lightweight Direct Access Protocol)/PAM (Pluggable Authentication Module) /Kerberos management
• Ambari backup and restore
• Miscellaneous options

2. Ambari Agent

Ambari Agent runs on all the nodes that you want to manage with Ambari.  This program periodically sends heartbeats to the master node. By using Ambari Agent, Ambari Server executes many tasks on the servers.

3. Ambari Web User Interface

Ambari Web UI is one of the powerful features of Apache Ambari. The web application is deployed through the server of Ambari program which is running on the master host exposed on port 8080. This application is protected by authentication. You can access and then control and view all aspects of your Hadoop cluster, once you log in to the web portal.

4. Database

Ambari supports multiple RDBMS (Relational Database Management Systems) to keep track of the state of the entire Hadoop infrastructure. You can choose the database you want to use during the setup of Ambari. Ambari supports these following databases at the time of writing:

• PostgreSQL
• Oracle
• MySQL or MariaDB
• Embedded PostgreSQL
• Microsoft SQL Server
• SQL Anywhere
• Berkeley DB

This technology is preferred by the Big Data Developers as it is quite handy and comes with a step-by-step guide allowing easy installation on the Hadoop cluster. Its preconfigured key operational metrics provide a quick look into the health of the Hadoop core, i.e., HDFS and MapReduce, along with the additional components such as Hive, HBase, HCatalog, etc. Ambari sets up a centralized security system by incorporating Kerberos and Apache Ranger into the architecture. The RESTful APIs monitor the information and integrate the operational tools. Its user-friendliness and interactivity have made it enter the list of top 10 open-source technologies for the Hadoop cluster.