Adoption of new technologies will be inevitable; the key factor is DBA mindset as it evolves over the experience gained from vast areas of data management. Technology advances on demand and if we see the means of it then it is 'DATA' which is going to be consumed, processed and managed out of it. So technicians having skills to manage data will have alike mindset. DBA in Hadoop ecosystem can leverage his Clustering, Resource Managing, Partitioning, Parallelism, ETL, SQL and Scripting skills as these are very much used in Hadoop and high-level concepts would differ scarcely for DBA mindset. For example, Fencing is one of the important mechanism used in clusters to provide High Availability by avoiding split brain syndrome - In Oracle RAC we use STONITH (Shoot The Other Node In The Head) concept by using odd number of Voting disks to ensure that node is a member of cluster when it is able access more than half of the Voting disks. Similarly in Hadoop we use STONITH concept for NameNode High Availability by configuring odd number of Zookeeper's to avoid split brain syndrome. DBA's having data warehousing experience can outperform in Hadoop world as Hadoop workloads resembles to what data warehousing workloads are, and controlling them in terms of resource consumption to ensure clusters are not choked down is an art. In Hadoop ecosystem YARN is next-generation compute and resource management framework to manage the resources which reconciles the way applications use Hadoop system resources by allowing interactive querying and streaming data applications to run simultaneously with MapReduce batch jobs. Administration of Hadoop requires operational expertise with strong scientific troubleshooting skills in areas like Memory, CPU, I/O, Network and Storage which we can assume every DBA knows it and if not then there is no reason that DBA can't deduce it. It is also important for DBA to have Linux skills as Hadoop is deployed on Linux platform, again we can say that most of the DBA's does have hands-on experience on working with Linux. If DBA has DevOps culture who builds and maintains automated database deployments, collaborates closely with development and takes responsibility of capacity planning to application scalability then the same skills sets can be used in Hadoop world as the deployment and configuration management tools like Chef and Puppet are universal. DBA's are blessed to have extensive diagnostic information in hand about the RBDMS systems which are designed precisely and thoughts are carefully applied to structure the schema for the data placement. On the other hand it is contradicting with Hadoop as it is not ACID complaint, schema is defnied during runtime and NoSQL is not relational, thus it demands deep technical expertise to manage data by leveraging varities of tools, analytical engines and flexibility available in Hadoop ecosystem. Hadoop requires intimate knowledge of the various components of the stack, some of which are still immature and require even more fine-tuning and understanding to get enterprise ready. It's certain that the roles and responsibilities of today’s DBAs are dramatically shifting, can DBA manage to survive this paradigm shift ?
↧
Forum Post: What will be the role of a DBA in NoSQL-Hadoop world? Is the role of a DBA diminishing?
↧