Usage of HIVE Tool in Hadoop ECO System with Loading Data and User Defined Functions
DOI:
https://doi.org/10.61841/2y2xdm74Keywords:
Hive,, -Import,, UDF,, -Map Reduce, Data Loading.Abstract
The general usage of Hadoop is to store the bulk data with Hadoop Distributed File System and to process the data with Map Reduce. Apart from this the eco system provides extensive functionalities like usage of query-based logics to import the data from local path and Hadoop distributed path. This article presents the usage of Hive in the context of loading the bulk data and some simple analytics applicability. The Hive User Defined functions (UDF) creation and running with eclipse is the additional context of the paper. The work explains the parameters involved in the processing of the data loading and working with UDF’s so as to simplify the Map Reduce (MR) process with HIVE commands.The context of Map Reduce requires the complex coding skills, and the problem is only HDFS path is known to the MR, there is no approach of working with local file system. The basic advantage of Hive is to work with local path files and as well as HDFS path files. Similarly processing wise Hive simplifies coding and functions usage with the implementation of the simple commands.The case study taken in this article deals with various parameters like page views data, system_IP, View_time, user_id and page_url. The other case study we have taken is loading of the bulk data in the less time.The outcome of the work is loading of the data in the context of local path and Hadoop Distributed Path. Loading of the bulk data within seconds and recording of the time taken is the other outcome. The creation of the UDF and running of the tasks in HIVE is the resultant of the work. Apart from these considerations the research issues and possible extension works can be observed in the article.
Downloads
References
1. U. P. K. Kethavarapu,S.Saraswathi, “Ontology based job recommendation system with dynamic source updates by slowly changing source detection”, International Journal of Knowledge Engineering and Soft Data Paradigms, vol. 5, no. 3/4, pp. 164-173, 2016.
2. K.UmapavanKumar, S. V. N. Srinivasu, A. Ramaswamy Reddy, “Hadoop Cluster Performance with MR and Pig Latin in the Big Data”, International Journal of Innovative Technology and Exploring Engineering, vol. 8, no. 7, pp. 83-87, May 2019.
3. K.UmapavanKumar, “DWH security encapsulation with Bitmap Indexing Mechanisms”,IJETCSE, vol. 11, no. 2, pp. 10-15, Nov. 2014.
4. K.Umapavan Kumar, “Various Issues in Hadoop Distributed File System and MR future research
directions”, International Journal of Pure and Applied Mathematics, vol. 120, no. 6, pp. 4441-4451, 2018.
5. U. P. K.Kethavarapu, “The ten ingredients of data base systems for improving performance and their review leading to research problems”, IFRSA International Journal of Computing, vol. 2, no. 2, pp. 409- 415, Apr. 2012.
6. U. P. K. Kethavarapu, “Various Computing models in Hadoop eco system along withthe perspective of analytics using R and Machine learning”, International Journal of Computer Science and Information Security, vol. 14, pp. 17-23.
7. C. P. Chen and C.-Y. Zhang, “Data Intensive Applications, Challenges, Techniques and Technologies: A Survey on Big Data”, Information Science, vol. 275, pp. 314-347,2014.
8. J. Y. Monteith, J. D. McGregorand J. E.Ingram, “Hadoop and its EvolvingEcosystem”, 5th International Workshop on Software Eco System ,pp. 57-68, 2013.
9. Mausam j. Naik (2019) mapksignalling pathway: role in cancer pathogenesis. Journal of Critical Reviews, 6 (3), 1-6. doi:10.22159/jcr.2019v6i3.31778
10. Craddock, T.J.A., Tuszynski, J.A. On the role of the microtubules in cognitive brain functions (2007) NeuroQuantology, 5 (1), pp. 32-57.
11. Georgiev, D.D., Papaioanou, S.N., Glazebrook, J.F. Solitonic effects of the local electromagnetic field on neuronal microtubules (2007) NeuroQuantology, 5 (3), pp. 276-291.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format for any purpose, even commercially.
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
- The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notices:
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation .
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.
