Oracle NoSQL Database is a distributed, flexible data model, key -value based database in which the value abstraction layers is either binary, JSON (Avro) or table structure. Oracle XQuery for Hadoop ( introduction ) provides a Oracle NoSQL Database adapter for storing text (UTF8 encoded), XML (UTF8) or XDK binary XML. Using the adapter Oracle NoSQL Database data may be decoded as UTF and returned as text, decoded as an Avro record and returned as XML, decoded as a sequence of bytes and returned as XML, or decoded as XDK binary XML and returned as XML. In this article we shall store UTF8 encoded text data in Oracle NoSQL Database using OXH and subsequently access Oracle NoSQL Database to get the text data also using OXH. The data fetched from Oracle NoSQL Database shall be loaded into Oracle Database 11g. The following functions are provided to access text data in Oracle NoSQL database and put and get text data. Function Signature Description kv:collection-text declare %kv:collection("text") function kv:collection-text($parent-key as xs:string?) as xs:string* external; Accesses a collection of text values. The parameter specifies the key whose sub key/value pairs are returned by the function. Two other versions of the function support specifying a hierarchy of kv pairs to be returned and a sub-range. kv:put-text declare %kv:put("text") function kv:put-text($key as xs:string, $value as xs:string) external; Puts a UTF-8 encoded key-value pair into Oracle NoSQL Database. kv:get-text declare %kv:get("text") function kv: get-text($key as xs:string) as xs:string? external; Gets a UTF-8 encoded key-value pair from Oracle NoSQL Database. This article has the following sections: Setting the Environment Oracle Linux 6.6 is installed on Oracle VirtualBox 4.3. In addition to installing Oracle Database 11g install Oracle NoSQL Database, which may be downloaded from http://www.oracle.com/technetwork/products/nosqldb/downloads/default-495311.html . Unzip the file to a directory, /oranosql for example. The environment is the same as for the previous OXH tutorials except that the Oracle NoSQL Database environment variables must also be added. Add the KVHOME env variable for the directory in which Oracle NoSQL Database is installed. Add the $KVHOME/lib/* to the HADOOP_CLASSPATH env variable. vi ~/.bashrc vi /oxh/hadoop-2.0.0-cdh4.6.0/etc/hadoop/hadoop-env.sh export HADOOP_PREFIX=/oxh/hadoop-2.0.0-cdh4.6.0 export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk export ORACLE_HOME=/home/oracle/app/oracle/product/11.2.0/dbhome_1 export ORACLE_SID=ORC export KVHOME=/oranosql/kv-3.0.5 export OLH_HOME=/oxh/oraloader-3.0.0-h2 export OXH_HOME=/oxh/oxh-3.0.0-cdh4.6.0 export HADOOP_MAPRED_HOME=/oxh/hadoop-2.0.0-cdh4.6.0/bin export HADOOP_HOME=/oxh/hadoop-2.0.0-cdh4.6.0/share/hadoop/mapreduce2 export HADOOP_CLASSPATH=$HADOOP_HOME/*:$HADOOP_HOME/lib/*:$OLH_HOME/jlib/*:$OXH_HOME/lib/*:$KVHOME/lib/* export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_MAPRED_HOME:$ORACLE_HOME/bin Create a lightweight Oracle NoSQL Database with the following command. java -jar /oranosql/kv-3.0.5/lib/kvstore.jar kvlite A kvlite store gets created. Create Oracle Database table OE.WLSSERVER . CREATE TABLE wlsserver (timestamp VARCHAR2( 55), category VARCHAR2(15), type VARCHAR2(55), servername VARCHAR2(15), code VARCHAR2(15), msg VARCHAR2(255)); Start HDFS NameNode and DataNode. Create the /oranosql directory in the HDFS. hdfs dfs -mkdir hdfs://10.0.2.15:8020/oranosql Put the Oracle NoSQL Database software from the local directory to the HDFS directory. hdfs dfs -put /oranosql/kv-3.0.5 hdfs://10.0.2.15:8020/oranosql Adding Text File into HDFS We shall be storing text file data in Oracle NoSQL Database with OXH Oracle NoSQL Database adapter and subsequently querying Oracle NoSQL Database using OXH also. Create the following text file base_domain_oranosql.txt in a local directory. Apr-8-2014-7-06-16-PM-PDT :Notice:WebLogicServer:AdminServer:BEA -000365:Server state changed to STANDBY Apr-8-2014-7-06-17-PM-PDT :Notice:WebLogicServer:AdminServer:BEA -000365:Server state changed to STARTING Apr-8-2014-7-06-18-PM-PDT :Notice:Log Management:AdminServer:BEA-170027:The Server has established connection with the Domain level Diagnostic Service successfully Apr-8-2014-7-06-19-PM-PDT :Notice:WebLogicServer:AdminServer:BEA -000365:Server state changed to ADMIN Apr-8-2014-7-06-20-PM-PDT :Notice:WebLogicServer:AdminServer:BEA -000365:Server state changed to RESUMING Apr-8-2014-7-06-21-PM-PDT:Notice:Server:AdminServer:BEA-002613:Channel Default is now listening on fe80-0-0-0-0-5efe-c0a8-147-7001 for protocols iiop,t3,ldap,snmp,http Apr-8-2014-7-06-22-PM-PDT :Notice:WebLogicServer:AdminServer:BEA -000331:Started WebLogic Admin Server AdminServer for domain base_domain running in Development Mode Apr-8-2014-7-06-23-PM-PDT :Notice:WebLogicServer:AdminServer:BEA -000365:Server state changed to RUNNING Apr-8-2014-7-06-24-PM-PDT :Notice:WebLogicServer:AdminServer:BEA -000360:Server started in RUNNING mode Create a directory /wls in HDFS if not already created. Put the text file base_domain_oranosql.txt to the HDFS /wls directory. hadoop dfs -mkdir hdfs://10.0.2.15:8020/wls hadoop dfs -put base_domain_oranosql.txt hdfs://10.0.2.15:8020/wls Putting Text File Data into Oracle NoSQL Database In this section we shall put the text file data from HDFS into Oracle NoSQL Database. Create a query script txt_oranosql.xq . Import the OXH text module and the Oracle NoSQL Database module. Access the text file base_domain_oranosql.txt in HDFS using the function text :collection ($uris) in the for clause of a FLOWR expression. Tokenize each line in the text file using the fn :tokenize function. Create a Oracle NoSQL Database key to store each line of text. Put the key-value pair in Oracle NoSQL Database using the kv:put-text function. The txt_oranosql.xq query is listed: import module "oxh:text"; import module "oxh:kv"; for $line in text:collection("/wls/base_domain_oranosql.txt") let $split := fn:tokenize($line, ":") let $key := "/wls/log/" || $split[1] return kv: put-text($key, $line) To be able to access Oracle NoSQL Database some connection properties are required to be specified either using the command line option –D or in a configuration file specified on the command line with the –conf option. We shall use a configuration file, oxh_config.xml . Specify the following properties in the configuration file. Property Value oracle.kv.hosts localhost:5000 oracle.kv.kvstore kvstore The configuration file oxh_config.xml is listed: oracle.kv.hosts localhost:5000 oracle.kv.kvstore kvstore Run the query script with the following hadoop command in which the configuration file is specified with the –conf ption. hadoop jar $OXH_HOME/lib/oxh.jar -conf oxh_config.xml txt_oranosql.xq Oracle XQuery for Hadoop gets started. The MapReduce application runs to access the text file in HDFS and put the text file data into Oracle NOSQL Database. The output from the hadoop command is listed: [root@localhost oxh]# hadoop jar $OXH_HOME/lib/oxh.jar -conf oxh_config.xml txt_oranosql.xq 14/05/22 15:40:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 14/05/22 15:40:31 INFO hadoop.xquery: OXH: Oracle XQuery for Hadoop 3.0.0 (build 3.0.0-cdh4.6.0-mr1 @mr2). Copyright (c) 2014, Oracle. All rights reserved. 14/05/22 15:40:31 INFO hadoop.xquery: Executing query "txt_oranosql.xq". Output path: "hdfs://10.0.2.15:8020/tmp/oxh-root/output" 14/05/22 15:40:35 INFO hadoop.xquery: Submitting map-reduce job "oxh :txt _oranosql.xq#0" id="564c55a3-a2bd-43c8-9b03-19796307fd02.0", inputs=[hdfs://10.0.2.15:8020/wls/base_domain_oranosql.txt], output=hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/564c55a3-a2bd-43c8-9b03-19796307fd02.0 14/05/22 15:40:36 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 14/05/22 15:40:36 INFO input.FileInputFormat: Total input paths to process : 1 14/05/22 15:40:36 INFO mapreduce.JobSubmitter: number of splits :1 14/05/22 15:40:37 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1576471189_0001 14/05/22 15:40:42 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 14/05/22 15:40:42 INFO hadoop.xquery: Waiting for map-reduce job oxh :txt _oranosql.xq#0 14/05/22 15:40:42 INFO mapreduce.Job: Running job: job_local1576471189_0001 14/05/22 15:40:42 INFO mapred.LocalJobRunner: OutputCommitter set in config null 14/05/22 15:40:42 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter 14/05/22 15:40:42 INFO mapred.LocalJobRunner: Waiting for map tasks 14/05/22 15:40:42 INFO mapred.LocalJobRunner: Starting task: attempt_local1576471189_0001_m_000000_0 14/05/22 15:40:42 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:40:42 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/wls/base_domain_oranosql.txt:0+1119 14/05/22 15:40:43 INFO mapreduce.Job: Job job_local1576471189_0001 running in uber mode : false 14/05/22 15:40:43 INFO mapreduce.Job: map 0% reduce 0% 14/05/22 15:40:43 INFO mapred.LocalJobRunner: 14/05/22 15:40:43 INFO mapred.Task: Task :attempt _local1576471189_0001_m_000000_0 is done. And is in the process of committing 14/05/22 15:40:43 INFO mapred.LocalJobRunner: 14/05/22 15:40:43 INFO mapred.Task: Task attempt_local1576471189_0001_m_000000_0 is allowed to commit now 14/05/22 15:40:43 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1576471189_0001_m_000000_0' to hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/564c55a3-a2bd-43c8-9b03-19796307fd02.0/_temporary/0/task_local1576471189_0001_m_000000 14/05/22 15:40:43 INFO mapred.LocalJobRunner: map 14/05/22 15:40:43 INFO mapred.Task: Task 'attempt_local1576471189_0001_m_000000_0' done. 14/05/22 15:40:43 INFO mapred.LocalJobRunner: Finishing task: attempt_local1576471189_0001_m_000000_0 14/05/22 15:40:43 INFO mapred.LocalJobRunner: Map task executor complete. 14/05/22 15:40:44 INFO mapreduce.Job: map 100% reduce 0% 14/05/22 15:40:44 INFO mapreduce.Job: Job job_local1576471189_0001 completed successfully 14/05/22 15:40:44 INFO mapreduce.Job: Counters: 23 File System Counters FILE: Number of bytes read=12552 FILE: Number of bytes written=19157824 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=18763376 HDFS: Number of bytes written=1640 HDFS: Number of read operations=206 HDFS: Number of large read operations=0 HDFS: Number of write operations=4 Map-Reduce Framework Map input records=9 Map output records=0 Input split bytes=115 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms )= 175 CPU time spent (ms )= 0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Total committed heap usage (bytes )= 25554944 File Input Format Counters Bytes Read=1119 File Output Format Counters Bytes Written=0 14/05/22 15:40:44 INFO hadoop.xquery: Submitting map-reduce job "oxh :txt _oranosql.xq#1" id="564c55a3-a2bd-43c8-9b03-19796307fd02.1", inputs=[hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/564c55a3-a2bd-43c8-9b03-19796307fd02.0/OXHI0xputtext-m-00000] 14/05/22 15:40:44 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 14/05/22 15:40:44 INFO input.FileInputFormat: Total input paths to process : 1 14/05/22 15:40:44 INFO mapreduce.JobSubmitter: number of splits :1 14/05/22 15:40:44 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1457307614_0002 14/05/22 15:40:48 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 14/05/22 15:40:48 INFO hadoop.xquery: Waiting for map-reduce job oxh :txt _oranosql.xq#1 14/05/22 15:40:48 INFO mapreduce.Job: Running job: job_local1457307614_0002 14/05/22 15:40:48 INFO mapred.LocalJobRunner: OutputCommitter set in config null 14/05/22 15:40:48 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.NullOutputFormat$2 14/05/22 15:40:48 INFO mapred.LocalJobRunner: Waiting for map tasks 14/05/22 15:40:48 INFO mapred.LocalJobRunner: Starting task: attempt_local1457307614_0002_m_000000_0 14/05/22 15:40:48 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:40:48 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/564c55a3-a2bd-43c8-9b03-19796307fd02.0/OXHI0xputtext-m-00000:0+1640 14/05/22 15:40:49 INFO mapred.LocalJobRunner: 14/05/22 15:40:49 INFO mapred.Task: Task :attempt _local1457307614_0002_m_000000_0 is done. And is in the process of committing 14/05/22 15:40:49 INFO mapred.LocalJobRunner: map 14/05/22 15:40:49 INFO mapred.Task: Task 'attempt_local1457307614_0002_m_000000_0' done. 14/05/22 15:40:49 INFO mapred.LocalJobRunner: Finishing task: attempt_local1457307614_0002_m_000000_0 14/05/22 15:40:49 INFO mapred.LocalJobRunner: Map task executor complete. 14/05/22 15:40:49 INFO mapreduce.Job: Job job_local1457307614_0002 running in uber mode : false 14/05/22 15:40:49 INFO mapreduce.Job: map 100% reduce 0% 14/05/22 15:40:49 INFO mapreduce.Job: Job job_local1457307614_0002 completed successfully 14/05/22 15:40:49 INFO mapreduce.Job: Counters: 23 File System Counters FILE: Number of bytes read=25157 FILE: Number of bytes written=38311851 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=37527273 HDFS: Number of bytes written=1640 HDFS: Number of read operations=420 HDFS: Number of large read operations=0 HDFS: Number of write operations=7 Map-Reduce Framework Map input records=9 Map output records=0 Input split bytes=168 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms )= 17 CPU time spent (ms )= 0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Total committed heap usage (bytes )= 25554944 File Input Format Counters Bytes Read=1640 File Output Format Counters Bytes Written=0 14/05/22 15:40:49 INFO hadoop.xquery: Finished executing "txt_oranosql.xq". Output path: "hdfs://10.0.2.15:8020/tmp/oxh-root/output" The first split in the text file in each line must not include any spaces. For example if field Apr-8-2014-7-06-16-PM-PDT is replaced with Apr 8 2014 7-06-16 PM PDT, the following error gets generated. java.lang.Exception: java.io.IOException: {http://xmlns.oracle.com/xdk/xquery/error }KVST0002 : Invalid key: /wls/log/Apr 8 2014 7-06-16 PM PDT Querying and Loading Text into Oracle Database In this section we shall query the Oracle NoSQL Database using OXH and load the data into Oracle Database 11g. Create a query script oranosql_oradb.xq in which import the Oracle NoSQL Database and Text modules . Specify a custom function to connect to Oracle Database and put column values in the OE.WLSSERVER table. Specify a FLOWR expression to access the text data in Oracle NoSQL Database get the text data. In the for clause invoke the kv :collection -text function( $key ) to access the KV pair associated with the /wls/log key. Tokenize the text data and create variables for timestamp , category , type , servername , code and msg in the log data. Invoke the Oracle Database custom function to put the log data into the database. The oranosql_oradb.xq query script is listed: import module "oxh:text"; import module "oxh:kv"; declare %oracle :put %oracle-property :targetTable ('wlsserver') %oracle-property :connection.user ('OE') %oracle-property :connection.password ('OE') %oracle-property :connection.url ('jdbc:oracle:thin:@localhost:1521:orcl') function local:myPut($c1, $c2, $c3, $c4, $c5, $c6) external; for $value in kv:collection-text("/wls/log") let $split := fn:tokenize($value, ":") let $timestamp := $split[1] let $category := $split[2] let $type := $split[3] let $servername := $split[4] let $code := $split[5] let $msg := $split[6] return local: myPut($timestamp, $category, $type, $servername, $code, $msg) Run the following hadoop command to process the query script with the configuration specified using the –conf option. hadoop jar $OXH_HOME/lib/oxh.jar -conf oxh_config.xml oranosql_oradb.xq Oracle XQuery for Hadoop gets started to process the query script. The MapReduce application runs to query Oracle NoSQL Database and load the data from Oracle NOSQL Database into Oracle Database 11g. 14/05/22 15:55:50 INFO mapred.LocalJobRunner: Starting task: attempt_local924222480_0001_m_000009_0 14/05/22 15:55:50 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:55:50 INFO mapred.MapTask: Processing split: oracle.kv.hadoop.KVInputSplit@12f817b 14/05/22 15:55:50 INFO mapred.LocalJobRunner: 14/05/22 15:55:50 INFO mapred.Task: Task :attempt _local924222480_0001_m_000009_0 is done. And is in the process of committing 14/05/22 15:55:50 INFO mapred.LocalJobRunner: map 14/05/22 15:55:50 INFO mapred.Task: Task 'attempt_local924222480_0001_m_000009_0' done. 14/05/22 15:55:50 INFO mapred.LocalJobRunner: Finishing task: attempt_local924222480_0001_m_000009_0 14/05/22 15:55:50 INFO mapred.LocalJobRunner: Map task executor complete. 14/05/22 15:55:51 INFO mapreduce.Job: Job job_local924222480_0001 completed successfully 14/05/22 15:55:51 INFO mapreduce.Job: Counters: 23 File System Counters FILE: Number of bytes read=203736 FILE: Number of bytes written=249845150 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=245352030 HDFS: Number of bytes written=18705 HDFS: Number of read operations=2983 HDFS: Number of large read operations=0 HDFS: Number of write operations=88 Map-Reduce Framework Map input records=9 Map output records=0 Input split bytes=1621 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms )= 280 CPU time spent (ms )= 0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Total committed heap usage (bytes )= 347054080 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=0 14/05/22 15:55:51 INFO hadoop.xquery: Starting "oracle.hadoop.loader.OraLoader" tool, with map-reduce job "oxh:oranosql_oradb.xq#1", inputs=[hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00001.avro, hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00002.avro, hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00004.avro, hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00005.avro] ... 2more, output=hdfs://10.0.2.15:8020/tmp/oxh-root/output/myPut 14/05/22 15:55:51 INFO loader.OraLoader: Oracle Loader for Hadoop Release 3.0.0 - Production Copyright (c) 2011, 2014, Oracle and/or its affiliates. All rights reserved. 14/05/22 15:55:51 INFO loader.OraLoader: Built-Against: hadoop-2.2.0-cdh5.0.0-beta-2 hive-0.12.0-cdh5.0.0-beta-2 avro-1.7.3 jackson-1.8.8 14/05/22 15:55:51 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class 14/05/22 15:55:51 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/05/22 15:55:56 INFO loader.OraLoader: oracle.hadoop.loader.loadByPartition is disabled because table: WLSSERVER is not partitioned 14/05/22 15:55:56 INFO loader.OraLoader: oracle.hadoop.loader.enableSorting disabled, no sorting key provided 14/05/22 15:55:56 INFO loader.OraLoader: Reduce tasks set to 0 because of no partitioning or sorting. Loading will be done in the map phase. 14/05/22 15:55:56 INFO output.DBOutputFormat: Setting map tasks speculative execution to false for : oracle.hadoop.loader.lib.output.JDBCOutputFormat 14/05/22 15:55:56 INFO loader.OraLoader: Sampling time=0D :0h:0m:0s:95ms (95 ms) 14/05/22 15:55:56 INFO loader.OraLoader: Submitting OraLoader job oxh :oranosql _oradb.xq#1 14/05/22 15:55:56 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 14/05/22 15:55:57 INFO input.FileInputFormat: Total input paths to process : 6 14/05/22 15:55:57 INFO mapreduce.JobSubmitter: number of splits :6 14/05/22 15:55:57 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local2007970092_0002 14/05/22 15:56:01 INFO mapreduce.Job: The url to track the job: http://localhost:8080/ 14/05/22 15:56:01 INFO mapred.LocalJobRunner: OutputCommitter set in config null 14/05/22 15:56:01 INFO mapred.LocalJobRunner: OutputCommitter is oracle.hadoop.loader.lib.output.DBOutputCommitter 14/05/22 15:56:01 INFO mapred.LocalJobRunner: Waiting for map tasks 14/05/22 15:56:01 INFO mapred.LocalJobRunner: Starting task: attempt_local2007970092_0002_m_000000_0 14/05/22 15:56:02 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:56:02 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00001.avro:0+735 14/05/22 15:56:02 INFO output.DBOutputFormat: conf prop: defaultExecuteBatch: 100 14/05/22 15:56:02 INFO output.DBOutputFormat: conf prop: loadByPartition: false 14/05/22 15:56:02 INFO output.DBOutputFormat: Insert statement: INSERT INTO "OE"."WLSSERVER" ("TIMESTAMP", "CATEGORY", "TYPE", "SERVERNAME", "CODE", "MSG") VALUES (?, ?, ?, ?, ?, ?) 14/05/22 15:56:02 INFO mapred.LocalJobRunner: 14/05/22 15:56:02 INFO loader.OraLoader: map 0% reduce 0% 14/05/22 15:56:03 INFO mapred.Task: Task :attempt _local2007970092_0002_m_000000_0 is done. And is in the process of committing 14/05/22 15:56:03 INFO mapred.LocalJobRunner: 14/05/22 15:56:03 INFO mapred.Task: Task attempt_local2007970092_0002_m_000000_0 is allowed to commit now 14/05/22 15:56:03 INFO output.JDBCOutputFormat: Committed work for task attempt attempt_local2007970092_0002_m_000000_0 14/05/22 15:56:03 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2007970092_0002_m_000000_0' to hdfs://10.0.2.15:8020/tmp/oxh-root/output/myPut/_temporary/0/task_local2007970092_0002_m_000000 14/05/22 15:56:03 INFO mapred.LocalJobRunner: map 14/05/22 15:56:03 INFO mapred.Task: Task 'attempt_local2007970092_0002_m_000000_0' done. 14/05/22 15:56:03 INFO mapred.LocalJobRunner: Finishing task: attempt_local2007970092_0002_m_000000_0 14/05/22 15:56:03 INFO mapred.LocalJobRunner: Starting task: attempt_local2007970092_0002_m_000001_0 14/05/22 15:56:03 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:56:03 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00008.avro:0+631 14/05/22 15:56:03 INFO output.DBOutputFormat: conf prop: defaultExecuteBatch: 100 14/05/22 15:56:03 INFO output.DBOutputFormat: conf prop: loadByPartition: false 14/05/22 15:56:03 INFO output.DBOutputFormat: Insert statement: INSERT INTO "OE"."WLSSERVER" ("TIMESTAMP", "CATEGORY", "TYPE", "SERVERNAME", "CODE", "MSG") VALUES (?, ?, ?, ?, ?, ?) 14/05/22 15:56:03 INFO mapred.LocalJobRunner: 14/05/22 15:56:03 INFO loader.OraLoader: map 17% reduce 0% 14/05/22 15:56:03 INFO mapred.Task: Task :attempt _local2007970092_0002_m_000001_0 is done. And is in the process of committing 14/05/22 15:56:03 INFO mapred.LocalJobRunner: 14/05/22 15:56:03 INFO mapred.Task: Task attempt_local2007970092_0002_m_000001_0 is allowed to commit now 14/05/22 15:56:03 INFO output.JDBCOutputFormat: Committed work for task attempt attempt_local2007970092_0002_m_000001_0 14/05/22 15:56:04 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2007970092_0002_m_000001_0' to hdfs://10.0.2.15:8020/tmp/oxh-root/output/myPut/_temporary/0/task_local2007970092_0002_m_000001 14/05/22 15:56:04 INFO mapred.LocalJobRunner: map 14/05/22 15:56:04 INFO mapred.Task: Task 'attempt_local2007970092_0002_m_000001_0' done. 14/05/22 15:56:04 INFO mapred.LocalJobRunner: Finishing task: attempt_local2007970092_0002_m_000001_0 14/05/22 15:56:04 INFO mapred.LocalJobRunner: Starting task: attempt_local2007970092_0002_m_000002_0 14/05/22 15:56:04 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:56:04 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00005.avro:0+518 14/05/22 15:56:04 INFO output.DBOutputFormat: conf prop: defaultExecuteBatch: 100 14/05/22 15:56:04 INFO output.DBOutputFormat: conf prop: loadByPartition: false 14/05/22 15:56:04 INFO output.DBOutputFormat: Insert statement: INSERT INTO "OE"."WLSSERVER" ("TIMESTAMP", "CATEGORY", "TYPE", "SERVERNAME", "CODE", "MSG") VALUES (?, ?, ?, ?, ?, ?) 14/05/22 15:56:04 INFO mapred.LocalJobRunner: 14/05/22 15:56:04 INFO mapred.Task: Task :attempt _local2007970092_0002_m_000002_0 is done. And is in the process of committing 14/05/22 15:56:04 INFO mapred.LocalJobRunner: 14/05/22 15:56:04 INFO mapred.Task: Task attempt_local2007970092_0002_m_000002_0 is allowed to commit now 14/05/22 15:56:04 INFO output.JDBCOutputFormat: Committed work for task attempt attempt_local2007970092_0002_m_000002_0 14/05/22 15:56:04 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2007970092_0002_m_000002_0' to hdfs://10.0.2.15:8020/tmp/oxh-root/output/myPut/_temporary/0/task_local2007970092_0002_m_000002 14/05/22 15:56:04 INFO mapred.LocalJobRunner: map 14/05/22 15:56:04 INFO mapred.Task: Task 'attempt_local2007970092_0002_m_000002_0' done. 14/05/22 15:56:04 INFO mapred.LocalJobRunner: Finishing task: attempt_local2007970092_0002_m_000002_0 14/05/22 15:56:04 INFO mapred.LocalJobRunner: Starting task: attempt_local2007970092_0002_m_000003_0 14/05/22 15:56:04 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:56:04 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00002.avro:0+458 14/05/22 15:56:04 INFO loader.OraLoader: map 100% reduce 0% 14/05/22 15:56:05 INFO output.DBOutputFormat: conf prop: defaultExecuteBatch: 100 14/05/22 15:56:05 INFO output.DBOutputFormat: conf prop: loadByPartition: false 14/05/22 15:56:05 INFO output.DBOutputFormat: Insert statement: INSERT INTO "OE"."WLSSERVER" ("TIMESTAMP", "CATEGORY", "TYPE", "SERVERNAME", "CODE", "MSG") VALUES (?, ?, ?, ?, ?, ?) 14/05/22 15:56:05 INFO mapred.LocalJobRunner: 14/05/22 15:56:05 INFO mapred.Task: Task :attempt _local2007970092_0002_m_000003_0 is done. And is in the process of committing 14/05/22 15:56:05 INFO mapred.LocalJobRunner: 14/05/22 15:56:05 INFO mapred.Task: Task attempt_local2007970092_0002_m_000003_0 is allowed to commit now 14/05/22 15:56:05 INFO output.JDBCOutputFormat: Committed work for task attempt attempt_local2007970092_0002_m_000003_0 14/05/22 15:56:05 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2007970092_0002_m_000003_0' to hdfs://10.0.2.15:8020/tmp/oxh-root/output/myPut/_temporary/0/task_local2007970092_0002_m_000003 14/05/22 15:56:05 INFO mapred.LocalJobRunner: map 14/05/22 15:56:05 INFO mapred.Task: Task 'attempt_local2007970092_0002_m_000003_0' done. 14/05/22 15:56:05 INFO mapred.LocalJobRunner: Finishing task: attempt_local2007970092_0002_m_000003_0 14/05/22 15:56:05 INFO mapred.LocalJobRunner: Starting task: attempt_local2007970092_0002_m_000004_0 14/05/22 15:56:05 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:56:05 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00006.avro:0+458 14/05/22 15:56:06 INFO output.DBOutputFormat: conf prop: defaultExecuteBatch: 100 14/05/22 15:56:06 INFO output.DBOutputFormat: conf prop: loadByPartition: false 14/05/22 15:56:06 INFO output.DBOutputFormat: Insert statement: INSERT INTO "OE"."WLSSERVER" ("TIMESTAMP", "CATEGORY", "TYPE", "SERVERNAME", "CODE", "MSG") VALUES (?, ?, ?, ?, ?, ?) 14/05/22 15:56:06 INFO mapred.LocalJobRunner: 14/05/22 15:56:06 INFO mapred.Task: Task :attempt _local2007970092_0002_m_000004_0 is done. And is in the process of committing 14/05/22 15:56:06 INFO mapred.LocalJobRunner: 14/05/22 15:56:06 INFO mapred.Task: Task attempt_local2007970092_0002_m_000004_0 is allowed to commit now 14/05/22 15:56:06 INFO output.JDBCOutputFormat: Committed work for task attempt attempt_local2007970092_0002_m_000004_0 14/05/22 15:56:06 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2007970092_0002_m_000004_0' to hdfs://10.0.2.15:8020/tmp/oxh-root/output/myPut/_temporary/0/task_local2007970092_0002_m_000004 14/05/22 15:56:06 INFO mapred.LocalJobRunner: map 14/05/22 15:56:06 INFO mapred.Task: Task 'attempt_local2007970092_0002_m_000004_0' done. 14/05/22 15:56:06 INFO mapred.LocalJobRunner: Finishing task: attempt_local2007970092_0002_m_000004_0 14/05/22 15:56:06 INFO mapred.LocalJobRunner: Starting task: attempt_local2007970092_0002_m_000005_0 14/05/22 15:56:06 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ] 14/05/22 15:56:06 INFO mapred.MapTask: Processing split: hdfs://10.0.2.15:8020/tmp/oxh-root/scratch/fd213abd-3949-40f1-aad2-2e37b305b1c0.0/OXHI0xmyPut-m-00004.avro:0+457 14/05/22 15:56:06 INFO output.DBOutputFormat: conf prop: defaultExecuteBatch: 100 14/05/22 15:56:06 INFO output.DBOutputFormat: conf prop: loadByPartition: false 14/05/22 15:56:06 INFO output.DBOutputFormat: Insert statement: INSERT INTO "OE"."WLSSERVER" ("TIMESTAMP", "CATEGORY", "TYPE", "SERVERNAME", "CODE", "MSG") VALUES (?, ?, ?, ?, ?, ?) 14/05/22 15:56:06 INFO mapred.LocalJobRunner: 14/05/22 15:56:07 INFO mapred.Task: Task :attempt _local2007970092_0002_m_000005_0 is done. And is in the process of committing 14/05/22 15:56:07 INFO mapred.LocalJobRunner: 14/05/22 15:56:07 INFO mapred.Task: Task attempt_local2007970092_0002_m_000005_0 is allowed to commit now 14/05/22 15:56:07 INFO output.JDBCOutputFormat: Committed work for task attempt attempt_local2007970092_0002_m_000005_0 14/05/22 15:56:07 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2007970092_0002_m_000005_0' to hdfs://10.0.2.15:8020/tmp/oxh-root/output/myPut/_temporary/0/task_local2007970092_0002_m_000005 14/05/22 15:56:07 INFO mapred.LocalJobRunner: map 14/05/22 15:56:07 INFO mapred.Task: Task 'attempt_local2007970092_0002_m_000005_0' done. 14/05/22 15:56:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local2007970092_0002_m_000005_0 14/05/22 15:56:07 INFO mapred.LocalJobRunner: Map task executor complete. 14/05/22 15:56:08 INFO loader.OraLoader: Job complete: oxh :oranosql _oradb.xq#1 (job_local2007970092_0002) 14/05/22 15:56:08 INFO loader.OraLoader: Counters: 23 File System Counters FILE: Number of bytes read=4096263 FILE: Number of bytes written=279525168 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=270330126 HDFS: Number of bytes written=125720 HDFS: Number of read operations=3798 HDFS: Number of large read operations=0 HDFS: Number of write operations=246 Map-Reduce Framework Map input records=9 Map output records=9 Input split bytes=1026 Spilled Records=0 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms )= 36 CPU time spent (ms )= 0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Total committed heap usage (bytes )= 208232448 File Input Format Counters Bytes Read=6514 File Output Format Counters Bytes Written=9662 14/05/22 15:56:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/05/22 15:56:08 INFO hadoop.xquery: Finished executing "oranosql_oradb.xq". Output path: "hdfs://10.0.2.15:8020/tmp/oxh-root/output" Run a SQL SELECT statement in SQL Plus to list the data loaded into Oracle NoSQL Database. The 9 rows of data from Oracle NoSQL Database, which is loaded into Oracle Database gets listed. The output from the SQL query is listed: SQL> SELECT * FROM OE.WLSSERVER; TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- Apr-8-2014-7-06-19-PM-PDT Notice WebLogicServer AdminServer BEA-000365 Server state changed to ADMIN TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- Apr-8-2014-7-06-22-PM-PDT Notice WebLogicServer AdminServer BEA-000331 Started WebLogic Admin Server AdminServer for domain base_domain running in Deve lopment Mode TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- Apr-8-2014-7-06-23-PM-PDT Notice WebLogicServer AdminServer BEA-000365 Server state changed to RUNNING TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- Apr-8-2014-7-06-21-PM-PDT Notice Server AdminServer BEA-002613 Channel Default is now listening on fe80-0-0-0-0-5efe-c0a8-147-7001 for protocol TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- s iiop,t3,ldap,snmp,http Apr-8-2014-7-06-24-PM-PDT Notice WebLogicServer AdminServer BEA-000360 TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- Server started in RUNNING mode Apr-8-2014-7-06-18-PM-PDT Notice Log Management AdminServer BEA-170027 TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- The Server has established connection with the Domain level Diagnostic Service s uccessfully Apr-8-2014-7-06-20-PM-PDT Notice WebLogicServer AdminServer TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- BEA-000365 Server state changed to RESUMING Apr-8-2014-7-06-17-PM-PDT Notice WebLogicServer AdminServer TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- BEA-000365 Server state changed to STARTING Apr-8-2014-7-06-16-PM-PDT Notice WebLogicServer AdminServer TIMESTAMP CATEGORY ------------------------------------------------------- --------------- TYPE SERVERNAME ------------------------------------------------------- --------------- CODE --------------- MSG -------------------------------------------------------------------------------- BEA-000365 Server state changed to STANDBY 9 rows selected SQL> In this article we used the Oracle NoSQL adapter in the Oracle XQuery for Hadoop 3.0.0 to store text data in Oracle NoSQL Database and subsequently query Oracle NoSQL Database to load the text data into Oracle Database 11 g.
↧
