DataFlow error – null value

Hi Team,

Please help us to resolve this error.

Thanking you in advance.

 

Regards,

Rama.

Error Details:

===========

INFO 11/25/2020 02:43:55 Test_DF_2 Controller Job Initializing [Run Id : 79711, Name : Test_DF_2, Run By : Kiran.Katkuri, JobId : 147981, AuditId: 79711]
INFO 11/25/2020 02:43:55 Test_DF_2 Controller Execution of job started
INFO 11/25/2020 02:43:55 Test_DF_2 Controller Status : Queue
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller DISuite Version [Controller] = 4.1.0.3112.000
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$$ARCHFILEDIR : ArchFileName]
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced params for RuntimeStats/statistics/others are
[$$LIBFILEDIR : -1 ]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$TFORM_SRC_DC_TRANSFERS : temp.T_SRC_DC_TRANSFERS_$$AUDITID_$$JOBID]
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$$ARCHFILEDIR : ArchFileName]
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$$ARCHFILEDIR : ArchFileName]
[$$AUDITID : 79711]
[$$UNITID : 1000]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$TFORM_SRC_DC_TRANSFERS : temp.T_SRC_DC_TRANSFERS_$$AUDITID_$$JOBID]
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$TFORM_SRC_DC_TRANSFERS : temp.T_SRC_DC_TRANSFERS_$$AUDITID_$$JOBID]
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:55 dt_transer_1000 1000 Controller Replaced Params for Unit: dt_transer_1000 are
[$TFORM_SRC_DC_TRANSFERS : temp.T_SRC_DC_TRANSFERS_$$AUDITID_$$JOBID]
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:56 TGT_FGL_POS_SHIPMENT 3 Controller DISuite Version [Controller] = 4.1.0.3112.000
INFO 11/25/2020 02:43:56 TGT_FGL_POS_SHIPMENT 3 Controller Replaced Params for Unit: TGT_FGL_POS_SHIPMENT are
[$DO_TEST_HV_DP_HADOOP_DEV_HV_FGL_POS_SHIPMENT_FGL_POS_SHIPMENT : FGL_POS_SHIPMENT.FGL_POS_SHIPMENT]
INFO 11/25/2020 02:43:56 TGT_FGL_POS_SHIPMENT 3 Controller Replaced Params for Unit: TGT_FGL_POS_SHIPMENT are
[$DO_TEST_HV_DP_HADOOP_DEV_HV_FGL_POS_SHIPMENT_FGL_POS_SHIPMENT : FGL_POS_SHIPMENT.FGL_POS_SHIPMENT]
[$TFORM_SRC_DC_TRANSFERS : temp.T_SRC_DC_TRANSFERS_$$AUDITID_$$JOBID]
[$$AUDITID : 79711]
[$$JOBID : 147981]
INFO 11/25/2020 02:43:56 TGT_FGL_POS_SHIPMENT 3 Controller Replaced Params for Unit: TGT_FGL_POS_SHIPMENT are
[$DO_TEST_HV_DP_HADOOP_DEV_HV_FGL_POS_SHIPMENT_FGL_POS_SHIPMENT : FGL_POS_SHIPMENT.FGL_POS_SHIPMENT]
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller Execution of Unit started
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller Unit Status : Queue
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller Unit Properties :
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller [Extract Type : FILE]
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller [Extraction Mode : File]
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller [Load Type : HVLOAD]
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller Unit Status : Active
INFO 11/25/2020 02:43:56 Test_DF_2 Controller Status : Active
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 Controller Load balance with Agent:[diagent1] from Agent Group : DIAgentWPSecured [diagent2,diagent1]
INFO 11/25/2020 02:43:56 dt_transer_1000 Controller Sending XML extract command
to Agent diagent2
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 Received xsd extract command from controller
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 ConnInfo [Name=Test_RK_DP, Host=, Port=, Database=, Schema=, User=,Id:147957]
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 Execution of xsd extract command started
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 Command –
xjc -d /apps/diagnt2/diyotta/agent/app/stage -p com.diyotta.xml147979 /apps/diagnt2/diyotta/agent/app/lib/147979.xsd
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 parsing a schema…
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 compiling a schema…
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 com/diyotta/xml147979/DCTRANSFERS.java
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 com/diyotta/xml147979/ObjectFactory.java
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 com/diyotta/xml147979/package-info.java
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 ExitCode:0
INFO 11/25/2020 02:43:56 dt_transer_1000 1000 diagent2 Command –
javac -d /apps/diagnt2/diyotta/agent/app/stage/com/diyotta/xml147979 /apps/diagnt2/diyotta/agent/app/stage/com/diyotta/xml147979/*.java
INFO 11/25/2020 02:43:57 dt_transer_1000 1000 diagent2 Note: /apps/diagnt2/diyotta/agent/app/stage/com/diyotta/xml147979/XMLHandler.java uses unchecked or unsafe operations.
INFO 11/25/2020 02:43:57 dt_transer_1000 1000 diagent2 Note: Recompile with -Xlint:unchecked for details.
INFO 11/25/2020 02:43:57 dt_transer_1000 1000 diagent2 ExitCode:0
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 Command –
jar -cvf /apps/diagnt2/diyotta/agent/app/lib/147979.jar -C /apps/diagnt2/diyotta/agent/app/stage/com/diyotta/xml147979 com
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 added manifest
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/(in = 0) (out= 0)(stored 0%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/diyotta/(in = 0) (out= 0)(stored 0%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/diyotta/xml147979/(in = 0) (out= 0)(stored 0%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/diyotta/xml147979/package-info.class(in = 355) (out= 264)(deflated 25%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/diyotta/xml147979/DCTRANSFERS$DCTRANSFER.class(in = 1078) (out= 576)(deflated 46%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/diyotta/xml147979/DCTRANSFERS.class(in = 1010) (out= 496)(deflated 50%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/diyotta/xml147979/ObjectFactory.class(in = 684) (out= 368)(deflated 46%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 adding: com/diyotta/xml147979/XMLHandler.class(in = 18091) (out= 8563)(deflated 52%)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 ExitCode:0
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 Command –
java -cp /apps/diagnt2/diyotta/agent/server/agent/lib/difla.jar:/apps/diagnt2/diyotta/agent/server/agent/lib/logback-slf4j-api-1.7.5.jar:/apps/diagnt2/diyotta/agent/server/agent/lib/logback-classic-1.0.12.jar:/apps/diagnt2/diyotta/agent/server/agent/lib/logback-core-1.0.12.jar:/apps/diagnt2/diyotta/agent/server/agent/lib/logback-jcl-over-slf4j-1.7.5.jar:/apps/diagnt2/diyotta/agent/server/agent/lib/commons-io-2.4.jar:/apps/diagnt2/diyotta/agent/server/agent/lib/commons-lang-2.5.jar:/apps/diagnt2/diyotta/agent/app/lib/147979.jar: -Xms1024m -Xmx4096m com.diyotta.fla.util.XMLIn /apps/diagnt2/diyotta/agent/app/stage/E::LOCATION::_79711_147981 /apps/dev/rfgl/landing/ “rvfl_DCTransfer_88888.xml” 147979 “,*****” /apps/dev/rfgl/landing//rvfl_DCTransfer_88888.xml “” “” null “” OFF “Do not replace” 79711_147981
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.369 [main] DEBUG com.diyotta.fla.util.XMLIn – csvFileLoc:: /apps/diagnt2/diyotta/agent/app/stage/E::LOCATION::_79711_147981 ,xmlFileLoc:: /apps/dev/rfgl/landing/ ,xmlFileNm:: rvfl_DCTransfer_88888.xml , dObj_Id::147979,colDelimiter::,,escapeChar::,natvType::null,lfInString::OFF,lfInStringVal::Do not replace
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.378 [main] DEBUG com.diyotta.fla.util.DIFunctions – Delimiter is already in TEXT Format
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.378 [main] DEBUG com.diyotta.fla.util.DIFunctions – ASCII value for Decimal Delim :,
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.378 [main] DEBUG com.diyotta.fla.util.DIFunctions – Out::,
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.378 [main] DEBUG com.diyotta.fla.util.DIFunctions – Delimiter is already in TEXT Format
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.378 [main] DEBUG com.diyotta.fla.util.DIFunctions – ASCII value for Decimal Delim :
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.379 [main] DEBUG com.diyotta.fla.util.DIFunctions – Out::
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.379 [main] DEBUG com.diyotta.fla.util.DIFunctions – Delimiter is already in TEXT Format
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.379 [main] DEBUG com.diyotta.fla.util.DIFunctions – ASCII value for Decimal Delim :
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.379 [main] DEBUG com.diyotta.fla.util.DIFunctions – Out::
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 Input xml file: /apps/dev/rfgl/landing//rvfl_DCTransfer_88888.xml
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 DC_TRANSFER
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 Input XMLs conversion to flatfiles completed successfully..
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 02:43:58.417 [main] INFO com.diyotta.fla.util.XMLIn – Out ::File Creation done.
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 ExitCode:0
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 Stage file :[ /apps/diagnt2/diyotta/agent/app/stage/EDC_TRANSFERS1_79711_147981.csv ] transferring to remote agent location :[ ##AgentStgFileDir/EDC_TRANSFERS1_79711_147981.csv ]
ERROR 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 Connection establishment with target agent host:10.18.57.224 and port: failed with error:For input string: “” at attempt 1
ERROR 11/25/2020 02:43:58 dt_transer_1000 1000 diagent2 ERROR_TRACE:EFA00048:Socket connection error.:For input string: “”
com.diyotta.fla.util.FileSender.createChannel(FileSender.java:146)
com.diyotta.fla.util.FileSender.sendFile(FileSender.java:159)
com.diyotta.fla.util.FileTransferUtil.nioFileTransfer(FileTransferUtil.java:341)
com.diyotta.fla.util.FileTransferUtil.transferFlatFileToRemoteAgent(FileTransferUtil.java:116)
com.diyotta.fla.execute.sql.ExecuteXsdExtract.execute(ExecuteXsdExtract.java:170)
com.diyotta.fla.execute.sql.ExecuteXsdExtract.run(ExecuteXsdExtract.java:110)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 Controller Status : Failed
ERROR 11/25/2020 02:43:58 dt_transer_1000 1000 Controller SubUnit execution completed :[Status: F ],[ReturnCd: EFA00048 ],[ReturnMsg: Socket connection error.:Socket connection error.:For input string: “” ]
INFO 11/25/2020 02:43:58 dt_transer_1000 1000 Controller Unit Status : Failed
ERROR 11/25/2020 02:43:58 dt_transer_1000 1000 Controller Unit execution completed :[Status: F ],[ReturnCd: EFA00048 ],[ReturnMsg: Socket connection error.:Socket connection error.:For input string: “” ]
ERROR 11/25/2020 02:43:58 TGT_FGL_POS_SHIPMENT 3 Controller Unit Execution Defered :[Status: D ],[ReturnCd: EPE00015 ],[ReturnMsg: Dependency Fail in dt_transer_1000 ]
INFO 11/25/2020 02:43:58 Test_DF_2 Controller Status : Failed
INFO 11/25/2020 02:43:58 Test_DF_2 Controller Execution of job failed
INFO 11/25/2020 02:43:58 Test_DF_2 Controller Sending HDFS files cleanup command to Agent :Agent [AgentNm=diagent1, AgentHost=localhost, Agent Data Receiver Service=N, DISuite Version [Agent] = 4.1.0.3121.021]
INFO 11/25/2020 02:43:58 Test_DF_2 diagent1 Received hadoop file cleanUp command from controller
INFO 11/25/2020 02:43:58 Test_DF_2 diagent1 Execution of hadoop file cleanUp command started
INFO 11/25/2020 02:43:58 Test_DF_2 diagent1 Registering agent service HDP_2.6.4_svc.diyotta@LABCORP.AD.CTC on port:9911
INFO 11/25/2020 02:43:58 Test_DF_2 diagent1 Received hadoop file cleanUp command from controller
INFO 11/25/2020 02:43:58 Test_DF_2 diagent1 Execution of hadoop file cleanUp command started
INFO 11/25/2020 02:44:11 Test_DF_2 diagent1 Agent service HDP_2.6.4_svc.diyotta@LABCORP.AD.CTC registered successfully with port :9911
INFO 11/25/2020 02:44:11 Test_DF_2 diagent1 Hadoop temp file cleanup request sent to agent service
INFO 11/25/2020 02:44:11 Test_DF_2 diagent1 Agent service already registered with port :9911, Hence using the existing service HDP_2.6.4_svc.diyotta@LABCORP.AD.CTC
INFO 11/25/2020 02:44:11 Test_DF_2 diagent1 Hadoop temp file cleanup request sent to agent service
INFO 11/25/2020 02:44:11 Test_DF_2 diagent1_agent_service Hadoop Temp File delete request received from agent
INFO 11/25/2020 02:44:11 Test_DF_2 diagent1_agent_service Hadoop Temp File delete request received from agent
INFO 11/25/2020 02:44:12 Test_DF_2 diagent1_agent_service Hadoop file handling process initiated
INFO 11/25/2020 02:44:12 Test_DF_2 diagent1_agent_service Hadoop file handling process initiated
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Deleted ::/user/svc.diyotta/tempId_SRC_DC_TRANSFERS_2_79711_147981
Hadoop temp file cleanup completed
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Deleted ::/user/svc.diyotta/tempId_SRC_DC_TRANSFERS_2_79711_147981
Hadoop temp file cleanup completed
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Execution hadoop file cleanUp command completed successfully
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Sending hadoop file cleanUp command result to controller
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Received hadoop file cleanUp command result from Agent [AgentNm=diagent1]
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Execution hadoop file cleanUp command completed successfully
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Sending hadoop file cleanUp command result to controller
INFO 11/25/2020 02:44:15 Test_DF_2 diagent1 Received hadoop file cleanUp command result from Agent [AgentNm=diagent1]

 

==================

Error details are :

Getting Input invalid or null error

=============

ERROR 11/25/2020 02:20:43 dt_transer_1000 1000 diagent2 Connection establishment with target agent host:10.18.57.224 and port: failed with error:For input string: “” at attempt 1
ERROR 11/25/2020 02:20:43 dt_transer_1000 1000 diagent2 ERROR_TRACE:EFA00048:Socket connection error.:For input string: “”
com.diyotta.fla.util.FileSender.createChannel(FileSender.java:146)
com.diyotta.fla.util.FileSender.sendFile(FileSender.java:159)
com.diyotta.fla.util.FileTransferUtil.nioFileTransfer(FileTransferUtil.java:341)
com.diyotta.fla.util.FileTransferUtil.transferFlatFileToRemoteAgent(FileTransferUtil.java:116)
com.diyotta.fla.execute.sql.ExecuteXsdExtract.execute(ExecuteXsdExtract.java:170)
com.diyotta.fla.execute.sql.ExecuteXsdExtract.run(ExecuteXsdExtract.java:110)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
ERROR 11/25/2020 02:20:43 dt_transer_1000 1000 Controller SubUnit execution completed :[Status: F ],[ReturnCd: EFA00048 ],[ReturnMsg: Socket connection error.:Socket connection error.:For input string: “” ]
ERROR 11/25/2020 02:20:43 dt_transer_1000 1000 Controller Unit execution completed :[Status: F ],[ReturnCd: EFA00048 ],[ReturnMsg: Socket connection error.:Socket connection error.:For input string: “” ]

 

DataFlow error - null valueSource XML – target Hive. –

Beginner Asked on November 25, 2020 in Diyotta Studio.
Add Comment
1 Answer(s)

Hi Ramakrishna,

Can you please do a test Datapoint on HIVE and let us know if it working as expected.

RE: DataFlow error - null value

Expert Answered on November 25, 2020.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.