RE: Hadoop job is failing with vertex failure when executed this in Diyotta , how can I overcome the vertex error
One of the Hadoop jobflow is failing with vertex issue error as part of the execution, This jobflow is having data ranges included as part of dataflow.
ERROR:Failed to Execute NativeSQL:: NativeSQL execution failed:Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1585896126081_186867_1_00, diagnostics=[Task failed, taskId=task_1585896126081_186867_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_e386_1585896126081_186867_01_000002 finished with diagnostics set to [Container failed, exitCode=1. Exception from container-launch.
John,
Can you try to include/add these optimization parameters in your jobflow and then try to run the jobflow.
set hive.tez.container.size=12288;
set hive.tez.java.opts=-Xmx9830m;
The vertex issue was causing as the container.size optimization property was missing at the Hadoop datapoint and jobflow optimization properties.
