Hadoop job is failing with vertex failure when executed this in Diyotta , how can I overcome the vertex error

One of the Hadoop jobflow is failing with vertex issue error asĀ  part of the execution, This jobflow is having data ranges included as part of dataflow.

ERROR:Failed to Execute NativeSQL:: NativeSQL execution failed:Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1585896126081_186867_1_00, diagnostics=[Task failed, taskId=task_1585896126081_186867_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_e386_1585896126081_186867_01_000002 finished with diagnostics set to [Container failed, exitCode=1. Exception from container-launch.

Advanced User Asked on November 6, 2020 in Diyotta Monitor.
Add Comment
1 Answer(s)

John,

Can you try to include/add these optimization parameters in your jobflow and then try to run the jobflow.

set hive.tez.container.size=12288;

set hive.tez.java.opts=-Xmx9830m;

The vertex issue was causing as the container.size optimization property was missing at the Hadoop datapoint and jobflow optimization properties.

RE: Hadoop job is failing with vertex failure when executed this in Diyotta , how can I overcome the vertex error

Expert Answered on November 6, 2020.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.