RE: When we create a Data Object from the file, we notice that the delimiters are not correct.

Currently having this issue with DB Command from job flow. The query is executing successfully and loading the data into a .dat file. However, when we want to create a Data Object from the file, we notice that the delimiters are not correct. Note: They are correct during the Show Preview section but not after that part.

Can you please provide some insight into how we can for example specify a delimiter like “||” or any special character that can make it easier to differentiate the columns in the Data Object phase.

 

RE: When we create a Data Object from the file, we notice that the delimiters are not correct.

Our use case is to load the data from DB2 source to BigQuery target after the DBCMD

TommyM Beginner Asked on April 21, 2021 in Diyotta Studio.
Add Comment
1 Answers

The suggested approach is to use the dataflow instead of DBCMD for pushing the file into the Big Query as we are inserting this into BigQuery and not into a Filesystem. Also in dataflow, you can include the column delimiters and text qualifiers as part of the extract properties.

For the use case: SELECT SQL in DBCMD and using that file to push it into a big query. You can use the below approach.

create a dataflow with source [DB2] and target [big query] and include the select statement in override query as shown below

RE: When we create a Data Object from the file, we notice that the delimiters are not correct.

Expert Answered on April 22, 2021.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.