How do I deal with spaces in my source Splice Machine table column names?

Answered

My Splice Machine source table has multiple column names that include spaces (e.g. First Name, Last Name, etc.). How do I work around that in Diyotta? I have tried several things, but my data flow continues to fail when it encounters First Name.

Beginner Asked on September 23, 2019 in Diyotta Studio.
Add Comment
2 Answer(s)
Best answer

Hi Breiney,

In Diyotta we do handle the spaces and reserved keywords as column names in the Table, by enabling the option Strict Name = YES.

which is currently included in the MSSQL, will add an enhancement to include the same property for Splice Machine.

MSSQL

RE: How do I deal with spaces in my source Splice Machine table column names?

As a workaround, please follow the below steps

  1. Create a job flow, for the splice machine dataflow and click on dataflow transforms.
  2. Open the override command present for the target as my dataflow is having only the source and target present in it.
  3. Enclose the column where there are spaces in double quotes.

RE: How do I deal with spaces in my source Splice Machine table column names?

Then try to execute the job flow.

 

Expert Answered on September 24, 2019.

I have a follow up to this question. Now that I have embedded the data flow within a job flow and made the override in the command, I have additional formulas and changes to the data flow that need to be made. I have made one of them to add an expression to calculate a new column value and updated the mapping within the Data Flow.

My issue now is that I cannot figure out how to get the Job Flow to recognize the changes to the Data Flow short of removing the Data Flow and re-adding it and then re-customizing the command to deal with the spaces in the column names. This is making iterative development difficult.

Is there a better way to get the Job Flow to see the updates in the Data Flow?

on September 24, 2019.
Add Comment

Hi Breiney,

Yeah, we should be able to override the query at the source instance present in the dataflow, as we have done it in the Job flow.

RE: How do I deal with spaces in my source Splice Machine table column names?

once we add double quotes to actual column names, the subsequent transformations present in the dataflow uses the source alias names

RE: How do I deal with spaces in my source Splice Machine table column names?

currently, we are having issues at source instance-level override.

Job override should still work, even if there are changes in the dataflow level for the changes made in the dataflow to be affected in the design job. Follow below steps

1. I added expression in my dataflow and saved it.

RE: How do I deal with spaces in my source Splice Machine table column names?

2. Then move to the Jobflow and click on validate which would update the design job with added transformations and then save it.

RE: How do I deal with spaces in my source Splice Machine table column names?

3. Once the save is done, the dataflow transforms shows the added transformation in the designjob.

RE: How do I deal with spaces in my source Splice Machine table column names?

4. Then click on the revert, and select the override icon to get the updated query in the target instance with added transformation in between.

5. Then try to put double quotes for columns at source columns only , the subsequent transformations added in the dataflow uses the source table column alias name.

6. Then try to execute the job flow.

RE: How do I deal with spaces in my source Splice Machine table column names?

 

Expert Answered on September 25, 2019.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.