How do I deal with spaces in my source Splice Machine table column names?
My Splice Machine source table has multiple column names that include spaces (e.g. First Name, Last Name, etc.). How do I work around that in Diyotta? I have tried several things, but my data flow continues to fail when it encounters First Name.
Hi Breiney,
In Diyotta we do handle the spaces and reserved keywords as column names in the Table, by enabling the option Strict Name = YES.
which is currently included in the MSSQL, will add an enhancement to include the same property for Splice Machine.
MSSQL
As a workaround, please follow the below steps
- Create a job flow, for the splice machine dataflow and click on dataflow transforms.
- Open the override command present for the target as my dataflow is having only the source and target present in it.
- Enclose the column where there are spaces in double quotes.
Then try to execute the job flow.
I have a follow up to this question. Now that I have embedded the data flow within a job flow and made the override in the command, I have additional formulas and changes to the data flow that need to be made. I have made one of them to add an expression to calculate a new column value and updated the mapping within the Data Flow.
My issue now is that I cannot figure out how to get the Job Flow to recognize the changes to the Data Flow short of removing the Data Flow and re-adding it and then re-customizing the command to deal with the spaces in the column names. This is making iterative development difficult.
Is there a better way to get the Job Flow to see the updates in the Data Flow?
Hi Breiney,
Yeah, we should be able to override the query at the source instance present in the dataflow, as we have done it in the Job flow.
once we add double quotes to actual column names, the subsequent transformations present in the dataflow uses the source alias names
currently, we are having issues at source instance-level override.
Job override should still work, even if there are changes in the dataflow level for the changes made in the dataflow to be affected in the design job. Follow below steps
1. I added expression in my dataflow and saved it.
2. Then move to the Jobflow and click on validate which would update the design job with added transformations and then save it.
3. Once the save is done, the dataflow transforms shows the added transformation in the designjob.
4. Then click on the revert, and select the override icon to get the updated query in the target instance with added transformation in between.
5. Then try to put double quotes for columns at source columns only , the subsequent transformations added in the dataflow uses the source table column alias name.
6. Then try to execute the job flow.
