Diyotta leaves this option to the user, either way is possible. Working with multiple different versions of Spark is also possible with Diyotta. Data point for Spark has these options “Dedicated Spark Application” and “Separate Spark Application for each Data Flow”.

• Choosing only the first option will spin off a separate agent service per data point, regardless of whether there is another data point that uses the same Spark version. All data flows using this data point will be sent to this service
• Leaving both options unchecked creates only one spark agent service per Spark version even if there are multiple different data points
• Choosing both options will create one spark agent service per data flow execution that uses this data point