How to load data into Azure blob storage from Snow flake using Diyotta?
How to load data into Azure blob storage from Snow flake using Diyotta?
Assume that Snowflake cloud platform is azure.
- Diyotta extracting snowflake data into snowflake stage location using COPY command as below.
COPY INTO @~/tmp/tgt_867166_1357430 FROM (SELECT
SRC_CUSTOMER.CUSTOMER_ID AS CUSTOMER_ID,
coalesce(SRC_CUSTOMER.CUSTOMER_SEGMENT_ID,”) AS CUSTOMER_SEGMENT_ID,
coalesce(SRC_CUSTOMER.FIRST_NAME,”) AS FIRST_NAME,
coalesce(SRC_CUSTOMER.LAST_NAME,”) AS LAST_NAME,
coalesce(SRC_CUSTOMER.ADDRESS_LINE_1,”) AS ADDRESS_LINE_1,
coalesce(SRC_CUSTOMER.ADDRESS_LINE_2,”) AS ADDRESS_LINE_2,
coalesce(SRC_CUSTOMER.CITY,”) AS CITY,
coalesce(SRC_CUSTOMER.STATE,”) AS STATE,
coalesce(SRC_CUSTOMER.ZIPCODE,”) AS ZIPCODE,
coalesce(SRC_CUSTOMER.COUNTRY,”) AS COUNTRY,
coalesce(SRC_CUSTOMER.EMAIL,”) AS EMAIL,
coalesce(SRC_CUSTOMER.PHONE,”) AS PHONE
FROM DEMO_DB.public.CUSTOMER SRC_CUSTOMER
) FILE_FORMAT=( type=’CSV’ COMPRESSION=’AUTO’ FIELD_DELIMITER=’|’ ESCAPE=’NONE’ ESCAPE_UNENCLOSED_FIELD=’NONE’ F IELD_OPTIONALLY_ENCLOSED_BY=’NONE’ DATE_FORMAT=’AUTO’ TIME_FORMAT=’AUTO’ TIMESTAMP_FORMAT=’AUTO’ BINARY_FORMAT=’UTF-8′ NULL_IF=” EMPTY_FIELD_AS_NULL=FALSE ) OVERWRITE=FALSE SINGLE=FALSE MAX_FILE_SIZE=16000000
- From that snowflake stage location downloading into diyotta stage location using GET command as below.
GET @~/tmp/tgt_867166_1357430 file:////app/cqacert/diyotta/agent/app/stage/tgt_867166_1357430 PARALLEL=4 ;
- Removing the stage location from snowflake cloud using remove command.
remove @~/tmp/tgt_867166_1357430;
- By using snowflake API uploading diyotta stage file to azure cloud location with user provided location and file name.
