This process involves creating a Parquet Data Source based on information sources and other views, and loading a .parquet file into the Azure Data Lake Gen 2 Storage folder.
Add new Parquet Data Source:
Through IFS Cloud Web, Parquet Data Sources can be created and saved. Selection of the Source Origin and Load Type, required columns can be done depending on the requirement.
Load Parquet Data Source:
Once the Parquet data source is created, performing an Explicit Load is necessary to create .parquet files in the Azure Data Lake Gen 2 Storage. During the source creation, Load Type can be selected, and during the Parquet Data Source refresh process, .parquet files are created based on the specified Load Type.
Edit existing Parquet Data Source:
Once the Parquet Data Source is Loaded, there is an option to Edit the Parquet Data Sources.
Once the editing is completed, performing an Explicit Load via IFS Cloud Web is a must, to reflect the changes in the ADLS Gen 2.
Import/Export Parquet Data Source:
Loaded Parquet Data Sources can be exported from one environment and imported to another environment (From DEV ,UAT/PROD).