ETL in three different countries across the globe, the

ETL ( Extract, transform and load) determines
how and when the source data moves. While extracting and loading is simply
moving the data, the process of transforming the data is more complicated. The
business insights are provided by how well the target databases give away the
information and how meaningful they are. How well these database perform are
the direct result of how well the ETL has performed on the data sources. If the
ETL system loads the data in unrecognised format , there will be no business
insights and thus it is useless to user. Thus ETL processes are very critical
in harvesting the data from the big data groups and effectively deal with high
volumes.These extracted data then havepl to be presented in a sensible and
meaningful format.
 
The extraction should ensure that all
required data are extracted with minimal resources and minimal load to the
source. The extraction could be in terms of updating the data or full extract
of the data. The full extraction of data would mean transfer of huge data in
several gigabytes or so.
 
After the data is extracted, the
presentation of the data is another challenge. Transforming the data needs to
ensure that the data are in same dimension along with same units so that they
can be combined with other sources . These data will need to undergo joining
from all data sources and validation.
 
During the load process, similarly as
the  transformation and extraction, the referral integrity is the
important factor to consider to ensure the consistency of the ETL process.
 
Even though the designing ETL process
is straight forward, there could be several limiting factors such as missing
extracts, or null values in reference databases , error in connection or
simpler situation like power outage. Without ETL process, there is no reliable
process to use data from several databases and thus ETL process has to be
designed failsafe.
 
Problems that may be encountered
while performing ETL within sunshine group
 
Sunshine group  being a
multinational business operating in three different countries across the globe,
the problems are huge. Firstly the secured transmission of data which could be
at different physical locations. Secondly, the data format, if different
locations use data in different metrics, their extraction and presentation might
be even more complicated. The financial burden of managing different data
centres and implementation of ETL across the system is even more challenging.
 
During extraction, the same level of
integrity may be hard to maintain in all data sources. Likely during
transformations, the requirement priority may be different depending on where
the data is accessed at. The challenge also  lies in getting all data in
the same dimension and units. While loading data, the loading process might
need more resources as different sources are operating

x

Hi!
I'm Brent!

Would you like to get a custom essay? How about receiving a customized one?

Check it out