Combining web and enterprise data for lightweight data mart construction
McCarthy, Suzanne, McCarren, AndrewORCID: 0000-0002-7297-0984 and Roantree, Mark
(2018)
Combining web and enterprise data for lightweight data mart construction.
In: 29th International Conference on Database and Expert Systems Applications (DEXA 2018), 3- 6 Sept 2018, Regensburg, Germany.
ISBN 978-3-319-98811-5
The Agri sector has shown an exponential growth in both the requirement for and the production and availability of data. In parallel with this growth, Agri organisations often have a need to integrate their in-house data with international, web-based datasets. Generally, data is
freely available from official government sources but there is very little unity between sources, often leading to significant manual overhead in the development of data integration systems and the preparation of reports.
While this has led to an increased use of data warehousing technology in the Agri sector, the issues of cost in terms of both time to access data
and the financial costs of generating the Extract-Transform-Load layers remain high. In this work, we examine more lightweight data marts in
an infrastructure which can support on-demand queries. We focus on the construction of data marts which combine both enterprise and web data, and present an evaluation which verifies the transformation process from
source to data mart.
Hartmann, Sven, Ma, Hui, Hameurlain, Abdelkader and Pernul, Günther, (eds.)
29th International Conference on Database and Expert Systems Applications (DEXA 2018), Proceedings Part 2. Lecture Notes in Computer Science
11030.
Springer. ISBN 978-3-319-98811-5