There was a time when a data warehouse architecture consisted of a chain of databases all running on one or two machines in our own data center. Handwritten ETL programs were used to copy and transform data from one database to another. But so much new technology offering innovative opportunities has become available, there are so many new BI requirements, and we have new ways to design our data warehouse architectures. Data warehouse architects are struggling with all these new developments. They have to find answers for an almost endless list of questions. Should the data warehouse be developed with Hadoop? Do we still need data marts if the BI tools read data into memory? Can we use Spark as query performance booster? What does it mean to design datavault-based data warehouses? How does data streaming and the IoT work together with the data warehouse? Should we move the entire architecture into the cloud? Can we replace the data warehouse by a data lake? What is the role of the logical data warehouse? Will an analytical SQL database server solve all our query performance problems? And so on, and so on.
This session discusses all the architectural and technical developments. How are they interrelated? How to migrate to a modern architecture? What are the pros and cons of all these developments?
Classic data warehouse architectures are made up of a chain of databases. This chain consists of numerous databases, such as the staging area, the central data warehouse and several datamarts, and countless ETL programs needed to pump data through the chain. This architecture has served many organisations well. But is it still adequate for all the new user requirements and can new technology be used optimally for data analysis and storage?
Integrating self-service BI products with this architecture is not easy and certainly not if users want to access the source systems. Delivering 100% up-to-date data to support operational BI is difficult to implement. And how do we embed new storage technologies, such as Hadoop and NoSQL, into the architecture?
It is time to migrate gradually to a more flexible architecture in which new data sources can hooked up to the data warehouse more quickly, in which self-service BI can be supported correctly, in which OBI is easy to implement, in which the adoption of new technology, such as Hadoop and NoSQL, is easy, and in which the processing of big data is not a technological revolution, but an evolution.
The architecture that fulfills all these needs is called the logical data warehouse architecture. This architecture, introduced by Gartner, is based on a decoupling of reporting and analyses on the one hand, and data sources on the other hand.
The technology to create a logical data warehouse is available, and many organisations have already successfully completed the migration; a migration that is based on a step-by-step process and not on full rip-and-replace approach.
In this practical seminar, the architecture is explained and products will be discussed. It discusses how organisations can migrate their existing architecture to this new one. Tips and design guidelines are given to help make this migration as efficient as possible.
Big data, Hadoop, in-memory analytics, Spark, Kafka, self-service BI, fast data, data warehouse automation, analytical database servers, data virtualisation, data vault, operational intelligence, predictive analytics, and NoSQL are just a few of the new technologies and techniques that have become available for developing BI systems. Most of them are very powerful and allow for development of more flexible and scalable BI systems. But which ones do you pick?
Due to this waterfall of new developments, it’s becoming harder and harder for organisations to select the right tools. Which technologies are relevant? Are they mature? What are their use cases? These are all valid but difficult to answer questions.
This seminar gives a clear, extensive, and critical overview of all the new developments and their inter-relationships. Technologies and techniques are explained, market overviews are presented, strengths and weaknesses are discussed, and guidelines and best practices are given.
The biggest revolution in BI is evidently big data. Therefore, considerable time in the seminar is reserved for this intriguing topic. Hadoop, Spark, MapReduce, Kafka, Hive, NoSQL, SQL-on-Hadoop are all explained. In addition, the relation with analytics is discussed extensively.
This seminar gives you a unique opportunity to see and learn about all the new BI developments. It’s the perfect update for those interested in knowing how to make BI systems ready for the coming ten years.