Source data models and data flow

The most complex part of managing the data models is how data flows into your data models. In the data modelling example, we introduced a domain data model and a solution data model, where the domain data model models your complete world view , and the solution data model gives you a subset of the domain data model for a specific purpose.

When loading data from source systems, keep the data in a format that resembles the structure found in the source system. This is called a source data model. Depending on how you want to control access, the source data model could live in a separate space or in a single controlled_data_models space (with generic and domain data models). In the example above, we split out data models into separate spaces. In the diagram below, all the data models are shown in the same space to simplify.

However, you should not load data into the same space as your controlled data models. Instead, you should create one space per data set that you want to control access to.

In the above diagram, the dotted lines show how the data (instances) live in spaces to control access to the data, while the solid lines show how transformations move data from one data space to another.

You can access data only through a data model (so by referencing one of the data models in the top square), while the actual data can be stored and structured physically in multiple "buckets" across spaces.

In the Advanced tutorial, you will learn more about the advanced mechanisms.