Manage old data with the reference date

First of all, what do we mean by "reference date" ?

It is basically a reference point around which data will be dealt with. It can refer to the date a data was created in a system. It can also refer to any given date at which data was altered. If the data is not dated, the use of a counter can provide a reference point. 

What's the use of this reference point ? 

During data integration operations, this reference point indicates the depth with which data will be read in the source system. To that regard, the data created or modified after this reference date (or which's ID is greater than the reference counter) will be read and then integrated into the target system in the case of a migration or synchronisation. That way, not only can you migrate or synchronise data that was registered a long time ago in your system but you can also alter data integration rules that impact data that has already been migrated. 

Let's imagine that you started your e-commerce business in 2005. You've recently decided that it was high time for a change : you have found a better performing e-commerce tool that allows you to increase productivity. You thus decide to migrate your data from your former e-commerce tool to the new one. You want to migrate all the data dating back to the year you started your business, so since 2005. The use of a reference date allows you to migrate not only recently registered data but also data that dates back to 2005.

Let's imagine now that you have found a tool capable of automating the migration of your client data as well as delivery adresses from your new e-commerce tool to your CRM. You establish a data integration rule that reflects this need and run it. Then you realise that you would have also liked to transfer your clients' email adresses to your CRM along with the other data. In that case, it is possible to correct the rule by adding the "email" field to the mapping. And for this change to be extended to your future data AND the data that has already been migrated, you can indicate a reference date that dates back to the beginning of your business or to the date at which you set the rule. 

The reference date : a data quality ally

The possibility of choosing a reference date has the benefit of easing the correction of your data integration rules but also of extending the induced changes to the data items of your choice, recent and/or old. This practice greatly unifies your data system and enhances its quality. 

If you refer to the previous example, the fact that you can establish a reference date and extend the amendment of an integration rule to your entire data system gives you a certain edge when it comes to strategic decision-making concerning your business. In fact, you can thus have a more global and comprehensive view of your clients, their situation changes and take better-informed decisions.

Finally, this practice enables significant efficiency gains given that in case you alter a data integration rule, choosing a relevant reference date automatically extends this change to the desired data set, without you having to make the corrections manually. This is rather interesting, isn't it ?

If you are looking for a data integration tool, we highly recommend you choose one that offers the possibility of setting a reference date. It will make your life so much easier !

For more information on the reference date, please go this page.