Data Engineering

It is often the case that many organisations get excited about the opportunities in Machine Learning and AI that they forget about the more fundamental challenges that they need to overcome first and that is data cleansing, preparation, and consolidation. Data engineering is an activity that is concerned with designing and building an end to end data pipeline by democratising siloed data sets and integrating those as part of cohesive enterprise wide data consolidation exercise. In this exercise, it is often the case that data needs to be cleansed, prepared, enriched and transformed into a format that is ready for Machine Learning algorithms to make best use of.