Data Preparation is the method for pre-processing unstructured or raw data to make it suitable for analysis. It consists of collecting, combining, editing, cleaning of data in the machine learning, data mining, and data science community. Some other terms used for it is Data Wrangling, Data Munging or Data Cleaning.
Components of Data Preparation
There are different components or processes in preparing the data. Pre-processing, profiling, cleansing, validating and transforming. Sometimes, data is pulled in from different sources and consolidated. This whole process together leads to preparation of data. The steps can be further classified into smaller sub-steps. They are:
• Checking Questionnaires. Here, we see if any instruction is incomplete or not. Missing pages, whether respondents are qualified.
• Cleaning. Data cleaning involves many functions. Such as: treating missing values, correcting incomplete or ambiguous answers.
• Statistical Adjustments. This happens when data requires scale transformation.
• Strategy Selection. This is the final step of data analysis. It is based on past performance and structure of the gathered data.
Importance of Data Preparation
Data preparation is the most vital step of data analysis. About 60-80% of the time is spent to collect, clean and prepare the data suitable for analysis. It is needed most when a business intelligence platform is required to import data from varied sources. Imported data is unfit for any analysis or visualization if it is not properly treated. This is where the tools come into play. There are some dedicated providers who offer tools for data preparation purpose. Raw data usually contains missing values, discrepancies, outliers, etc. Presence of these attributes is quite critical for accurate analysis.
Benefits of Data Preparation
Data preparation tools exceed our expectations and have a lot many benefits.
• Decision making becomes data driven
• Improved analytical efficiency
• Improved operational efficiency
• Reduction in analytical silos
• Improvement in cost efficiency
• Increase in revenue
Different techniques have developed for data preparation. But it is still under research. Data scientists are looking for even more novel ways to explore and prepare data ready to analyze. The real value of data is recognized, but the ways to make it useful are short of supply.
