After collecting data, the next important step is to clean and validate it. Raw data is often messy. It may contain errors, missing values, or even wrong entries. If we use this data without checking it, our analysis will give wrong results. Remember that earlier we collected data using surveys and other methods. Now, we need to check if that data is correct, complete, and ready for use. This step ensures that the data we analyze later is trustworthy.
Raw Data
Raw data is the original information we collect from people, sensors, websites, or apps. It has not been checked or cleaned yet. For example, assume that 20 students filled your survey about favorite sports. Some entries may look like as:
| ID. | Name | Favorite Sport | Frequency per week | Study Time of Day |
|---|---|---|---|---|
| 1 | Ahmad | cricket | Daily | Evening |
| 2 | Ali | Weekly | Afternoon | |
| 3 | Sara | Football | Daily | |
| 4 | Sana | Football | Daily | Evening |
Table 5.2 contains:
- spelling errors i.e., Football (ID 3), Daily (IDddd 4)
- missing entries (ID 2 & 3)
- Mixed Capitalizations i.e., cricket (ID 1), Football (ID 4). Such kind of data needs cleaning.
Common Data Errors (Missing, Duplicate, Outliers)
Table 5.3 shows some common issues found in raw data.