4.3 Stages of the Data Science Life Cycle

Data science life cycle is an iterative process. It has various stages, each has certain steps to be taken to move forward. It requires a structured approach to solve data driven problems. The stages of data science life cycles are as follows:

Problem Definition: At this stage the business problem or research question is defined with clear objectives and outcomes. The scope and limitations of the problem are determined. Preliminary research and literature reviews help the stakeholders to understand their needs and expectations. Key performance indicators (KPI) are defined to measure the success of proposed solutions. KPIs are measurable targets that help us track progress and success of a process. For example, increasing sales by $15\%$ in two months can be set as a KPI of some business problem. Another example of KPI is improve the speed of website loading by $20\%$ in 30 days. By setting KPIs, we can focus on finding the right solutions and track our progress. The outcome of this stage is a hypothesis or well- defined problem.

Data Collection:

At this stage, data is gathered by using various reliable and useful sources. The method of data collection depends on the kind and complexity of problems. For simple problems, we can observe and write down what we see, while more complex problems we might need surveys, experiments, or even using online services through APIs (Application Programming Interface). APIs are small programs that act like messengers between two or more software.

These are like waiters in a restaurant who receive an order from us and bring back the food as response to our order. The data collected should be relevant and accurate, as the quality of the data directly affects the results at later stages

Data Cleaning:

It is also called data preprocessing, this step is about fixing the data to make sure it's ready for analysis. In this step we remove errors, such as wrong information or duplicate values. If some values are missing, which might cause problems, we fill in with estimated values.

During this process, data is organized into an easily workable format. Sometimes, new features or attributes are created from existing data. For example, if we have dates, we might create a new column for the day of the week. Clean and well- organized data is essential for making good predictions later on.

Data Analysis:

After cleaning the data, we have to analyze it. Statistics or machine learning techniques are used for data analysis. These techniques help in finding patterns, trends, or important details. By looking at the data closely, we can test early ideas or assumptions about the data.

For example, we might want to know if a certain factor is causing an outcome, like is the weather affecting sales of certain products? At this stage, visualization tools like graphs or charts can be used to understand what's happening in the data.

Data Modeling:

In this stage, the data is organized into a structured model. It helps to create diagrams that show the relationships between different entities (like customers and the products they purchase) and their attributes (like names, prices, or dates).

We also focus on how to store and retrieve this data efficiently by setting up databases or other systems. Once the model is ready, it is tested using historical/previous data to see if it performs well. The goal is to create a structure that can be easily used for predictions or other analysis.

Model Evaluation:

At this stage we evaluate the performance of our machine learning model. This is done by measuring how accurate and reliable the model is, and whether it meets the performance goals we set earlier.

For example, we can test how often the model makes the right predictions or how fast it delivers results. If the model doesn't perform as expected, we might need to go back to the data modeling stage to make improvements. This stage also includes making sure that the data and the model are secure and respect privacy rules, especially if sensitive information is involved.

Model Deployment:

Once the model is working well, it is time to deploy it for use. This stage is called deployment, and it means the model is set up in the system or infrastructure where it will be used. It means to integrate the model on a website, application, or database.

Sometimes, deployment is done in small steps (partial deployment) to test if everything works fine before launching it completely. Other times, the model is deployed all at once. After deployment, the model starts working with real data, providing insights or making predictions.

4.3.1 Case Study: Understanding a real- world business problem and its formulation to data science problem

We will learn to apply the data science life cycle in real- world business problem. Suppose we have a task to optimize the inventory of a smart phones.

Problem definition: Suppose you own a super store and you want to ensure that you want to have good enough stock of popular smart phones. The objective is that you don't run out of stock to meet the demand of your customers. Keeping in mind that you also don't want to make unnecessary purchases that would be difficult to sell.

Data Collection: The first step is to gather data related to the smartphone stock and sales. The data includes:

Current stock levels for each smartphone model (how many phones of each type are in stock). Sales data showing which models are selling fast and which are slow- moving. Supplier information about new models and upcoming promotions or price changes.