top of page

Data Scientists, Please Don’t…

Updated: Mar 2, 2022

A famous quote by Sherlock Holmes well defines that the role of a data scientist in business is as similar as to that of a detective:

"My name is Sherlock Holmes. It is my business to know what other people don’t know."

As a data scientist, whether rookie or experienced, is reliable on data which is hardly ever flawless. To ensure that the model you've built and the analysis you've done on the data are both valid, it's critical to tackle several typical data quality concerns correctly. Here, we'll go through how to prevent some of the most prevalent cases.

1. Lacking of an organized problem-solving approach

Any problem-solving strategy requires a goal and a plan for moving ahead. This is where the majority of people struggle. All of the difficulties start when people go blindly into the pre-set procedures of analysis and modelling without first taking a step back and defining a clear purpose for the problem they're attempting to solve.

Following are some essential stages to include in any problem-solving strategy:

  • Identify the project's assumptions

  • Map all accessible data sources

  • Create a list of performance indicators that translate the business requirements

  • Provide a realistic timeline in order to achieve deadlines.

There's no magic here: a successful data science project requires a thorough grasp of the issue and a methodical approach to solving it.

2. Skipping Exploratory Data Analysis

Exploratory data analysis refers to the critical process of performing initial investigations on data so as to uncover patterns, detect anomalies, test hypothesis and verify assumptions with the help of summary statistics and graphical representations.

Data scientists that are in a rush to get to the machine learning stage or just satisfy business stakeholders quickly tend to either completely skip the exploratory step or undertake extremely shallow work. It's a really severe and, unfortunately, all-too-common blunder. Like techniques may result in skewed data, outliers, and a large number of missing values, as well as some disappointing project results, such as

  • Models that are inaccurate

  • Creating accurate models with incorrect data

  • Creating accurate models with incorrect data

  • Inefficient use of resources, including the need to redesign the model.

Majority of individuals don't realize that successful exploratory data analysis (or EDA) enables you to identify or even define the questions you're seeking to answer with your data.

Now I see how EDA may be a time-consuming and often unpleasant process since it involves creating charts and visuals as well as spending the time to comprehend them. Fortunately, there is be a fantastic solution that can automate a significant piece of this job.

It's referred to as pandas-profiling. It's a Python module that accepts data in the form of a DataFrame object and creates a detailed report on it automatically. Distributions, statistics, bar charts, outliers, correlation matrices, missing data, and so on are all included in this report. What would normally take a day or two is created in a matter of minutes, allowing you to concentrate on understanding the issue and devising the best solution to address it.

3. The utter lack of data annotations and continued use of corrupted data

Data labelling is a highly critical step; even the tiniest mistake may cause havoc. To train machine learning models, data scientists need a huge volume of accurately labelled data, especially in case of image and video data.

Working with tainted data without data annotations is analogous to attempting to bake a cake without the proper ingredients. Will your cake be fluffy, soft, and delicious? No!

4. Not consulting domain experts

As data scientists, we may feel that our tools and algorithms can answer any business challenges with a single click of a Jupyter cell and without leaving our comfort zone. As appealing as this may seem, it is seldom the case.

Interacting with domain experts is an important component of a data scientist's work, and there are at least two reasons for this:

  • Domain specialists are essential since they provide insights and hints that aren't visible in the data.

  • Domain experts must also engage with you since you are developing a solution for them, and they must learn how to utilize it from you.

In my own experience working on an industrial data science project, we had to spend three months working with the CSO, plant heads, safety heads, and stakeholders simply to understand the existing information and how we might utilize it to formulate the relevant questions to be addressed with the data.

Having error-free and high-quality data will aid in improving the accuracy and dependability of your model.

5. Not assessing all relevant datasets when designing the model

A professional data scientist should assess all of the datasets in your problem statement and try to correlate the information between them. Data is often broken up into different datasets to make it more understandable. A data scientist's objective while developing a model is to create a link between the data set, analyze it, and draw a proper image of it. However, you should not use all of the data without first evaluating which qualities are critical to solving your problem. Using dimensionality reduction, this may be done precisely.

The practice of changing a dataset such that only the most significant properties are picked for training is known as dimensionality reduction. The importance of feature selection and dimensionality reduction may be explained in three ways:

  • Prevents Overfitting: Overfitting may occur when a dataset has a large number of dimensions and features (model captures both real and random effects).

  • Simplicity: A model with too many characteristics might be difficult to comprehend, particularly when the features are connected.

  • A model trained on a lower-dimensional dataset is computationally efficient (execution of algorithm requires less computational time).

6. Disregarding Analysis

The most fascinating aspects of becoming a data scientist are data visualization and analysis. Some data scientists may leap right to predictive modelling, however in real-world circumstances, this method will not address any machine learning issues properly. Data scientists must go further into the information gleaned from the data.

We can extract more useful insights from the data by paying closer attention to data analysis, researching trends and patterns, and asking questions.

7. Misunderstanding Correlation for Causation

"Correlation is not Causation."

Even though two things seem to be correlated, this does not imply that one causes the other. For any data scientist, confusing correlation with causation may be catastrophic.

When dealing with large datasets, most individuals think that correlation equals causality, which is seldom the case.

Correlation is a statistical approach for describing how two variables move in lockstep (ex: if variable x changes, so does variable y), while Causality is the study of cause and effect.

Getting this relationship wrong can be costly, as demonstrated by Freakonomics' example of mistaking correlation for causation, which nearly led the State of Illinois to send books to every child in the state because studies showed that having books at home was associated with higher test scores. Later studies revealed that children from families with a lot of books fared better even if they never read, causing researchers to rethink their ideas and realize that households whose parents acquire books create an atmosphere that encourages and rewards learning. Illinois, like today's business, didn't have money to throw away by heading in the wrong path.

8. Focusing Too Much on Accuracy

High accuracy does not always mean a better model.

For a good model accuracy should not be the sole criterion. The customer is not interested in a black-box model that just provides high accuracy. Accuracy is desirable, but it is not sufficient.

A Data scientists should describe how the model achieves accuracy, which characteristics are significant, why they picked a certain method, how other algorithms behave, and more. This helps your client better understand and accept your model.

When creating a model, it's also important to consider the specifications of the live production unit. Otherwise, the job will be a waste of time, and it may need to be repeated to match the actual settings of the real environment.

For example, consider a 1000-data-point dataset with 900 negative data points and 100 positive data points. Our model predicts that all data points will be negative. It signifies those 900 points out of 1000 are correctly anticipated. As a result, our accuracy will be: Accuracy = (900/1000) * 100 = 90%

Even if the model achieves 90% accuracy, it is still a dump model.

9. Lack of Consistent Model Validation

Some data scientists believe that building a good machine learning model is equivalent to reaching the pinnacle of accomplishment. Having built a right model is just half the battle won; you still have to make sure the model's predictive ability is maintained.

Many data scientists neglect or refuse to acknowledge the need of re-validating their models at regular intervals. Some data scientists make the error of believing that the prediction model is perfect because it matches the observational data. Predictive power of the built model can disappear instantaneously based on how often the modelled relationships keep changing. To prevent this, the ideal practice for any data scientist is to score their data models with fresh data every hour, day, or month, depending on how quickly the relationships in the model change.

Selecting the iteration frequency is critical for maintaining the predictive strength and validity of the created models, and failing to do so might result in erroneous findings.



bottom of page