Photo by Jan Huber on Unsplash

I recently completed a course on ‘Natural Language Processing using Probabilistic Models’ by deeplearning.ai on Coursera. This course covered a wide range of topics that are Spelling Correction, Part of Speech tagging, Language modeling, and Word to Vector. All the topics were covered in-depth and with detailed practical exercises.

But, we all know if we don’t practice the concepts learnt we are sure to forget about them in no time. So, I thought of writing this blog about one of the very important metrics that was covered in the course — ‘Edit Distance or Levenshtein Distance’.

The following topics will…


Use Statistical techniques to select the right features for your model.

Photo by Katarzyna Pe on Unsplash

This is the second part of a two-part series. You should read the first part first.

We’re talking about a way to compare the bag of words across the categories without building machine learning models and to do feature engineering.

So, far we have looked at:

  1. To treat the frequency of words across the categories as separate distributions
  2. Apply the Mann-Whitney U Test — a non-parametric test on the distributions of each word, to test for their significance
  3. Analyze the results i.e. compare the frequencies and the p-values of the significant words

After looking at the results we definitely got…


Photo by The Matter of Food on Unsplash

Every creative idea needs inspiration and in my case, it came from a beautiful young woman (*^_^*)

I sat down with a pen, a notebook, and my thoughts, thinking for hours about — what to write or from where to start or even is there anything in the world that I can use as a simile?

Then my voilà moment appeared and the result was —

Peach, is it even a good theme?
Will it cover the whole scheme?
It’s colors go well with the outfits,
Will it also represent thy habits?

I realized after pondering for hours,
that it perfectly…


Compare the frequency distributions across labels without building an ML model.

Photo by Luke Chesser on Unsplash

One of my friends asked me a problem — “how can we compare the BOW across different categories or labels? Where categories or labels could be sentiment or state or some customer segment.

My intuitive response was — to create a Bar graph of the frequency of words for each category. This really is a simple to implement solution, but have various drawbacks, some of which are:

  • The data scientist/analyst working on this will be required to compare words and their frequencies across all the categories, which in some cases like countries or could easily get over 100
  • Comparing the…

One of my friends asked me a problem — “how can we compare the BOW across different categories or labels? Where categories or labels could be sentiment or state or customer loyalty.

My intuitive response was — to create a Bar graph of the frequency of words for each category. This really is a simple to implement solution, but have various drawbacks, some of which are:

  • The data scientist/analyst working on this will be required to compare words and their frequencies across all the categories, which in some cases like countries or could easily get over 100
  • Comparing the frequencies…

Image is taken from https://www.desktopbackground.org/

Everyone knows about the story of the Little red riding hood, but that’s only a small part of it. But let me tell you the part of the story that nobody talks about.

Long before the girl encountered the wolf, the part of the story you all are aware of, she met an owl. Don’t even think of comparing the owl and the wolf as their natures were like the opposite poles of the magnet. The wolf was greedy and foul, whereas the owl had a loving and caring nature.

The owl loved to sing songs even though he had…


Photo by Viktor Smith from Pexels

Hello friends,

I hope you all are doing well and that you all are enjoying the work I have been sharing with you.

Today I am going to write on one of my friends, she really is a big workaholic but this does not mean she does not relax or spend time with others.

From what I know about her, she just wants to give her best whether it’s about the work or her hobbies, which is something I really appreciate about her.

So, below is a short poem dedicated to her.

Ants are all black,
which don’t know how to…


I know it’s been too long since I’ve written anything, but I warned you about this in my last post — “A workaholic ant”, that if my friend finds about the comparison between her and an ant (BUG!), my writing days are over. :(

Now, since my writing spirit and the hand are finally healed, I thought of sharing an incident happened with me some time ago. It was not at all funny, at least for me, but all my friends had a great laugh at it. You know how friends are, the little devils…

So, let me tell you…

Prateek Jain

Data Scientist | Mentor | Retail | Telecom | Social Media

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store