A practical guide for aspiring Data Scientists
I happened to receive many calls from friends, and friends of friends on how to break into a Data Scientist job. So I thought, for the benefit of all those who have similar questions, this would be a definitive place to find all the answers.
Data Scientist: Roles and Responsibilities
Though the exact scope of roles and responsibilities of a Data Scientist may vary a little depending on what type of company you join, the core responsibilities are pretty standard:
- Data Analysis and Visualisation
- Building AI/ML Models
- Cross-functional Communication Skills
- Basic to Intermediate Software Development Skills
Apart from these you may also be expected to know the basics of Agile Software Development principles if you work for a company that follows the principles. You may not be expected to, but will have to learn version controlling using Github or similar tools. If you work for a startup, you may be expected to own the end-to-end software development, meaning you may need to know about software architecture, micro-services, etc. …
With the Roberta Sentence Tokenizer
The Requirement
I was recently exploring multiple models for building a cognitive search bot for open-domain question answering. The bot should be capable of returning the appropriate answer to the question posed by the user. Of course, the efficacy of the bot is limited by the content of the training dataset. However, this bot should be robust enough to handle misspelled words — especially named entities (proper nouns such as names, places, animals, things).
For my prototype, I chose the WikiQA dataset from Microsoft Research. …
Learning from a practical NLP project
The problem of incident ticket classification is one of huge impact to IT companies. When users raise a ticket, the ticket needs to be directed to the right team as quickly as possible, to ensure speedy resolution. Sending it to the wrong department leads to longer resolution times since it takes time for the ticket to be redirected to the right team. Currently, in most of the companies, ticket classification is done manually, which is prone to errors, and is tedious as the volume of tickets increases. Hence, we need a better solution to handle this problem.
Auto-ticket assignment: Problem
In order to reduce the possibility of misclassification of a ticket to the wrong assignment group (referred to in this post as ‘class’), we explore various classification methods and embedding methods and evaluate their performance on a sample dataset. …
An intuitive guide to calculating input shape and complexity of neural networks
While building neural networks, a lot of beginners and non-beginners alike, seem to get caught up in figuring out the input shape that needs to be fed into the neural network.
But why should we know the input shape, and why should we feed it? Can’t the Neural Network figure it out on its own?
The answer to this question lies in the basics of matrix multiplication.
Suppose we have two matrices A and B. Let the dimensions of B be m rows x n columns. Now, for the two matrices to be compatible for multiplication, the column dimension of A should be the same as the row dimension of B. …
An introduction to Pattern Exploiting Training
The overwhelming world of mammoth language models
Ever since the advent of transfer learning in Natural Language Processing, larger and larger models have been presented, in order to make more and more complex language tasks possible.
But the more complex the models, the more time and the more amount of data it needs to train. The latest GPT-3 model achieves state of the art results in most natural language tasks, but it has close to 175 billion parameters to train, and takes years to train!
So is there a way around it?
Timo Schick and Hinrich Schutze came up with an ensemble masked language model training method which has proven to be as potent as Open AI’s revolutionary GPT-3 model, but requires only 0.1% of the parameters required by GPT-3! …
An Introduction to Cisco’s Mindmeld Conversational AI Platform
Chances are, if you are working on building conversational agents (a.k.a chatbots), you have explored various platforms such as RASA, Kore.ai or Microsoft’s QnA Maker. Each one is composed of very similar algorithmic components but different technological choices. However, each of them comes with different features, different levels of customisation, post-analytics, etc.
And now, there is a new kid on the block: Mindmeld!! It is based on Elasticsearch, is completely open source and is offers in-built ‘blueprints’ that correspond to various domain (food orderig, HR assistant, Home assistant, etc.) …
Product Management for Aspiring Product Managers
Every Product Manager is faced with one important question every release cycle: which feature(s) to prioritize for this release?
Some features might look super savvy, and having them on your marketing material might be super attractive, but it may take a whole lot of development effort from multiple teams. On the other hand, there might be some low-hanging fruit that may just be some minor bug fixes. How would you choose which one to prioritise?
First Principles
If I had to choose a feature to work on first, I would pick the one with the biggest bang for the buck — the bang being impact on the customers, and buck being the effort my development and testing team need to put in. …
Web Services 101
If you work in the Software Domain, you can’t not have heard about web services. Every company that relies on technology is moving towards a web service framework. They are easy to work on, they make the entire code modular, and since they integrate pieces of code written in different languages, it is also more efficient and easy to maintain.
What is a web service
A web service is a piece of software/code which offers a functionality to any other piece of code that wants to interact with it.
Why web services
As tech companies try to make their product or service better and quicker, they try to transition towards newer technologies and platforms. In this case, having a code that is tightly coupled to the technology or to the platform on which the older piece was built could be a tedious and sometimes a dangerous job. …
Most Human Interpretable Model Explanation Method
Let’s say Paul applies for a loan at ABC Bank. He receives a mail from the bank expressing deep regret to inform him that his loan application has been rejected. Rejected!
After being shattered for a while, Paul may be curious to know what happened. So he goes to the bank and asks for the person who takes decisions on loan applications. He is directed to Jane, who takes the decisions on loan applications. Paul asks Jane why his application got rejected. …
Introduction to JMeter and basics of Unit Testing
All ML stories begin with loading datasets and end with serving them as APIs. However, if you want your models to go into production, you need one more step before you put your machine learning applications into production — Load Testing.
What is Load Testing
Load testing is the part of testing effort where we determine how much load (how many requests) the model API and the device on which it will be deployed can take at a time. We usually compare it with a set SLA (service level agreement). …
