Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Ajit Rajasekharan
ChatGPT - an epochal event
The training procedure to create InstructGPT, a precursor to ChatGPT. Given the similarity of the training process in the InstructGPT paper illustrations and the ChatGPT blog, it is likely ChatGPT's training procedure is similar to this. ChatGPT paper has not been released yet. Image by Author Definition of "" in ChatGPT's own words - A quiet shift has been in progress over the last few years in the NLP ecosystem driven by large language models(LLMs).
A Quiet Shift in the NLP Ecosystem
It is already in motion… A paper published a few days ago (15 June 2022) offers a key insight into the impressive task-solving capabilities of the recent spate of billion+ parameter language models.
Almost Any Image Is Only 8k Vectors
An image representation that’s a closeanalog for words in generative tasks The search for an answer to the question “what is the analog of words in images?” appears to be broadly proceeding along two paths (with a few exceptions) driven by multiple factors — the nature of the task being solved (discriminative or generative ) how a model is trained (self-supervised vs supervised) the nature of the training process (e.g. BERT style masking input and predicting masked inputs) etc.
SSL could Avoid Supervised Learning
Figure 1. Named entity recognition (NER) is solved in this post with self-supervised learning (SSL) alone avoiding supervised learning. The approach described here addresses the challenges facing any NER model in real-world applications.
Is Naive Bayes Classification Supervised or Unsupervised?
MACHINE LEARNING FUNDAMENTALS Naive Bayes classification is a form of supervised learning. It is considered to be supervised since naive Bayes classifiers are trained using labeled data, ie. data that has been pre-categorized into the classes that are available for classification. This contrasts with unsupervised learning, where there is no pre-labeled data available.
Quantitative evaluation of a pre-trained BERT model
Figure 1. Quantitative evaluation of a pre-trained BERT model. The test quantitatively evaluates a pre-trained model’s (a) context sensitive vectors by the models ability to predict a masked position and (b) the [CLS] vector quality by examining vector quality of the masked phrase. The clustering quality of underlying vocabulary vectors, particularly the separation of entity types into clusters plays in implicit role in this.
Swiss army knife for unsupervised task solving
BERT is a prize addition to the practitioner’s toolbox Figure 1. Few reasons why BERT is a valuable addition to a practitioner’s toolbox apart from its well known use of fine-tuning for downstream tasks. (1) BERT’s learned vocabulary of vectors (in say 768 dimensional space) serve as targets that masked output vectors predict and learn from prediction errors during training.
Unsupervised synonym harvesting
A hybrid approach combining symbolic processing with distributed representations Figure 1. A hybrid approach combining symbolic processing with distributed representations for unsupervised synonym harvesting (acronyms also harvested during this process). Input is a domain specific corpus of interest and the output for each extracted synonym candidate sentence fragment is a synonym pair. The aggregate output across sentences is a set with a pivot term element and its synonym variations.
Maximizing BERT model performance
An approach to evaluate a pre-trained BERT model to increase performance Figure 1. Training pathways to maximize BERT model performance. For application domains where entity types — people, location, organization etc. are the dominant entity types, training pathways 1a-1d would suffice.
Unsupervised creation of interpretable sentence representations
For sentence similarity/document search applications Figure 1. Unsupervised creation of sentence representation signatures for sentence similarity tasks. Illustration uses BERT (bert-large-cased) model. To date, models learn fixed size representation of sentences, typically with some form of supervision, which are then used for sentence similarity or other downstream tasks. Examples of this are Google’s Universal sentence encoder (2018) and Sentence transformers (2019).
Deep Learning beyond 2019
Progress in (slow) conscious task solving? This figure is synthesized from recent talks by Yoshua Bengio (NeurIPS 2019 talk), Yann LeCun and Leon Bottou.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Ajit
Contact Ajit, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack