Advaith Malladi

Undergraduate Researcher

Under the guidance of Prof. Shashank Srivastava, I am pursuing research in NLP. I am working on interpretable and explainable machine learning in NLP and LLMs.

Undergraduate Researcher

Under the guidance of Prof. Bapi Raju, I am pursuing research on the combination of NLP and Cognitive Science. I am working on probing LLMs and understanding how their representations align with the representations in the human brain.

Undergraduate Researcher

Under the guidance of Prof. Rahul Mishra, I have been working on converting unstructured documents to a structured format using the planning and reasoning capabilities of language models.

Machine Learning Intern

I will be joining Enterpret as a Machine Learning Intern in December this year.

Machine Learning Intern

I worked on utilizing the planning capabilities of LLMs to generate synthetic datasets. I used these synthetic datasets to train smaller models and evaluated their performance compared to models trained on real human-generated data.

NLP Intern

I worked on building the in-house Retrieval Augmented Generation (RAG) pipeline with the NLP team at Subtl.ai, focusing on evaluating the retrieval pipeline and fine-tuning LLMs to ensure text quality as part of the generation process. I also built synthetic datasets using LLMs to evaluate the retrieval step of the pipeline.

Teaching Assistant

• I am a Teaching Assistant for the Advanced Natural Language Processing (CS7.501) course taught by Prof. Manish Shrivastava for the Monsoon ’24 semester.
• I was a Teaching Assistant for the Introduction to Natural Language Processing (CS7.401) course taught by Prof. Manish Shrivastava for the Spring ’24 semester.
• I was a Teaching Assistant for the Algorithm Analysis and Design (CS1.301) course taught by Prof. Suryajith Chillara for the Monsoon ’23 semester.
• I was responsible for designing assignments and evaluating them by conducting vivas and grading quizzes and assignments, while also leading tutorials for classes of 200 students, teaching topics such as Language Modeling, Word2Vec, and Contextual Word Embeddings (ELMO).

IIIT Hyderabad

• Introduction to NLP
• Advanced NLP
• Information Retrieval and Extraction
• Cognitive Science and AI
• Computational Linguistics
• Computational Psycholinguistics

IIIT Hyderabad

Prompt-tuned vs Fine-tuned models Which Better Account for Brain Language (and Vision) Representations?

• We used decoder language models to compare which representations better account for the brain's representations: fine-tuned or prompt-tuned.
• We extended the same idea to vision encoder models, implementing vision prompt tuning along the way.
• We learned that prompt-tuned text representations are similar to the brain, while fine-tuned vision representations are closer to the brain.
• Link: Fine Tune vs Prompt Tune

Retrieval Augmented Multimodal Factual Verification

• Given a claim, we built a pipeline to retrieve the relevant paragraphs, tables and images from Wikipedia. These serve as Evidence.
• After the retrieval of Evidence, we made use of two kinds of models for the claim verification part: FactBERT+DINO and Bridge Tower (presented in AAAI'23).
• Link: Multimodal Factual Verification

Parameter Efficient Prompt Tuning of GPT-2

• Implemented Parameter-Efficient Prompt Tuning on GPT-2 for Question Answering, Machine Translation and Summarization.
• Link: Parameter Efficient Prompt Tuning

Attention is all you Need

• Implemented the Encoder-Decoder architecture presented in the "Attention Is All You Need" paper from scratch without using the ready-made encoder-decoder modules.
• Build a Machine Translation System for English to French.
• Link: Eng2French

Decoder Language Model

• A decoder-only transformer based language model built from scratch using PyTorch.
• Link: Transformer Language Model

Evaluating Discourse Coherence in Paragraphs

• A stacked LSTM model which can detect topic coherence and temporal/sequential coherence in a paragraph with an accuracy of 0.8.
• This model was taught 2 kinds of discourse coherence, temporal discourse coherence and topic discourse coherence by inducing different kinds of negative sampling.
• Link: Textual Coherence

Embeddings from Language MOdelling (ELMO)

• Trained a Stacked Bi-LSTM model on Masked Language Modelling to learn contextual word embeddings for downstream tasks.
• Link: ELMO