Date of Award

2024

Publication Type

Thesis

Degree Name

M.Sc.

Department

Computer Science

Keywords

BERT; Fine-Tuning; Large Language Models; LLaMa; Loss Functions; Sentence Embeddings

Supervisor

Alioune Ngom

Supervisor

Jianguo Lu

Abstract

Large language models have revolutionized the field of natural language processing due to their extensive scale, complex architectures, and training on vast datasets, allowing them to capture intricate and detailed language representations. These models have shown remarkable abilities in understanding and processing language across a variety of contexts and applications. With the objective of advancing natural language processing, this study conducts an extensive analysis of large language models fine-tuned with diverse loss functions to evaluate tasks in SentEval (an evaluation toolkit for sentence embeddings) classification and Semantic Textual Similarity (STS). The BERT model was fine-tuned using various types of loss functions including sentence-triplet, sentence-pair and cross-entropy based losses, in both transfer and non-transfer settings. The fine-tuned model was then assessed through classification accuracy on SentEval classification tasks, and Spearman's Rank correlation coefficient calculation on Semantic Textual Similarity (STS) tasks. By systematically evaluating these loss functions, we aim to develop a framework that yields superior sentence embeddings, facilitating more accurate and robust downstream tasks. Our approach seeks to improve the inherent capabilities of large language models and provides valuable insights into the synergistic effects of different loss functions and how each one guides the model into optimizing sentence embeddings. It is observed that triplet-based loss functions perform better in classification and pairwise losses perform better in semantic similarity tasks. However, the results demonstrate significant improvements in benchmark performance after fine-tuning the model, also showcasing the potential of our framework for various downstream tasks.

Share

COinS