Date of Award

2024

Publication Type

Thesis

Degree Name

M.Sc.

Department

Computer Science

Keywords

deep sequential mining; language models; neural networks; Recommender systems; semantics

Supervisor

Christie Ezeife

Supervisor

Mahreen Nasir

Abstract

Sequential recommendation (SR) focuses on predicting the next product a user will interact with based on their previous sequence of interactions. Incorporating more semantic information (e.g., product description) of type of products interacted from these sequences plays a pivotal role in the predictive performance of the recommendation systems. Sequential Pattern Mining (SPM) identifies frequent purchase subsequences from user-item interactions for product recommendations. Deep sequential mining uses deep neural networks to mine semantic relationships and contextual meanings of these interactions, which traditional SPM fails to capture. For instance, while SPM might recognize that customers often buy “milk” after “bread” and “butter,” deep sequential mining using neural networks can recommend more specific items like “lactose-free skimmed milk” by learning product descriptions, thus providing richer contextual recommendations. Existing SR systems such as SEMSRec20, ADNNet21, and RDNBR23 utilize various neural architectures but have some limitations. SEMSRec20 employs prod2vec for product embeddings and PrefixSpan for pattern mining but relies solely on product IDs, resulting in low-quality embeddings. ADNNet21 combines Convolutional Neural Networks (CNNs) for long-term interests and Gated Recurrent Units (GRUs) for short-term needs, but it struggles to recommend new or non-frequent interacted products. RDNBR23 uses binary-encoded transactions to capture repetitive purchases. However, it cannot recommend new items that a user has never interacted with, as it heavily relies on users’ repetitive behavior and historical transaction data. This thesis proposes a deep learning-based SR model called BERT based Sequential Mining for Richer Contextual Semantics E-commerce Recommendation (BERT-SEMSRec), an improved SEMSRec20 that replaces the prod2vec model, by enriching existing BERT model to find embeddings from products description and purchase sequences of products. A two-phased architecture is outlined where the first phase, MetaBERT2Vec involves a BERT model that processes description of each product to generate embedding that represent the semantic(meaning) of products. The second phase, SeqBERT2Vec fine-tunes another BERT model to generate positional embeddings of each product in the purchase sequence that represent how the products are appearing in the sequence. These embeddings are then concatenated to generate top-K recommendations. Our experimental results show BERT-SEMSRec consistently outperforms SEMSRec20 and other models like GRU4Rec, SASRec, and BERT4Rec across four categories of Amazon dataset (Office Products, Electronics, Health_and_Household, and Beauty), with up to 12% higher HR@10, 15% higher MRR, and 18% higher NDCG@10, demonstrating superior ability to capture both semantic and sequential product information for recommending products.

Share

COinS