Date of Award
2024
Publication Type
Thesis
Degree Name
M.Sc.
Department
Computer Science
Keywords
deep sequential mining; language models; neural networks; Recommender systems; semantics
Supervisor
Christie Ezeife
Supervisor
Mahreen Nasir
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Abstract
Sequential recommendation (SR) focuses on predicting the next product a user will interact with based on their previous sequence of interactions. Incorporating more semantic information (e.g., product description) of type of products interacted from these sequences plays a pivotal role in the predictive performance of the recommendation systems. Sequential Pattern Mining (SPM) identifies frequent purchase subsequences from user-item interactions for product recommendations. Deep sequential mining uses deep neural networks to mine semantic relationships and contextual meanings of these interactions, which traditional SPM fails to capture. For instance, while SPM might recognize that customers often buy “milk” after “bread” and “butter,” deep sequential mining using neural networks can recommend more specific items like “lactose-free skimmed milk” by learning product descriptions, thus providing richer contextual recommendations. Existing SR systems such as SEMSRec20, ADNNet21, and RDNBR23 utilize various neural architectures but have some limitations. SEMSRec20 employs prod2vec for product embeddings and PrefixSpan for pattern mining but relies solely on product IDs, resulting in low-quality embeddings. ADNNet21 combines Convolutional Neural Networks (CNNs) for long-term interests and Gated Recurrent Units (GRUs) for short-term needs, but it struggles to recommend new or non-frequent interacted products. RDNBR23 uses binary-encoded transactions to capture repetitive purchases. However, it cannot recommend new items that a user has never interacted with, as it heavily relies on users’ repetitive behavior and historical transaction data. This thesis proposes a deep learning-based SR model called BERT based Sequential Mining for Richer Contextual Semantics E-commerce Recommendation (BERT-SEMSRec), an improved SEMSRec20 that replaces the prod2vec model, by enriching existing BERT model to find embeddings from products description and purchase sequences of products. A two-phased architecture is outlined where the first phase, MetaBERT2Vec involves a BERT model that processes description of each product to generate embedding that represent the semantic(meaning) of products. The second phase, SeqBERT2Vec fine-tunes another BERT model to generate positional embeddings of each product in the purchase sequence that represent how the products are appearing in the sequence. These embeddings are then concatenated to generate top-K recommendations. Our experimental results show BERT-SEMSRec consistently outperforms SEMSRec20 and other models like GRU4Rec, SASRec, and BERT4Rec across four categories of Amazon dataset (Office Products, Electronics, Health_and_Household, and Beauty), with up to 12% higher HR@10, 15% higher MRR, and 18% higher NDCG@10, demonstrating superior ability to capture both semantic and sequential product information for recommending products.
Recommended Citation
Dhar, Sudipta, "Bert Based Sequential Mining for Richer Contextual Semantics E-Commerce Recommendation (BERT-SEMSRec)" (2024). Electronic Theses and Dissertations. 9621.
https://scholar.uwindsor.ca/etd/9621