Browsing by Author "DJAMAI , Abdelbasset"

Now showing 1 - 1 of 1
  • Thumbnail Image
    Item
    Arabic abstractive text summarization via supervised transfer and deep reinforcement learning
    (Faculty of sciences , 2024) DJAMAI , Abdelbasset ; BOUGAMOUZA , Fateh
    The rapid growth of digital content has created a pressing need for efficient text summarization techniques, particularly for languages with complex features like Arabic. Although progress in Arabic abstractive text summarization has been limited compared to languages like English, recent advancements leveraging transfer learning with pretrained transformer-based language models (PTLMs) have shown promise. These approaches have primarily relied on supervised learning techniques for finetuning, which typically focus on maximizing next-word prediction like lihood rather than summary quality. They also suffer from in consistencies between training and inference conditions, as well as exposure bias. Notably, reinforcement learning (RL), which offers potential solutions to thes issues, especially its ability to directly optimize non-differentiable objectives, remains largely unexplored in Arabic ATS. This thesis proposes a novel approach combining transfer learning and RL to address these limitations and advance Arabic ATS. We introduce a novel two- phase framework: (1) supervised finetuning (SFT) of a PTLM, followed by (2) RL-based optimization via Proximal Policy Optimization. Our approach is evaluated on the XL-Sum and AraSum datasets, using reward functions derived from different automatic evaluation metrics and textual entailment. Experimental results demonstrate that our RL-finetuned models out perform our supervised baseline across multiple metrics, while demonstrating enhanced factual consistency. Comparison with prior work shows that our models achieve new state-of-the-art performance in terms of BERT Score, while achieving competitive and more balanced ROUGE scores on both datasets. Moreover, a test-only transfer evaluation on a new dataset reveals that our RL-optimized models exhibit superior generalization capabilities compared to the supervised baseline. We also conduct ablation studies to analyze the contributions of some critical components in our approach