Date of Submission

5-2026

Document Type

Thesis

Degree Name

Master of Science in Computer Science

Department

Electrical & Computer Engineering and Computer Science

Advisor

Mohamad Nassar, Ph.D.

Committee Member

Saida Elmi, Ph.D.

Committee Member

Muhammad Aminul Islam, Ph.D.

Keywords

Lightweight open-source language model, word-association games, Supervised Fine-Tuning (SFT), LLaMA-3.1-8B-Instruct, Claude Opus 4.6, GPT-5.2

LCSH

Artificial intelligence--Data processing , Open source software, Lateral thinking puzzles, Supervised learning (Machine learning)

Abstract

Recent progress in large language models has produced reasoning-oriented systems that allocate additional computation at inference time (e.g., deliberation, verification, and self-correction), raising the question of whether these gains can solve tasks requiring creative and lateral thinking. In this thesis, we study such capabilities using two popular word-association games—LinkedIn Pinpoint and New York Times (NYT) Connections—which require identifying non-obvious patterns or relationships. We first benchmark some light-weight open models and analyze their performance trends with respect to puzzle release dates and approximate model knowledge cutoffs to distinguish memorization effects from genuine inference. Our results show that accuracy of traditional embedding-based baselines remains low, while stronger instruction-tuned models perform substantially better, with LLaMA-3.1-8B-Instruct achieving 31.50% accuracy on Pinpoint and 19.33% on NYT Connections. Second, motivated by the scarcity of large, high-quality labeled datasets for creative reasoning tasks, we develop a scalable pipeline for generating Pinpoint-style puzzles by seeding categories from Wikipedia and producing candidate puzzles using Claude Opus 4.6 as a teacher model. We fine-tune three lightweight open-source models such as LLaMA-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct on this synthetic dataset via Supervised Fine-Tuning (SFT) and evaluate puzzle quality using GPT-5.2 Thinking as an automated judge. Our results show that SFT substantially improves puzzle quality across all three models, with fine-tuned models preferred over their base versions in 65–73% of pairwise comparisons, demonstrating that smaller open-source models can be meaningfully specialized for creative generation tasks without access to proprietary infrastructure.

Download

Share

COinS