Transformers Struggle to Learn to Search

This study investigates transformers' search capabilities using graph connectivity, revealing that while they can learn to search, performance declines with larger graphs, unaffected by model size or in-context learning.

https://arxiv.org/abs//2412.04703

YouTube: https://www.youtube.com/@ArxivPapers

TikTok: https://www.tiktok.com/@arxiv_papers

Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016

Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers

--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support