The Whisper Challenge Is Harder Than It Looks