The issue that the GP is pointing at is that the ability to reason in the abstract the author is implicitly stating to be the marker of intelligence.
It may be one form of intelligence, but certainly a brilliant writer, a gifted musician, or an exceptional artist can all be considered intelligent even if their ability to grok logical constructs is limited compared to those that spend their waking hours doing just that, and almost certainly have been honing this skill for their entire lives.
I think the second essential part of the GP's marker for intelligence is the ability to form sentences that convey information, and do it efficiently.
Ability at abstract reasoning is invisible to outsiders unless the bot can also transmit their information to others, as well as understand transmissions from others and react appropriately (constructively or entertainingly).
AFAIK, up to now, none of the measures of synthetic intelligence have tried to measure the flow of information from and into a bot -- its efficiency, coherence, or relevance. I think the rise of master aper bots like GPT-2 and Q&A bots like Watson that beautifully model syntax and rhythm yet no semantics may finally force this issue to the surface. To wit, information matters more than style.
Frankly, I welcome the arrival of bot overlords like these. Maybe they'll motivate us humans to pay more attention to the meat of what we hear, read, and say, and therein act less robotic ourselves.
It may be one form of intelligence, but certainly a brilliant writer, a gifted musician, or an exceptional artist can all be considered intelligent even if their ability to grok logical constructs is limited compared to those that spend their waking hours doing just that, and almost certainly have been honing this skill for their entire lives.