Both LLMs and most humans treat symbols as determined. Symbols are maps, not territory. LLMs just inherit our ignorance to this and stubborn attempts make circle from a square.