Puzzling and prospering the United States certainly still is, but the country through which I recently travelled is now also cynical and concerned, quizzical and querulous, brash but bluffed. In 44 years' visiting the US, I have never before known Americans to be embarrassed or abashed about their own nation or its leaders. Even during Watergate, when those trips began, American friends – cautiously, incrementally but proudly – rejoiced in the way their system worked, in discovering once more that theirs was "a government of laws and not of men".

Wherever I went recently in the US, locals were buying two books about themselves. One, Ron Chernow's Grant, amounts to bare-knuckle nostalgia. The biography is a plodding chronicle, overshadowed throughout by the 336,000 words in Grant's own stunning memoir. Nonetheless, Chernow is responding to Americans' appetite for stories about a genuine American hero, albeit a frequently drunk, brooding, scandal-ridden, brutal one.

The other, Kurt Andersen's Fantasyland, seeks to prove that the great American motor has been not Grant and his ilk but rather a bizarre miscellany of nutters, outcasts and fantasists. Andersen concludes: "Americans have given themselves over to all kinds of magical thinking, anything-goes relativism, and belief in fanciful explanation, small and large fantasies that console or thrill or terrify us". That appraisal answers readers' awareness that something has gone badly wrong ("haywire", Andersen would say) and their concomitant wish to establish where and when the sorry tipping point arrived.

Other explanations of America's predicament (J. D. Vance's Hillbilly Elegy or Robert Packer's The Unwinding especially) contain more worthy counsel but no book can supply coherent answers. After all, what would we read to explain Australia to ourselves? Ned Kelly's Jerilderie letter? 'Seven Little Australians"? Henry Parkes' Tenterfield speech, or John Curtin's "look to America"? Waltzing Matilda?

To look for America, immersion therapy is required. I tried a dose of that during a road trip through four states (Texas, Arkansas, Tennessee and New York). Driving myself was essential, for driving is both a right and a rite in America. America's highways are funnels and tunnels, designed for speed, pandering to the unfettered individual, with any roadside stop likely to prove lethal for anyone wanting edible food or unsugared drinks. When the woman I love drew unwanted attention to dripping fat and grease on her plate, the waiter dismissed her worries by noting that "we're not known for healthy food 'round here". Locals' grizzles were focused elsewhere, on any disturbance to the right-rite of the road. They were annoyed at trucks on the highway, vexed at roadworks, grumpy about bends on mountain roads. Living free had been mistaken for driving fast.

Any detour or digression, though, promised real America within half an hour's drive. More texture and context are to be found than, say, in trying to decode Australia by examining RSL clubs, Shell servos and commercial hotels in country towns. We exited at Fort Worth stockyards and parked next to the chapel, advertised by parishioners "Stepping out for God" while a herd of horses were "Galloping for His glory". Away in the Ozarks, the controversy about football players taking a knee during the national anthem was reprised outside a church as: "We stand for the flag and kneel before the cross". In the liberal enclave of Austin, a clothes shop served Vegan Zen chocolate bars. Inside Memphis' pyramid, a shop stocked Micro Vigilante camo overalls for any young juvenile with a gun in his hand. On Fifth Avenue, a homeless person had scribbled the most poignant of cardboard signs: "I don't matter to anyone".

America is not sideshow alley. There are still better angels to its nature. Outside that pyramid in Memphis, past city authorities had decided to lock up the Mississippi waterfront now and forever, for the pleasure of the public. A couple of kilometres away, anyone who cared to do so could watch a 1960s civil rights video of training for sit-ins at lunch counters; that is, training to sustain non-violent resistance when confronted with abuse, spitting, kicks and slaps. When I thanked a taxi driver outside Memphis, she solemnly advised me that "it's been a blessing having y'all today".

Richard Nixon had a point, when he observed that, as president, he had hoped to be able to look at America as though through a telescope. Instead, he found himself peering through a kaleidoscope. Is there a pattern to the jigsaw, which makes it possible to govern effectively such a large, entitled, powerful and fissiparous society? The public-policy conundrum underpinning that query has to do with how well institutions – sometimes sclerotic, sometimes cosseted, sometimes partisan – can adapt and coordinate. In turn, that question circles back to worries about the fact that Americans, for more than a quarter of a century now, have elected a succession of presidents, each with different flaws, but all limited in their capacity to exercise the full powers of their office for the public good. Now that trio (Bill Clinton, George W. Bush and Barack Obama) need do nothing but watch ruefully, as the Trump presidency retrospectively elevates their talents and enlarges their legacies.

America can invariably be trusted to come up with something new. I am talking not merely about new generations of smartphones or robot workers but consumers' lust for novelty. This time in Manhattan, those with iron stomachs could buy a sushi burrito, a papaya dog or a mug of Irish coffee on tap. Novelty, though, may need to be blended with a capacity to make things (by which I mean institutions) work. Perhaps the failure to manage that was Obama's limiting flaw.

Institutions do not have lives of their own. The animating spirits are an eclectic bunch: leadership; a sense of public purpose; structured objectives; consensus support. Their combination, and hence their force multipliers, are now under strain. It would be a miserable day if we lost faith in Americans, but a truly tragic one if Americans were to lose faith in themselves.

Mark Thomas is a Canberra-based writer.