This time yesterday, Gillian Duffy was an ordinary pensioner from Wigan. By mid-afternoon, the prime minister was in her house, begging for his political future. Her ordinary, white front door was broadcast live on TV across Britain and the world, via the internet, for 45 minutes or more.

Her world changed when Gordon Brown referred to her as a ‘bigoted woman’ in a private conversation in his car, seconds after a seemingly friendly, if sometimes spirited, exchange in the street. Her ‘offence’ was to question the level of immigration in the UK, especially from countries in central and eastern Europe.

Mrs Duffy is a typical, white working-class voter. People like her are the bedrock of Labour support – despite the middle-class liberal views of Gordon Brown and others at the top of the Labour Party (as well as many ordinary members, such as myself). Her concerns matter – even if they are sometimes based on a false perception, or years of right-wing newspaper headlines.

The truth is that Labour has done far too little to address these concerns, or to communicate the benefits of immigration, for fear of the press. And Labour will be electorally punished for it.

The comment was bad (as well as the attempts to blame an aide for the encounter. However, it could happen to anyone – hence the muted reaction of the other parties. (In fact, which one of us in our own lives can say we haven’t been nice to someone in public and bad-mouthed them once their back is turned?)

Yet for me, the management of the comment was worse. Gordon Brown doesn’t apologise well – it’s always ‘if I said that’, ‘if I caused offence’. When the clip was played to him on Radio 2 (see video), his reaction spoke volumes – blaming the media for broadcasting a private conversation (Sky controlled the radio mic for a pool of broadcasters – notably, it is said that Tony Blair always had his radio mic controlled by a Labour official on these occasions); claiming that he was under pressure to leave, and couldn’t answer the question. Neither excuse was credible – but typical of Brown’s disconnection from his own actions.

Then he returned to apologise in person (something Mrs Duffy said she was not bothered about). This public self-flagellation merely perpetuated the story. It was poorly conceived, and also poorly executed. He claimed he had misunderstood her point – but as others have said, how? It was another flimsy excuse, to cover up another gaffe.

Three years ago, I opposed a ‘coronation’ for Gordon Brown. He is a very good political thinker – but not a good 21st-century politician. He is uneasy with people. He is uneasy with the media. He struggles to communicate in human terms, instead talking in statistics. Could you imagine, for example, Alan Johnson getting into such difficulties?

There are still seven days to go to polling day. But it’s going to be hellishly difficult for Gordon Brown to lead Labour to a win from here – or even to stop a Tory win.

Labour members that I’ve spoke to are annoyed and angry. We work hard, only to have our slim chances of victory dramatically reduced by an amateur hour mistake.

If our own members aren’t motivated, and our own natural constituency doesn’t believe that we’re on their side, that we can deal with their concerns, then the game is up.