What is the Novel for, a correspondent wonders in ‘Knowing French’, one of the several stories about the contemporary condition of aesthetic production in Julian Barnes’s collection – expectably wry, dry, witty, and of course essayistic – The Lemon Table. Is it for padding out the philosophising with stuff about cigs and drinks and what the waiter said? And for ‘undemanding’ loss of self ‘between about 10pm and bedtime’?

A lot of this year’s British and Commonwealth fiction shares a dyspepsia about artistic work. Patrick McGrath’s peripatetic-gothic Port Mungo does – its painters busily messing up lives, their own and other people’s, in the Caribbean and New York. So does Colm Tóibín’s The Master, devotedly recreating Henry James’s anguished turn in the 1890s from failed drama to knotty, difficult, self-reflexive novels. Tóibín’s fraught, crypto-gay James knows he’s not ‘living all he can’ – for the sake of his writing. (It’s a wonderful novel, this, for those readers up for Tóibín’s nicely hinting replay of the germinatings of The Turn of the The Screw, The Ambassadors, and the rest.) The literary types in Peter Ackroyd’s The Lambs of London are involved in literary fraud (the faking of a hoard of Shakespeare manuscripts) as well as in Ackroyd’s usual delve into old London mayhem, madness and murder. Rachel Seiffert (one of our very best newcomers) has an architect in her new collection Field Study breaking down on the big vision thing and finding contentment in designing a tomato-plant house on his dad’s allotment. Philip Hensher’s The Fit exuberantly mocks the conceptual artists its indexer-hero gets tangled up with (Hans-Kleist Klugg’s used-condoms photography project; the art student Hope’s two-hundred plaster penises of all the men she’s ever slept with). Howard Jacobson’s wonderfully comic novel The Making of Henry, about elderly jewish folks in London’s St John’s Wood (all loving, and gorging on gossip, morbidity and expatriated Viennese pastries) indulges its one-time literature teacher in his great un-pc riffs about the awfulness of literary theory especially in its feminized or gynocratic branches. ‘What happens when you give your life to teaching the literary history of girls?’ Where are the syllabus’s books about men? And so forth. Henry’s monstrous rants work, as ever with Jacobson, precisely through their fired-up rhetorical excess.

Port Mungo by Patrick McGrath

What hope – this seems to be a going gist – remains, then, for old plots and forms, those one-time stabilizers of contemporary anxieties and distresses that the Novel was supposed to be so well adapted for? Not all that much, this year’s crop seems to reply. In Val McDermid’s latest, and very superior, take on the Crime Story, The Torment of Others, for instance, the crimes are impressively up-to-the-minute (paedophile rings, vile S-M mutilations), as are the tools of criminological knowledge (profiling, techno wizardry, reading the eco-scene of the crime), and the new detectives – given to lesbian jealousies, getting mown down in the cause – are ones the tradition of god-like salvific crime-solving would not recognise at all.

Clearly the old generic way and workings are being carefully shown tottering and trembling, even if they haven’t quite fallen yet into utter ruin. Take Alan Hollinghurst. His superiority lies precisely in reworking the serious traditional love-story and Bildungsroman as precarious, apocalyptic even, gay-sex adventurism. It’s no accident that the three main young protagonists in his long, finely tuned, intricately revelatory and utterly stylish The Line of Beauty (watch those awesome adjectives and adverbs!) are called Nick, Toby and Catherine, named of course for Iris Murdoch characters (in The Bell actually), nor that Nick is writing a thesis on Henry James. Hollinghurst knows well the line of fiction he’s quietly undoing.

Not all that dissimilarly, Justin Cartwright’s The Promise of Happiness treads through the familiar spaces of the bourgeois family saga with lovely close-up attentiveness to the Judds’ doings (parental retirement to Betjeman’s Cornwall, international dot.com fortune for the son, glam art-historical career in New York for one daughter, ritzy life in TV-commercial production for the other), but as traditional plots of fortune go these are awfully blunted by modern catastophe (father kicked out of City job in takeover kerfuffles, coked-up filmic daughter tormented by goatish older producer boyfriend, aesthetic daughter jailed for involvement in US scam with stolen Tiffany windows). The novel climaxes in a great big English wedding for millionaire son and his gorgeous shopaholic South American bride, but this reads like a conscious satire on the old happy-ending formulae. Luxurious flowery-bowery nuptials can no longer be made to work as resolution for a novel’s throng of terribilia.

Dry Bones by Richard Beard   The Fit by Philip Hensher

And this year’s novels are indeed mainly preoccpied with terrible things of one sort or another. To be sure, comedy, and the knowing grin of satire, do help keep at bay some of the gloomiest current ponderings on melancholic histories, bad prime ministers and the cruelties of war. Howard Jacobson, master of the one-liner and the Black Jewish joke, has to be saluted as the now certain successor of the late great Mordecai Richler, the most delighting squeezer of laughs out of the otherwise desponding case. Some gimmickry of form or plot is a great alleviator too – as in Richard Beard’s Dry Bones, about the hunt for profitable relics in present-day Genevan churches and graveyards (Thomas-à-Becket’s toe, Calvin’s bones, Richard Burton’s skull, and such) – a compellingly zany engagement in a sort of secular resurrectionism in the face of profound religious decline; or as in Philip Hensher’s The Fit – offering us an actual index to its indexer-hero’s encounter with the vagaries of modern marriage, female clothing and conceptual artists; or, most impressively indeed, in Iain Sinclair’s latest genre-busting factual-fictional arm-wrestling of the topography and political-geography of contemporary Britian into meaningful form, Dining on Stones – an autobiographical, intertextual, Conradian travelling meditation about walking southwards out of London down the A13 road, which makes a startling, astounding successor to Sinclair’s cultic Lights Out for the Territory and his great London Orbital, his book about walking around the M25 motorway.

But what really presides hereabouts tends to be some form of the melancholic – often lovely of course as in the masterly broodings of William Trevor, the Irish doyen of the doleful short-story still attractively at work in the tales of A Bit on the Side – but it is after all the making of aesthetic goods out of what the greatly influential Sinclair-admiree and guru J. G. Ballard has cannily labelled the modern ‘library of extreme metaphors’. The Now appears in the novels of this year so commonly as Apocalypse Now. In Amitav Ghosh’s extoically brooding The Hungry Tide, it might be (a half-Indian cetologist, and her Indian male acquaintance who’s on a mission to delve into his uncle’s diary record of past violences, get engulfed in a nightmare typhoon in Bengal’s watery mangrove badlands). Or in Maggie Gee’s The Flood (featuring watery bad-times for a city and culture oppressed not least by its untruthful warfaring Prime Minister Mr Bliss: guess who he’s meant to be). (‘Blair’s Britain’: it’s taken over now almost completely, as our novel’s preferred site and sign of the real and the ordinary gone plightful, from the ‘Thatcher’s Britain’ which proccupied so muuch fiction of her time, and which survives now only in consciously retro corners like David Peace’s large angry documentary about the ‘Great Miners’ Strike’ of 1984, GB 84.) What’s specially good about Maggie Gee’s present-day dystopia is how the London of Dickens and Woolf is made to blend most artfully into more recent Sci Fi-type forebodings as patented by J. G. Ballard.

Badlands all. And bad histories – Apocalypse Then, as one might say – impinge with equally nightmarish force – as, variously, in Robert Harris’s Pompeii, his very gripping recreation of the most notorious of ancient volcanic eruptions; and in Louis de Bernière’s really very good epic Birds Without Wings, about the early 20th century struggles between Turkey and Greece (how excellent de Bernière is at recreating modern warfare, his Gallipoli is as fetchingly good as anything by Sebastian Faulks or Pat Barker); and in The Secret Purposes, third novel of David Baddiel (professional comic, unexpectedly metamorphosed into a very intelligent and compelling fictionist), about Jewish refugees from Hitler unkindly and stupidly interned en masse on the Isle of Man during the Second World War. Our novels just can’t stop the massive remembered violences done to the human from keeping on poking through – as hot war, say, as in de Bernière or in Maggie Helwig’s aweingly grim take on the Bosnian conflicts of recent times Between Mountains; or as Cold War reheated, as in John Le Carré’s most superior Absolute Friends (how odd it now seems to recall those doomsters who thought the fall of the Berlin Wall would dry up his subject-matter: old Cold War footsoldiers never die, not yet anyway); or as recent colonial exploitations coming destructively home to roost in London around the person of a returned radical missionary priest in A. N. Wilson’s My Name is Legion (Wilson’s best novel for some time).

All told, and one way or another, dementing memories and histories wash compulsively over our fictions like this, making for a real dominance of stories of mental, spiritual and bodily distress. How might survivors, inheritors of corrupt knowledge, tell what they’ve remembered and learned: so wonders the daughter-in-law of an old Stasi-informer in Rachel Seiffert’s story ‘Dimitroff’ (in Field Study). It’s a dominant question for these novels and novelists. Our fathers have, to coin a phrase, eaten sour grapes, so it’s no wonder the children’s teeth are set on edge – and I’m thinking not least of Liz Jensen’s most moving presentation of estranged boyhood in The Ninth Life of Louis Drax.

Meanwhile, do look out for Liz Jensen and for many of the above on the Man Booker long-list. If I had my way, Jacobson, Hollinghurst, de Bernière and Tóibin would all be shortlisted. And Iain Sinclair would win. Fat chance, of course, of that.

Valentine Cunningham is Professor of English Language and Literature at Oxford University, and Tutor and Senior Fellow in English Literature at Corpus Christi College, Oxford. He has twice been a judge for the Booker Prize (1992 and 1998), and was a Regional Chair for the Commonwealth Writers Prize (Eurasia Region) in 1999 and 2000. His most recent book is Reading After Theory.

Editor's note: Achmat Dangor, Sarah Hall, Alan Hollinghurst, David Mitchell, Colm Tóibín and Gerard Woodward are the six authors shortlisted for the Man Booker Prize for Fiction 2004. The winner will be announced on 19 October.