We have far more oil, coal and gas than we can safely burn. For all the millions of words written about climate change, the challenge really comes down to this: fuel is enormously useful, massively valuable and hugely important geopolitically, but tackling global warming means leaving most of it in the ground – by choice. Although we often hear more about green technology, consumption levels or population growth, leaving fuel in the ground is the crux of the issue. After all, the climate doesn’t know or care how much renewable or nuclear energy we’ve got, how efficient our cars and homes are, how many people there are, or even how we run the economy. It only cares how much globe-warming pollution we emit – and that may be curiously immune to the measures we usually assume will help.

There are three facts that tell you all you really need to know about climate science and politics. One: for all the uncertainty about the detail, every science academy in the world accepts the mainstream view of man-made global warming. Two: virtually every government, recognising the profound danger of tampering with the climate that allowed human society to thrive, has agreed the world must limit the global temperature increase to 2C – a level which isn’t by any means “safe” but may be enough to avoid the worst impacts. Three: the amount of warming we will experience goes up roughly in proportion to the total amount of carbon that global society emits – cumulatively.

Here is the rub. Even if we gave up on all the obscure and unconventional fossil fuel resources that companies are spending billions trying to access and just burned the “proven” oil, coal and gas reserves – the ones that are already economically viable – we would emit almost 3tn tonnes of carbon dioxide. No one can say exactly how much warming that would cause, but it is overwhelmingly likely that we would shoot well past 2C and towards 3C or even 4C of warming.

Four degrees might not sound much but at the planetary level it is. It is about the same as the temperature increase observed since the ice age’s “last glacial maximum”, when much of the northern hemisphere was trapped under ice as thick as the world’s five tallest skyscrapers stacked on top of each other. It is impossible to say what changes another three or four degrees would bring, but the impacts could very plausibly include a collapse in global food production, catastrophic droughts and floods, heatwaves and the beginning of ice-sheet melt that could eventually raise the sea level enough to wipe out many of the world’s great cities.

Impact of climate change: flooding in India. Photograph: Gideon Mendel/Corbis for Actionaid

Sceptics argue that this doomsday scenario might not come to pass – and they are right. If we are lucky, the impact of burning all that oil, coal and gas could turn out to be at the less severe end of the plausible spectrum. But that is hardly reassuring: it’s akin to saying that it is fine to walk blindfolded into a main road since you can’t be sure there are any cars coming. After less than 1C of temperature increase so far, we are already seeing some profound changes, including a collapse in Arctic sea ice coverage more severe than even the most pessimistic predictions from just a few years ago. (Brits secretly hoping for a hotter future, be warned: that collapsing sea ice may have caused the freakish jet stream behaviour that made 2012 the wettest English year on record and obliterated this year’s spring, both mere amuse-bouche for the feast of climate impacts expected in coming decades, even from the carbon we’ve emitted so far.)

Given what is at stake, it is no wonder that governments agree global warming must be stopped. But that is where the common sense ends and the cognitive dissonance begins. Because to have a decent chance of not exceeding the already risky global target, we need to start phasing out fossil fuels now at a fast enough rate to bring down emissions globally by a few percent a year, and continue doing so for decades to come.

Now compare that with what is actually happening. As with the climate, to understand the situation properly it is necessary to zoom right out to see the long-term trend. Doing so reveals something fascinating, worrying and oddly overlooked. As scientists from Lancaster University pointed out last year, if you plot a graph showing all the carbon emissions that humans have pumped into the air, the result is a remarkably clear exponential curve stretching all the way back to the mid-19th century. Zoom back in on the past decade and it is clear that for all the mounting scientific concern, the political rhetoric and the clean technology, nothing has made a jot of difference to the long-term trend at the global level – the system level. The growth rate in total carbon emissions in the past decade, at around 2% a year, was the same as that of the 1850s.

CO2 emissions since 1850 (red); exponential growth (blue); cuts to hit climate target (dashed). Photograph: guardian.co.uk

That might sound hard to believe. After all, thanks to green policies and technologies, emissions have been falling in Europe, the US and many other countries. Wind turbines and solar panels are ever-more common, not just in the west but in fast-growing China. And the energy efficiency of cars, light bulbs, homes and whole economies has been improving globally for decades. So why isn’t the carbon curve showing any let up? Some might instinctively want to blame the growing population but that doesn’t stack up. The rate of population growth has dropped like a stone since the 1960s and is no longer exponential, but the carbon curve doesn’t appear to have noticed that any more than it has noticed the Kyoto protocol or whether you cycled to work this morning. For whatever reason, cutting carbon has so far been like squeezing a balloon: gains made in one place have been cancelled out by increases elsewhere.

To understand what is going wrong, it is necessary to consider the nature of exponential growth. This type of accelerating trend crops up when there is a feedback loop at work. For example, a credit card debt grows exponentially because interest gets applied to ever more interest. The number of algae in a jar grows in the same way: as long as there is food and air, there will be more algae and so they can breed faster.The fact that our carbon emissions have followed the same accelerating trend suggests that our use of energy is driven by a similar kind of feedback loop which is cancelling out apparent green gains.

That certainly fits with history. The industrial revolution that kick-started the human impact on the climate was driven by just such a feedback. The steam engine enabled us to drain coal mines, providing access to more coal that could power more steam engines capable of extracting yet more coal. That led to better technologies and materials that eventually helped ramp up production of oil as well. But oil didn’t displace coal, it helped us mine it more effectively and stimulated more technologies that raised energy demand overall. So coal use kept rising too – and oil use in turn kept increasing as cleaner gas, nuclear and hydro came on stream, helping power the digital age, which unlocked more advanced technologies capable of opening up harder-to-read fossil-fuel reserves.

Seen as a technology-driven feedback loop, it is not surprising that nothing has yet tamed the global emissions curve, because so far nothing has cut off its food supply: fossil fuels. Indeed, though our governments now subsidise clean-power sources and efficient cars and buildings – and encourage us all to use less energy – they are continuing to undermine all that by ripping as much oil, coal and gas out of the ground as possible. And if their own green policies mean there isn’t a market for these fuels at home, then no matter: they can just be exported instead.

Impact of climate change: ice melt in Antarctica. Photograph: Peter McBride/Barcroft Media

This extraordinary double-think is everywhere to be seen. Take the US. Obama boasts that American emissions are now falling due to rising auto efficiency standards and gas displacing dirtier coal in the energy mix. But the US is extracting carbon and flowing it into the global energy system faster than ever before. Its gas boom has simply allowed it to export more of the coal to other countries such as China – which of course uses it partly to produce goods for US markets. Not happy with increasing US carbon extraction, Obama is also set to approve the Keystone XL Pipeline that will enable Canada to flood the global markets with crude produced from dirty tar sands. So much for carbon cuts.

Or take Australia, which in the same year introduced a carbon tax and started debating plans for a series of “mega-mines” that would massively increase its coal exports, helping build confidence among the companies and governments planning no fewer than 1,200 new coal-fired power stations around the world. Even the UK, with its world-leading carbon targets, gives tax-breaks to encourage oil and gas recovery and has been growing its total carbon footprint by relying ever more on Chinese factories – and therefore indirectly its reliance on American and Australian coal. And not just that. Although it rarely gets commented on, Britain – along with other supposedly green nations such as Germany – regularly begs Saudi Arabia and the other Opec nations to produce not less oil, but more. As journalist George Monbiot once put it, nations are trying simultaneously to “reduce demand for fossil fuels and increase supply”.

It is not just governments that are in near-universal denial about what needs to happen to the fossil fuel sector. Blithely ignoring the fact that there is already far more accessible fuel than can be safely burned, pension fund managers and other investors are allowing listed fossil fuel companies to spend the best part of $ 1tn a year (comparable to the US defence budget, or more than $ 100 for every person on the planet) to find and develop yet more reserves.

If and when we emerge from this insanity, the carbon bubble will burst and those investments will turn out to have been as toxic as sub-prime mortgages. Don’t take my word for it. HSBC analysts recently concluded that oil giants such as BP – beloved of UK pension funds – could have their value cut in half if the world decides to tackle climate change. Coal companies can expect an even rougher ride, and yet our financial regulators still allow them to float on stock markets without mentioning in their share prospectuses that their assets may soon need to be written off.

But for now, the fuel is still flowing freely. And for as long as that continues, the global energy feedback loop will ensure that many of the things we assume will help may be ineffective – or even counterproductive. More efficient engines may simply enable more people to drive more cars over greater distances, triggering more road building, more trade and indeed more big suburban houses that take more energy to heat. New renewable or nuclear power sources might just lead to more economic activity, increasing demand and supply of all energy sources, including fossil fuels. And local carbon cuts caused by green choices, population decline or even new economic models may simply free up more fuel for use elsewhere.

Of course, oil, coal and gas use will level off eventually no matter what we do. Fossil fuels are a finite resource and each year they get more expensive relative to renewables and nuclear. But given the continued acceleration not just in fossil fuel extraction but in the production of cars, boilers, furnaces and power plants that need oil, coal and gas to function, there is zero prospect of that happening of its own accord any time soon. Forget peak oil caused by dwindling supplies. At least until we’ve cracked cheap carbon capture, we need to bring about peak fossil fuels. Voluntarily. And soon.

We know how to do it. A properly designed global cap and trade scheme is one option. Stiff taxes on the production or sale of carbon-based fuels is another. Or we could simply oblige companies taking carbon out of the ground to arrange for a rising share of what they extract to be buried again. Any of these models could bring down global emissions and stimulate an explosion of investment and innovation in clean and efficient energy systems. But there is no avoiding the unpalatable side-effects: spiralling fuel and energy prices; a write-off of fuel reserves worth many trillions of dollars; and a fierce global squabble about how to share out the fuels we do decide to burn.

How would all this affect the global economy, or pension funds, or the financial health of the Middle East, the US and other carbon-rich nations doing most to resist a global climate deal? For all the confident opinion on both sides, no one can say for sure, just as no one can be certain how human society will fare in a warming world. But with so much money and power bound up with oil, coal and gas, one thing seems clear: constraining global fossil fuel supplies will take bigger thinking, harder politics and – crucially – a whole lot more public pressure. Voluntary carbon cuts are a great start but they are no match for a system-level feedback in human energy use.

Globally, the vast majority of people want climate change dealt with. But can we bring ourselves to prioritise a safe planet over cheap fuels, flights, power and goods? Can we face calling on our leaders to end the double-think and constrain oil, coal and gas supplies on our behalf? Can humanity muster the restraint and cooperation needed to leave assets worth trillions in the ground?

This article is based on the book The Burning Question by Mike Berners-Lee and Duncan Clark, which is published on 20 April by Profile Books, price £9.99. To order a copy for £7.99 with free UK p&p, go to guardian.co.uk/bookshop or call 0330 333 6846