Staff Contributors
Guest Contributors

Calling all economists

Are the CGE models useful for predicting the effects of climate policy?

Posted by Clark Williams-Derry (Guest Contributor) at 11:03 AM on 05 Jun 2008

Read more about: economy | business | carbon trading | climate

sandcastle
Photo: StuSeeger via Flickr.

My pal Peter Dorman is looking for answers: Does the class of economic forecasting tools known as "computable general equilibrium models" (aka CGE models) have any documented track record of success?

This may seem like an arcane point, but it's quite relevant to climate policy. Government agencies throughout North America are using CGE models to forecast the economic impacts of various cap-and-trade proposals. But many academic economists -- Dorman among them -- think that the CGE models are built on sand. Says Dorman:

I think these models are so dubious theoretically and unreliable in practice that there is no case for using them.

Peter's a smart guy, and he's looked and looked. But he's found no evidence that anyone's actually tested the reliability of CGE models, to see if their predictions match up with real events as they unfold.

All it would take is a retrospective study, lining up CGE-based forecasts made in, say, the early 1990s, and comparing them with what actually happened. But search as he might, he's found nothing of that sort in the literature.

Worse, he suspects that commodity traders and hedge fund managers -- the sorts of people who get paid a lot of money to predict the future -- simply ignore the results of CGE models. Sure, policy makers and academics use CGE models all the time, but there's no real personal harm to them if the models lead them astray. For people with skin in the game, though, a bad guess is a costly one, and a good guess is worth a mint. If market traders don't use CGE models in their work, then that's a decent piece of evidence that those models just aren't all that useful for predicting the future.

So for anyone out there who's economically inclined, Peter hopes you'll spread his challenge: If you have evidence that CGE models are worth their salt -- and that their predictions are any more reliable than, say, informed guesswork -- please let him, or us, know.

Here's a trail

A trail to follow.  Did LTCM use a similar model?  Maybe that's why hedge fundies are loathe to follow down the academic economist garden path.

Hedge funds mainly rely on illegal inside information to make "investment" decisions.  A wink of the eye and a new bubble is off to the financial sewer, clogging everything up until the shit hits the fan.

As has happened in global credit markets targeted by the sub-prime mortgage bundling scam.  And is happening now in fertilizer, farm land, and energy.

Are carbon emission permits next?  Yep, as soon as cap and trade gets put into place.

http://amazngdrx.blogharbor.com/blog John Schneider, Northern Wisconsin

Models

Clark makes a good point, and is the problem with economic forecasting in general.  The truth is that virtually every economic forecasting tool has a crummy track record.  Economists are about as good at predicting the future as any given carnival gypsy.  (A good friend at a DC energy think tank summed up the problem to me by saying that "economists would be more disciplined if they were forced to eat the shards of all their shoddy crystal balls".)

That's not to say that there isn't value in economic modelling - just that as a discipline, economics is much better at explaining what happened than what will happen.  And yes, damn few policies ever factor this in.

As your wikipedia link notes, CGE models make neo-classical assumptions of economic optimums that almost never exist outside of economic theory.  They assume capital is optimally allocated (meaning that any incremental change in capital allocation is sub-optimal, in an economic sense.  In other words, the reason that you don't own a more fuel efficient car is because if you did, society would be economically disadvantaged.)  It takes about 3 seconds of thought to realize how goofy this is.  But if you don't make that assumption, most economic theories don't work.  It's innate to their math.  Moreover, any mathematical model by its nature can only predict those things which is was structured to predict.  

(Dr. X: this is the reason LTCM collapsed, not because of any malevolence.  They built massive models based on options pricing that assumed certain risks associated with price volatility.  Their assumption was wrong.  But had their assumptions been correct, their model would have worked - as indeed it did, initially.)

This means that the models are blind to precisely those things which would prove them wrong.  New technologies.  Rapid shifts in productivity.  9/11.  Rapid changes in immigration.  And so on, and so on.  Moreover, those models are generally much better at calculating costs than revenues, which makes many wealth transfers appear to be economic losers rather than net-neutral to positive events.

One fun fact I learned this past week is that the Annual Energy Outlook - the DOE's model of future energy prices - has a massive mathematical model that is ultimately all driven by a fixed assumption about the ratio of capital:labor:energy use in society.  One wonders how a model like that would ever capture de-industrialization, growth of energy efficiency or productivity gains.  It can't.  And indeed, the AEO is consistently wrong.  

But - and this is sad to say - it's the best model we've got.  So when all is said and done, I totally agree with you, but in a rather pessimistic way.  The models aren't worth a damn.  We make far too many decisions as if they were predictive.  But unfortunately, we don't really have any better options.  So either we use those, in spite of their faults, or else we confess our ignorance.  The former is more comfortable, if a bit less honest.

Ok Sean

I'll buy that.  I don't know wether I will actually bother to try to understand the economic model though.  Economics could permenantly alter one's consciousness, it should only be studied by prescription, under a (meta)physician's care.

Basing any kind of dispensation or investment of scarce capital available to revive our economy and fight GHG disater?  Not so good.  Hehey.

http://amazngdrx.blogharbor.com/blog John Schneider, Northern Wisconsin

But Sean,

If we make policy based on the models, and models are systematically biased in favor of inaction, then our policy is systematically biased toward inaction. Doesn't seem like we should be fatalistic about that! Let's get mad as hell and not take it any more and so forth.

grist.org
Fit the curve

"...any mathematical model by its nature can only predict those things which is was structured to predict."

Is this why allegedly "anomalous" results that don't fit the statistical curve expected are thrown out?   I understand this is the norm in prescription drug testing studies.

Is this the same statistacal model allowed in economics?  If a particular part of the model comes up "wrong" it is discarded?

For instance if solar or wind power suddenly exhibit huge consumer demand pressure (as they have been for the last few years) that would indicate probable exponential growth. Are supply constraints ignored in economic modeling?  Capital tied up by traditional energy industries who protect their own market positions, with their power over capital markets?

This would indicate that money invested in say, solar PV foundries or wind power manufacturing, would result in an immdeiate bump in renewable growth, outside the norm of economic models.

I know from trading that even computer backtesting of a particular market model cannot guarantee a future market result.  There is no sure thing.  That is what economic models offer us, a sure thing.  An unlosable bet on the future.

That's "Twilight Zone" material.  The horse racing tout episode.  It's how all stock touts work.


http://amazngdrx.blogharbor.com/blog John Schneider, Northern Wisconsin

David

One solution is not to frame these as economically consequential policies.  We are quite good at passing economically disastrous policies without first building economic models.  (Think Iraq, or anything else that is introduced as "The War On _".)  Those are all generally framed as some moral or global obligation, without first having to prove that the economic consequences are acceptable.  It's a thought...

Obviously, these models are flawed

but I don't think they're any more flawed than any other scientific models- the phenomena is simply too complex to predict with accuracy- doesn't mean the information isn't useful as a guide, a way to examine ranges, to see how different variables may influence projections, etc.

They shouldn't be the end all be all but if we don't use any modeling what are we going to do, sit in a room and use tarot cards? We need some baseline to begin the conversation.

And economists are pretty good at some types of predictions. For example, monetary policy is quite effective- Ben Bernanke has done a tremendous job helping us avert major economic catastrophe- he based his analysis on some degree of predictions- not CGE models- but on, if I do A, B will happen. I'll take that any day of the week if it works.

Economic Illiteracy Harms The Planet! www.voicesofreason.info.

I was hoping you'd weigh in, Jason!

And my only point of disagreement is that I like E.O. Wilson's definition of science: namely, that it is predictive.  Economics really doesn't pass this test - not because of any laziness on the part of economists, but because it simply isn't sufficiently advanced.  (As Wilson notes, medicine fails this test as well, so don't take any undue offense!)  But we have about as much ability to predict how regulation X will affect GDP as we do to predict how a 10% increase in one's selenium intake will affect one's risk of prostate cancer.  Do we have some vague, directional answer?  Sure.  Is it sufficient to conclude that we now have a cure for prostate cancer.  Nope.

Optimal Allocation of Capital

Sean, on your first point that the model's assuming optimum allocation of capital means: "In other words, the reason that you don't own a more fuel efficient car is because if you did, society would be economically disadvantaged"

It's worth noting that the fuel efficient vehicle example is so clearly goofy because it represents an environmental externality.  Since the cost of carbon pollution is not reflected in individuals' transactions, neither is the benefit of averting carbon pollution.  Fundamentally this is more a problem with the regulations governing our economy than with the model.

The 'optimum allocation of capital' assumption is really two assumptions: 1) Individuals allocate their capital in the way that is optimum for them; 2) The best societal result will follow from everyone acting in their own economic self interest.  

The first assumption, while not perfect, is hardly ridiculous.  And the second assumption is of course dependent on the rules that govern the marketplace.  But at core its pretty aligned with your general advice that the government should set goals and let the market (which is the collection of individuals acting in their own self interest) find a way to achieve them.

If the cost of carbon pollution were internalized, it would make more (if not perfect) sense to assume that someone's not buying a more fuel efficient car was society's optimum allocation of capital.  Because if they 1) know what's in their own financial interest, and 2) correctly spend that extra money on something that saves them more than the car would in lower fuel prices, then they have successfully (if accidentally) lowered the carbon output of the economy more than they would have if they bought that extra car.

Ultimately, the answer in cases like these is not changing the model's assumptions, but in changing the rules (laws) of the market.

Also, your fun fact reminds me of a similar one I recently heard - apparently the CPI exaggerates inflation, or at least its burden on consumers, because it assumes a fixed ratio of goods in the 'basket.'  That is, if gas prices double, presumably people will start to use less gas, but the model assumes that those consumption levels (as percentage of overall consumption) stay fixed.  
So obviously sometimes there are really poor assumptions built into models that really are just the fault of the model.

Max - I think the problem is deeper

There's no shortage of evidence that neo-classical economics oversimplifies human behavior, in a pretty erroneous way.  Behavioral economics is the most recent example, but far from the only one.

When an industrial facility tells it's employees that non-core investments will not get funded unless they achieve <2 year simple paybacks, they are behaving rationally from a managerial perspective (in the sense that they are encouraging employees to focus on their core activities), and they aren't constrained by regulation per se.  But they are also behaving sub-optimally in a neo-classical sense, since it means that they are consciously choosing not to invest capital in projects that will pay 40% dividends.  When a mutual fund manager who spends her day chasing 11% returns chooses not to put double-paned windows in their new McMansion, squandering 30% returns in the process they are behaving no less rationally (in the sense that the windows are out of their expertise, and therefore "riskier", or worse, simply don't save enough money to matter to the individual's bank account, notwithstanding the return on capital.)

Examples like this abound in the energy space, and are all conveniently ignored by neo-classical theories.  Of course there are regulatory barriers as well, and I don't in any way mean to suggest that pricing externalities wouldn't help - but the idea that capital is optimally allocated is rarely heard outside of academia.  

Economics a science?

That is going way too far.  Hehey.

http://amazngdrx.blogharbor.com/blog John Schneider, Northern Wisconsin
Good Point Sean

As usual.  I definitely didn't mean to suggest that pricing externalities solves all problems.  There are plenty where its not feasible because of being more expensive to monitor/collect than its worth.  There are also plenty of market disconnects, like where residents who consume electricity aren't (directly) paying utilities, which means an increase in price won't affect their incentive to conserve.  

Or, like your windows example, plenty of appliances we buy we just don't even look at power efficiency (except for maybe grist readers), so a company doesn't feel it could recoup higher prices for a more efficient product that might cost a bit more.  Even though it would sell great if we all took time to calculate at the counter what our economically rational purchase would be.  These lack of incentives when things just dont seem 'worth it' at small scales definitely add up to suboptimal capital allocation economywide in the aggregate, and I think you're right that thats especially with energy.  So thanks for the pointer on optimum allocation of capital.

The fact that these CGE models can have trouble reflecting reality really hit home in 2 recent conversations I had with EIA analysts.  One was about the IEO which predicts $65 a barrel oil in 2030 (in 2006 dollars).  The other was about a part of the Lieberman Warner model that figured 'international emissions allowances' from some foreign cap&trade program would be around $30 by 2020.  In each case, I pointed out that the models seemed kinda outta line with reality (the oil case is obvious, on the foreign permit price I noted that ETS EA's are already above that today).  In each case, the answer I got was (at least the beginning of it), pretty near verbatim, 'well, these are general equilibrium models, so...'


You are not logged in. Thus, you cannot post a comment. If you have an account, log in. If you don't have an account, well, by all means go make one! Meet you back here in five.
sign in
Search Gristmill
Subscribe
  • subscribe via RSSStay updated with the Gristmill RSS feed.
  • Add to My Yahoo!
  • Subscribe with Bloglines
  • Subscribe in NewsGator Online
  • Subscribe in Netvibes
  • Subscribe in Google
Using Gristmill
  • What is Gristmill?
  • Posting rules
The comments of Gristmill users reflect the opinions of those individuals only, and do not necessarily reflect the viewpoints of Grist, its staff, its board members, their psychotherapists, or their aestheticians. Got it?

Gristmill is powered by Scoop.

ADVERTISING POLICY


About Grist | Support Grist | Job Board | Archives | Grist by Email | RSS | Podcast
Gristmill Blog | In the News | Ask Umbra | Muckraker | Victual Reality | 'Tis the Season | The Grist List | The Bottom Line



Grist: Environmental News and Commentary
a beacon in the smog (tm) ©2008. Grist Magazine, Inc. All rights reserved. Gloom and doom with a sense of humor®.
Webmaster | Sitemap | Privacy Policy | Terms of Service | Trademarks