speculation in commodities

  • Hedgers, Speculators and ‘Excessive’ Speculation in Futures Markets*

    The summer 2008 spike in crude oil prices to $147/bbl, followed by a steep correction in late 2008/early-2009 and subsequent sharp rebound over the last two years have jolted the world economy and pinched consumers at the fuel pump. Given the predominance of crude oil in the world economy, the pace of change of prices generated substantial attention from regulators, legislators and commentators who have decried the existence of ‘excessive’ speculation in the crude oil futures markets. Indeed, the rise in participation by non-commercial traders during the preceding ten years provided further ammunition for those seeking causal connection with concurrent price increases.

    Two of the most important functions of futures markets are the transfer of risk and price discovery. In a well-functioning futures market, hedgers interested in reducing their exposure to price risk find counterparties. In a market without speculative interest, long hedgers must find short hedgers with an equal and opposite hedging need. In fact, many traditional hedgers have dual liquidity needs, intending to offset their futures positions before physical delivery of crude oil. Speculators enhance liquidity and reduce search costs by taking the opposing positions when long hedgers do not perfectly match short hedgers. Speculators provide immediacy and facilitate the needs of hedgers by mitigating price risk, while adding to overall trading volume, which contributes to more liquid and well-functioning markets. In this regard, both groups can contribute to price discovery in futures markets. Futures prices reflect the opinions of all traders in the market.

    An extensive body of theoretical and empirical literature has analysed whether returns in futures markets are related to commodity-specific risk (reflected in hedging pressures) or systematic risk (measured by the covariance between futures and stock returns or other macroeconomic variables). The majority of the empirical studies conclude that idiosyncratic risk (commodity-specific risk) is priced, with futures prices biased downward (upward) in commodity markets where hedgers' net position is short (long).

    When speculators trade with one another in addition to trading with hedgers, the greater liquidity resulting from this ‘excess speculation’ should decrease hedgers’ trading costs. Intuitively, then, the impact of hedging pressures on commodity returns should itself depend on the number and cost structures of speculators willing to take a position opposite that hedging pressure. This begs the question of whether there is any optimal level of speculation.

    If long and short hedgers’ respective positions in a given commodity futures market were exactly balanced (i.e., of the same magnitude at all maturities at market-clearing prices), then their positions would always offset one another and speculators would not be needed in that market. Because long and short hedgers do not always trade simultaneously or in the same quantity, however, speculators must step in to fill the unmet hedging demand.

    In this restricted sense, the amount of speculative activity in commodity markets could be quite small. In practice, of course, speculators hold a range of views about the future and take positions on both sides of the market. As a result, speculative activity almost always substantially exceeds the level required to offset any unbalanced hedging. As suggested by Working (1960), the level of speculation is meaningful only in comparison with the level of hedging in the market. Increased speculative positions naturally arise with increased hedging pressure. In order to assess the adequacy of speculative activity in the crude oil market relative to hedging activity, we calculate Working’s (1960) speculative index for all maturities as follows:

    where SS is short speculator (non-commercial) positions, SL is long speculator positions, HS is short hedge (commercials) positions and HL is long hedge positions. The speculative index value has risen over time to an average of 1.40 in 2008, implying that speculation in excess of minimal short and long hedging needs reached 40%. Of course, this increase can also result from speculators increasing spread trades as we observed in CFTC position data. While this rise in the speculative index to 1.40 may appear alarming, in fact it is comparable to historical index numbers observed in other commodity markets. Further, it is interesting to note that while a sharp rise in the speculative index was visible at the time crude prices rose to record highs in 2008, such a relationship is much less apparent for the 2010/2011 period. Indeed, it can be argued that the speculatative index has been on a declining trend since 2008. As Working (1960) also notes, the speculative index measures excess speculation in technical terms, but not necessarily in economic terms.

    speculative index

    Diverse Opinion on the Causes of Volatile Prices

    Despite scant evidence of excessive speculation in the crude market since 2008, excessive speculation itself has the potential to disrupt markets. Shleifer and Summers (1990) note that herding can result from investors reacting to common signals or overreacting to recent news. As de Long et al. (1990) show, rational speculators trading via positive feedback strategies can increase volatility and destabilise prices. However, findings by Boyd et al. (2009) and Brunetti et al. (2010), respectively, suggest that herding among hedge funds is countercyclical and that it has not destabilised the crude oil futures markets during recent years. Some argue for the possibility that speculative trading might lead to higher prices if speculators increase their accumulation of inventories (e.g. Kilian and Murphy (2010) and Pirrong (2008)). Alquist and Kilian (2010) formally link forward looking behaviour and inventory building. Their model predicts that increased uncertainty about future oil supply shortfalls will lead the real price of oil to overshoot in the immediate short-run with no response from inventories. The real price of oil then gradually declines as inventories are slowly accumulated. Kilian and Murphy (2010) develop a structural model for crude oil that allows for shocks to the speculative demand. In their model, a positive shock to speculative demand increases both the real price of oil and oil inventories. They find no evidence that the 2003-2008 price surge had much to do with speculative demand shocks.

    As suggested by Hamilton (2009b), crude oil inventories were significantly lower than historical levels in late 2007 and early 2008, when crude oil price changes were most dramatic. On the other hand, Davidson (2008) argues that the absence of higher inventories does not necessarily indicate the absence of excess speculation in the market. Using the Marshallian concept of ‘user cost’, Davidson argues that if oil prices are expected to rise in the future more rapidly than current interest rates, then producers can enhance total profits by leaving more oil underground today for future production. If oil producers do take the user costs of foregone profits into account in their profit maximising production decisions, then they may limit current production and above ground inventories may not rise. In this regard, Davidson (2008) points out that traditional hedgers, such as oil producers, might be involved in speculation. A similar argument is made by Parsons (2009), who argues that the change in the term structure for oil to a long lasting and deep contango late in 2004 can explain steady above-ground stockpiles of oil. Kilian and Murphy (2010), however, find no evidence that a negative oil supply shock played an important role in the spike in crude oil prices. In assessing the balance of these findings, however, it is important to note the lack of timely and detailed physical market data, including inventory data, especially for non OECD countries, as well as the lack of OTC data.

    Kilian (2008) and Kilian (2009) propose a structural decomposition of the real price of oil into three components: shocks to the flow of supply of crude oil; shocks to global demand for all commodities; and demand shocks that are specific to the crude oil market. Their empirical analysis provides evidence that the 2008 price hike was primarily a result of global demand shocks rather than supply shortages or crude oil-specific demand shocks. Hamilton (2009a) suggests that both factors – stagnant production and low short-run price elasticity – are needed for speculation to drive prices too high, but that financial speculation (by non-commercial entities) would also cause inventories to rise. He concludes that supply and demand fundamentals provide a more plausible explanation for the 2008 price spike. However, Hamilton (2009b) also suggests that it is possible for speculators to drive up prices without any change in inventories if the short-run price elasticity of demand is close to zero (an approximate condition of the oil market). Kilian and Murphy (2010) estimate the short-run oil price elasticity of demand at a significantly negative -0.24, much higher than existing estimates in the literature and, if accurate, further undermines speculation as an explanation for oil price increases.

    Buyuksahin and Harris (2011) and Brunetti et al. (2010) also find that speculative activity in crude oil futures markets does not lead price changes, but reduces volatility by enhancing market liquidity. On the other hand, Tang and Xiong (2010) and Singleton (2011) find a significant impact from investment flows by non-user participants on prices and volatility of commodities (see my blog on Singleton's paper). Tang and Xiong (2010) further argue that the average correlation between indexed commodities and between indexed commodities and equities increased due to the growing presence of commodity index investors after 2004. However, Buyuksahin and Robe (2011) suggest that increased correlation is a relatively new phenomenon and that hedge funds, rather than passive long-only investors’ positions, might explain the increased correlation between equities and commodities. Specifically, they find no persistent increase in co-movements between returns on equities and investable commodities until the Lehman demise in September 2008. A corollary is that, in ‘normal‘ times, commodities provide benefits in terms of portfolio diversification. However, commodity-equity return correlations jumped immediately after the Lehman collapse, reaching levels after November 2008 never before seen during 1991-2008. Furthermore, cross-market linkages have remained exceptionally strong since autumn 2008. Equity and energy markets did drift apart in February and March 2011 due to tensions in the Middle East. However, this drift was temporary. Equity-commodity return correlations have since rebounded – they were again higher as of April-May 2011 than at any point in the 1991-2007 period.

    In short, academic opinion remains highly polarised on the respective roles of hedgers and speculators, and on the concept of ‘excessive’ speculation in the crude oil market. Despite this lack of a clear consensus, however, significant new regulatory measures aimed at reducing systemic risk in financial markets are being developed, which may substantially limit the participation of non-commercial players within commodity derivatives markets.


    Note: IEA Medium Term Oil and Gas Report, June 2011.


    • Boyd, Naomi, Bahattin Büyükşahin, Michael S. Haigh, and Jeffrey H. Harris (2009). “The Prevalence, Sources and Effects of Herding.” CFTC Working Paper.
    • Brunetti, Celso, Bahattin Büyükşahin and Jeffrey H. Harris (2011). “Speculators, Prices and Market Volatility” IEA and CFTC Working Paper.
    • Büyükşahin, Bahattin and Jeffrey H. Harris (2011). “Do Speculators Drive Crude Oil Prices?” The Energy Journal 32: 167-202.
    • Büyükşahin, Bahattin and Michel A. Robe (2010). “Speculators, Commodities, and Cross-Market Linkages.” CFTC and IEA Working Paper.
    • Davidson, Paul (2008). “Crude Oil Prices: Market Fundamentals or Speculation?” Challenge 51, 110 118.
    • DeLong, J. Bradford, Andrei Shleifer, Lawrence H. Summers and Robert J. Waldmann (1990). “Positive Feedback Investment Strategies and Destabilizing Rational Speculation,” Journal of Finance 45(2): 379-395.
    • Hamilton, James D. (2009a), “Understanding Crude Oil Prices.” Energy Journal 30:179-206.
    • Hamilton, James D. (2009b). “Causes and Consequences of the Oil Shock of 2007-08” Brookings Papers on Economic Activity, Spring: 215-259.
    • Kilian, Lutz (2008). “The Economic Effects of Energy Price Shocks.” Journal of Economic Literature 46(4): 871-909.
    • Kilian, Lutz (2009). “Not All Oil Price Shocks are Alike: Disentangling Demand and Supply Shocks in the Crude Oil Market” American Economic Review 99(3): 1053-1069.
    • Kilian, Lutz, and Dan Murphy (2010). “The Role of Inventories and Speculative Trading in the Global Market for Crude Oil," University of Michigan Working Paper.
    • Parsons, John E. (2009). “Black Gold and Fools Gold: Speculation in the Oil Futures Market. ” Center for Energy and Environmental Policy Research, 09-013.
    • Pirrong, Craig (2008). “Stochastic Fundamental Volatility, Speculation, and Commodity Storage.” University of Houston Working Paper.
    • Shleifer, Andrei and Lawrence H. Summers (1990). “The Noise Trader Approach to Finance.” The Journal of Economic Perspectives 4 (2): 19-33.
    • Singleton, Kenneth (2010). “Investor Flows and The 2008 Boom/Bust in Oil Prices.” Stanford University Working Paper.
    • Tang, Ke and Wei Xiong (2010). Index Investing and the Financialization of Commodities. NBER Working Paper No. 16325, September.
    • Working, Holbrook (1960). “Speculation on Hedging Markets.” Stanford University Food Research Institute Studies 1: 185-220.

    Related Links

    Bahattin Buyuksahin My Blog
    Lutz Kilian Street Professor (Craig Pirrong Blog)
    Ken Singleton Econbrowser
    Scott Irwin UNCTAD Commodities
    Michel Robe Markets Wiki
    James Hamilton JLN
    Wei Xiong My Facebook Page on Commodities