Terry Odean
UC Berkeley

Benjamin Graham once said that an investor's chief problem, even his worst enemy, is likely to be himself. I will spend most of my time here talking about things that investors do that make them their own worst enemies.

First, though, I would like to spend a few minutes talking about why I think that finance is more difficult than accounting. For accounting, you only need to be able to add. For finance, you need to be able to understand probability. Let's look at a simple example of this.

Everyone knows what two plus two equals. Kids know this. If you suggest to a six year old that two plus two is something other than four, she will laugh at you. Of course, if you ask an accountant what two plus two equals, he will probably respond by asking what you want it to be. In general, though, people know what two plus two is. We have a very good understanding of simple addition. It is very intuitive for us.

Probability is different. If I flip two coins, what is the probability that both will come up heads? Many of you might know the answer to this one. If you don't have any idea, then you should stay away from the gaming tables! The probability is one in four that the two coins will both come up heads. I believe that I could convince a lot of people, by the way, that the probability was one in three. I could tell them that there were three possible outcomes - two heads, two tails, and one of each - so the probability of two heads was one in three. We do not have such an intuitive grasp of probability as we do of addition.

Let me give you another, slightly more complicated example. Two plus two plus two plus two is a more complicated addition problem, but most of us can quickly see that the answer is eight. We don't need to think very hard about this. It is very intuitive - human beings have a very good grasp of quantities.

Now, let's make the probability example more complicated as well. If we flip four coins, what is the probability that three coins will be heads and a fourth coin will be tails? Generally only people deeply involved in math, like mathematicians and physicists, know the answer to this. Without training, we do not have a good intuitive grasp of probability. At is turns out, the probability for getting this result is one in four (four in sixteen).

One of the reasons that investing is difficult is that you need to have a good understanding of probability. Most investors do not have an innate understanding or intuition for probability. Many of the mistakes that investors make come from this lack of understanding. They also use a lot of mental shortcuts as a way to deal with probability. These shortcuts work reasonably well in many situations in life, but there are situations like investing in which these shortcuts do not work so well.

I would like to talk to you about some things that investors do and some of their habits. Investors tend to trade too much. They hold onto their losing investments, and they chase the action when deciding what to buy.

Why do investors trade too much? The first paper I wrote as a Ph.D. student was a theoretical paper about overconfidence. I looked at what would happen in a market if investors were overconfident. Overconfidence means believing that you have more ability than you actually do, in this case, for investing. It is a fairly common human trait, and it is not really such a bad trait. People who have self-serving biases like overconfidence tend to be somewhat happier than others, they are more willing to go to work, they work harder, and the world looks like a rosier place to them. These people do not, however, make better investors.

There are a number of ways to demonstrate overconfidence. In my MBA class every year, I hand out a survey that asks the students to rate their own driving skills compared to the other students in the class. Are you in the top 10%, the top quartile, the top half, the bottom half or the bottom quartile? Normally about a quarter of the students say that they are in the top 10% of the class. Half of the students are in the top quartile. All but a very few are above average. This is a very typical outcome. A couple of years ago, one student rated herself below average, and I had to ask her why. She said that she was going to put herself in the top quartile, but then she realized that in the last year she had been in three accidents, had received two speeding tickets, and was going to court to try to prevent her license from being suspended. With all of this evidence, it occurred to her that perhaps she was only about average. Only once have I ever had someone rate himself in the bottom 10% of drivers. I was shocked when it happened. When I asked him why, he answered that he did not drive because he was from a foreign country. The rest of us consider ourselves above-average drivers, and most investors see themselves as above-average investors.

What happens to the market if the market is full of overconfident investors? Overconfident investors trade more because they think that they're right. They think that their idea is a sure thing, and they are willing to bet on it. As a result, they tend to earn less. They under-diversify because when you're sure you're right, there does not seem to be a need to hedge. Market volatility increases, and market depth increases as well. These are the results of a theoretical study, but I needed to find a way to test it.

The most robust aspect of the study could be tested - the more overconfident investors would trade more and that trading would hurt their returns. We do not know, of course, how much people should trade. There are no clear guidelines for what the appropriate amount of trading is. I could get my hands on data for a particular group of investors, however, and apply a rule about each investor's confidence. Each time that an investor trades a stock and replaces it with another stock, he believes that the stock he buys will outperform the one he sells by enough to at least cover the trading costs. If you are not at least covering your trading costs on average, then you are better off not trading. Based on the trades of 10,000 investors at a large discount trading firm, I looked at the performance of stocks after these kinds of trades took place. On average, the stocks that these investors bought went on to underperform the stocks that they sold by about 3% over the next year. This is before taking out commissions. On average, these people did not do very well in picking stocks.

There are reasons to trade besides speculation. An investor might sell for liquidity purposes - a person might sell assets to help pay for a child's college tuition, for example. You might also sell assets for tax purposes - you might sell some of your assets that have lost value in order to offset your tax bill. I culled through my data and tried to remove trades that were made for this kind of non-speculative reason. I wanted to see how well these investors were doing when they were focused on stock-picking. I excluded liquidity trades by only including transactions in which an investor sold a stock and bought another stock within three weeks. (If you need money for less than three weeks, then there are better ways to raise it than by trading stocks.) I was also able to exclude tax-motivated trades by looking only at trades where investors were selling at a profit.

When I filtered out the non-speculative trades, I assumed that the investors would perform a little bit better. What I discovered, however, was that they actually did worse. The stocks that they bought underperformed the stocks that they sold by over 5% in the next year. In addition to that loss, they also paid commissions. The harder that investors tried, it seemed as though the worse they did.

I got a second data set with about 60,000 investors from an online trading firm. I worked with Brad Barber at UC Davis. Our first study was a variation on the last study. Our theory was that overconfident investors would trade more and earn less. We calculated the turnover rate in each of these 60,000 accounts - how actively the investors were trading. Then we sorted the accounts by their trading activity. Then we calculated the net return for each account - including commissions and spreads. The more actively people traded, the lower the net returns. The very active traders were underperforming the buy-and-hold traders by about six percentage points a year.

We took one more look at this topic. If you wanted to show that overconfidence leads to higher trading and lower returns, you would like to be able to split a large group of people into an overconfident group and a less overconfident group, and then watch the trading activity and return for each group. How can we possibly divide a group of people into a "more overconfident" group and a "less overconfident" group? We used the criteria of gender to divide our population. Men and women are not equal when it comes to overconfidence - I can tell that this comes as a surprise to about half of you. Deaux and Ferris in 1977 said, "Overall, men claim more ability than do women, but this difference emerges most strongly on... masculine tasks." Masculine tasks are all of those things that we were told in school that boys were good at - math, science and finance. The boys grew up, for some reason, being more overconfident in these areas than women.

We divided our data into men and women based on the gender of the person who opened the account. Our prediction was that men would trade more actively than women, and that this trading would hurt their returns. Before we had the results, a colleague of mine pointed out a potential problem with the study, which turned out to be beneficial. He asked how we could know who is making the trading decisions for couples. His wife has inherited some money a few years before, it was in an account under her name, but he called all the shots. It is true that we do not know who is making the trading decisions for couples. We would not be able to tell how the couples were influencing each other, but we did know the marital status of each account. If our original prediction is right, that men are more overconfident than women, then we predicted that the differences between men and women in general would be even larger between single men and single women because they won't have that spousal influence.

In this study, we found that men traded 45% more actively than women, and single men traded 67% more actively than single women. This is a striking difference. We also wanted to evaluate the returns of these groups. Our theory was that overconfident investors would trade more and that that trading would hurt their return. We looked at what each investor held in his or her portfolio at the beginning of the year, and we calculated what that investor would have earned for the year with a buy-and-hold approach. Then we calculated what the investor actually earned after all of their trades. We then subtracted the buy-and-hold return from the actual return. We found that on average, both men and women underperformed a buy-and-hold approach. Men, however, underperformed by more. Women underperformed the buy-and-hold return by about 1% on average. Men underperformed by 1.4%, and single men by over 2%.

I would like to talk next about selling. The best selling advice I ever heard came from Will Rodgers. "Don't gamble. Take all of your savings. Buy a good stock. Hold it until it goes up, and then sell it. If it don't go up, don't buy it." It's pretty good advice if you can follow it. I've done some studies that have found that individual investors will buy a stock, and if it doesn't go up, they do not sell it. Individual investors tend to hold onto their losing investments, and they tend to sell their winners. We are doing some further studies on this bias, and we are starting to discover that it reaches to institutional investors as well. This tendency has been called the "Disposition Effect". Throughout the year, investors sell off winners faster than their losers, and then only in December do investors start to sell off more losers for tax benefit. The losers that investors hold onto generally go on to underperform the stocks that they sold.

We did another study on this topic based on a data set from Taiwan. For a five-year period, we have records for every trade, every sale, every purchase, and every order placed in Taiwan. We know who placed each of these orders. We are able to look at the entire market, including about 1.2 billion purchases and sales. It has been quite a computer-taxing study. We wanted to know how pervasive the Disposition Effect really is. In Taiwan, individual investors have a very strong tendency to hold onto their losers, much stronger, in fact, than do US investors. Corporate investors in Taiwan also hold onto their losers almost as much as individual investors do. Dealers tend to hold onto their losers at a slightly lower rate. The mutual funds are just about even, and only the foreign investors have a slight tendency to sell off more losers than winners. When you look at the market as a whole, about 94% of the market strongly holds onto their losers and sells their winners. The Disposition Effect is a very pervasive tendency.

Why do investors buy the stocks that they buy? How do they choose stocks? We recently wrote a paper called "All That Glitters". In this paper, we argue that investors buy the stocks that catch their attention. The reason this happens is that there is a huge search problem when you want to buy an individual stock. There are between 5,000-7,000 stocks out there to choose from. Human beings are not very well equipped to deal with thousands of options. Hundreds of options or even dozens of options are still too many for us to have to search through. It would be a very difficult task to try to put all of, say, the S&P 500 stocks in order of preference. We can program computers to handle tasks like this if we give them the decision criteria, but it is very difficult for humans to search through dozens or hundreds or thousands of choices. We believe that many individual investors solve this search problem informally and perhaps unconsciously by only considering the stocks that catch their attention.

Suppose that there are a dozen attention-grabbing stocks one day. These are stocks that are in the news, or something big is happening with them. Investors can still have their preferences between these stocks. A momentum investor likes to buy stocks that are on a roll. He will buy one of these twelve stocks that appears to be on a roll. A value investor or a contrarian might be more drawn to a stock that has fallen out of favor, but he will still choose one of these twelve stocks. Attention is one of the dominant factors in determining which stocks an investor will buy. Attention does not, however, affect the selling behavior of individual investors, at least not to the same extent. This is true because there is no big search problem when it comes to selling. The average investor in our data sets owns three or four stocks. If you own less than ten stocks, you can easily consider each of them when you want to make a sale. Attention is not as critical an issue for selling as it is for buying.

When do stocks attract attention? Stocks attract attention when they experience extreme price moves - going up a lot or down a lot. Stocks also attract attention when they are in the news. Our theory is that individual investors will be net buyers of attention-grabbing stocks. For every buyer, of course, there is a seller, but for any particular class of investments, there can be net buyers on a particular day. Attention is more of a problem for individuals than for institutional investors, in part because institutions have a lot more attention to give to stocks with more dedicated time and more teams of people focused on picking stocks. With these resources, institutions can sort through a wider universe of stocks. So individual investors will be net buyers of attention-grabbing stocks on days where the stocks are in the news or there are extreme price moves.

Each day we sort stocks based on our measures of attention - extreme price moves or presence in the news - and then we evaluate whether individual investors were net buyers or net sellers. Our data included investors from a large discount brokerage, a small discount brokerage, and a large retail brokerage. All three groups were net buyers of stocks which had performed very well or very poorly the previous day. They tend to ignore stocks that didn't do very much on the previous day. All three groups also tend to be net buyers of stocks which are in the news, and then tend to ignore stocks that are not in the news. Investors tend to be net buyers whether or not the news is good.

I have been talking about individual investors, but I would like to touch on what happens when you get individual investors together into groups. I will talk briefly about investment clubs. This statement comes from the Chicago Tribune, back in December 1994. "The queen of all investment clubs is a group known as the Beardstown Ladies, a group of women from the central Illinois town of Beardstown whose average age is close to 70 years old. From 1983 through 1992, they averaged a 23.6% annual return, bettering the S&P 500 by more than 8%. In 1991, their portfolio grew by a whopping 59.5%. The Beardstown women have become celebrities. They starred in their own video, 'Cookin' Up Profits on Wall Street'."

How do they do it? Well, we found out a few years later in the Wall Street Journal. "The Beardstown Ladies, the investment club of grandmotherly investors that became popular with the media, said that an audit by Price Waterhouse Coopers shows that their ten-year annual rate of return was 9.1%, not the 23.4% promoted on the cover of their best-selling book." As I understand it, this group had a small accounting problem. If you have an investment club, then every month or quarter, members put some more money into the club. The Beardstown Ladies were counting this money on the profit side of their books. It did amazing things for the bottom line, until they got audited.

Brad Barber and I did a study on investment clubs. We didn't do it to pick on the Beardstown Ladies, but we were interested in statements that we'd read in various sources that 60% of investment clubs beat the market every year. We had never seen many academic studies done on these investment clubs, but we'd seen lots of academic studies done on money managers' performance. On average, money managers have a difficult time beating the market, even though there are exceptions. It is a rare year when 60% of money managers beat the market. We began to wonder how it was that people who got together in their spare time once a month were able to better than the money managers who are very smart and very well-trained. Our simple answer is that we don't think it happens. In our study, we found that investment clubs didn't actually do very well. Their gross returns were slightly lower than an index fund, but their transaction costs were much higher. Individual investors in this time did well in terms of gross returns, but they did not do as well with their net returns. So if investment clubs in our study did so poorly, where did these reports of high performance come from? I don't have a definitive answer, but I think that the reports came from voluntary surveys that were sent out to investment clubs. I don't think people lied on the surveys, but the groups that had done well were more likely to respond to the survey than those who had done poorly. Out of fairness, perhaps we should allow money managers to self-report their returns as well! I would bet that more than 60% of them would beat the market as well!

I believe that investment clubs are a good thing. My father was upset with the results of my study, and he brought me volumes of records from his investment club, assuring me that they had good performance. Investment clubs bring people together, they teach people about the market, they encourage savings, and they are a good social venue. I do not believe, however, that they represent the key to beating the market.

Let's talk for a moment about mutual funds. Our study confirmed the work of other people. Investors pour money into the mutual funds that did well the year before. In our study 39% of the purchases were made in the 10% of the funds that had performed best the year before. Over half of the purchases are from the top 20% of funds from the previous year. This is all true, despite numerous academic studies that show that a single year's performance does not tell you very much about a money manager's ability.

This raises a question - why do investors pour money into funds that did well the previous year, if this strategy will not necessarily help them? I would like to answer this question with a story. Imagine that Bill Miller and I went to the Amazon rain forest. We got separated from our group. We were not hurt and we had fresh water, but we were very hungry. We see a lot of plants that we have never seen before. I select one plant to eat, and Bill selects another. For the rest of the day, I am feeling very sick, but Bill is feeling great. The next day I am feeling better, but you can be sure that I will not eat the same plant I ate before.

Two weeks later, Bill and I have been rescued, and we decide to celebrate in Las Vegas. We head straight to the roulette table. I put my money on red, and Bill puts his on black. The ball lands on black and Bill wins. The next time I place a bet, what should I do? It does not matter. Bill won on black last time, but it doesn't matter for the next spin.

What's going on here? We are learning by induction and observation, but we also learn by deduction. We must learn how deterministic of a process that we are dealing with. When a plant makes you sick, it is very likely a deterministic process - it is very likely that the plant that makes you sick today will make you sick again tomorrow. It is not sure thing, but it is very likely. If you observe pattern, then you should change your behavior. In both situations in this story, Bill and I make opposite decisions. In only one of the situations, though, does the outcome radically change our behavior in the future. The plant story is a deterministic situation, but the roulette wheel is random. One outcome at the roulette wheel does not tell you much about the next outcome. Investors must ask themselves where they believe the stock market lies on the continuum between totally deterministic and totally random. Most investors make the mistake of thinking that the market is more deterministic than it really is. This causes them to look for patterns that they believe are meaningful and that will repeat themselves. They expect short patterns to replicate themselves.

There are a lot of areas in life where this works, but it does not seem to work very well in investing. Why did we evolve to be such poor investors? I suppose it is because we did not evolve in a market economy. Think back to hunter-gatherer times. Some evolutionary psychologists think that at least some of our psychological make up was determined in hunter-gatherer times although clearly a lot of our psychology pre-dated that time. Imagine two hunters go out one day, and they hear a roar behind a big boulder. One hunter runs around the boulder first and gets eaten by a tiger. The other one runs away. A few days later, the second hunter is hunting again when he hears a roar. In his mind, he associates the roar with a tiger, and he leaves. That is a good decision. He might not be as well off if he stops to think, "The last time I heard a roar, my friend was eaten by a tiger, but I don't want to make a decision on a sample of one. That is statistically insignificant!" In that environment, it is a good thing for him to turn and leave when he hears a roar.

In this environment, what is the down-side of seeing a pattern that is not there? A few weeks later, the hunter goes out to hunt and brings a rabbit's leg along with him for lunch. He has a fabulous hunting day. He comes home with a lot of kill, and he is a proud man. All of his lunch that he brings home is the rabbit's foot, and he decides that this foot is his lucky charm. When he has this rabbit's foot with him, he will have luck hunting. He decides to wear it with him everywhere. He has seen a pattern that we would say is spurious. But what is the downside? He may burn a few more calories carrying this foot around, but it doesn't really hurt anyone. The foot might even be moderately attractive - it might help him find a mate. There is not much downside for him to have seen a pattern that was not real. For investors, however, there is a downside to seeing patterns that are not real.

One of the reasons for this increase in trading is that investors may not understand the real costs of trading. Another reason that trading increased after investors went online is what I call "friction", or "no chocolate chip cookies in my cupboard." I often work at night after my daughters go to bed. Around 11pm if there are chocolate chip cookies in the house, the girls will call out to me, and we will meet in the kitchen over a glass of milk and some cookies. I think that I should not eat so many chocolate chip cookies, so have a simple solution. I don't buy them and bring them home. There is a 7-Eleven about a half-mile from my house, but I do not go out in the middle of the night to buy cookies. But if the cookie is right there, I will eat it.

It might be that some investors, when they wake up at 3am with a brilliant idea or a panic about their portfolio, would be better off if they cannot trade and if they had to wait until morning. They might make better decisions after they've had a cup of coffee, read the paper, or talked to someone. One reason some people may trade more online is simply that it's too easy. They no longer have the friction to help them with their self-control.

When people switch from telephone trading to online trading, they dramatically increase their trading activity. In the short term, that's just a reflection of trying out a new toy. Over time, however, they settle down to a plateau where they are more active traders than they were before making the switch to PC trading. These traders' returns were severely hurt by making the switch. They were doing very well before they went online. After they went online, their gross market-adjusted returns were basically flat. Their net returns were negative, however, because they were churning their accounts. I do not think that these people "lost their knack" when they went online. In any given period, some investors among a large population will be getting lucky. These traders will tend to believe that they have a special skill at trading, and they will decide to become more active. So these more successful traders switched to online trading, and their "special skill" turned out to be a fabrication.

Now, can the biases of individual investors affect asset prices? There are a couple of criteria that are necessary for this to happen. One important issue is whether the biases of individual investors aggregate in such a way that you end up with systematic trading by individual investors. We found that the answer is a very emphatic yes. When we looked at groups of individual investors in a given period of time, the correlation between what two such groups was buying was about 75%. If you know what one group of investors is doing, that gives you a very good sense of what other groups of investors are doing. It also carries out in time-series. If you know what investors are doing this month, then will have a pretty good idea what they'll be doing next month.

Why does trading volume drop in a bear market? Self-attribution bias is one reason. When you are successful, you tend to take too much credit for the success, and you don't attribute enough of your success to luck. There is an old Wall Street adage, "Don't confuse brains with a bull market." In a bull market, we have a lot of genius investors, and genius investors tend to trade more actively than they would in a bear market.

Another reason for trading volume to decrease is the disposition effect. People tend to hold onto their losers and sell their winners. In a bear market, they don't have very many winners to sell.

A third reason is attention. People tend to buy the stocks that catch their attention. People pay more attention to a bull market. People pay a lot of attention in the moment of a crash, but in general people pay more attention to the market when it is going up. After a talk that I gave recently in Boston, someone compared this phenomenon to his relationship with the Red Sox. "When the Red Sox are in the pennant race, the first thing I do every morning is open to the sports section and read about the game. Even if I watched the game the night before, even if I was at the game the night before, the first thing I want to do is read about it. But when the Red Sox are in the cellar, I don't even want to see the sports section." When the market is in the cellar, people don't want to look at the investment section. If they don't look at it, they don't get ideas, and their buying volume will decrease.

A fourth reason is the "House Money" effect. This idea comes from gambling. Say you walk into a casino with $100, and you have agreed that you will play until the $100 runs out. Suddenly you find yourself ahead $400 or $500. You will start to play a little looser with that money. You take the original $100 and stick it in your back pocket, and the rest of it is just fun money. You will use it to gamble more aggressively or buy things you wouldn't normally buy. In those boom years when the NASDAQ was up 82%, a lot of investors were probably thinking that they had made money they had never expected to make. They were willing to play with some of that money and drive up trading volumes. In a bear market, investors are back to playing with their own money, and trading volumes decrease.

A fifth reason in liquidity. Vernon Smith, one of last years Nobel Prize winners for economics, has done some experimental market studies about bubbles. When you inject more liquidity into a market, people trade more actively, and they tend to run the market up higher for awhile. In one of our studies of online trading in 1998 and 1999, the amount of borrowing that investors were doing on their margin accounts was much higher relative to their income than it had been for the previous twenty or thirty years. As liquidity goes up, you will get more trading.

A sixth reason is representativeness. This is the idea that people follow short-term trends. People tend to buy stocks that have been going up. In a bull market, more stocks are going up so there are more things people like to buy. In a bear market, there are fewer stocks that people like to buy so trading volume goes down.

The final reason that trading volume goes down in a bear market is entertainment. Some people trade for entertainment, and it's a lot more fun to trade when you're making money.

You can see all of my research papers on my website. www.odean.us

Robert: Have you thought of doing studies to see if people can learn to change their behaviors, if they see that their prior behavior didn't work?

Terry: That is a very good idea. The difficulty is getting the data to test this. What I would like to do is get a longer-term study in which I could interact with the investors whose records I had access to. I would not interact with them personally, but I would like to send them a survey to get a sense of whether people are learning, and which people are able to tolerate risk. When investors lost money in the downturn, I would like to know which investors were able to stay the course and which investors panicked and pulled out of the market. I would like to identify the characteristics that allow people to stay in the game, and then I would like to find ways for people to learn those characteristics.

Robert: What steps could an investor take to overcome the problem of overconfidence?

Terry: I suggest that they buy a mutual fund!

Robert: Have any of your studies identified an optimal turnover ratio?

Terry: We have not identified an optimal number, but your turnover should be low, low, low. You do not want a high turnover ratio for several reasons - it will run up your taxes and your trading costs. An optimal turnover ratio would fall between that of an index fund and that of an active trader. It should be closer to that of an index fund. I would want my fund to trade not very much more actively than an index fund. I would want them to trade a little bit more frequently so that I know the fund is being actively managed.

Robert: If buy-and-hold is generally a good strategy, then why is it a mistake to hold onto a stock that is underperforming? What criteria should an investor use to define a "losing stock"?

Terry: I think buy-and-hold is a good strategy. When I say that it is a mistake to sell the winners and hold the losers, that is conditional on their making a sale in the first place. If you are going to sell, you cannot predict which one is going to go up and which will go down. Anyone can tell, though, which investment will give you a tax write-off and which will increase your tax bill. The average investor spends a lot of time worrying about picking stocks, and they don't have the tools to do that well. They do have control over a few things, though. They should be well-diversified, they should pay attention to trading costs, and they should pay attention to taxes. Those are things you can control.

Robert: Bill and Lisa maintain that the market is functionally efficient. What do you think about market efficiency?

Terry: The market is not completely efficient, but it is very hard to make money trying to beat the market. It is not impossible, but it is not easy. It is probably out of the reach of the average investor to do so through anything but luck.

Robert: Has your research looked into winning and losing in investing as it relates to one's level of investment expertise?

Terry: I have not look at that. I have looked at research that other people are doing, and the results seem to be mixed. An important question is how you measure "expertise". Do you go on what someone thinks their own expertise is? I did look at investors who had been in the market longer, and I did not find any effects. We looked at people's self-evaluations when they opened an online brokerage account, and we found no effects on that, either. It is possible that there are some effects. What I worry about is that investors with experience tend to split into two populations - active traders and buy-and-hold investors. I don't have a lot of confidence in the active traders.