Monte Carlo - wrong tool for the wrong job?

msf

I've been asked how I felt about using Monte Carlo (specific tools? general concept?).

As a concept, I'm already on record as saying that there are many things that MC methods can help with. The technique itself is just a matter of "rolling dice". Based on a model, those random numbers get mapped to different events (e.g. gaining 10% one year). Do that enough times and you'll get a pretty good histogram (probability plot) of possible outcomes. When implemented in a tool, the MC part is just a few lines of code.

The guts, and where things go wrong, are in the model. Simplistic models (e.g. normal or even fat-tailed distributions of returns) ignore history. According to these models, business cycles don't matter, valuation/CAPE/PE doesn't matter. Regardless of where the market is now or how it got there, 1/8 of the time the US stock market will have a negative cumulative return over the next three years. That's according to PortfolioVisualizer with statistical returns and no withdrawals.

That can be addressed by tools that build better models. The problem here is with the tools, not the technique.

The broader problem is that "safe withdrawal" tools don't answer questions like: What's my optimal allocation/withdrawal plan? They're not optimizers. They just examine a very simple question: If I have $X and I invest with allocation A, how much will I be able to withdraw monthly? These tools expect you to do the mechanical trial and error work of testing different allocations. You'd think they would at least automate that.

PortfolioVisualizer does contain optimization tools, but they're of limited help with real world planning. I'd like to know how much I can improve my odds with different strategies. For example, instead of following a constant 4% (or whatever) withdrawal schedule, I'd like to know how my odds for success improve if I'm willing to cap my withdrawals at $X (inflation adjusted) when the market soars, thus saving during fat years to have enough for the lean years.

More generally, I'd like an optimizer that could accept multiple objectives. Perhaps I'd be willing to take a little more risk in order to improve the likely amount I'll have to bequeath to heirs.

Monte Carlo is a technique for generating possibilities. It's only as good as the models it uses. Even then, it doesn't help you select an asset allocation, let alone give you an idea of how different ways of adapting to bad returns can improve the odds of meeting your objective(s). That's what optimizers are for.

Old_Joe

Oh-oh. @MJG is going to have heart failure.

davidrmoran

Have you used the Welch optimal retirement planner -- ORP ?

msf

davidrmoran said:

Have you used the Welch optimal retirement planner -- ORP ?

Have you?

[Starting 2035] OASI income would be sufficient to pay 77 percent of scheduled OASI benefits. Since this is fact, until Congress changes the law, ORP reduces Social Security income by 23% beginning in the year 2035.

https://www.i-orp.com/LumpSum/help/ORPHelpQ.html#SocSec

It figures out how much you can withdraw (or equivalently, the percentage of your starting investment) by assuming your investment returns the same percentage each year. For this one needs a tool?

ORP generates a deterministic plan for the full term of retirement. This plan uses a constant rate of return (ROR) on savings assets and a constant rate of inflation. This assumes active asset management so that savings value volatility is considerably less than the more volatile S&P 500 market index typically used by retirement calculators.

https://i-orp.com/gamma/help/ORPHelpQ.html#Assessment

Ask it to compute how much you can withdraw annually, starting with $1M in a Roth (so that taxes are ignored), no SS, no pension. Its answer: 4% exactly ($40,000). All supposedly without risk, since it assumes there's no volatility in your portfolio.

On the "Extended ORP" page there is a Monte Carlo button that does run simulations and comes up with slightly lower numbers, as expected. It gives you a few figures on the odds: the withdrawal amount for which 50% of its runs fail, for which 16% fail, for which 3% fail, and worst case (less than $10K in all the trials I ran). Hard to pick a starting value (or percentage) with such a sparse report.

The way it randomly selects an annual rate of return for the stock market is to divide historical returns (since 1955) into a bucket of returns following a negative year and into a bucket of returns following a positive year. When it runs the simulation, if the last simulated year is negative, it randomly selects one of the historical returns from the negative bucket for the next year.
https://www.i-orp.com/modeldescription/montecarlo.html

An obvious issue with this, and one that I've raised in other places, is that there's nothing magical about zero. Assuming that one wants just two buckets (why not partition the historical results into, say, quarters?), wouldn't it make more sense to split the returns into those that exceeded the average (not zero) and those that underperformed? Instead, what you've got here is one bucket of 501 data points, and another of 195 (according to Appendix C on the page cited above).

Now, disregarding all I just wrote, does it answer the simple question I posed originally: what asset allocation enables me to safely withdraw the most money? ORP doesn't answer that. It's not even easy to figure out what asset allocation it's assuming.

davidrmoran

Other than that, Mrs Lincoln.

>> Have you?

Sure, I use it all the time. It helps me think about RMD overage wrt taxes and how much to consider augmenting from Roth.

Yes, it will return somewhat different results for the same inputs.

>> ORP doesn't answer that. It's not even easy to figure out what asset allocation it's assuming.

I hesitate to ask a question, but is this not specified via one's detailed inputs?

Welch seems an interesting and thoughtful guy, as you might infer from his long footnotes, so you might consider posing these concerns to him directly; he responds.

msf

davidrmoran said:

Other than that, Mrs Lincoln.

>> Have you?

Sure, I use it all the time. It helps me think about RMD overage wrt taxes and how much to consider augmenting from Roth.

Yes, it will return somewhat different results for the same inputs.

>> ORP doesn't answer that. It's not even easy to figure out what asset allocation it's assuming.

I hesitate to ask a question, but is this not specified via one's detailed inputs

First, let me say that this looks like a tool well designed for exactly what you're using it for, as it focuses heavily on taxable/trad/IRA allocations and income taxes. It's not that well focused on projecting returns or optimizing what you can draw down.

In particular it doesn't figure out what asset allocation one should use; as you stated, one has to tell it. That virtually guarantees a suboptimal allocation and thus lower withdrawals. I stated originally that a basic problem with many tools (apparently including this one) is that they "don't answer questions like: What's my optimal allocation/withdrawal plan?" You've just confirmed this by saying that one must guess (input) the asset allocation.

Not to mention the glide path. Curiously, the "essential" version and the "extended" versions default to different allocations and different glide paths. Regardless, the tool doesn't tell you what glide path it follows based on your starting and ending allocations. Linear? Exponential decay? How do you implement the glidepath plan if it doesn't tell you what that is?

While I'm fairly confident that the tool does what it says it does (to the extent that it actually says what it's doing), I'm less confident about attention to details. For example:

If you go back to Bengen's original paper you will see that the 4% rule is very narrowly defined and applies only to a specific set of assumptions including exactly a 30 year retirement period and not one year more.

https://www.i-orp.com/3-PEAT/faq.html

Actually, if you go back to Bengen's original paper you will see that the 4% rule applies not only to 30 year retirement periods, but to periods up to 33 years. Further, Bengen considered various different allocations in determining that 50/50 optimized (maximized) the amount that could be safely withdrawn.

Assuming a minimum requirement of 30 years of portfolio longevity, a first-year withdrawal of 4 percent ... should be safe. In no past case has it caused a portfolio to be exhausted before 33 years...

Tools like ORP or Portfolio Visualizer or ... do not seem to solve a very simple question: disregarding taxes, what allocation maximizes the amount that may be withdrawn safely?

Portfolio Visualizer will optimize allocations for various objectives such as minimum volatility, but it doesn't incorporate retirement withdrawals into this optimizer, so its optimizer is designed for accumulation not retirement spending.

MJG

Hi Guys,

I am an enthusiastic supporter for adding Monte Carlo tools to your investing tool kit. I do recognize it has its limits (like its input uncertainties and the ranges explored) when applying it. A user must be aware of its shortfalls.

But it is a quick and effective way to explore and quantify investing return uncertainties and risk. Hopefully, with some insights into the performance uncertainty magnitudes, more rewarding outcomes will be generated under somewhat controlled risk circumstances. That's sort of the name of the game. Monte Carlo analyses do not make a decision; that remains your difficult task.

Risk analysis is embedded in just about every investment decision made. We are always challenged by uncertainty and ambiguity, Nobody consistently projects the future accurately. Monte Carlo simulation helps define boundaries. It helps ( note it only helps) to improve decision making under uncertainty. The call is still yours.

I encourage MFOers to learn more about this tool. Give it a few test runs. You just might find it as useful as I do and add it to your investment tool kit.

Best Wishes

Old_Joe

Well, judging by the above commentary, Monte Carlo would seem to fall somewhere between a Ouija board and a direct visit from the gods revealing the future. Can't hurt, and is useful as long as the limitations are understood. Not recommended for designing bridges in Genoa.

msf

@MJG wrote "this tool. Give it a few test runs".

I tried to draw a distinction between a technique (or technology, if you prefer) and tools, which are implementations of an approach. What is this tool? Perhaps you mean Portfolio Visualizer? Aside from cost (free), what is it that you especially like about this particular MC implementation, or whichever tool you have in mind?

(FWIW, Portfolio Visualizer's asset allocation optimizer doesn't implement MC. "The optimization is based on the monthly return statistics of the selected portfolio assets for the given time period.")

We are in total agreement that playing around with tools like these can give people a better sense of possibilities. I agree that one benefits from "adding Monte Carlo tools to [one's] investing tool kit ... [so long as] a user [is] aware of its shortfalls."

MJG

Hi msf,

Thanks for your comments. There are numerous Monte Carlo codes accessible on the Internet. Portfolio Visualizer ( now spelled correctly) is just one example. Each has advantages and limitations. I really don't have a favorite. Another free Monte Carlo is available on the Vanguard site. Here is a Link to it:

https://retirementplans.vanguard.com/VGApp/pe/pubeducation/calculators/RetirementNestEggCalc.jsf

Indeed this version can help in retirement planning. It is very easy to use with a minimum of input requirements. Using tools like these, a lot of what-if scenarios can be explored in terms of portfolio survival probabilities. This allows a user to test the impact of candidate portfolio changes on that survival likelihood. Nothing very clever or deep, but useful nevertheless.

Best Wishes

MikeM

Well, judging by the above commentary, Monte Carlo would seem to fall somewhere between a Ouija board and a direct visit from the gods revealing the future.

Not true Joe. The community above is msf, msf and msf, who could find the weakness in any argument - either side. But weakness and shortcomings shouldn't mean it isn't useful to many. Not sure what you guys have against this "tool", but your comparison is way off the mark in my view. I still believe it is the best tool I use (other than my own spreadsheet) to test the probability I'm on tract - or not.

Old_Joe

@MikeM- Yeah, I know. Just rattling chains a little...

msf

MikeM said:

Not sure what you guys have against this "tool" ... I still believe it is the best tool I use (other than my own spreadsheet) to test the probability I'm on tract - or not.

Thanks for putting "tool" in quotes. As I keep writing: "As a concept, I'm already on record as saying that there are many things that MC methods can help with."

My concern is not with the technique, but with the application - the model, or the tool forcing that model onto the user. Maybe this simplistic example will help illustrate my concern:

Suppose I want to know the odds of rolling snake eyes (1,1). Assume I don't know the rules of probability, so I don't know that to compute the probability of two independent events happening I just multiple the probability of each. So instead I model each die, and simulate rolling dice.

If my model is good - a 1/6 chance of rolling a one on a die - I'll get good results. Say I simulate rolling the dice 720 times. I may get 22 snake eyes. I try the simulation again, I get 19 snake eyes. Nearly all the trials will have around 20 snake eye rolls, so I conclude that the odds are 1/36.

But maybe I'm thinking that on the opposite side of a one on the die is a six, and those six dots are heavy. So the die is more likely to land six dots down (one dot up) and I mistakenly estimate the odds of rolling a one are 1 in 5. I feed that model into my simulator, and simulate rolling the dice 700 times. Maybe the simulator reports 29 snake eyes. A second simulation results in 26 snake eyes. And so on. After awhile I notice that they're all around 28 snake eyes out of 700 iterations. So I conclude, incorrectly, that the odds are 1/25.

The problem is not with the MC method, nor with the simulator. The problem is with the model.

Different tools take different approaches to constructing their models. Most care in varying degrees about relationships between returns. Some care about trends (or cycles) from year to year. Others may not care about trends over time but may still care about the correlation between returns of different classes (they're not totally independent). The pure random model, where each event is totally independent (no correlation between asset classes, no persistence of performance) seems to me to be the worst of all possible worlds, and the worst one on which to base these tools.

MJG was critical of using historical data, even if the years were selected randomly. Given this opinion, it is curious that he declined to express a preference for one tool (using one type of model) over another.

To reiterate: my concerns are over the quality of model that a tool imposes on the user and over the quality of the data used to build that model, but not over the use of MC simulations.

Portfolio Visualizer explains the benefits in using historical data, whether simply randomly selecting a year from history, or randomly selecting a sequence (block) of years from history:

Using a single year as the bootstrapping model retains the cross asset correlations for the configured portfolio allocation for each simulated year and avoids overweighting any specific year. Using block bootstrapping selects a random sequence of annual returns and better captures the serial correlation and mean reversion of assets.

To its credit, PV does apply historical asset correlations to its statistical (mean and std dev) models so that these models at least incorporate cross asset correlations.

Any of these tools gives some crude measure of whether you are on track. But when they tell you that you're off track, its up to you to figure out whether you need to save more or just reallocate your investments (and how).

MikeM

@msf, you are a very intelligent person and you explain things very well, but this is all I needed to know (already knew):

Any of these tools gives some crude measure of whether you are on track. But when they tell you that you're off track, its up to you to figure out whether you need to save more or just reallocate your investments (and how).

I, and I'm guessing most people, aren't expecting anything more than that from Monte Carlo. Just a crude measure affirming if we're on tract - or not

hank

Those of us fortunate enough to have a DB pension may be able to “wing it“ without MC or any other particular scheme. A lot depends on how frugally you’re willing to live and how disciplined you are in managing your nest egg.

But, it’s a different story for someone without a good pension. Has to be very difficult doing that kind of long-range planning. Don’t think I could. Almost forgot to mention: Although I despise annuities (from everything I’ve seen about them), I guess that would be something to look at.

msf

Old_Joe said:

Well, judging by the above commentary, Monte Carlo would seem to fall somewhere between a Ouija board and a direct visit from the gods revealing the future. Can't hurt, and is useful as long as the limitations are understood. Not recommended for designing bridges in Genoa.

While this was clearly said somewhat in jest, it's a lot more accurate than people realize, which explains my concern about these predictions being taken too seriously.

Cited in a page that was quoted by MJG is a paper where the authors build a more robust model.

Under then current (2013) market conditions and history, using this model and Monte Carlo simulation it concludes that 4% withdrawals from 40/60 portfolio has only a 48% chance of success over 30 years.

Portfolio Visualizer, using statistical returns (MJG's preference), says that 40% total US stock, 60% total US bond portfolio has better odds of success than Ivory soap (99 44/100%)

48% vs. 99½%.