The providers of retirement calculators vie to improve the forecasting accuracy of their models
By Robert Stowe England
See companion article as this link.
The models, in essence, aim to raise the degree of fidelity of the model to real life. Fidelity, in this case, “is defined as the ability of a calculator to potentially produce reality,” according to retired software developer Darrow Kirkpatrick, who has published a list of The Best Retirement Calculators.
The constant testing of existing and new calculators for their strengths and weaknesses has been at the heart of the race to attain high fidelity.
Those working to build more sophisticated models, including some that now require customers to pay licensing fees, are unlikely to publish the results of their tests of their own calculators and those of others.
Fortunately for consumers, expert independent analysts like Kirkpatrick have done sophisticated tests of the many calculators now available and have published some of their results. So far Kirkpatrick has reviewed 82 calculators and estimates there may be another 40 or so calculators out there. “A few of the newer ones are on my internal list still to review,” he says.
Kirkpatrick sees value and merit in both basic simple calculators as well as complex and sophisticated models. The simple ones are appropriate for use early in one’s working career, he says. The more sophisticated calculators are better “when you need precise answers about your retirement date or tax moves.”
Kirkpatrick says credit for the articulating the concept of fidelity in retirement calculators belongs to Stuart Matthews, a retired electrical engineer who in 2011 developed the Pralana Retirement Calculator, offered by Pralana Consulting LLC, Plano, Texas.
Matthews, who retired in 2009, developed his own retirement calculator to plan out the last ten years of his working life. In 2011 he read a review of online retirement planning tools by Consumer Reports and decided to test them and see what results he got. “Wow! What a disappointment,” hewrote and set out to build a better model.
Not satisfied with his original Pralana model, Matthews worked to improve it. As part of his research, Matthews took a small set of actual test examples of people saving for retirement. He ran the data from those text examples through several models and compared the results.
From these findings, Matthews rated the calculators as having high fidelity or low fidelity. He has since developed a more comprehensive set of tests and evaluations of eleven calculators and published his findings.
Matthews has been careful to note that low fidelity calculators should not be disregarded because they are simple or sponsored by an asset management company.
“If you can summarize your financial situation with relatively few numbers and assumptions and want to get a quick estimate of whether you’re on track or not, these tools can probably meet your requirements at no cost and with a minimum of effort on your part,” Matthews has written.
To improve the accuracy of their forecasts, developers of calculators are using more complex formulas for model’s assumption on the rate of return on investment assets.
Most simple calculators use a measure of the average rate of return to forecast how assets will perform in the future. This approach can provide an overly optimistic result because it may not reflect the impact of market volatility on investment returns, according to Kirkpatrick.
To address volatility, some calculators perform a Monte Carlo simulation of a range of outcomes randomly chosen. However, Kirkpatrick is not sure that Monte Carlo simulations accurately incorporate the volatility in the real world.
Some models base their rate of return on historic market data on the performance of asset classes over the past century. If the future is not like the past, then the accuracy of this approach will suffer, Kirkpatrick stated.
In the end, users of retirement calculators will likely have to make judgment calls on such things as which method to use to measure the rate of return.
Kirkpatrick has suggested that consumers should try all three methods – average, Monte Carlo and historic – and compare the results. Then, taking into consideration the results from each method, they should rely on their own judgment to decide on a rate of return they feel will be realistic.
