How Accurate are Monte Carlo Forecasts in Retirement Income Planning - June 2023

thank you good to see everybody thanks for joining I'm going to pull up our slides here so this uh is you know one

0:07

of our two monthly webinars um the the one we had last week is more of

0:12

a q a new features and and so on um and the second one the one that we're

0:18

doing right now is it tends to be on you know broader topics maybe things that Derek and I have done research on

0:25

recently um and so this is really a presentation about some of the uh the material that

0:33

we published on kitchiss.com a few months back

0:38

um about about forecasting in uh in retirement

0:43

income planning and um some of the results that we that we found in in looking at different ways of forecasting

0:50

what kinds of you know forecasts how they perform uh in comparison to each other and in comparison to what we would

0:56

what we would hope and so although you know there are some statistics and things in this presentation it's

1:03

actually incredibly practical for people actually doing retirement income planning with real clients

1:11

um so hopefully we can talk a little bit about that and we have Derek here um to talk about the the places where

1:18

this touches on on his practice and maybe things that that he's done to um

1:24

to accommodate these these findings so um you know really since probably the

1:30

90s um we have used probabilistic forecasts and retirement

1:36

planning um and what's a probabilistic forecast it's really just saying well we don't really know exactly how things will turn

1:43

out but maybe we're not kind of 50 50 either maybe it's not a coin flip maybe we have a feeling hey it's more likely

1:49

to be this or it's more likely to be that and so we make these forecasts and they can help guide our our decision

1:56

making but behind any kind of probabilistic forecast is the assumption that the the

2:02

forecast is is accurate in some in some meaningful sense so you know for example

2:08

if we forecast that a retirement income plan has an 80 probability of success

2:14

and you know uh if you've been on our webinars in the past or if you use income lab software that we don't like

2:20

to frame things in terms of success and failure it for for many reasons probably the most important being that's not

2:26

actually the way that retirement works out people don't succeed or fail they adjust but in this presentation we will

2:32

use that language since it is so common um outside of income lab

2:38

um to kind of frame the discussion so if you predicted an 80 probability of success you know you'd expect that uh if

2:46

if you did that lots and lots of times about 80 of the time they would quote unquote succeed

2:52

um you wouldn't want to predict an 80 chance of success but actually have only

2:58

40 of those cases if you kept you know putting people on plans that had 80 probability of success you would not

3:05

expect 60 of them to quote unquote fail right that would be a really bad outcome that's underestimating risk

3:12

but you kind of also wouldn't want to forecast say a 40 probability of success

3:17

um but actually 70 of those people succeed so that's underestimating risk and which leads to probably giving

3:23

advice that is overly Frugal on the spending side right so we kind of want

3:28

things to be roughly accurate so how could you measure the accuracy of probability

3:35

probabilistic forecasts uh well weather forecasting is a reasonable analogy here

3:43

it's not perfect no analogy is um but we could do a categorical forecast say yes or no will it rain

3:49

tomorrow yes or no um but because the world is messy and

3:55

there's just there's just too many things going on we can't really hope for probabilistic forecasts to be

4:01

um to be correct you know to be the best we can do um and so we we tend to go for

4:07

probabilistic forecasts so that's why on the you know the Weather Channel you'll get percentages

4:13

um so if the if it doesn't rain tomorrow which of these is more accurate well the

4:18

75 right versus being certain um what about if it doesn't rain

4:24

tomorrow sorry what if it does rain tomorrow

4:30

again offering kind of a low but non-zero chance of rain is is better than saying

4:39

um no no it won't rain and then it does rain okay so how can we evaluate

4:44

probabilistic forecasts um what we do is we we take the forecast themselves which are you know

4:51

percentages and we compare it to the actual outcome so this shows a good

4:57

model of rain forecasting um which is really just kind of you know higher numbers uh it predicts higher

5:05

chance of rain on the days that it actually does rain lower chance of rain on days it doesn't versus the opposite the bad model is really getting things

5:12

getting things wrong here and then what we do is we we look at kind of the

5:17

average um the average error uh in the in the model and the higher the error the worse

5:24

okay so for those of you who are stats nerds this is a briar score it's the the

5:31

uh uh the the mean squared error

5:37

um but all you really need to know is ideally we would have zero zero error right but that's when we're doing

5:43

probabilistic forecasts that's that's never going to happen so we just want really low errors and the higher the

5:48

error the worse the model is um there's another way to measure how

5:55

good our our probabilistic forecasts are and that's using something called calibration so again this is saying

6:03

let's let's group together every time we predicted you know a an 80 chance of

6:08

rain and say okay on how many of those days did it rain and if it was eighty percent then you're

6:15

perfectly calibrated so the perfect calibration line is just a diagonal on this right if I pre if I group together

6:22

all the times I said 90 and 90 of the time it did rain I'm perfectly calibrated

6:27

um now if I'm over this line um I have what we're going to call a wet

6:35

bias right so actually here in order to make it match probability of success I'm I'm showing this as predicting no rain

6:41

predicting um uh dry days right which is analogous to success quote unquote in retirement

6:48

income planning so above the line is a wet bias I'm predicting more rain than

6:53

there actually is so I'm being conservative below the line is a dry bias where I'm predicting things will be

6:59

better than they actually end up being and actually in in weather forecasting

7:05

it wouldn't be uncommon for us to have a wet bias uh maybe even on purpose

7:11

um because people tend to prefer positive surprises right so people

7:16

aren't going to be as upset with you if you say it'll rain and it doesn't uh

7:22

that's just hey great versus you you say it'll be dry but it rains right you

7:28

didn't bring the umbrella okay so we're going to use those two tools the you know the kind of overall

7:35

error level of the forecast and there will always be error and calibration to test different ways of doing retirement

7:41

forecasting um and we tried to use models that are

7:48

available and or commonly used um so that again this could be really

7:54

um this could be uh usable in your in your practice so

8:00

we have uh traditional Monte Carlo which is by far the most commonly used uh

8:05

approach reduced Capital Market assumption Monte Carlo which just took our approach to traditional Monte Carlo

8:10

and reduced because return assumptions uh regime-based Monte Carlo

8:17

which uses two sets of Capital Market assumptions one for the near term and one for the long term and historical

8:23

analysis which uses historical sequences of returns and inflation to model

8:29

um to model the situation so uh in order to test forecasting we

8:37

have to have a a completely systematic way of producing Capital Market

8:43

assumptions in order to test them right because we're we're going to be testing forecasts that would have been made in

8:49

the past against what really happened in the past so we can't cheat we can't cheat by saying well I know what

8:54

happened so I'm going to create really good Capital Market assumptions um so this is how we did that how we

9:00

created them formulaically you know no cheating no putting your hands on the scale so for traditional Monte Carlo we

9:08

created the Capital Market assumptions using the preceding 30 years of of

9:15

returns and inflation okay so for each point in time we looked back 30 years took the average and

9:21

standard deviation and correlations for for that for reduced Capital Market assumption

9:26

Monte Carlo we just reduced that by two percent um and for all these we're using a 60 40

9:33

portfolio so if you're kind of wondering how large of a reduction that is it's

9:38

compared to a 60 40 portfolio for regime based um we use the approach that we actually

9:44

use in the income lab software to produce our default Capital Market assumptions

9:50

um which is we take all of history and for and then we filter out

9:57

any of the periods that are least like um each point in time from an economic

10:03

standpoint so think of that as inflation PE Cape things like that so uh you know

10:10

unemployment so we have a kind of a set of of things to um

10:16

to use as economic filters um and and that's what we apply for this study we actually simplified it even

10:22

more and we just used Cape an income lab we use a little bit more than that and what we do is then we so we filter out

10:29

um I believe for this study it was it was half of history and then using the remaining history the kinds that are

10:36

closest to um to to um you know each point in time we use

10:41

the uh we we produce two sets of Capital Market assumptions one for the first 10 years one for

10:48

um the remainder of the plan and then for historical at each point in

10:53

time we're just using all historical returns and inflation that's available up to each point in time so all this is

10:59

completely formulaic there's no opinion uh in here and I think it probably

11:05

um you know it's it's as Fair as as you can get um for representing how producing

11:11

Capital Market assumptions at each point in time could work um this is just a look at the how

11:17

regime-based Monte Carlo would apply because this is a maybe a little less familiar to a lot of people

11:24

um so if you applied the near-term Capital Market assumptions to the first

11:29

chunk our default is 10 years and actually often Capital Market assumptions that are available to

11:35

advisors really are more for like a 10-year period than a 30-year period so we apply the near term and then

11:41

everything else is the long term so again not to not to belabor this

11:46

point but I think it's really important to understand what this what this uh study is doing so let's say for example

11:52

we're looking at forecasts that would have been made in June 1960.

11:59

at that point it's going to look at the 30 years preceding that point to create

12:05

the Capital Market assumptions and then it's going to forecast probability of success for the period after that all

12:12

right and then we'll do it again for July and now we have one more month in the preceding period and we're pushing a

12:18

month forward in the forecast period than August and so on for regime based

12:24

Monte Carlo doing a very similar thing except now because we're using all of history up to

12:30

that point but filtering it um in June we have you know everything

12:35

from 1871 which is when our data began up through um up through May of 1960.

12:43

and then we moved to July August and so on

12:48

and again the filter is we're just excluding half of History where Cape is furthest from the value at the forecast

12:54

point so it's not you know does is Cape similar to today it's in this case is

13:00

Cape similar or not to August 1960 right so it's always trying to recreate what

13:06

would forecasting really have been like at each point in time for historical again we're using all

13:13

preceding historical returns and inflation and then as time goes on we're getting a little bit more a little bit

13:19

more data Okay so using that approach

13:26

um how do these models perform um and and again the first way we measure this is with error and our

13:34

reminder is lower error is better higher errors are worse

13:40

um we the way we if we kind of tested all this was we said all right we're

13:45

going to take um every monthly point from 1951 to 2002

13:53

and we're gonna give each model um 200 different systematic withdrawal

13:59

plans from a 60 40 stock and bond portfolio very simple plans nothing it's

14:04

just withdrawals um it's it's flat inflation adjusted withdrawals we're not applying the smile

14:10

or things like that that we often people do on income lab there's no social

14:16

security or anything very very stripped down simplistic approach

14:21

um obviously we would want to you know apply these to to more complex plans as well but it's often uh really useful

14:29

just to start with something simple um and the 200 different systematic withdrawal plans are between

14:36

um you know very conservative low withdrawal and higher withdrawal plans

14:41

um that we you know we pre-baked them to be a range that was from extremely safe to to extremely risky uh and so this

14:49

allowed these models to have a chance to say oh this one will be risky this one will be safe this one will be in between

14:56

um the reason that it only went to 2002 was we were having it predict

15:02

um success or failure of a 20-year plan so the last 20-year plan at the time that we did this work would have ended

15:09

in 2022 um so we couldn't go further than that um we did test 30-year plans as well and

15:16

then looked at the correlation between those results and the 20-year plans and it was quite good so we're we're

15:22

relatively confident that this these results extend to longer plans as well the reason we started in 1951 was we

15:30

wanted to give those those approaches that needed a lot of history we wanted to give them a good amount of History to

15:35

be able to work with right it goes from 1871 um so if we had started them in 1900

15:41

they would have had very few years to actually um to actually use in there in producing their their model

15:48

of the world okay so what were the results um there was really a clear set two sets

15:57

of results traditional Monte Carlo and reduced uh Capital Market assumption Monte Carlo which are by far the most

16:04

commonly used approaches had I had a lot higher error rates than

16:09

regime-based Monte Carlo and historical so there's a way of comparing these and basically the two on the right had 25

16:16

percent lower error um than than the than the two on the left

16:21

so pretty pretty large pretty pretty important difference here

16:27

what about calibration um remember this is about hey you know is the model particularly good in

16:34

certain spots particularly bad in other spots is it is it um you know consistently above the line or

16:39

consistently below the line and so on and probably because most people aren't

16:45

uh you know producing plans with a you know 10 probability of success uh or at

16:50

least you know following plans like that we can just you know focus on the upper side of this

16:57

um where interestingly the traditional Monte Carlo

17:02

uh gets a pretty big dry bias meaning it's underestimating risk right it's

17:08

telling people hey leave the umbrella at home it's going to be fine but then it rains

17:15

um it's actually interesting because on the on the lower end which again nobody's really using uh it it has a wet

17:21

bias but then in the important area it goes to a dry bias

17:26

um regime based Monte Carlo on the other hand which is the green line consistently has a wet bias

17:33

um so this is overestimating risk meaning it's going to say whoa whoa whoa

17:40

take the umbrella take the rain coat but then you know more than is predicted it

17:45

turns out to be a sunny day so it's overestimating risk

17:51

what does that mean for clients um in reality or for advisors

17:57

uh let's pick the uh the predicted 85 probability of success that's a very and

18:04

I think most people would agree even though you know again at income lab we we try to reject this success failure

18:10

framing as much as possible but you know for the purpose of this I'll use it um if we looked at plans that that had a

18:18

predicted 85 probability of success uh for every 100 people that were set

18:24

off on their Journey with one of these plans the traditional Monte Carlo

18:30

um you would have expected 85 of them to be fine and you know 15 to uh to

18:36

quote-unquote fail which you know in our world means adjust downward uh but in

18:41

fact 27 of them per 100 would have had to decrease spending over time because this was underestimating risk so it was

18:47

predicting 85 but in reality it was 73. for regime-based Monte Carlo predicted

18:53

85 in reality it was over 91. so uh we

18:59

we expected 15 out of 100 to need to adjust downward but in fact only nine uh had to so uh basically under promising

19:07

over delivering for the green line for regime-based Monte Carlo um over promising under delivering for

19:14

the red line traditional Monte Carlo um and I chose 85 both because I think most people would agree that's a

19:20

reasonably strong plan and because it's kind of it is sort of a middling place for both

19:26

of these um you know there are obviously you can see that these lines are quite Jagged so are there spots where they're closer to the line spots where they're

19:32

farther away so this just kind of one representative um spot here

19:39

um I think before we we get on to the next Point um and Derek I think uh you know you

19:45

have some some thoughts on this uh we're we'll come back to this wet and dry bias

19:52

and talk about whether um one or the other would be you know

19:57

preferable um given that we know there are errors when you're making forecasts what kinds of errors are better as an advisor and

20:04

as a client um and I think you know most would agree that a a wet bias an overestimate at

20:11

risk um under promising and over delivering is a is a better safer way to

20:16

go for for the advisor on the client and it also leads to better client experiences because again you're going

20:22

to set out expecting that eight that 15 out of 100 will need to adjust downward

20:29

um but in fact only nine will and that probably also means uh you know more people will get pay raises than than you

20:35

had expected so surprising to the upside okay

20:40

let's look at um go back to errors now and and I think

20:45

it's interesting to look at not the entire uh time period all at once again that

20:52

was from the 50s to the to the early 2000s for when we're making those forecasts but to look at them in Rolling

20:59

five-year periods to kind of see okay are there environments where some models

21:05

perform really well some models perform really poorly do they flip ever and so on

21:12

um and here we see our two uh worst

21:17

performing models overall in the in the Bold in the orange and the red that's traditional Monte Carlo and reduced uh

21:26

Capital Market assumption Monte Carlo um they do have brief periods where the

21:31

error is quite low especially the reduced um Capital Market assumption Monte Carlo

21:36

which kind of makes sense basically if we look through these periods so at each point in time we're looking at the

21:43

errors uh for the forecast that in the five preceding years so in

21:50

1956 it's the the forecast from 51 to 56. so we see through the 60s and into the

21:57

70s reduced Capital Market assumptions did really well why because that turns out in retrospect

22:04

that that was kind of the worst sequence of returns for systematic withdrawals

22:10

um in the period we're looking at right so of course having used lower Capital

22:16

Market assumptions during this time would have been better what we're seeing here is reduced Capital Market assumption Monte Carlo it's a one-trick

22:23

pony it does Well when things are going to turn out to be rough

22:28

um traditional Monte Carlo has a couple spots where it's low you can see basically uh the late 50s

22:35

and then again it does well um kind of in the late 70s the reason is

22:40

it's always used in the preceding 30 years and you know the period in the 60s and 70s was had low

22:47

inflation-adjusted returns you can see everything did pretty poorly in the 80s

22:53

um because we were you know the 80s actually turned out to be a really good time to retire because sequence of

22:59

returns was going to be really good um none of the models got that quite right but

23:06

you can see that the historical and regime based uh did not you know kind of

23:12

just rise to the moon on their on their errors and overall these two approaches

23:18

did um did fairly well across periods at least comparatively

23:25

um you can see regime based actually probably because of that wet bias right

23:30

it's kind of over over predicting risk in a lot of cases it does really well through that worst period in this in

23:36

this study from the 60s um and again it actually does really

23:41

well in the uh into the.com bubble era area

23:46

um probably because of that Cape um filter right so Cape was really high so it was predicting low returns

23:54

um so we can see here that these two historical and regime based are are a

23:59

little bit closer to being kind of uh all environment um uh good performers at least in

24:06

comparison to the others that we looked at so more consistent across that range of environments and this is the thing

24:11

when we're doing forecasting when we're doing retirement income planning we don't really know what world people are

24:17

going to live through so we would like models that actually have a have a hope of doing okay whether things turn out

24:22

really good or really bad or somewhere in between so um

24:29

what can we take from from this um I I think you know there's a lot of

24:34

stats in here but let's let's back up we tried to look at

24:40

um at approaches to forecasting approaches to planning approaches to

24:45

kind of you know scoring a plan you know seeing whether it was uh you know really

24:50

safe or or really aggressive that people actually use and traditional Monte Carlo which uses

24:58

one set of Capital Market assumptions is certainly um the most common way to do this

25:04

um you could you could uh you know argue that our use of the preceding 30 years

25:09

you know is not representative how people do this um and maybe that's true for you

25:15

um we also looked at uh reducing those Capital Market assumptions which I think I think that a lot of people will do

25:21

especially on kind of some of those forward-looking um Capital Market assumptions that you can find they do tend to say

25:28

um okay let's reduce our our assumptions going forward um certainly if we had a big database of

25:35

the Capital Market assumptions that were actually made at each time at each point in time that would be an even better way

25:41

to do this study but I think the results here are are fairly clear that

25:46

traditional Monte Carlo um using one set of Capital Market assumptions and using historical

25:52

averages and reducing those Capital Market assumptions

25:57

um really don't perform well compared to other options that are available probably the most commonly used

26:06

um of those other Methods at least in research is historical analysis so

26:13

um you know like the uh Bill bangin's uh work in the 90s that led to the you know

26:18

four percent rule that was all historical analysis um I think one worry people have often

26:24

had about historical analysis is hey you're only looking at things that have actually happened you're not looking at things that haven't happened but could

26:31

happen um so this is and fair enough that philosophically certainly uh Rings true

26:36

and makes a lot of sense um as a as a critique of historical analysis but what really matters with

26:43

our analysis with our forecasting with our planning is what would the actual impact on our advice and on client

26:50

outcomes be right so this isn't about kind of some just purely academic

26:55

discussion it's what's the impact and it looks like historical analysis even

27:00

though it's only using you know what's actually happened is actually still providing better

27:06

um lower error rates than you know traditional Monte Carlo at least done

27:12

here um regime based Monte Carlo has ever so slightly higher error rates across the

27:17

board um oh no sorry I'm wrong uh regime base has lower error rates across the board

27:23

but again this is a very small difference we don't know for sure um whether this will will be sustained

27:29

um and it provides a little bit more of a of a

27:35

consistent error meaning it's it's almost always I think maybe truly always yeah uh erroring on the Wet Side on the

27:44

over predicting risk on the under promising and over delivering side whereas historical you can see

27:51

um is a little bit more of a mixed bag it's the blue line here so it does it it stays a little tighter but it tends to

27:57

kind of go to either side of the of the perfect calibration line here

28:03

so I I think Derek and I were both you know somewhat surprised um by these results

28:10

um but also hardened in a way that there are you know there are ways of doing forecasting that improve on the status

28:18

quo so using historical using um regime based Monte Carlo do does

28:24

reduce errors you know by 25 um and that's that's truly important for

28:31

you know these are consequential things that we're advising people on um on how much they can spend in

28:36

retirement what the chances are that they'll have to adjust and so on so really using models that have lower

28:42

error uh you know is kind of a no-brainer I think

28:48

um probably the maybe the most important uh conclusion here though is that no model

28:55

was even close to being error-free so these these error rates uh or these

29:03

error scores are high to give you a comparison

29:09

if I had you know if I was predicting rain um and I just always said there's a 50

29:15

chance of rain all right I'm just going to take the I call it fence sitting right I'm sitting on the fence every day that error rate

29:23

would be 0.25 so uh and that's essentially like I'm

29:30

not going to do any work I'm just going to predict 50 50 on everything okay I would get an error of 0.25 so

29:37

these error rates are high um you know reduced Monte Carlo 0.18 I

29:43

mean that's just that's that's really bad um on these five-year rolling periods

29:49

our error rates get far above that we're just you know like the prediction is worse than just flipping a coin

29:55

um so and and even with uh some of these better performing ones regime based and

30:01

historical they they do get high occasionally like in the 80s um so what do we do about that

30:08

this shows that we really can't um we can't just take our prediction

30:15

probability of success or you know in an income lab we talk we think about it as risk and you know when would we adjust

30:21

and so on um we can't just take those from the beginning of our plan and and run with

30:28

them um and never and never come back to it never adjust this shows that the errors

30:33

are high enough that we really need a plan for adjustment a systematic plan for

30:39

adjustment and there's a couple reasons for that one is just that we know that we will be wrong

30:45

we're going to try you know to use the best models possible to have the lowest errors um but we're going to need a hand on the

30:51

steering wheel right so it's um you know it's a little bit like uh

30:57

like driving down the road you know you might be able to go for a while just just pressing the accelerator and hoping

31:02

the wheels stays straight but you as things come up as potholes come up as you know little deviations in the road

31:08

come up you gotta you gotta adjust because now you're learning what road you're actually driving down what what adjustments are needed

31:15

um and that's the other part of this is as time goes on let's say somebody retired in I don't know 1980

31:23

well things would have felt pretty bad at that time they've just come through stagflation they didn't know when

31:30

inflation was going to end it really didn't wind down until a few years later or start winding down and so

31:37

as time goes on the planner and the model that you're using um is learning

31:43

more about what environment it's in so it'd be very silly to go from 1980 to 1986 and not use you know what you've

31:50

learned between 1980 and 1986 to improve uh your advice for a client and so

31:58

that's another reason that um that it's not just about forecasting

32:04

success and failure or you know spending capacity and guard rails at the

32:09

beginning of retirement it's really important that it be updated over and over and over again as you learn more

32:15

another way to look at it would be you know you could predict um you know rain or not rain uh a week

32:24

out right I um just went camping with my family this weekend and I was believe me

32:29

I was looking okay what's it going to be like what's it going to be like I was looking a week out but I knew as we got closer I was going to have better

32:35

predictions right I was we were going to learn more about you know what the weather patterns were and would have

32:40

been very silly of me to um you know ignore the the radar

32:46

um as we were you know driving into the mountains um that's new information that's

32:51

information that's that's crucial for planning so I think if you're going to

32:57

take one thing away from this uh it is that not only do we need good

33:02

forecasting models best that we can have but since no forecasting model will be perfect

33:07

in fact far far from it we need ways to um to update our models update our

33:15

forecasts and adjust as as we learn um what's happening

33:21

so Derek I wanted to before we take questions um I wanted to kind of get your

33:27

um your thoughts and input on what a you know a financial planner with you know

33:33

who's who's doing planning and retirement for clients like what the the takeaways would be for you and how it

33:39

affects your your processes now for me I mean it's really comes down

33:46

to you know which which model here do we want to use actually working with clients I think I lean heavily and I

33:53

like regime based Monte Carlo or historical um I I don't you know we've gotten into a lot of weeds here in terms of

34:00

technical details this is something I almost never discuss with clients like this isn't something I'm really talking

34:06

to them about but in terms of as a professional selecting from different methods what do I want to use

34:13

um I I think there are some just on a practical level I mean one thing I like about historical uh results is that you

34:20

are going to get more consistent between areas you update a plan things like that so that is one factor I consider and one

34:28

reason why I actually do kind of like historical I think historical if a client does ask you know what is this

34:34

based off of it's also easier to explain you know we're looking at actually you

34:40

know kind of what you could have spent going through historical markets you know I think that's a little bit easier

34:46

um just conceptually then especially regime based Monte Carlo which is a fairly

34:51

technical type of uh consideration but I think you know at the same time if you if certainly there's advantages to the

34:59

regime based Monte Carlo um you know I think the market we're in to can dictate some

35:05

of that like you know if you rewind the clock when you know interest rates hadn't risen stock valuations were

35:12

higher that was probably an environment where I was definitely even more inclined to use a regime based Monte

35:18

Carlo just given the environment we're in and kind of the unique risks I feel like you know that's maybe

35:24

now we've seen interest rates come up we've seen evaluation you know that we at least have a little bit of a dip

35:29

there right it's not as uh maybe doesn't feel like as unique of a

35:34

time as it did before but at the same time um you know they'll just finding that

35:42

most relevant sort of Market environment that we're in and projecting based on a short-term basis and then a

35:49

long term uh basis that conceptually that makes a lot of sense to me too so really to me it comes down to I want to

35:55

use one of those two methodologies personally when I'm actually running a plan but this is something I'm not I'm

36:03

just doing this internally as a as a professional I'm not actually communicating much of this to to clients

36:09

unless they really want to dive into those weeds I think that's a really good point this is like uh I don't know a doctor

36:16

choosing uh you know a blood test approach or a I don't know a um you know

36:22

choosing a medical device a replacement hip or something like that um there are you know this is where you

36:30

know kind of your expertise comes in to choosing the methodology uh occasionally it comes up it sounds like Derek I don't

36:36

know to what if you have a a guess of how often a client has sort of asked for

36:43

more on uh on where's this stuff coming from very rare I mean you know for me and

36:50

it's even a lot of times if I do get a question right because sometimes if you're presenting something and it feels

36:56

too kind of black boxy somebody's just you know where are these numbers coming from to me it's more like what are the

37:02

questions I get are more what are the return assumptions or you know how does this you know what what's driving this

37:07

certainly not down to the level of historical versus Monte Carlo versus regime based Monte Carlo like that's a

37:14

level of detail I don't think I've ever had a client push me to I'm sure there are some out there that would but um I just

37:21

personally haven't ever run into that yeah um

37:27

do you have um when when you're considering what to use sounds like you've used historical

37:32

and regime based um what it sounds like one consideration

37:37

there is you know in particular maybe uh when valuations were extremely high when

37:43

interest rates were extremely low you liked to use regime based Monte Carlo

37:49

um are well actually before I ask my question I wanted to bring up another example I saw some people last year

37:57

um who um were I talked with them about plans that they were doing that had um

38:05

that had a lot of pension income that wasn't adjusted for inflation

38:10

and so those plans were were really subject to inflation risk

38:15

and I uh what we discovered was there there really is something about regime-based Monte Carlo there where you

38:22

can really Express um a risk or Express an opinion in a way that you can't with historical so they

38:29

were able to say well let's really test this plan or produce a plan that assumes inflation will be high at least at the

38:35

beginning of the plan which was important for these plans in particular because if inflation was high and it

38:41

looked like it was going to be um that would that would really change how much someone could spend so that was

38:47

another place where so not well I guess it was a mix of the environment because we saw inflation coming up and the plan

38:54

itself seemed to to really have a weakness which was that inflation risk

38:59

and so they were able to say all right let's make sure we can handle inflation risk because we think we will see it

39:05

with regime base whereas historical analysis will include High inflation periods but it'll also include low inflation periods it'll include

39:11

deflation right so it's a really nice representation of what the world can be

39:16

like but it is not so focused on a particular risk or a particular plan

39:22

um so I agree with you there yeah I think that's a good point on that having the ability to maybe fine-tune

39:30

some of that we're with the historical record it is what it is versus being able to go in and actually Express

39:36

here's a concern for this particular plan it could be a little bit more precise or Surgical and kind of your

39:42

what you're trying to look at with regime based yeah yeah

39:47

um I know when we were doing this uh you also have some thoughts on this uh you

39:53

know web bias or overpredicting risk or under under promising over delivering

40:00

um how important do you think that is for people especially when you're doing

40:05

ongoing advice rather than just point in time advice yeah it's it's something I try to

40:11

balance because you know you don't want so much wet bias in the sense that you

40:16

know you're just you're telling somebody to spend way too little I do think there is you know a certain degree that's nice

40:22

to kind of have that positive uh you know they're more likely positive income

40:29

experience going forward so I do I do think a little bit of that is nice

40:35

um you know at the same time though you know there's multiple ways to get at that I mean you can introduce a little

40:40

web bias even by shading your your recommendation down a little bit or whatever that might be but if you want

40:46

to use the full power of you know income lab and kind of you know leveraging the

40:51

software to automate a lot of that that is something that I think is nice that you get with the the regime based Monte

40:57

Carlo I'm also comfortable with historical just kind of being in best

41:02

guess in a sense like I that doesn't cause me much concern

41:08

um it does I do find it concerning when you know we see particularly with the traditional Monte Carlo uh actually you

41:16

know having the dry bias that to me is a is a real concern because that's you know especially somebody who's not using

41:23

a risk-based guard rails type approach and that's particularly concerning when you're running a plan and targeting say

41:28

90 probability of success and maybe uh not realizing that underestimation of

41:35

risk is there but um you know for me that's something I really want to stay

41:40

away from so I think a little bit of uh web bias is good um also okay with kind of hitting the

41:47

hitting the mark but really do like to stay away from that dry bias

41:53

yeah that's a good point I both are bad um you know it's just harder to see the the cost of the wet bias is reduced

42:00

standard of living right um which is uh that is a that is a cost

42:05

it's a really important cost we tend to focus on the the scarier costs

42:10

um but um you're right trying to trying to hit it as close as possible um probably as a a good goal at least

42:18

um okay I think we've got a whole bunch of questions here so I don't know if uh that's right

42:26

um I've been keeping track of them so I've kind of broken them up into some categories here um but to first start off uh just a

42:33

reminder to our users um that at the end of the webinar we'll have that survey where you can give us

42:38

feedback and also put in your cfp number so that way we can get you the credit um but their congestion we had one great

42:45

feedback great presentation uh users are wondering if you guys are planning on writing an article on this topic and as

42:52

well as providing the slides uh afterwards yeah so I will drop the uh

43:01

the uh let's see here how do I do that can everybody see the chat I think

43:09

I can change it to everyone okay so here's a link to um

43:15

through the article on this that um that Derek and I um

43:20

posted on the kids's blog a little while back so you know very

43:26

similar similar content to here if you want to um dive in um I also see some

43:33

some folks asking for the slides happy to make a PDF of these so we can send it out

43:39

okay these uh first group of questions are around uh regime based uh Monte

43:44

Carlo um we had a few upvotes and maybe Derek you might be able to answer this um just wondering you know do the major planning

43:50

software companies provide means to model regime based like right Capital uh money guide Pro e-money

43:58

yeah I I haven't done a recent check on

44:03

the other software but last I knew I don't believe that's something that you can do

44:09

um in any of the major uh Monte Carlo platforms that are out there

44:14

gotcha gotcha great and then um next question here is uh just wondering why do you use two

44:21

sets of Capital Market assumptions in the regime based Monte Carlo method

44:29

yeah I mean the whole concept there is really when you look at you know how do

44:36

you what can what can we actually best predict with economic information I

44:41

think there's more evidence that that shorter term window right we can kind of take advantage of say a a 10-year period

44:47

but based on the current environment we're in to change our assumptions but then you don't want to use that

44:55

um that short-term window long term right because we know there's going to be some reversion back to kind of the

45:01

long-term average so that's the conceptual ideas basically take advantage of what we can say about the

45:07

current market we're in that's more going to be short-term longer term we're going to get back to more with that

45:13

long-term average would be yeah if you think about you know maybe

45:19

you had really low you know Bond return assumptions

45:24

there's a big difference between assuming those will be with you for five or ten years and something they'll be with you for 30 years especially if

45:31

you're heavily invested in bonds um so yeah that's the that's definitely a concept of having two different sets I

45:37

mean you could certainly uh imagine a world where you had even more but there's also kind of a practical

45:43

um aspect to this and you know how much is to be gained uh by by the extra work

45:49

if you had three four five uh achieves um and I think the other point which Derek just made is you know there are

45:54

Capital Market assumption forecasts out there uh um and those don't tend to be for 30 or 40

46:01

years they they tend to be for five or ten years um so really there is a mismatch between

46:07

putting those in for a plan you know to use on plans that are longer than that

46:14

um the way that it works in our software is you if you're using regime-based

46:19

Monte Carlo the first uh period of the plan will be covered by the first set we

46:24

call them near-term Capital Market assumptions by default that's 10 years but you can change it so if you had a plan that was 12 years it would be you

46:32

know most of it would be using the near term um if you had a plan that was 30 years most people were using the long term

46:40

just an interesting one do you guys know the inspiration behind the name regime based

46:46

I think I um my understanding is really it's just saying that there's multiple regimes

46:52

right periods that we're in of assumptions so regime based myocardial being you know two or more regimes

47:00

instead of just using one um one set of assumptions yeah I think it's it's sort of like

47:06

economic regimes or something like that is what the thought is there I don't actually know who coined it

47:11

um there there's been a fair amount of there's been sort of a popular tool in

47:17

the research world so I know kids this is written on it a bunch of times I think Wade fowl and um

47:24

uh there was a good article um years ago I'm trying to remember who co-authored that um also looked at this

47:30

kind of thing it just hasn't been available to advisors um until recently

47:38

and um speaking of that we have a few uh folks who aren't income lab users and they're just wondering you know are all

47:43

these methods available um to select within the software they are yeah so we even do have a

47:50

traditional Monte Carlo with one set of Capital Market assumptions um we actually use a 50-year average

47:55

instead of a 30-year but otherwise it's the same for our defaults but you can you can always edit on a market assumptions so yeah all of them are

48:01

available right and then um these next group are more about kind of communicating with

48:06

clients um so you know how would you recommend it be communicated to a client while using guard drills or dynamic

48:13

distribution strategy results in such significantly higher income level than using Monte Carlo

48:19

and if you have a few talking points here yeah for me I'd say one first question

48:25

that comes to mind is you know what what was the Monte Carlo plan itself

48:31

shooting for right because I I mean that could be part of the reason why you see different results depending on you know

48:37

if you're shooting for 99 probability of success right that's going to be part of what's what's driving that but I think

48:43

you know if you're looking for just really you know quick easy talking point there is something to the fact that you

48:50

can spend a little bit more with confidence if we know that the plan is to make adjustments when needed right

48:56

that's the how I would kind of distill that down because it does allow you to spend a little bit

49:02

more if you're willing to make an adjustment and this is you know really actually a good opportunity for a

49:07

broader conversation to look at something like the retirement stress test and take somebody through

49:13

historical periods you know show them what the downside was like and see if that's the right level of risk for them

49:20

because somebody might you know that's that's one thing you're not going to get with a traditional Monte Carlo simulation at all not not a projection

49:29

using traditional Monte Carlo I'm just saying like a traditional Monte Carlo software you're not going to be able to see what that income experience was like

49:35

because there's no assumed adjustment um you're just going to charge forward blindly but in this case you know seeing

49:42

okay if somebody actually followed that strategy is this a reasonable level of downside

49:47

risk and if somebody's willing to make those cuts back then that might be the right strategy or maybe you need to

49:54

fine-tune the strategy and dial in the risk level of the guardrails to make sure that matches better

50:02

yeah I think um that yeah there was another point about

50:07

you know we were saying well adjusting is important because we know that there will be errors but adjusting is also

50:13

important because it allows um clients to you know if it's if it's

50:18

appropriate for their kind of temperament right they can accept a higher chance of having to pull back

50:25

um in exchange for spending more so you know this is the income lab software um this you know this spending level for

50:32

this particular plan which has all sorts of stuff in it uh Social Security uh

50:38

portfolio all sorts of things and that's what's producing this over twenty thousand dollars and monthly uh income

50:44

but really there's a range you could you could go more conservative

50:50

um spend less today and have a larger gap before you'd have to adjust or you

50:56

could say no I wanna I wanna spend a lot more but I have a smaller gap before I would have to adjust so yeah that is

51:02

another another real positive of of planning with adjustments not only does

51:07

it mean you know as you learn things over time you can kind of make up for your errors right like maybe you didn't think it was going to rain but then you

51:13

know dark clouds appear on the horizon well okay we're going to adjust a little bit but also it allows people to kind of

51:19

match their spending to their to their temperament to their wrist tolerance

51:25

and maybe just this kind of a touching on on that point a little bit but um one

51:31

of our users was curious as to you know is there a general ballpark range of how much higher spending can be in

51:36

retirement using a dynamic strategy versus um a static strategy

51:42

10 20 higher um for example in this do you know if there's any formal or informal research done in that area that

51:49

maybe an advisor can kind of uh get pointed to

51:55

I think it would depend on how much the plan depends on portfolio withdrawals um there is using the smile which this

52:02

example plan I'm showing um does we know that that tends to add

52:08

for someone in their 60s that adds probably 15 to 20 percent um spending capacity basically because

52:14

you're letting them spend you're kind of stealing from the latent plan and putting it earlier right we're just kind

52:19

of shoving it toward the the younger years um so that's I think pretty well established that the smile does that now

52:24

that's not Dynamic adjustment based planning that's just using the smile but that's

52:30

definitely um a factor that people do enjoy I don't know Derek do you have thoughts on that

52:35

I mean once you know you can adjust does it provide more spending capacity I think this is a situation where the

52:43

the unique client circumstances are going to Trump any sort of generalized

52:48

statement you can make because you know depending on how much guaranteed income somebody has versus portfolio income

52:55

um that's going to be a factor of what risk level they're using on their guard rails is going to be a factor so for me

53:01

I you know rather than you know gut feeling says that you know maybe those

53:07

aren't you know maybe 10 is not a terrible level that you might just expect to see but at the same time I

53:13

don't even really give that much thought I'd rather build out the plan and see for that particular client you know what what do we see

53:20

because you know there can be spending um like Justin's talking about in terms of spending smile but it could also just

53:26

be somebody's goals maybe they have large goals at certain times that have unique circumstances for their plans so

53:32

I would really rather build that out and see what we get from there yeah Mike Amore into play if somebody is

53:39

you know if their plan is more of a reach right if somebody has lots of resources and they spend frugally

53:45

you know I'm not even sure it comes into play whereas if somebody is saying I really want to retire

53:52

um this is how much I need um you could have a conversation with them that says well in order to do that

53:57

we got we're gonna have to take on a little bit more risk that you are going to have to you know pull back from that

54:03

point so just Eyes Wide Open yes we can do it but understand you've there's a

54:09

decent chance I'm gonna call you and tell you it's time to tighten the belt so maybe in that situation it could give

54:14

people some permission to to try it out uh to try out retiring I could see that

54:19

coming into play um and real quick our tax rates modeled

54:25

into Monte Carlos uh they

54:31

yeah they typically are I mean there's ways to do it with or without um but yes

54:37

I mean it the reason that kind of waffle a bit is because um let's take an IRA for example the

54:45

withdrawals you can take the risk of a withdrawal rate from an IRA does not depend on your tax rate right like your

54:53

IRA doesn't care what your tax rate is um what it will sustain in withdrawals does not care what your tax rate is so if taxes go down you'll be able to spend

54:59

more of it but you won't be able to take more of it um at the same risk level so it the

55:04

interplay of taxes and risk in retirement is unfortunately not like just perfectly easy and clear

55:12

and just just to jump in there if the question there was getting at more just like does income lab take taxes into

55:18

consideration absolutely that's that's there so um that uh is something to be accounted

55:24

for we do have quite a few questions just want to point out uh most of those

55:30

things it looks like they're about getting the slides as Justin mentioned we'll get you guys the slides um PDF form and a follow-up and I think you did

55:37

drop the link in here as well um but maybe maybe this last one while we have two minutes

55:42

um just some questions on you know is income lab only designed for the distribution phase of retirement I know

55:47

we get that question a lot maybe there Justin you guys can maybe use our last few minutes to kind of help our new

55:53

folks maybe understand that piece I'll jump in with my my take on that

55:59

question I guess in my opinion um certainly there's you know a lot of really practical application for

56:05

somebody's who's inner approaching that distribution phase of retirement um there are ways that I still use it

56:11

with accumulators so if somebody wants some sort of projection kind of what income trajectory they might be on can I

56:18

I don't you know running projections for somebody in their 30s there's so many assumptions there that it's almost not that useful in in my opinion but

56:25

somebody wants some of that I do think it's still better than running a Monte Carlo simulation for somebody early on

56:31

um but then particularly getting into life Hub that's an area where I could have a lot of different things that I'm

56:37

doing with somebody who's more of an accumulator um and farther off from retirement just in terms of

56:43

touching in making sure or making sure we're covering everything's up to date I have that all information have that all

56:50

captured but then also some of the projection capabilities and saying you know you're on track to pay off your mortgage at this time or here's the date

56:57

uh here's where you're you know it kind of project some account balances out things like that I do think are very

57:02

useful for somebody outside of a traditional retiree yeah I would Echo that I would say this

57:09

the actual you know what can you spend and what would make that change and what would the change look like

57:15

that is typically more useful close close to at and in retirement I think of

57:20

it as a GPS so like when would you find you know starting up Google Maps or ways

57:27

to be useful when you're going on a trip it's not necessarily just as soon as you get in the car it might be hours before

57:34

right but but it's probably not months before Perfect all right guys thank you we are

57:41

up on our time to all of our users thank you again for joining today as I mentioned again please fill out the

57:47

survey we look forward to hearing your feedback and making sure you get your cfp credit um for the folks we didn't get to your

57:53

questions feel free to reach out to us in the survey too you can uh request a one-on-one meeting and that's a great

57:59

time to get to know the software have any questions answered and see a demo as well

58:04

um and then please be on the Lookouts for our next webinar uh next month that we will be sending out the invite to

58:10

that as well Justin and Derek as always thank you guys for the time and effort you put into these webinars you'll see

58:16

in the comments that we had a lot of uh folks just saying a really great presentation and sharing a lot of good

58:21

feedback so I want to make sure you sought those as well coming in all right well thank you everyone we will send the

58:27

email out tomorrow with the reporting uh so you will see that uh in your inboxes tomorrow have a wonderful day and take care

58:34

everybody thanks everyone