How and why to move beyond "Probability of Success" in retirement planning
Learn why it's important to go beyond traditional success metrics when planning for retirement.
Last published on: August 29, 2025
PANEL: Jamie Hopkins, David Blanchett, Jeff Brown
Monte Carlo simulations and the concept of "Probability of Success" were significant innovations in financial planning more than two decades ago. However, it's now clear that retirement isn't a pass/fail proposition and that today's retirees need guidance, not scores. Our panel of experts will discuss why it's essential to move beyond "probability of success" scores in retirement planning and how to communicate effectively with clients without focusing on this misleading statistic, ultimately leading to improved client outcomes, experiences, and communication.
Â
Video: How and why to move beyond "Probability of Success" in retirement planning
Webinar Transcript
Failure
Monte Carlo
Good Advisors
Carcin Ecosystem
The Problem with Probability of Success
The Problem with Dynamic Spending
Analogy Google Maps and Waze
People cant adjust their spending
Saving vs Spending
How do you get people to shift
Monte Carlo simulation
Confidence
Analogy
Spending
Enjoying your savings
Building a fixed income floor
thank you matley and a big thank you to our panel David Blanchette
0:05
Jeff Brown Jamie Hopkins we got a star-studded panel here today um David from pgim
0:12
uh Jeff from Stratos private wealth and uh Jamie Hopkins from the Carson group and I think it's a perfect
0:19
um combo to discuss our topic today which is um probability of success and really how
0:26
and why uh to move beyond that score that statistic in uh retirement planning
0:34
um you know this this group uh jointly and probably even individually has talked with thousands of advisors and
0:40
clients um over the years thought hard about this topic thought hard about find experiences client outcomes the research
0:48
behind the analytics Behind these issues um so I'm really excited to hear
0:55
um their their thoughts and ideas on this hopefully some some practical takeaways for uh the advisors in the
1:02
audience so I'll kick it off with just a quick intro but try not to talk myself too much today either given our our
1:08
star-studded panel um so I I thought it kind of just lay the groundwork
1:13
um and and admit that Monte Carlo simulations they were a really important
1:19
innovation in financial planning a couple decades ago and this kind of simulation analysis would
1:26
be brought more broadly what it is it it the important thing is it allows you to take risk into account I'm certainly had to account
1:32
um and along with that we kind of got this
1:37
basically now ubiquitous score of probability of success I think it's important to actually understand those
1:43
two things Monte Carlo analysis and probability of success are different things they're related but probability
1:49
of success is just a way of of uh kind of reporting on Monte Carlo uh
1:54
simulation so you know I don't think participants should should take this we'll probably get into a bit as Monte
2:01
Carlo is terrible that's that's not what we're saying what we're talking about here is probability of success um and just to show my own hand those of
2:08
you who followed income lab over the years know that our software will certainly never have terms like success
2:13
and failure in it we don't believe those Concepts belong in retirement income planning a lot of reasons for that I think we're
2:20
going to get into um Jeff Brown you've written about kind of the bad gamification of client behavior um leading them to restrict
2:27
spending um you know we've heard about anxiety and worry that's triggered by this
2:33
um and these are kind of the things that we're going to be talking about today so with that
2:38
thank you again to our panel for being here um and actually Jeff since you're probably closest to the action
2:44
um on this I thought I'd start with you and get your thoughts on kind of over the years with with your clients with
2:50
your teams how has a focus on probability of success
2:56
um affected your clients experiences your own experiences your team's experiences whether that's
3:02
um you know on their decisions the outcomes people have in planning have there been good ways to use it bad ways
3:09
to use it what what have you seen yeah thanks Justin I
3:14
think 99 5 date myself a bit so when when I first started it was a lot of
3:19
linear type of financial planning right so so inflation's constant returns are constant and Monte Carlo entered with
3:26
very complicated software at the beginning uh to really what I what I would Define now looking back on it was
3:32
was just talking about the sequence of returns risk the the re the fact that things are not linear so it kind of
3:37
matters when you know when these returns happen um so if you get bad returns up front it's much it's much worse than
3:43
having bad returns later and so probability of success is how I learned how to describe it and um as I've grown
3:50
my firm it's all my advisors learned how to describe it because that's what I did
3:56
and over the years I I because I wasn't in the room and I've come in back and forth I've I saw the anxiety from a
4:03
different point of view and because what I would do is I would drop into some of these some of these higher Network meetings and and the score would drop so
4:10
this is this is where it started to happen I think um the financial crisis was the first time although it was so chaotic back
4:17
then that it was it wasn't as I would say everyone was in bad shape you know for a period of time so everyone got it
4:24
but there was other periods of time when people's scores would drop for various reasons and they would get really
4:29
freaked out and and the software we used before a ways back was saying it said green if
4:36
you were above 85 percent okay so 85 percent was good below that was yellow
4:41
then red So Below 80 if we're below 85 we're
4:46
fixing it right so so and and probability of success is that we've talked about it's it's really you know
4:52
we've changed the way we explain it now but back then people didn't know okay if
4:58
I don't succeed am I out on the streets you know and and it's this mathematical you know game to get back into
5:04
profitable probability success which is as we know now leading people kind of short you know on their life experiences
5:10
so I think from the advisor's point of view is that you know advice knowing it
5:16
from a client they don't know what a pass or fail looks like they don't know what it means so we've changed that to
5:21
now we've kind of moved along to the probability of adjustment the soft the softwares don't they don't
5:27
it doesn't say that but that's what we describe it as the probability of making a change and I think the issue becomes
5:33
that if we buy into the the nomenclature change okay sounds better
5:39
from a client's point of view well what does that mean and what that really means from an advisor's point of view is that
5:45
we need to then go back behind the scenes and answer those questions because the client's going to ask well what does that mean if I need to make an
5:51
adjustment so we have to create all this math behind the scenes to answer a question that might come at us and that's going to change for every meeting
5:58
so it's just a lot of work and and to be honest when you and you have to explain well what is Monte Carlo analysis I mean
6:04
how I mean how does that go for you right so you're getting into this deep dive about two standard deviations and
6:10
it's just become it's become more stressful and I think from my point of view what I what I've learned is when
6:16
you're deep in it and working in it it just seems totally fine to do it it's much different when you can remove yourself and then watch an advisor do it
6:23
and see the client's face and realize there's got to be a better way it's been 20 years there's got to be a better way
6:30
I think that's an interesting point too about um you know how an advisor can get deep in the analytics and then feel like they
6:37
actually need to um you know explain the methodology and they would be like a doctor explaining
6:42
how eegs work or something to you like that probably isn't the the right kind
6:47
of level of communication Jamie I saw you nodding a bunch during uh what Jeff was saying there what are your your
6:54
thoughts on kind of the goods and bads of probability of success yeah and uh I was hoping we'll we'll get
Failure
7:01
to this probably a little bit later uh which is you know I I have three point set I'll
7:07
start with the first one which is like the the whole idea of success and failure though makes it like you know a
7:14
one or a zero game which is just not the world right so it frames people up as if
7:20
you know to just point that if it's 84 you have failed but like 84 is like
7:25
passing in most there is the world like you become a doctor with an 84 right like it's actually pretty good
7:31
um but that's a failure and you know it's also just not life right like people don't fail if they fall a dollar
7:37
short at age 100 but the model would suggest right that that is a failure but
7:42
if you really dove into that you'd be like we've got a great plan for you but you fall one dollar short at the end and
7:49
you know you might die a dollar in debt I'd be like that's that's a pretty good for
7:55
good for magnitude accessor failure which I know you know David's worked on before and we used to teach a lot that
8:00
isn't usually captured in a pure Monte Carlo or success or failure one's always an issue right because even in if it
8:08
succeeds I used to tell this one when I'd go meet with clients was you know if you you could have a success or failure
8:14
model where it showed you 90 success but in the 10 that you failed you ran out of
8:19
money in year two of retirement right or you could have one that said 70
8:24
30 but all the 30 was after year 100 right well that was actually a way safer
8:30
plan even though that probability of success or failure does not paint that picture so like that part you know and I
8:36
think the the one the last point of just part is that you then have to get into explaining a lot of that which is
8:42
probably wasted energy and getting clients focused on all the wrong things so you know I think a lot of what we've
8:47
done at Carson is we moved to like a timeline based approach on a lot of our stuff which is you know we're more
8:53
focused on like events that we've done in the past and things you want to achieve in the future and less away from
8:58
that you know pure hey here's where the the money's going to ebb and flow based off of Monte Carlo we still have it but
9:05
we focus a lot less on that in the client Communications than we did you know just like Jeff when Ron was back
9:12
like 1993 kick it off Carson wealth like that was a big part of everything and it's it's definitely shifted you know
9:18
recently even more so is it I mean basically if your
9:24
clients trust your process and your focus on their you know the things that matter to them
9:29
you don't you don't have to focus on the analytics and things like that David I know you've probably been uh as
9:37
Jamie uh hinted probably one of the one of the uh most recognized researchers in
9:43
this space um what's your take uh are there places probability success still works
9:50
um should we get rid of it in retirement completely yes um so I I think I want to I want to
Monte Carlo
9:57
make a point probably like every time I talk that both you and Jeff of me is that is that Monte Carlo doesn't have to
10:03
do anything with with success rates right I see all these people like like oh Monte Carlo you have to use historical returns or like Monica no no
10:10
like Monte Carlo is like infinite like you could do anything there like I think that door a is like this pure
10:15
deterministic projection where stocks go up seven percent a year for like 70 years and and door B is like everything
10:22
else like multiple runs I mean we need to find it differently but like Monte Carlo is just like hey no one knows
10:28
what's going to happen so I think like the first thing is to divorce yourself from this idea that you know like Monte Carlo has to do anything a lot of the
10:34
tools do things that I think are are not great but that's not Monte Carlo right and so I think that like like the thing
10:39
about success rate that like really bothers me is that it only focuses on the marginal role of savings when it
10:45
comes to accomplishing a goal so your goal is like to have a hundred thousand dollars a year and you get like 99 000 a
10:50
year from Social Security and pensions you could have a zero percent success rate right it is solely the marginal
10:56
role of savings when it comes to finding a goal who doesn't capture that kind of like distribution of outcomes right and
11:02
then so I think that there's better ways to do it like as Jamie mentioned it's a binary objective function it's ones and zeros life is not ones and zeros right
11:08
and then like to me like I I actually increasingly don't think we should tell our clients numbers at all
11:14
um maybe it's like bunnies or rainbows or like stop signs I don't know like I'm not like a I'm not like a pictures guy
11:20
but like but I think I think we could all agree like this is like a 50 or 60 or 70 year projection like it's going to
11:26
be so wrong it's absurd and so like let's not freak people out with like a you have a you know an 82.7 successful
11:33
like who who cares I think I think you know telling them like you know you know Jeff to your point like if you're in the green you're in great shape right now
11:40
you're tracking well let's come back next and we'll talk about it I think that you know like I loved I love to quantify things but I think we can make
11:47
the ways that we quantify outcomes more complicated behind the scenes to make them simpler to clients so that they
11:53
don't like freak out with all the number stuff because that's what number guys like to do
12:00
makes sense and I think you mentioned there kind of a couple of
12:05
well he's gestured towards some some Alternatives um to this kind of foregrounding of
12:10
probability of success one thing I I think that was a really interesting point about how you're really it's a
12:16
score but it's a score based on your goal being having money left in your savings at the
12:23
end of this arbitrary time period right which is not actually the goal of retirement right
12:30
there's a lot potentially lots and lots of goals right one the main one being Thunder standard of living
12:36
um anyway so maybe we'll go back around the horn what has alternatives to foregrounding or you know maybe you're
12:42
getting rid of it completely what are other ways of talking about retirement plans their their level of
12:50
risk because that's really kind of what you're trying to get at right it's like is this a risky plan is it not are you you know already built in suspenders so
12:57
Jeff what do you tell your advisors or how do you coach them on how to talk about things yeah so so it's a lot of
Good Advisors
13:04
good advisors in the industry here we steal our best ideas from other people um so I happen to admire and be friends
13:11
with Carl Richards and I think he has an excellent way of looking at this um we view ourselves as guides and the
13:19
clients come along with us and and good guides need to know first and foremost we need to know who we're guiding right
13:25
we need to know the clients secondly we need to know the landscape that we're dealing with here
13:31
um so take it the retirement and taxes and Investments and all that we need to know the landscape very well and and
13:38
third we need to get we need to have a backpack full of cool stuff that we can break out when things go straight when
13:43
things go awry right and that could be soft different types of software tools because different tools are going to get
13:49
the job done differently Monte Carlo might be excellent for accumulation of
13:54
assets because it's almost like a wealth maximizer tool right so it helps you looking at that perspective do I pay off
13:59
the mortgage or not when I'm 40 years old it's going to give you pretty radical differences sometimes depending upon interest rates so that might be
14:06
useful information when you're growing and accumulating your assets when you're in distribution or close to distribution
14:11
phase you're going to need a different tool like income Labs tool or other tools that can help you spend the money
14:16
better so um and I think when the way we explain it is look we're going to work you want to take a route and let's just
14:23
say we're climbing Everest right that's a pretty aggressive route to take we're going to take the the fastest way up
14:28
think about the most highest return way up or for a client that may not be the highest return it's spending as much
14:33
money as you possibly can given the resources that you have therefore on the Monte Carlo score it would be a lower
14:40
acceptable score maybe 70 is the right number you know you're taking that aggressive route and we would be as
14:46
advisors helping you determine when we need to not take that route when we need
14:52
to take a different route that's safer for a period of time before we go back to the original route
14:57
explaining it that way to clients I think helps them understand it better and the context I mean remember if we're
15:03
using Monte Carlo as our only tool that we have we have to make sense of the number somehow so we make it we make it
15:09
we explain it that way that it's a risk-based number you know you're getting 95 probability
15:15
you're going to take your own sweet time to get there there's probably very little chance of failure whatsoever but
15:20
it's going to take you a long time you're going to leave stuff on the table you may not experience it as well so that's how we try to explain it to our
15:26
clients and removing like David said Stoke soak so well let's take the number out of it making it not the main point
15:33
of conversation makes sense and like you said last year
15:38
it sounds like you have these situations where you know somebody had an A or an A minus right they're in the 90s and then
15:45
through no fault of anybody maybe that score went down to 60 or something by removing the score completely you're not
15:51
you're just saying hey we're your guide maybe there wasn't evasive action needed
15:57
but for some people maybe it was hey let's take this deal or let's go let's get ready to take it right let's let's
16:02
just get ready to we might get ready to take it or we might need to know like what kind of tool we need to pull out of
16:07
either your backpack or ours maybe hey you're my client Justin and you don't want to change something just yet I
16:13
think that's the problem is that some people don't they didn't know ahead of time what the change might look like that's the most
16:20
stressful situation they don't know okay if something goes wrong what do I do well we're not I mean it changes all the
16:25
time if you're using Monte Carlo they don't have a defined change to make so it's really stressful to say Justin you got to cut your spending by a thousand
16:30
dollars a month starting tomorrow to get you back on track oh I wasn't quite ready for that so we might have to wait
16:36
so I think knowing some of those things ahead of time is Key but if you're just using Monte Carlo I think just explaining look get ready if it stays
16:43
like this we're going to need to make some changes and here's what they might need to be Jamie you mentioned that Carson you're
16:51
one of one of your strategies is to kind of focus more on these milestones and things like that are there other tools
16:57
alternatives to using this score have you do you generally coach people to just jettison the score entirely
17:03
um what what do you do no so I'd say we've very much inside of the carcin
Carcin Ecosystem
17:08
ecosystem we've kind of adopted three different approaches however you can use all three at once it's just I would say
17:15
most people don't right like you just pick one and you stick to it so if you think about uh like a Monte Carlo
17:21
probability based one and some of the traditional planning softwares we have a lot of advisors that still just stick to
17:27
that it's the way they grew up is the way they learned and they follow through with it and they feel fine with it I
17:32
don't know though that it you know they would even agree it's the best way to do it but it's the way they learned and it
17:38
works well enough to serve their clients so that's obviously one uh we have uh like two proprietary you know CX
17:45
experiences so one is the like lack of a better term the bucketing approach which is more of a needs wants and wishes you
17:52
know liability or time Horizon matching approach and so that one gets you very much out of this you don't talk at all
17:59
about success or failure or any of that there you're really just saying hey here's the allocations or spending
18:04
coming out of different time Horizons and where that money is going to come from it gets a very different
18:09
conversation very different focus and to David and just one like you can kind of get away from performance and returns a
18:17
little bit more there and get more into like visualizing spending versus you
18:22
know assets that you have or income sources which is I think a more probably more fun conversation then the third one
18:28
is more of that uh a goal or you know time period Focus but that one we don't
18:35
usually show in a like bucketing approach it's more here are these time Horizons and things you want to
18:40
accomplish on along the way but we don't really like we at one point thought about like assigning percentages to
18:48
those and we actually there was some beta version of that that existed that got totally scrapped so it was like
18:53
here's 75 of the way to funding your kids college but the problem was that like those things didn't necessarily
18:59
like equal each other so you might look behind on something then visually but like if not really the important goal
19:06
like if you've only hit 75 of College funding for your kids that actually might be a great success and not look
19:11
but like you're kind of painting it then like well you didn't hit that one so you failed so we actually took that away at
19:17
some point and said like it's not the best way to Showcase it and then you know when people were talking earlier
19:23
one of the interesting things about the successor failure version is I think the one of the main reasons it popped up
19:30
though is the main fear for retirees is that running out of money so that was like the way we addressed that as an
19:36
industry was like well let's just show them that fear-based answer and it like that was kind of the solution to some
19:43
degree right it's not the only reason but if you think about that that's generally the number one stated fear and
19:48
then here is some projection around that top fears that I somehow run out of money but that's not actually people's
19:54
goals like people's top goal is not run out of money it is actually to do something in retirement like five other
20:00
things and then not run out of money
20:06
that's a great kind of like what David was saying the score then is actually
20:11
it's a score about your fear it's not a score about your goal I mean that's a really great great way to put it
20:17
um David you kind of already answered the question about what the alternatives are I think you said uh rainbows or
20:23
frogs or something like that um I need to actually recommend frogs but I like that I like green eggs I think we're moving the same direction
20:31
um but I know I we've we've talked before I know we have very similar thoughts about what you know retirement
20:37
experiences are really like I mean you said it's not ones and zeros um I think you even you talk the same
20:43
way that that we often do at income lab about you know it's adjustments it's giving good advice on when that's prudent and the size and and so on
20:52
um I wonder if you could talk a little bit more about kind of what the experience of retirement
20:57
really is like what um you know what risk is like like how how we should talk
21:04
about it with clients because I do think this running out of money thing it can be
21:10
um it seems so so drastic and scary and catastrophic right that it's like it's
21:17
the shiny object that I think we've kind of we've taught people to pay attention to like how how can we do better there
21:24
what's the reality I mean you know I'm sure Jeff sees a lot of this than I do but like if you're driving towards a cliff you're probably
The Problem with Probability of Success
21:30
going to slow down and not drive off of it right like shocker right absolute shocker but like the thing is that you
21:35
know in in almost every tool it exists right you just you just keep driving off that cliff and you just and you just
21:41
explode right and that and that's the outcome and I think the problem with that is it's it's obvious and advisors
21:46
will say you know like you know so I know that income lab does dynamic adjustments they say well I said well
21:51
that's what I do I'm going in every year and I'm gonna I'm gonna tell them what to take out and I'm like I
21:56
congratulations that's what you should be doing but unless you consider it as part of your plan you're not going to give someone the right advice today like
22:03
incorporating the idea that you will make changes in the future just even basic changes is is pretty significant
22:09
like like I I'm trying to be nicer online and you know I see these posts about like I ran a Monte Carlo and I
22:15
have I'm gonna have there's a one percent chance a lot of 500 million dollars in 50 years and I'm like no your model stinks like because if you
22:21
actually experience these like fantastic returns in real life you're going to spend more money right and incorporating
22:27
that kind of like that that response people make to good and bad outcomes I think can just radically change this
22:32
definition of what they should be saving how much they should be spending kind of all you know that's interesting that even
22:40
though it's called probability of success It's really because it's focused on the fear
22:45
um it's all focused on the downside which I understand like we want to understand how to avoid the bad stuff
22:50
the scary stuff but what you just mentioned is there's this upside
22:57
um right like Jeff how do you deal with situations where actually there's good news and you know and and avoiding the
23:06
failure of someone who wants their last check to bounce but actually ends up with 50 million dollars just I'll tell
The Problem with Dynamic Spending
23:11
you that that is the hardest part of our job like the to get people to spend more
23:17
or feel more comfortable about doing that um despite the fact we get deep into
23:22
people's intentions about what's important to them and it's it's strange they get stuck on
23:28
spending a certain amount of money we can show them the score we could show them how much they can spend it's really
23:33
hard to get people okay we've figured it out let's say we figure it out in our onboarding process and they could spend
23:39
a hundred thousand dollars a year and let's just say that you know markets rallied let's just say that was at the bottom last year and markets have rally
23:44
back and they could spend if you used your software you could show that they'd be spending you know able to spend more you know five ten percent more and they
23:51
want to but they just don't feel comfortable doing that and I think that as as David said so well
23:59
by not having that Dynamic spending as part of the plan at the beginning it makes it really hard to adjust it in
24:05
either direction as time goes on because they didn't feel like that was part of the plan so they kind of feel like oh
24:12
we're just going to get our inflation and oh I don't know about that right things were down they came back people don't you know obviously we know that
24:18
markets go up over long periods of time right all of that um and and so having been in this practice for for so long
24:26
hey we've never had anybody run out of money right we've never had anyone explode off the cliff as David said we've had people like have to change and
24:34
that isn't the result of the markets actually that was a result of them overspending relative to what they said they were going to usually significantly
24:41
so okay and not listening to advice on that or B something radical happening in
24:47
their personal situation which hasn't happened so often but there was always a plan B for those folks
24:53
we've seen far more people um you know because we there you know we worked with people when I was younger
24:59
and they were older pass away with far too much money relative to their plans relative to what they say they wanted
25:06
um and and honestly in many cases it didn't it wasn't that beneficial to the overall family because the kids were
25:13
already successful it's in fact the parents should have either given it away which we advised or spent it which we
25:18
also advised but it was so hard for them to do that because we didn't have Dynamic income planning back then when
25:25
we started and I think having it up front is the key you know Derek darpan and Michael kitsus
25:30
did I don't think they've published this yet unfortunately but they did a survey study where they showed people
25:37
probability of success you know pass fail type Framing and then adjustment framing up front so I think that's your
25:43
point is like if you set the expectation that the the journey is going to look like this we've got a steering wheel I'm
25:49
going to tell you to change and there will be changes um versus you know here go do this have a nice
25:58
life um people far um preferred the adjustment based
26:03
Framing and when it came time for the advisor to tell them to change which as I forget who it was said most advisors
26:09
probably are will be doing that anyway um the advisor was perceived as much more
26:15
credible because they'd already said that's what we're going to do as opposed to probability of success kind of
26:20
implies that there's no more work to be done like here's your final exam score uh enjoy
26:26
um David I know you've done um hey Justin if you don't mind I got one more analogy I don't mean to
Analogy Google Maps and Waze
26:32
interrupt but but so if you if you like we we're all so accustomed to like Google Maps or Waze or what have you and
26:38
and we're so adjust so if you say I'm going from point A to point B and it tells you to get off the freeway
26:45
when you're supposed to be on the freeway because there's a thing ahead we're so used to using it we're getting off the freeway we're trusting this
26:51
device to do it okay if we didn't if we never used that before it was the first time we're not getting off the freeway
26:57
we're going to get stuck in that 30 minute Jam because we're so used to doing it a certain way so I think it's as you said I think it's a good analogy
27:03
to to give clients is look this is this works it gets us there on time it gets us there faster and it's just getting
27:09
training people on how to use it correctly that's right David I I think you've done you've uh
27:16
published some things recently talking about this adjustment concept and specifically on kind of evidence of of
27:23
retirees willingness to capacity to 10 mc2 adjust spending and retirement
27:29
what's the what's the evidence tell us on that that people can't adjust their spending I mean it's an absolute shocker
People cant adjust their spending
27:35
right people don't have to spend that same amount you know I'm Jeff I'm sure all your clients call you up on January 1st to say hey the CPI went up 3.7 last
27:42
year I'm going to spend three I mean like I mean I get it like 30 years ago like these these model this modeling was
27:47
pre-revolutionary the problem is is that like our tools haven't caught up with what we can do I think Jeff like your
27:53
analogy about like like Waze and Google Maps is really on point and like and so like the point that I make what I said earlier is that like when you when you
27:59
when you acknowledge the fact that you can get off the freeway for example then you can maybe leave a little bit later
28:05
right it should be like there's questions already coming in like do you like do you use it for accumulation or retirement well you know like whatever
28:10
you tell that person they're gonna spend in that first year of retirement is going to become an anchor for them to some extent and so when you when you
28:16
when you when you overlay Dynamic decisions into that you start them off and maybe a very different place than if
28:22
you just use a static methodology like that also attaches to saving so to me like there's like there are like like
28:28
True full life cycle considerations here that if you're using stuff models you're gonna you're gonna you're gonna
28:33
universally result in spending too little over time
28:39
yeah it's actually interesting if you it if you still use Monte Carlo models
28:45
um and I think when they were talking a minute about you know some of your uh comments David on how to improve those but
28:52
um if you didn't view it as success and failure instead if you viewed it on hey here's a range of things that could happen to you
28:57
um and each of those has an a spending level of associated with it if I asked
29:03
you well what's the what's the best guess you'd pick the middle one you'd pick the
29:08
median right if it was like Vegas go put something you're gonna you're trying to get the best odds right well it's the one where half the time it's higher half
29:14
the time is lower now that's probably not the way that most people would prefer to to do their plan where you
29:21
know loss of hers risk-averse and and so most people probably start lower than that but it really once you jettison
29:27
this idea of success and failure you just view it as well here's a range of things that that might happen you know
29:32
if it's if it's a good enough model um Jamie on that
29:39
um are there any any reaction to to uh you know Jeff and and David there yeah I'll
Saving vs Spending
29:47
time to just two themes that you know I really like throughout that which was uh
29:52
one of the ones you brought up and Jeff hit on it too which is actually how you present something and I guess to David's
29:58
point the saving side like in the front end of client Communications it actually becomes really hard if you spend 30
30:05
years with a client to then shift them to a different approach right because it's like anything else in the world
30:11
like we train somebody to understand something and that was a you know I I kind of came to that realization back
30:17
when I was teaching at American college that like all of the world tells you about how to save and that spending is
30:23
bad right like spending is essentially demonized in the retirement plan World
30:28
until you get to retirement then we're just like hey I know we told you to not spend for 40 years but we're just
30:35
kidding like you have to now spend like it doesn't matter like best of luck right and that's a really big Challenge
30:43
and if you start thinking about like why is it so hard for a lot of Americans if
30:48
you look at the data to spend down as we told them it was a bad thing so they want to live off the interest the growth
30:55
it's why they adjust to it so one of the things a couple years ago I started talking about was to get better at the
31:02
spending right near retirement so and I think Justin you and I have talked about this one before but like I actually
31:08
encourage a lot of people to stop saving the last two or three years before retirement and to actually spend that
31:13
money because it actually has almost no impact on like their portfolio longevity like and it's interesting because the
31:21
secure 2.0 doubled down on that we said save even more the three years before you retire and it's like it actually it
31:27
has like almost no one like an extra three grand or five grand a year for three years it's like so meaningless
31:33
like it just doesn't move the needle and what I told people is like go buy a new car like go on a vacation any of those
31:39
things and actually if you do that and spend more and stay in the workplace three months six months longer you have
31:46
vastly improved your outcome and probably just enjoyed life a little bit more
31:51
and for a while I was thinking I don't know if this message resonates with anyone at least have one person now that
31:56
has told me when they that I talked to him about that they did go on vacations and I think that was in 2018 and they
32:03
were playing or retiring the next year and they are still working right now so they have now stayed in the workplace for like four more years and he's like
32:09
we go on really nice vacations now and you know I've been able to stay employed and it has brought a much higher quality
32:16
of life because he's like I look forward to that now and I stopped all my 401k contributions like I'm done with them
32:21
and you know but he's like otherwise you wouldn't have given me permission to do that and I think the interesting part is
32:27
what are other ways we can encourage the spending Behavior pre-retirement it's like testing anything right like we all
32:34
tested ways once and said did it actually work before we decided to always test it right and you know how do
32:41
we test spending and is it part-time I've even tried to Think Through is there a world in which people
32:48
you know it'd be more of a policy thing but let's just say you're saving 25 Grand a year into your retirement
32:54
account do you take distributions of 25k each year too and just like start to get
32:59
used to what a withdrawal is and so you're not really adding new money but you're putting money in and you're
33:05
taking the same amount out so you're just tax wise and everything kind of breaking even but you start getting clients used to this withdrawal notion
33:12
of pulling money out of an account and just you know maybe start that three years before retirement so I think
33:17
there's things like that I don't know that there's a best practice there yet but I find those are interesting ways to
33:23
how do you just start that conversation get people used to it because I see we've got like three questions about that in there too
33:30
I I really like that idea um about kind of it's like uh semi-retirement or something like that
33:37
Jeff what what are your thoughts I think I mean Jamie that is I wrote that down that is that is one of the coolest ideas
How do you get people to shift
33:43
that I've heard in a long time that's that's new um I think that yeah training people to
33:48
do that is is huge and um I I you know I kind of just kind of maybe diving into a
33:53
couple questions that could relate a little bit as Jamie was mentioning here so uh one of the things is like how do
33:59
you get people how do you get people to shift like unless you say hey you've been using this for so long and and I got this new this new thing
34:05
like how do you know it works and we actually I work with a lot of Engineers either fortunately or unfortunately um
34:11
depending upon how you look at it so they can they get deep into it and you've actually done a call I think with one or two of them Justin in terms of
34:17
kind of delving into the mechanics behind how it all works because they don't wait it shows me I could spend more is it is it true is it right I
34:24
think one thing for advisors out there because there's a few of them that's worked really well for us especially if you're structured in a team where you
34:30
have like a lead advisor and an associate advisor we've actually trained our younger associate advisors on the
34:35
software they're younger they know Tech pretty well and they come in and explain
34:41
it and so a the lead advisor doesn't necessarily need to know it as well the associate does it's this person that
34:47
actually instantly builds credibility with something new and you think about like how the how it works like I trust
34:52
my my teenage son on my phone far more than I trust my wife trying to figure something out on it right so it
34:58
definitely it adds that credibility oh this is new this is exciting and so you get the double whammy of sort of upping
35:03
the game for the associate advisor and then having a good way to introduce it to to the firm that's worked really well
35:10
and I think also another thing that in terms of getting another question is like hey do I when do you use a software
35:17
program like Dynamic spending versus traditional Monte Carlo like how do you how do you draw the line I don't know
35:22
that there's a best practice either I'd be interested to see if anybody else has this idea but as I mentioned earlier I think
35:28
Dynamic spending works really well a if you're in if you're in distribution
35:34
obviously right so if you're taking money out B we call at least call I forget we were at um I started at
35:40
Prudential and they called it the retirement Red Zone right so within five years away from retirement Davis so uh
35:46
yes the little take to you there so so within five years of that is when you should start that and if you could
35:51
combine that with what Jamie just said oh my gosh like how great would that be right getting people used to spending a little bit more within that Red Zone
35:57
with some new tactics to do and I think the third thing that actually I personally looked at is if you're going
36:02
to make a big decision with your money like we looked at taking on a Capital Partner how does someone take a
36:08
controlling interest buying a piece of our firm it's a big difference for me right it's my in my profitability goes down but I have a bunch of money to
36:14
invest how does that change my spending can I still live My Lifestyle that's a big decision and this and and income lab
36:20
actually helped me make that decision around wow that's a big difference this is really cool I can I can do it whether
36:26
it's stock options selling a business so for me those three things are excellent places to use Dynamic spending like like
36:32
income lab Dynamic planning around that and for younger people like my lead advisor Colin who's in his mid-20s it's
36:40
got a long Horizon you know what Monte Carlo works pretty well for him he doesn't even know what retirement looks like at this point in time it's more
36:46
around wealth maximization do I save this here do I put it in a mortgage do I put it in a 529 see the long-term
36:51
impacts of that I think that's where you kind of draw the line for it
36:57
I like a lot of that and I I hopefully there'll be maybe maybe more and more ideas about how to make that shift in
37:05
mindset um you know along with Jamie's idea about starting to spend even when you're still working
37:11
um we actually we we had a webinar panel webinar with a bunch of advisors who did talk about that transition
37:18
um so I can try to we can try to find the recording of that and maybe put that out
37:23
um to the attendees here I think the main response was really it wasn't a big deal like everybody just took to it like
37:29
a duck to water um and in terms of when to do it I actually think the GPS uh analogy is a
37:35
pretty good one like when would you put something in Google Maps
37:40
like I mean we live in Colorado I might actually do it a couple hours before we
37:45
go skiing right just see hey what's the traffic looking like should we wait should we go now so it's not that it's when you get in the car but I wouldn't
37:52
do it a week before who cares right that's that's not the traffic I'm going to encounter so I I do think actually
37:58
the analogy goes pretty far they all break down eventually but that one goes pretty far um David any thoughts on uh this you
38:04
know when is the shift um is probability of success still useful in kind of early
38:09
accumulation years I mean it can be I just I mean we're always making choices right you're
38:15
making choices about you know you go on the vacation you buy the new car and so like like no one likes to save money
38:21
like it isn't like a life goal to saving your 401k we do it because we want to replace our you know our money when we
38:26
retired so I I like the idea of kind of framing something forever just because it can't have important implications
38:31
around like how much you have to save now you know like here's the thing though so if you want to use probability okay like the like the one place that I
38:39
like that I I think it might be okay is is if you reframe it from success to
38:44
like if you can do this in your tools every tool is different but like in the worst one in ten runs you will have 45
38:53
000 in income at age 95. that's the only time you know if you want to use a probabilistic lens because if you can't
38:58
do Dynamic you know things get weird telling someone of the worst one in five one in ten whatever but give them an
39:04
income number and even then it's not Dynamic I get it but that's better than a success rate because success rates
39:09
give you no context for when you're broke what the income is going to be so if you want to use you know for someone
39:14
out there that's been like drinking the the success rate Kool-Aid now for 20 years you know and you want to like a
39:20
like a temporary shift I would say okay just pivot towards telling them you know the worst one in ten runs you're gonna
39:26
have eighty thousand dollars here at 95 and see how they respond to that versus a success rate
39:31
I also think like what was the shift like um for those of you who who are doing planning at the time from you know
39:39
this to Monte Carlo like my guess is it wasn't you didn't really wonder are my
39:46
are my clients going to worry that I go to something more sophisticated and realistic right
39:53
um David before we go to questions um I I did wanna you you kind of mentioned earlier that
39:58
there are some places that you do worry about just the the analytics themselves you know are there
40:05
um like you said you can do anything you want with with uh with Monte Carlo with simulation analysis there you can get a
40:11
sophisticated unsophisticated as you want are there in the in the actual practice of 99 of people are there
40:20
worries that you have and things that you would do because of those worries um in doing retirement planning
Monte Carlo simulation
40:26
so I guess I'm not quite sure I follow so like in in how Monte Carlo simulations are actually done today well
40:33
I mean so again I want to I think we need to repeat I I wanted to say it five times earlier right so I'm going to say it again you know Monte Carlo can do
40:39
anything like so for those of you that think about it like you know Divorce Yourself from what the calculators do
40:44
right now like if you have a crappy calculator like that's kind of on you not on calculators in general don't hate
40:49
calculators because there's a Super Bass okay so Monte Carlo right is this like infinite thing right I think that there
40:55
are like all these things that I I like to see we move to as an industry like moving away towards static withdrawals
41:01
moving towards single goals and then moving towards better outcomes metrics I mean you know like I don't want to get
41:06
too wonky but like like I think that that you know like this idea of a single goal can really lead to bad advice and
41:12
guidance right if you think about like Jamie mentioned needs wants and wishes if you can kind of quantify the like the
41:17
paying of each of those and give someone guidance like I think that you know if you wrap that together like I've just
41:23
seen that you can get materially different recommendations around everything savings asset allocation
41:28
spending annuities all that when you use more realistic models so I just think our tools need to evolve because they
41:34
haven't evolved in the last like 30 years which is kind of sad if you think about how like our your phone like your calculator your phone today is a lot
41:41
cooler than your phone five years ago most of the key assumptions and all of our tools are identical what they were
41:46
20 years ago quickly all right Jamie David Jamie
41:54
Jamie wants to go I will wanna have a funny story about a bad calculator so when I took my
41:59
investments class for the cfp exam I went to Walmart and bought a calculator for 199 or 198 I think it was 198. and
42:08
when I got there and I hit the two times eight and it showed me 24. I was like I
42:14
don't think I can rely on this one for the exam but I brought it in there it's the only thing I had with me so I did
42:19
that whole one I just like I was doing present value calculations on up that little whiteboard thing that you have to
42:25
erase hand and I was like this is awful [Laughter] uh but uh the other one is those uh when
42:33
when I was at the college with Wade we built a whole uh competency in ricp about this which was uh really it it was
42:40
less so about success or failure and Monte Carlo but about the outcome that we were testing and that's also another
42:46
really interesting thing is very like we actually test and show uh that about
42:52
very few outcomes so again most of those tools and calculators out there are using an analysis to project running out
42:58
of money but way it is also like look like some people want to know like what's the odds that I have an
43:04
increasing payment stream and like in how many scenarios do I you know go through retirement with increasing
43:10
income or decreasing income or average Legacy amount at the end right and those
43:16
actually are all different ways to look at it and for some people you might say look like you know I'd be pretty happy
43:21
if my income goes up in nine out of 10 times and I'm less concerned about the you know you know running out of money
43:28
but that's right like I'm super well funded I'm not going to run out of money but I want to know that I just keep generating more and more and I can keep
43:35
giving away more and that's a totally different approach but we we have not built very many tools to show those
43:41
other versions of what an outcome might be which I think is really interesting right like we just don't have good views
43:47
into a lot of that you can do it if you're smart and kind of rig it yourself but most of the general tools out there
43:55
are not showing you those other ones and I forget the name of that paper because it's been a while but I know Wade published that somewhere and I think he
44:02
tests like 17 different outcomes that he talked about and like how you would compare those through different
44:07
retirement income strategies too that like if you cared about like the consistency of spending right then you
44:14
annuitize more and if that's your goal right you you might run out of money in some cases in the sense of a true one
44:20
but you have this consistent spending all the way through so even though you've depleted your savings if your
44:27
goal was a consistent spending amount you did achieve that Jamie said the a word not me just to be
44:33
clear all right I think we'll uh we'll lifetime income sources
44:41
we get some uh uh questions going I think maybe some of the panelists have
44:46
been uh somehow also reading while talking which is truly impressive so if
44:51
there are any uh is there any questions that are jumping out at you otherwise I can I can pick one there's one with five
44:56
up votes so I feel like maybe we should hit that one um this is probably mostly for Jeff
45:03
um do you prefer to use income lab for retirees and another tool for pre-retirees not in income mode
45:12
yeah I think I hit on this so that's what I tried to hit on earlier is that I think the income lab income lab is really good because of the dynamic
45:17
spending in distribution in the Red Zone the retirement Red Zone which we classify as three to five years from
45:24
retirement we're making a big Capital decision that's really going to have an impact one way or the other
45:30
um and and I think that's important even if you're far away from this because ultimately if for time we didn't even
45:35
talk about like if retirement is your goal right so but but um you know whatever spending might be we assume
45:41
it's retirement but it could be giving it away or whatever so um just the ability to be able to use your savings
45:46
for whatever sources that they are I think it's really good to to use and and before that we currently and I I wish
45:53
I'm looking forward to David's software release where he's going to address those other ways of dealing with accumulation but um currently we are
46:00
using just a one of the Monte Carlo types of softwares where we can just show them because we look at that as
46:06
earlier you know more of like it's an in most cases it's like how do you maximize your assets in most cases especially
46:13
when you get farther away from it um yeah I'd say that's that's really how we do it
46:19
makes sense let's see here
46:26
so this uh this question we may have addressed a little bit but but maybe it's got two outputs so a couple of
46:32
other comments on it might be nice which is um basically saying that how do I
46:38
introduce a different concept or how do I move away from probability of success I don't
46:44
want to come across as discounting that strategy or methodology like do you have
46:49
a talk track for that just do it what what would you recommend
Confidence
46:57
I mean my vote is you probably just do it I also would probably say 99 of the time we over concern ourselves with what
47:04
clients are actually going to think about the particular thing we put in front of them and like a lot of times like if you do it with confidence they
47:11
don't really care I mean that's that's what I see more often than not and the the reason that I have that perspective
47:17
is we have you know since I've been at Carson we have transitioned about 150 firms about 400 advisors from some other
47:25
world some other investment philosophy other planning tool into our ecosystem and that the clients ultimately as long
47:33
as the advisor goes in and talks about the new one with confidence that like here's why we're doing this and gives a
47:38
little bit of that story and like clients are they're just appreciative that you're thinking about new things
47:44
and bringing things to them I mean that's part of the advisor's job is not to just like hey I do one thing and only
47:50
one thing forever that you are looking at the landscape you are seeing things that are coming because most
47:57
clients are going to be delegators and the fact is they don't want to look at the new things that are coming down the pipe and that's why they hired you the
48:03
do-it-yourselfers aren't your clients right like they're the ones that are reading Barons and things and grabbing
48:08
stuff and doing it themselves and testing stuff out so most of the time I think people are fine so I would you
48:14
know practice it think about it be be thoughtful I mean I like Jeff's idea of like if you have different teammates
48:19
that can show expertise in different ways so you don't have to pretend like well I'm now the new expert in this but
48:26
you have somebody else on the team that can come in um and you know the other fun thing is like income lab it's a new technology
48:33
like you don't you you can you don't have to sell them we didn't use this 10 years ago for some reason it didn't
48:39
exist 10 years ago like we couldn't have used it if we wanted to this is what we've all been waiting for
48:45
yeah and you're really you're not saying the old way was bad you're just saying now I have a better way right so it's just about getting getting better and
48:52
better um I could I could add to that a little bit is that um because I have had this
Analogy
48:57
question uh first off I'm gonna go back and I'm going to use my analogy from now on on the on the ways because I think that's a
49:03
good one like before we were using the Thomas guide and now we have Waze so we can now move to the better software that
49:09
because of computing power we just couldn't do this before and and also at some level the people that were doing
49:15
deep planning on the Monte Carlo they were trying to come up with hey if this happens like we have to create a bear
49:22
Market scenario we have to adjust your spending lower to get you back to a probability of success that we deem is
49:28
correct we were trying to build this into the plan anyways we were just having to manually Do It
49:34
um so we were this makes it a lot easier for us to stay on top of it on a regular basis and I think that's that shift has
49:41
been pretty easy for us to to overcome because most of our longer term clients that have been doing it we were doing those things anyways which was just
49:47
taking a lot of time and but we weren't setting the framework up up front
49:56
all right um probably have time for maybe one or two more
50:01
um so let me see here
50:09
all right well it's possible this would be a very quick answer but it is the most outvoted one I don't know this book
50:15
but if anyone has read die with zero Bill Perkins book
50:20
um comment I can read the whole thing if you want
50:26
here so I like his philosophy uh do you how do we think about spending at
50:32
our highest utility years with them saving for a long-term care event such
50:38
as a huge unknown but can happen which has a huge unknown but can have a significant expense
50:43
um actually that's that is a pretty interesting way to think about those like the years when you can spend your highest utility years
Spending
50:50
yeah it's a it's a decent book I've read it um it's pretty popular actually it's like uh I mean and that's the whole
50:55
point the philosophy is like getting the most out of your money in life so you spend and there's a lot of good
51:01
conversations about that or we talked about gambling and if you were to gamble and when you spend your money right like
51:06
you would spend it in the more likely years that you'd actually be around um so there's like there's kind of a
51:12
little bit of that too right like if I was really being smart I wouldn't take over a 30-year time period and put equal
51:19
dots of money over all 30 years right because actually my first five are super
51:24
likely that I'm there and then the next five the last five aren't all that likely that I'm there right like maybe a
51:30
10 chance so if you were thinking about spending in a like rational probability
51:36
approach there I would shift money to the more likely years to exist right and
51:42
that also talks about you can enjoy it more and it is a little bit of how people just live their lives anyway right this spending smile right type of
51:49
conversation Nation too Yeah we actually we actually do to this
51:55
um so uh and to be honest I mean the income lab allows you to do this pretty well so the way that we do this is we we
52:01
set it up we we build plans so that you have your your basic leaving needs that
52:07
that is an item we call it Baseline living expenses and that could be your you know your property taxes your
52:12
utility blah blah blah we all know what those are right and then you have then you have your discretionary expenses but
52:18
we do it by year so we have discretionary expenses for your life discretionary expenses for the you know
52:25
kind of the the the 20 years then we have discretionary expenses for ten and then five so it's a stair step we call
52:31
it stair stepping retirement so it actually accomplishes this type of goal because you you know what your your
52:36
basic is going to be we don't do it for everybody but for the people that want to kind of the YOLO live your you know
52:42
you only live once type of people this works really well for them um to allow them to spend a little more money up front when it matters the most
52:49
yeah I mean I think just back in the envelope stuff just using the retirement
52:54
smile um like exactly as you put it in I know there are different versions of it but like exactly from your article David
53:01
I mean just think of it as like it's like a swimming pool you get all your friends together and like push the water to one side it's the same amount of
53:07
water but it's it's going to be momentarily at least deeper in one side it gives you like close to 20 percent
53:13
more spending capacity just kind of for an average person obviously things vary
53:18
um and if you've got um things like Social Security an extra dollar of Social Security adds more than
53:25
a dollar of current spending in that situation so it's a pretty sweet way to like you said give people more when they
53:31
need it the most and Justin so like if you if you if you do this spending smile and you do
Enjoying your savings
53:36
Dynamic spending you're talking like 40 more potentially for someone based upon the structure their assets for someone
53:42
who's young retirement and I think the point is James Point that's when you can actually enjoy it like I mean yes like you want to have money there's a
53:48
question about like long-term care shocks but like like like you you know saving money is not fun like you want to
53:54
do stuff with it and I worry that a lot of the metrics and the plans for today or you know advisors talk about 99 success rates like that doesn't help
54:00
people actually enjoy their savings you're just plenty for that like absolute crazy workout
54:08
yeah I mean that's an interesting point I know you you actually mentioned that earlier but the idea that also just
54:13
blending with planning with adjustments in mind lets you spend more up front right it's like
54:19
um if you have a steering wheel you can get places right like if you you can take more risk right oh it's risky to
54:24
get on the road yeah but you got breaks you got a steering wheel like you'll be able to adjust we must riskier like to Pinewood Derby you know your car down
54:31
the road right um for those who are not in Boy Scouts that's it you just let the let the car
54:37
go um so yeah I think that's a really good point in terms of this you know utility maximizing
54:43
um concept all right um Mockley do you see like a short one
54:48
we could hit in in two minutes here
54:55
I guess we could we could hit this one again because I do think it's a it's an important thing which is just any other
55:01
ways to encourage clients to spend more any hacks on that
55:08
you can you can allocate to an annuity I'm just gonna throw that out there but like you know advisors help with this and I get that and annuities are it's a
55:15
it's a it's a vast interesting ocean but like when you're when you're faced with uncertain longevity and Walker returns
55:21
it's very difficult to spend money from a portfolio so like I just I I believe that individuals should have their needs
55:28
covered with a super high level of certainty you know it can be guaranteed doesn't mean annuity but I think you
55:33
know helping individuals understand no matter how long they lived on what happens they're covered makes it an easier conversation
Building a fixed income floor
55:39
yeah and I I second that a little with David and it's just the there was some research that shows that right when
55:45
people have a higher fixed income floor they're willing to spend more and whether it's annuitization pensions
55:51
Social Security deferral whatever one you decide to pick but building that floor and Jeff approaches that in a
55:58
different way it's taking care of the right essential expenses and making sure that that's off the table and then that
56:03
gives people permission to spend in the discretionary stuff too because uh the J that JP Morgan data though from them it
56:10
actually shows a bunch of people with plenty of money that could spend but they're they don't give themselves permission and then I will also say when
56:17
people overly self-insure um we didn't go a lot into long-term care but there is data that also shows
56:23
that people who quote unquote save their own assets for all long-term care and end-of-life expenses don't spend as much
56:29
because essentially you are you're trying to protect and insure against every future potential risk and
56:35
therefore you can't spend that money even though you're like I didn't lock it up with an insurance practice what will you do did you locked it up in your own
56:41
insurance bucket and it was actually you had no risk pooling so it was more expensive to actually do that
56:48
yeah and I agree with that it's I think on both of those cases right the more you can make people feel secure about
56:53
whatever income they have and whatever potential disaster could be down the road um I also think that uh you know by
57:00
having strategies to gift most people want to take care of their children well let's let's make some strategies around
57:06
that now right giving to your kids like hey we already banked it away whether it's an eyelid or something like that getting it out of here say they're done
57:12
they're good you can spend the rest and I think two other things do this one is I think stories make a big difference
57:17
like what you know David said I like the story you know you we have never had a client go flying off the cliff and crash
57:25
right we've had people come close we've made adjustments we've had far far more people that have that have died with
57:31
more assets than they need and you know honestly your software has helped our clients spend more money because in many
57:40
cases they can and they see what they need to do to get back and now they have a plan to get back on track and I think
57:45
that's been a there's been a bit of a paradigm shift for some people that we've transitioned over
57:52
right seems like a great place to stop um again want to thank our panel this
57:58
has been awesome I feel like we could talk for uh another hour about this stuff
58:04
um I uh really thank you for taking the time um I know you're all super busy and uh
58:09
thank you to uh everybody who attended knockly got any uh taking us out here
58:16
yeah thanks everyone uh for our attendees please fill out the survey afterwards we'd love to hear your
58:22
feedback and then let us know how we can connect with you going forward and to our panelists thank you guys again this
58:27
is awesome take care take care
Â
Â