Minimizing retirement taxes through orderly withdrawals

Larry Swedroe, Author of “Your Complete Guide to a Successful and Secure Retirement” and Chief Research Officer of Buckingham Wealth Partners, says the order in which retirement accounts are drawn out of can have a big impact on the taxes owed.

Video Transcript

[MUSIC PLAYING]

- All right. As part of our retirement segment brought to you by Fidelity Investments, we want to talk about the strategies and the tax implications of withdrawing from your retirement accounts. And we're going to do that with Larry Swedroe, the author of "Your Complete Guide to a Successful and Secure Retirement." Also the Chief Research Officer of Buckingham Wealth Partners.

Good to have you here on this New Year's Eve. Is it a little too late to be looking at tax planning, especially when it comes to retirement withdrawals on the last day of the year with essentially financial offices-- I mean, they've been at home and in bed for a week now-- but closing shop in 30 minutes?

LARRY SWEDROE: Yeah, I'd say you're probably past your bedtime [LAUGHS] on this one. Yeah.

- OK. But for 2022, a lot of people are going to be looking at what should be some pretty hefty gains this year in their retirement accounts if they're tied to equities. Looking forward, is it-- the issue of withdrawals and the tax implications really depend on where you are in the age spectrum, right?

LARRY SWEDROE: Well, the tax-efficient withdrawal is really an important part of the winning investment strategy, because you can extend the life of your portfolio by getting the asset location withdrawal right. And many people make mistakes here.

What you want to do, the general rule is first take your withdrawals from your taxable account. The second place in order would be is your tax advantaged accounts like a 401(k) or a traditional IRA. And then the third and last place you want to take it from is the tax-exempt accounts like a Roth, because you want that to continue to grow, either tax-deferred, or in the last case, tax-free as long as possible. And too many people make mistakes and don't follow this order. That's one of the things that I would point out.

But generally, the biggest mistake I see is that people try to minimize taxes in the current year instead of focusing on minimizing taxes over the lifetime. And sometimes it's better to pay some taxes now if your tax rate is low in order to save taxes in the future.

- I am so grateful for your response to that first question in which you talked about how to plan, because I've always wondered, where do you take first? As I'm staring at the off ramp in about 15 years, maybe less. Do you go to the Roth, 401(k)? Do you go to the traditional IRA? So thank you for explaining that.

But then-- and I realize this is a question probably better suited for my financial planner on my issues, but a lot of people wondering. When you are exiting, when you're on the off ramp and you're delaying Social Security, you're going to be forced to take withdrawals from IRA by the government. Does it work against you if you've been a good saver for 30-plus years? And do you get forced out of being able to do a Roth Conversion does it become cost-prohibitive at that point in the retirement cycle?

LARRY SWEDROE: Well, this is really part of a longer term planning strategy. What we often see-- I'll give you a good example-- is a lot of people are retiring early. Partly they've been able by very strong markets, as you've indicated. And if you're also a healthy person, likely to live long, the best strategy, because it buys you longevity insurance, is to delay Social Security as long as possible.

So you would enter what we refer to in my book as a blackout period where you have little or no income. Certainly no earned income. You're retired. You're delaying your Social Security.

And what you want to do then is withdraw from your taxable account, number one. But you also want to take advantage of the fact that you're in a low tax bracket now. And you want to fill up the lowest tax brackets, depending upon how much wealth you have.

But for higher net worth people, I would suggest that at least up to the 24% or 26% tax bracket, take money out of your traditional IRA and convert it into a Roth where you then get all of the growth tax-free so you're reducing your RMD, your Required Minimum Distribution later.

So that's kind of a good strategy. Live off of your taxable income. Take some money out of your traditional IRA or 401(k) as long as you're in the lower bracket. And you can also take advantage of if you're in a low tax bracket to, again, not minimize current taxes, to actually harvest gains and take them if you're in the lowest bracket as well. So there's another opportunity that many people miss.

- Larry, we appreciate that advice. A buddy of mine just did the off ramp, and he's not-- he's 58 years old, but he's been having the discussions you've been having with his financial planner. He's not touching anything, and he's too young at this point. But it really is about coordinating into the future.

Larry Swedroe is author of "Your Complete Guide to a Successful and Secure Retirement." And I just have to tell you, I'm envious of my buddy. 58 and he's already having-- he's on the golf course and on the off ramp. All the best to you, and a Happy New Year.