True to his word, in 1848 Polk reiterated his intention to retire at the end of his single term, although he could easily have been nominated for a second term. He confided in his diary that he felt "exceedingly relieved" to be free from public duty. Unfortunately, he was able to enjoy less than three months of retirement, the least of any former President. After handing over the reins of government to his successor, Zachary Taylor, the Whig candidate whom he had correctly suspected of planning to use his Mexican War heroism to seek the presidency, Polk embarked on an extensive tour of the southern states. His trip took him from the Atlantic seaboard, west along the Gulf states, and up the Mississippi River to Tennessee. Everywhere he traveled, the crowds were large and festive, and he felt overjoyed with the proclamations of affection and thanks. At the end of the trip, he moved into his recently purchased estate in Nashville, which the former President named "Polk Place." He spent his final weeks there remodeling the estate and sorting through his presidential papers.

Seriously ill during the last days of his tour, possibly from cholera that had broken out in New Orleans while he was there, Polk cut short his trip because of fatigue and bouts of diarrhea. At first he thought that it was just more of the same symptoms that had plagued him throughout his life, but his weakness grew progressively worse. He died on June 15, 1849. Polk left most of his estate to his wife, with the request that she free their slaves upon her death. In 1893, Polk's body was transferred from the Polk Place cemetery to a tomb at the state capitol in Nashville.