Franklin and Eleanor Roosevelt had five sons and a daughter, although one son died in infancy. FDR was not deeply involved in raising his children, in part because he was so occupied with his work. But he also believed that childrearing was his wife's (or the family nanny's) task. When FDR entered the White House in 1933, his oldest child, Anna, was in her late twenties and his youngest child, John, was in his late teens. Nonetheless, the children played important roles in their father's life while he was President, offering him emotional comfort, tending to the physical needs of a man withered by polio, and, in some cases, helping him execute his daily duties as chief executive of the United States government.

Anna moved into the White House in 1933 while going through a divorce from her first husband and stayed for more than a year. In 1943, she returned to the White House to serve as her father's confidential assistant. She accompanied him to the Yalta Conference in 1945 and was with him in Warm Springs Georgia in April 1945 when he suffered a fatal stroke. Anna's brother James (the second oldest child) oversaw his father's presidential campaign in Massachusetts in 1932 and later served as an assistant (and later secretary) to his father in 1937. James stayed at the White House for less than a year, departing because of the stress his job entailed and because FDR's political opponents had charged that he received his position through mere cronyism. During the war, James joined the Marines, while his brothers John and Franklin enlisted in the Navy. His other brother, Elliott, served in the Air Force.

FDR's life at the White House was consumed largely by the heavy duties of leading the United States through the Depression and the Second World War. But the President still found time to relax. He enjoyed collecting stamps, bird-watching, playing cards, or swimming in the pool he had built at the White House. FDR was an extremely out-going man who relished the company of others, and hosted a cocktail hour each day for those in his inner circle. He regularly entertained friends and acquaintances, as well as political allies and visiting dignitaries, at the White House.