James Patrick McGranery was born on July 8, 1895, in Philadelphia, Pennsylvania. He received a law degree from Temple University in 1928 and became active in Democratic politics in the Philadelphia area. McGranery was first elected to the U.S. House of Representatives in 1936, remaining there until October 1943 when he became an assistant to the attorney general. In 1946, McGranery became a U.S. district court judge in Pennsylvania.

He stayed in that post until April 1952, at which time President Harry S. Truman made McGranery his attorney general following the resignation of J. Howard McGrath. McGranery remained in the cabinet until 1953, when Dwight D. Eisenhower ascended to the presidency. McGranery died in December 1962 while vacationing in Florida.