James Paul [sometimes listed as Patrick] Sullivan was the third of six surviving children born to Irish immigrants Jeremiah Sullivan, a grocer, and his wife Elizabeth Flynn, who arrived in the United States in 1870 and 1875 respectively. Young J.P., or Jim, was born on February 28, 1885 and spent his early years in Manhattan where he attended P.S. 14 on East 27th Street and joined the track team of the St. Bartholomew Boys Club and, later, of the 22nd Regiment. By that time, six foot tall Jim could run a mile in 4:50. In September 1905 at the New York Metro Senior American Athletic Union Championships, Sullivan ran the mile in 4:22 creating a new American record. After impressive showings at the Canadian Championships and the New York Athletic Club Games that same month, he became known in the press as "4:22 Jim" and by March 1906 had been selected for the American Olympic team.

Athens, Greece was the site for a special international athletic meeting known as the Intercalated Games scheduled between the 1904 and 1908 Olympic years. The American Olympic team sailed from New York City on the S.S. Barbarosa on 31 March 1906 with James P. Sullivan aboard, on leave from his position as a junior clerk with the Department of Education's Bureau of Buildings.

In Athens, Sullivan competed in the 800 meter race but did not qualify for the final. He also ran in the 1500 meter race, finishing fifth. His second place win at the Olympic Trials in Philadelphia in June 1908 made him one of thirteen members of the Irish American Athletic Club selected to represent the United States at the 1908 Olympic Games in London.

But, once again, Jim Sullivan did not bring home an Olympic medal. Nevertheless, he was part of an historic track-and-field team competing under the American flag – IAAC teammates Martin Sheridan, John J. Hayes, and Mel Sheppard, for example, brought home the gold in discus throw, Greek discus, the marathon, and the 1500 meter respectively – a team that was fêted upon their return to New York with a parade down Fifth Avenue and a reception at Sagamore Hill on Long Island with President Theodore Roosevelt.

Jim Sullivan continued running competitively until 1912, despite surgery for a tendon injury and times for the mile – like 4:24 or 4:29 – that never beat his personal best. In 1910 and 1911 his image was selected for two series of tobacco cards (Hassan and Mecca) featuring Champion Athletes and Prizefighters.

He married Grace Burke in 1913 – the daughter of Irish immigrants, Richard (a broker at the Custom House) and Mary – and they raised two children, Richard and Elise, on his job as a clerk for the Kings County Court, a position he held until 1958. James P. Sullivan died on 9 April 1965 at age 79.