Metronome Magazine had an annual poll during 1939-1961 that picked who their readers considered the top jazz instrumentalists on each instrument for the year.
More than that, Metronome actually recorded the all-stars on a fairly regular basis with sessions of the 1939-1942, 1945-1950, 1953, and 1956 contests.
In most cases, the group recorded two songs with short solos from practically all of the participants.