Between the program, the television monitors, the highly-visible number of a horse's saddlecloth and the track announcer's call, it is pretty easy for the average fan to follow the progress of a particular horse throughout a race.
But when horse racing first began in the early 18th century, there were no such things as program numbers, public address systems or closed circuit television. So when King Charles II (he's the one with the long, curly wig) first assembled race meets on the plains of Hempstead, the dukes and the barons had trouble figuring out which horse was which. So, they adopted racing silks - or colors - to distinguish their jockeys for easier viewing.
During the time of King Charles II, the silks were simple -- red for one duke, black for another duke, orange for one earl, white for another earl, and so on.
The tradition of the silks remains today as jockeys wear the colors of the horse owners, but since there are so many owners, they have become even more colorful. Some of the most famous silks are the devil's red and blue of Calumet Farm, worn by the jockeys of Kentucky Derby winners Citation and Ponder and Allen Paulson's star-spangled red-white-and-blue colors, carried by the champion racehorse Cigar.
The jockeys' room at Churchill Downs houses hundreds of silks which are hung on pegs in the order of each jockey's races for that day.