A community known as Prairie Point, located at the crossroads of two stagecoach lines near the site of present Rhome, was established in the late 1850s by a group of Missourians. One of these new settlers, Samuel Sheets, owned land near the crossroads and in 1858 began to lay out and sell town lots. In the same year a post office opened, and by the early 1860s Prairie Point, with a hotel, a school, and several businesses, was the second largest settlement in Wise County.
By 1863 Prairie Point was virtually deserted. By 1882 the Fort Worth and Denver City Railway had laid tracks nearby, and by 1883 the new settlement was named Rhome, in honor of Col. Byron Crandall Rhome, a prominent local rancher who was credited with bringing the first Hereford cattle to Wise County.
The Rhome post office was established in 1883, and by the early 1890s the town had 175 residents, a school, a church, and two flour mills. One of these mills was the first roller mill built in the county, and possibly the first in Texas. In operation by 1898, this more technologically sophisticated mill used rollers instead of stone burrs to grind wheat into flour. In 1939 Highway 81 was built through Rhome, and by 1942 the town was incorporated.