Bridge's family were craftsmen. Both his great-grandfather, Benjamin Bridge, and grandfather, William, were shoemakers. William Henry Bridge (1845-1928), Frank's father, also began his working life in a modest way, as a lithographer, although his main interest lay not in printing but in music. During middle age he abandoned his career as a master lithographer, becoming a violin teacher, and from his home base in Brighton he travelled round to the local boarding-schools, both in town and further afield at Seaford and Eastbourne, to coach his pupils. In addition, he became the musical director of the Old Oxford Theatre, subsequently renamed the Empire Theatre, a music-hall in Brighton, conducting the orchestra and providing the musical part of the entertainment from eight o'clock onwards each evening.

Although in his dealings with people William Henry was direct and to the point, he was not an altogether likeable person. He could be irascible, quick to flare up over some point or other, imperious, difficult to approach, stern. As a Victorian paterfamilias, he brought up his twelve children with not a little severity and then turned them out, both girls and boys, when they reached their mid-teens. He had three wives, the second, Elizabeth Warbrick (1849-99), being Frank's mother, and it was she who had to look after all the children as best she could, bringing to the household a gentleness and fund of good nature that was otherwise lacking. As a child, Frank (born 26 February 1879 at 7 North Road, Brighton) was closest to his half-sister Alice (1867-1951), Nellie (1881-1965) and William (1883-1956), although in later life, several of the brothers and sisters became estranged from time to time. Reconciliation was not often favoured by the Bridges.