About 60 years ago, he overthrew the grand shapers of the Five Kingdoms and took control as sole ruler. To do so, he pretended to kill his five daughters (Destiny, Honor, Miracle, Constance, and Elegance) while in reality stealing their shaping power and imbuing himself with it. His wife, Harmony, helped her daughters to escape and go into hiding.
Stafford was envious of his wife and daughters because of their shaping powers being superior to his own natural abilities, and this envy was exploited by Owandell, a high-ranking member of the shapecrafters, who convinced Stafford to betray his family.
Book 3: Crystal Keepers