I agree that all of the factors you point out contributed to the rise of the middle class on some level. But I think the effect of university education and birth control was probably greatest on white Americans. People of color were far less likely to take advantage of these, especially during the 1940s-1960s when the middle class experienced some of its highest growth.
Also, unlike cheap energy, investment in infrastructure, and a two-parent income, home ownership gave many people the ability to maintain a net worth that kept pace with inflation, especially during the 1970s when the middle class took a big hit.