The famines preceeded the cultural revolution, and thus the student guard, by a very long time.

In fact, a good argument is that Mao was sidelined after the debacle of the famine, and the Cultural Revolution was him doing a low level insurgency to regain power.