Make your support known
Another huge part of being an ally is being a visible, vocal supporter of anti-oppression work. That means more than just agreeing with non-privileged members and remaining silent yourself. You’ve got to join the struggle.
This is not easy, right? For male allies of feminists, speaking up against sexism can generate adverse reactions from other men, because it threatens the collective performance of masculinity. Allies risk accusations of being feminine or possibly even gay. As for white people, bringing up racism is taboo in ‘polite’ conversation. They can be chastised for bringing up problems, making waves, being divisive, getting stuck on the past of racial inequities. In all instances of challenging privilege, you carry the risk of social disapproval, ostracization, and even hostility. Of course this stuff isn’t easy.
Now imagine what women and people of color have to go through, all the damn time.
I would add that an actual act of imagining this is something allies ought to be doing, and regularly. It takes effort, and act of will, to continually put oneself in the shoes of others as much as one can; it takes some imagination, I think. One thing that can help people who are allies deal with the adverse reactions of other men (and women) to speaking out is to remind ourselves that those in various non-privileged groups have to deal with adverse reactions all of the time, and don't have a lot of choice in the matter a good deal of the time.