For a while I had settled on Fritz Graf's formulation that, historically, magic was the personalized aspect of religion that was done in private, "at night", for selfish ends - basically a parallel "shadow side". And this was seen as bad by the community religion because those ends were suspect, a perversion of rites, and the Gods might punish everyone for the activity of just one person. Graf was countering Frazer's idea that magic was "proto-science" and "proto religion". Still, Frazer has some good mechanical ideas, and I think there's truth in more than one formula.
Lately, I have been more inclined to think that, again historically, "magic is what your enemies do". It could be a legit religion in a broader sense, but because the dominant culture doesn't like it, it gets a label. I mean, etymologically speaking, the word "magic" itself for the Greeks wasn't exactly a friendly term - they were sneering at the Persians, old grudges and such.
For all practical purposes, though, magic as present in religion can be applied to the structure of rites. A typical example is the Eucharist, with transubstantiation as literal transformation, or even consubstantiation for the protestors - Zwinglian memorialism as heresy must have been so boring because it was too reasonable and far less fantastic. The dogmatic difference is still in intention (personal vs. communal).
But you can slice the cake so many ways and still come out with cake.