To be perfectly honest, I don't think you can flag it down to one reason alone. There is probably an entire set of reasons as to why magick is an obscure subject among people. History provides some reasons and in fact, the particular set of reasons also depends on the time period that you are in. The mental attitudes and perspectives at a particular phase in history determines much of why magick was obscure. By extension some of those institutions that are influenced by those mentalities also drove the obscurity even further.
Here is my ChatGPT-like, but totally self-written response to the OP's question:
1. Scientific / Materialistic Skepticism - The rise in the scientific method and all of the advances that it has brought us has promoted a very materially oriented view of our reality. While not every scientist or researcher is an atheist, many people today and for at least much of the last century have started to gravitate towards a vision of reality that is purely dictated by the laws of physics and by what we can measure or directly perceive. Subtle things like magick are regarded as fantasy in the same bin as religion. People simply choose to believe more in the material world rather than accept that the magical / spiritual world can exist alongside it.
2. Religiously driven fear - This is a commonly known one, but religious institutions throughout history have hated the occult and magick in general. It is ironic given that prayer is a completely valid form of magick for most occult minded people and that many of the primary texts of religion actually contain direct acts of magick within them. But for purposes of control, subordination, and power, religious institutions have always promoted magick as being something to fear and hate. The followers of these religions usually end up blindly going along with this and you then get a mass perpetuation of hatred against magick. This leads to a suppression of magick among the religious and its active denouncement on a societal level and this has continued throughout centuries of our history.
3. Gatekeeping - This does not necessarily apply to magick as a whole, but many occultists do have a doctrine of silence that they adhere to. Some books even actively promote it and there are sometimes good reasons why. Many who adhere to this doctrine gatekeep magick from society at large because they think that magical practice could bring harm to the layperson, encourage karmic issues, or perhaps simply that the average layperson is not ready to practice. So there are not many occultists out there who will actively try to spread magical lessons or thoughts to other people. This ends up leaving the world a little less filled with open magical thought.
4. Survival and fear - This is the reverse of 2 and related to 3. Those who practice the occult have been demonized throughout history by the religiously minded. For their own survival throughout history, they kept their own magical practice and resources a secret due to fear of being persecuted. Consequences like witch burnings, executions, and such come to mind here. Even today in western countries where occult practice can be talked about more openly, there is active demonization from society due to media and religious influence which tends to keep occultists a bit more on the quiet side. I run my own occult group where I live quite publicly and have experienced some critical people who have discouraged me from discussing magic openly. I live in a pretty liberal and accepting country, but it happens even here. So people people will keep their magical practice, resources, and thoughts to themselves unless a certain level of trust is established.
5. Culture and media portrayals - For the modern day, this is a big one. Movies such as "The Exorcist", "The Conjuring", and many more are pumped out by Hollywood and give the supernatural a much darker edge than it really has. The media regularly talks about stuff like "ritual murders", cults, and other events that they link to the occult in general. All of these negative portrayals end up making people think that magical practice involves sacrificing people to unknown gods, giving blood sacrifices to the devil, or summoning unholy spirits to cause havoc in society. The reality of magical practice is, on average, much lighter and more good-natured than that, but this reality never gets reflected in media portrayals. This adds on to the survival factor I mentioned and is partially related to religiously driven fear. Either way, such portrayals makes it tough to speak freely about magick without suffering from negative connotations and interpretations by your average person.
To me, these are the biggest reasons why magick is so obscure. The funny thing is that we are much more numerous than we individually think. I live in a place that is cultural flat and not very dynamic, but the number of people who are into magick is more than I thought. Lots of folks were into witchcraft, grimoires, or even chaos magick, but they kept themselves hidden due to a combination of these factors I mentioned. The moment I started publicly announcing meetups for magick related discussion, I had a lot of people sign up. They just needed a lightning rod to come up. And when you consolidate people together like this, magick ends up being less obscure. By breaking the fear and the gatekeeping, you even get people who are curious to join on the discussion. I actively share magical resources in case if they are interested in their own practice. This helps open it up further and make it less obscure.
Anyway, if you read all the way up to hear, thanks! I hope this helps answer the question!