One of the main reasons for the appearance of Yoga being like religion, is really a confusion between religion and mysticism. Within most religions, there are shades of gray within two poles, that are called Exoteric and Esoteric.
The Exoteric, or external religious practices are what most of us see as dominant in our cultures. Within all of the Exoteric religions, there are also Esoteric practitioners that are seeking direct experience. These seekers of direct experience are often outcasts from the Exoteric organizations of their own religion. This has happened throughout human history. The examples are easy to find in books and articles, and are not presented here.
Yoga is itself a mystical seeking of the direct experience of inner realities or truths. However, Yoga itself is not telling an aspirant what religion to follow. It is involved with training in the process of working with senses, body, breath, and mind, such that the inner can be experienced directly. Yoga deals with the Esoteric, rather than the Exoteric. In this light, some of the practices of Yoga are contained within religions, while religion is not contained within Yoga.
What appears to be conflict between a particular religion and Yoga is often not really valid, because of comparing an Exoteric religion with the Esoteric practices of Yoga. This same appearance of conflict also occurs within religions themselves, usually due to the fact that the Exoteric followers have little understanding of the Esoteric seeker. The Exoteric follower has simply not traveled far enough to feel the deep-seated Esoteric longing of the mystic and yogi to know the deeper teachings and direct experience.
In the principle of spiritual life being like climbing the steps of a ladder, or staircase, it is easy to see the steps behind you, while it is difficult to see the steps still in front of you. Thus, in the view of the Exoteric religionist, the Esoteric seeker or Yogi is seen to have strayed from the path, to be an evil person or sinner, or to have a psychopathological problem.
Like it or not, the practitioners of Yoga and the mystic seekers in our modern world, like those throughout human history, must accept that there is rejection by others who do not understand the subtler path. This is not likely to change, for the simple reason that the seekers of inner experience and truth are consistently in a minority.