I'm not allergic to it, but after having smoked for nearly 3 decades, I now cannot stand the smell. If I even walk past someone who is smoking and the smoke wafts my way, my lungs tighten up. I try to hold my breath while I walk through the plume, just to keep from having my lungs grab, even for an instant.
When I quit, I lived in a house that was openly accepting of smoking and drinking, where I rented one bedroom and shared the common areas with up to six or seven others. The only time I smoked was when I drank and couldn't stand to be around smokers when I was sober.
After I quit drinking, and the smoking went with it, I quickly searched for another place to live and moved to another apartment where smoking wasn't allowed. I was, again, renting a bedroom and sharing the common areas.
Even if your neighbors moved their smoking to the outside, you'd probably still have issues depending on the prevailing winds.
The best solution would be if you could find a place that doesn't allow smoking. Until then, if it's not one neighbor, it might be another. Good luck.
Stay well.
Lynn