It is likely your hip.
I have this problem too, and have struggled with it for years. My left side looks completely like a chair seat while my right side is in perfect alignment.
In the past, just letting my legs hang in the saddle at a walk - allowing my lower body to be bumped along by the horses's walk REALLY helped. In addition to strengthening my abs. A few years ago, I also went to an instructor who helped me with doing exercises in the saddle- taking the legs away from the saddle as well we up over the flaps, bicycle movements, rolling the ankles, and then relaxing the muscles at a slow trot or jog, first without stirrups then with. She also had me do upper body exercises in the saddle - those are a bit harder to explain. The horse was very smooth and slow - i was on the longe line, and it was amazing what 45 minutes would do- by the end my legs were lengthened by several many holes in the stirrups, completely stable yet flowing. I have some before/after pictures somewhere taken in one lesson- simply amazing.
That instructor, however, has moved away

Everything was great for years after a few of those lessons.
Then...just recently, it has gotten worse, worse than it has ever been. I took a few months off from November- April. Bad move.
I'm actually going to start seeing a massage therapist this week to help

Now I can't hardly let my hip relax on the left side when I'm in the saddle - sends shoots of pain in all directions and stifles my breathing. Doesn't seem to matter which horse or which saddle, I've tried all sorts of combinations. So, time to seek help for me.