Get some pics of the interference and what's wearing, where... That'll help. Sounds like it's all at the back of the upper receiver, or bolt-to-buffer.
Just so you know, the bolt DOES push the buffer off the buffer retainer pin, ever time you close the receivers. When the rifle is closed, that buffer never touches the buffer retainer pin. If it did, during operation, it would shear that pin right off as soon as you fired it, and it recoiled, and closed on the next round.