Who made your upper and lower? If you have so much movement in your upper and lower, that it is causing an elongation of your takedown pin hole, you have some fitment issues. I have an AR built in .338 Lapua magnum, which delivers considerably more recoil than any standard cartridge, and the receiver shows no signs that it has been fired.
My theory is, A) your lower receiver and your upper receiver do not mate well, or B) your lower receiver is not built to perform under the cycling stress of the caliber you're shooting. It’s entirely possible that your lower receiver is not strong enough (be it poor design or poor choice in metal) and that is causing a flex between the pistol grip and the stock. That little flex is causing the strain at the pin hole. This was a common area of failure for the original polymer lower receivers.
You really only have two options… 1. Separate yourself from the rifle and make it somebody else’s issue. 2. Find a machinist that’s willing to do the work, and have him bore the hole out to a larger diameter and re-shim it with some kind of a bearing or other insert. JP makes a pin that will tighten that up, but that’s just a remedy to the symptoms and not a fix to the root issue.
If your upper and lower are from the same company, or if you have an off-the-shelf rifle, you might be able to contact the manufacturer and get the parts replaced??????