Accuracy Issues With My .257 Condor

I need some help interpreting some measurements that likely have to do with some accuracy and barrel leading issues with my .257 Condor. I slugged the barrel (it’s a 700 mm TJ’s) and took some measurements as follows. I am casting bullets from soft sheet roofing lead using a 2 cavity Lyman 257420 mold. They drop at about 72 grains. Here’s what I measure:

Three slugged bullets each measured .2573 and Groove measures .2528 to .2530.

I measured 3 bullets that came out of my custom Lee .257 sizing die and they measured .2575 to .2581. As cast they measure .2584 to .2596

I have spent lots of time polishing the bore with a Wheeler kit using only the 320 and 600 grits and finishing it off with some JB’s bore paste and Flitz bore polish. I finally have it to where I can clean the leading with Hoppes and a patch and can skip the bronze brush but still only get about 20 shots between cleaning. My best groups at 50 yards is about 1″ but there is always a flyer or two that opens the group to 2 to 3″. What I am thinking is that the Lee sizing die does not size the bullets small enough and the rifle bore is doing some of the sizing as evidenced by how the slugged bullets come out. One more sign of this Lee die being the problem is that I can mix the soft roofing lead with wheel weights at a 50/50 mixture and some of these actually stick in the bore about 5 or 6 inches down the pipe and I have to drive the bullet out from the muzzle end. Any thoughts would be appreciated.
Thanks
Ray