I initially screwed up my e-steps calculation. Since I was overextruding I had it in my mind that esteps would go down which normally they should in that case, but due to the problem I'm having (where nothing made sense) the numbers I had from the tests would have me increasing e-steps. In any case my esteps and the values when extruding 100mm are all over the place.
So I took apart the BMG and found the first problem:
So I removed the extruder, cleaned everything, and set it up so the hot end would be removed from the path. Then I made sure everything was tight (especially the tensioner), and ran filament through several times. The results were slightly more consistent but still not right. I found that I could stop the filament from moving by holding onto it, though the extruder did not click or complain, and the stepper shaft did not stall or miss a step - which means the gears are not biting strongly enough on the filament. I cranked the tensioner as far as I could turn it easily and tried again, but I can still stall the filament by hand. I'm guessing that the tollerances in the BMG clone are just not quite there and there must be either a misalignment or a gap possibly. I checked over the gear alignment as best I could with the gate swung open and it seems OK though.
So now I least I know that this is not a problem with the stepper, vref for the e-stepper and probably not the e-steps. It looks like a problem with the extruder which is allowing the filament to slip.
I dug further into the extruder and next found this, which I overlooked the first time I had it apart (and apparently also missed it when I put it together initially):
I did have to adjust the esteps again, and starting over from 415steps/mm which is the default for the BMG, I eventually landed on 421 steps/mm. That was at least a consistent value, which I confirmed with two tests that both hit on exactly 100mm when I requested 100mm from the stepper/extruder.
So although really dum, this was a success, and maybe even more so considering I actually have some really nice prints from the printer, even with the screwed up extruder. That was until recently at least, when I tried to run PETG and noted the inconsistent extrusion and flow values (which I track using a Google Sheet). I was also seeing issues with wild retraction settings when checking hatchbox wood previously (a post I will update to point here as a likely cause of that problem).