So after more poking around this issue with the filter values for voices overwriting each other also seemed to be present in the harmonic filter. It's very easy to recreate I don't know why it hasn't come up before, I think not many people are using voice start mods with poly filters or something.
I've updated the PR with the additional fix from Claude. In the process I also got it to fix a smoothing issue and a display issue.