When you go back and play in an Arena lower than the highest one you've unlocked, your weapon stats may be capped to the maximum values for that lower Arena in order to maintain a fair experience for the players who are still climbing the ranks. You might still have an advantage, but your weapon won't be overpowered.