The arbitrage they make is what becomes the LP holders impermanent loss plus transaction fees. In other words, the arbitrageurs "skim" a little of the profit out of the LPs so the LP holders don't make as much.
This is 100% false. Many articles that try to explain impermanent loss do phrase it in this weird way though. It's not suddenly possible for people to extract value out of an LP pool just because we get a centralized exchange listing. Impermanent loss 100% occurs because coins in the LP are for sale and they are being bough and sold at market value on the curve.
Think about it this way: Impermanent loss is impermanent. If the price goes up, LPs take the loss, but if the price goes back to where it was, there is no loss. Did the arbitrageurs give the money back? lol... I don't understand why so many websites try to push that obviously false narrative.
Arbitrageurs don't make any money off of the LPs: they make it off of the liquidity located on the centralized exchange. This is very obvious just from the fact that impermanent loss is impermanent.