To me, the smoother the barrel, (1) the less potential damage from friction and rust, and (2) the easier to clean -- brushes and patches have less resistance and wear & tear going through.
One of the reasons they say to only run patches breech-to-muzzle is that's the direction the bullets travel, meaning you're less likely to snag a jagged edge. It's like rubbing a cat the "right " way vs. the "wrong" way.
Accuracy may not be the only factor. I think a rough surface just makes keeping it clean, oiled and rust-free a little more challenging.
As mentioned, once it's broken in, all these rough sections ought to be taken care of. During the break-in period, you may see changes in accuracy, too. Those changes may be a problem for you if accuracy is a big concern for you and you aren't expecting them.
Anyway, those are my thoughts. I'm not a gunsmith nor professional marksman. I've read that most new barrels don't require much break-in at all, but that doesn't mean it's not going to improve your results.