Turns out I was just confused by the UI: I assumed that it would show a line for each parallel download, but it rotates between them on a single line. Adding fastestmirror=1 significantly increased speed.