Support multiline strings in user configurations

Closes #1868 (closed)

I hit the 1000 line limit when updating the tests, so I shortened them with subtests in 69673edd first. The actual tests were added in 84d99c01.

Adding subtests (and thus making proper assertions all the time) made me catch a mistake in the way we returned the user configuration errors (as a list instead of a dict) so I also updated this in 429c3711.