The dumped variables don't have html entities encoded. So if your variable contains < in the data, your browser will see it as the start of an html tag. Additionally the dump has lots of => in it, which are also not escaped, although your browser does manage to figure out to display them as is. But they are errrors though.
Wrapping it in PRE doesn't help. Using the XMP tag will work, but it's a depreciated tag. I'm using it anyway, since it does what I need, and this function isn't (usually) used for production code anyway.
<?php
echo "<XMP>";
var_dump(get_defined_vars());
echo "</XMP>";
?>
XMP is your best bet, or use output control and the htmlspecialchars function.