I don't mind the border appearing, its just the strange bit at the top that I dislike. I know I can work around of this, getting rid of the border altogether by using outline: 0, but I also know that's bad.
The HTML setup is
<li>
<a href='..'><img alt='..' src='..'/></a>
<a href='..'>...</a>
</li>
An extract of the CSS:
li{
    display: block;
    width: 9em;
    margin-right: 1em;
    height: 12em;
    text-align: center;
}
li img{
    height: 8em;
    display: block;
    margin: 0 auto;
}
(The effect is similar if the two are combined in a single <a>.) 
I think this is related to using display: block on the image. I've reproduced the issue here: http://jsfiddle.net/pvd69wce/

If you don't want to remove the outline, one solution is to give display:inline-block to the <a>.
li {
  display: block;
  width: 9em;
  margin-right: 1em;
  height: 12em;
  text-align: center;
}
li a {
  display: inline-block;
}
li img {
  height: 8em;
  display: block;
  margin: 0 auto;
}<ul>
  <li>
    <a href='#'>
      <img alt='' src='https://upload.wikimedia.org/wikipedia/commons/e/e2/Merton_College_Oxford_Coat_Of_Arms.svg' />
    </a>
    <a href='##'>Merton</a>
  </li>
</ul>To be honest though, I'm not entirely sure why this works exacty. Nor indeed, where the extra lines do come from when the <a> is not an inline-block.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With