In "The Color Yellow", a photo of him is seen when Lisa is looking for some nice ancestors, only to be disappointed by everything she has found about her known ancestors. So, Hugo was likely a crook and/or had a bad reputation.
Interestingly, his first (and so far only) appearance outside of The Simpsons Uncensored Family Album is in "The Color Yellow", an episode that heavily contradicts the book's established Simpson family tree. This shows that the book is to an extent canon and has not been completely retconned. It is also possible that Abe's story about Virgil was false due his dementia or created to cheer up Lisa, with Hiram and Mabel being related to the Simpsons but not direct ancestors, thus leaving the previously established family tree unchanged.