I have several albums of the same artist on my Lumia (I'm not using the Xbox Music Cloud).
When I'm accessing [Collection] -> [Music] -> [Artist] -> [My Artist] -> [Albums] I don't get all the artist's albums that are on the device, yet when swiping to [Songs] I get a list of all the songs that are on the device.
In addition [Collection] -> [Music] -> [Albums] also offers a complete list of all my albums.
The artist's page looks similar to this (currently can't take pictures of the device):
Is this a bug in WP8 or am I missing something?