It seems that getting album art to show up for Ogg Vorbis files is an ongoing struggle on every phone I get.
At least the source code for Android seems to be readily available.
So, first clue: in 2009 code was removed from src/com/android/music/MusicUtils.java that would read AlbumArt.jpg: "Remove code for getting and saving album art from the music app and use the media provider instead.".
Which probably means that I should go hunting in the media provider for more clues.
It looks like it does scour the filesystem for albumart.jpg - so why doesn't it work?!?
Update: Ok, it seems to work if the albumart.jpg file is there from the beginning. It doesn't seem to work to add it later, not even if I remove the entire directory and then copy it back.
I wonder how I can "flush the album art cache".
Update 2: Looks like adding albumart.jpg before transferring the files to the phone and having the image square (NxN pixels) works.
Update 3: If the phone has decided that the albumart isn't there, it seems a reboot is needed to change that decision (at least; clearing Music data might also be needed; moving the files off+back on is not necessary).
Update 4: The image doesn't have to be square - I just got three images that are higher than they are wide to show up, so I am down to Android being picky about the JPEG-format; opening the image in Gimp and saving it again seems to help (with a clear Music data (possibly) and a reboot (for sure) afterwards).
Also, did you notice that the stock ringtones, notification-, and alarm-sounds are all Ogg Vorbis files? Nice touch.