You might look into a SanDisk Sansa Clip+. (I don't own one, but I know someone who does.)
This: - supports playback of Ogg files (amongst other things) [1,2,3]. - can record audio (although I believe this may be in WAV format [2,4]) - has USB, can behave as a mass storage device, so works fine with Linux - has a microSD card reader. - is the size of a matchbox
I've read a suggestion [4] that Rockbox [5] might be installed as a way to allow the device to record in MP3 format [6,7].
One caveat: European versions have their volume limited, due to EU health and safety rules. However, this can be circumvented by resetting the firmware and choosing another locality, possibly modulo some loss of FM radio functionality.
1. http://www.sandisk.com/products/sansa-music-and-video-players/sandisk-sansa-... 2. http://en.wikipedia.org/wiki/SanDisk_Sansa 3. http://ubuntuforums.org/archive/index.php/t-917677.html 4. http://forums.sandisk.com/t5/Site-suggestions-Product-feature/Voice-and-FM-R... 5. http://www.rockbox.org/ 5. http://www.rockbox.org/wiki/SansaClip#Clip_43_variants 6. http://download.rockbox.org/daily/manual/rockbox-sansaclipplus/rockbox-build...