Not all bookshelves are containers, but the ones that are can be looted by walking close to them.
Bookshelves look like this:
North/South facing
East/West facing
