A Problem picture can be a painting of obscure meaning depicting people involved in a situation or event of mysterious or anomalous nature.
It is a term referring to everything but the picture being the problem. But in the light of e.g. an art historians work, it poses a problem.
Alternatively, it can mean that the picture depicts refers to a problem or paradox little understood by anyone other than the originating artist. For a lay person, Surrealism springs to mind, but here there is usually no problem in the sense that art aficionados tend to decode that as subconscious thoughts and dreams imposing their meaning on perceived reality.