I think there has no such mothed to find out the encryption.
First. there has so many encryptions include user define encryption that you even cann't know.
Second. encryptions just convert a binary stream to another. It has no feathers in it.

Maybe you can get some clue by the binary stream. But it is very difficult and not exactly.