I think the main difficulty of wildlife photography rests largely on getting to the ideal location. Some places simply don’t have a lot of wildlife to be photographed. Sometimes, the photographer has to be willing to take the time and hike into the animals’ habitats, places not always easily accessible. Since you are often carrying heavy telephoto lenses and other photographic equipment, patience and endurance will certainly be factors contributing to the difficulty. However, the rewards are also high and come as you witness true nature in action.