As a rider and a walker I can see it from both sides. I love exploring with my four legged man and there are times when out of nowhere he finds a monster in the undergrowth! I always apologise if there are walkers near by as the look on their faces normally speaks volumes. When a horse spooks it is mostly as much a surprise to the rider as it is to others. If the rider is familiar with the horse though, there is a very good chance they will know what will happen over the next 10 seconds or so after a spook, so although they may sound anxious, where possible, it is worth following what they ask you to do as trust me they will be thinking of you as well as themselves and the horse. Riders may sound anxious if they see walkers near by panicking and causing more worries for the horse.
From a walkers point of view approaching a horse I do not know I always call out to the rider if we have to pass in close proximaty to see if they are happy for me to pass. I also bike ride and do the same, as a horse rider it makes such a difference if a bike rider makes themselves known to me as I do not always hear them. A rider jumping out of her skin as a bike whizzes past is enough to then send the horse into the bushes, even if the horse is one step ahead of his rider and knows the bike is there!
I always tell my friends that are wary of walking through fields with horses to remember that horses are flight animals, not fight animals. If you go into a field looking for a fight (in the horses eyes) with clenched hands and starring them in the face, the horse is likely to react, if they are in an enclosed field they can not run away from you so may start running around the field etc, which could be intimidating for a walker. However if you go into the field and keep your body language soft then chances are the horses will either ignore you or they may wonder over to say hello, again if you keep your body language soft they are likely to say hello and then wonder off. One tip I also have is to not eat anything if walking through a field with horses. I had one walker go crazy at me (I arrived at the field just as she was exiting it) as my horse followed her the whole way across the field nudging her with his nose as she was eating an apple!
If you do have any promblems with horses, contact the landowner or owner of the horse (most will keep their details nailed somewhere to the fences in the event their horse is found injured). As an owner we are not always aware if our horses are causing problems as we are not with them 24-7.
I hope some of the above may be helpfull. Horses are lovely animals even though they are the size of small houses!