I think what some of us was irritated about was that he has used his stuntdouble for fightscenes in previous movies, and he should be able to do fightscenes on his own. Besides that I dont think anybody has a problem with him using stuntdoubles in the the actual *stuntscenes.*
Seagal should take an example from Schwarzenegger. Read on e-online that he has done ALL the stunts on his latest movie himself, he's 64 years old and has been out of making films for years. Deep respect for him.
Peace