Exercise is exercise and if a kid is interested at 4 or 5 so be it. We have soccer tots and t ball at 4- why not lax as well? I agree that the skills are more advanced, but have we all not seen those kids that can throw a lax ball at 4? Let them enjoy the movements and learn from it. I am sure it is age appropriate.
My youngest son started in kindergarten with community clinics and it did back fire because he hated the helmet and gloves and found it hard. It could back fire, but that is like anything else as well.