I mostly agree with Southern. If you really stick well with your mare, and don't tend to loose your balance easily (those little guys can *really* take the turns!), then I wouldn't worry so much about the height. Ponies are great at barrels for exactly what Southern stated, and if you can stay on just fine, I wouldn't worry about it too much. Just stay close to her around the barrels and y'all shouldn't have problems throwing each other off balance.
However, if you feel awkward on her just doing rail work, etc, trying for barrels may not be in your best interest. What has your trainer said?