I beat it boys
Two things I learned is that they can't hear at all, so you don't have to sneak around. You can walk right behind them full speed.
Also they don't look up at all. You can climb around them and glide right over top of them and they won't see you.
After picking up those two things, I was able to cheese the whole thing, grabbed all the loot and beat it easily, using just one banana for the last guy (he actually caught me but I was able to get to the boss stage before they caught up to me).
I hate stealth in games so much. It's never done well.