Problems with Pathfinder

I am trying to create a logo. The basic idea is that the name is in the shape of a chili pepper with the varigated colors of the pepper running throughout the letters. I started by high fidelity image tracing a photo of a chili, it came up with beautiful results. However, then when I got the letters placed inside the chili (and converted them to outlines) I couldn't get the color of the chili to stay only inside the letters. I tried having the letters in front, I tried having them in back. I've tried minus front and back, intersect, exclude, divide and others. The only thing that would happen is that my letters would disappear. So then I unfilled the letters and outlined the strokes and tried it all again. Nothing. So then I created a my own chili with the gradient mesh tool and tried all combinations again. And nothing worked. This was supposed to be a quick job for a friend and I've now spend way more time on it than I should've. I've done something similar to this before but with a solid color, so I don't know what the problem is or how to fix it. Could someone please help point me in the right directionImage may be NSFW.
Clik here to view.