Excellent advice given thus far!!! I couldnt agree more!
I do my portraits in Pastels as well! Try useing the textured side of your Canson paper, but you do need to try putting down a layer with soft pastel sticks first, then go in with your pencils for details. Try blending the first layer in with your fingers, then spraying a light fix if needed and then add your pencils for detail if you choose.
After experimenting a bit more with the pastels, Im sure youll find a groove that fits your personal style.
Hope to see some of your work soon!