Re: cutout feature of photoshop
Photoshop filters look different on different photos. The contrast of the original photo plays a key role in the outcome of the filters. If your photo has low contrast, filters will not work very well. I have even found that adding contrast to photos that are weak just doesn't work that well so you have to start with a very good crisp contrasty photo for filters to work.
If yo don't have a lot of contrast, try using "Select Color Range" or magic wand with a tolerance around 30 to 50 and then fill your selection with each tone of color starting with the darkest colors and working toward the lightest. It might be a good idea to save your selections as alpha channels so you can go back and change the density on each area as needed. You may have to de-select previous selections as you are selecting each tone just so they don't overlap. Hope that makes sense, It's not easy to translate the things going on in my head!