The single biggest reason I can think of for it to not work is simply that you'll never get the "true" color of something (hue, saturation, and especially value) by either taking a photo or scanning it, since the "color" of anything changes from moment to moment as the lighting conditions around it change. For example if you turn on another light in the room or open another window suddenly everything has a lighter value. If you're in a room lit by a normal lamp everything will be more yellow. Our eyes/brain just adjusts to compensate for how yellow regular lamps are so a lot of times people don't notice or realize it. At different values various hues will appear more or less saturated.
The reason that it might work would be if everything was affected the same and you just organize things relative to each other in the photo. Basically it might not matter if everything was darker or lighter than its true value so long as everything was darker/lighter by the same amount.
You might want to try looking here
to see if the pastels you use are already given digital values for their color. I can't guarantee complete accuracy with all their numbers, since I was recently looking up numbers for daniel smith watercolors there and they showed carbazole violet as being a dark green, but it's an option.