Right, Dhenton's got it!
It may also depend on whether you're painting/viewing in 256 color mode instead of 16-million (or more). That would definitely add to the "pixelation". Saving as a GIF would do it, too. (That sounds elementary, I know... I only mention this because when someone else asked me a similar question, it turned out that that was the problem!)
Now, I'm going to turn this into a dissertation on DPI and PPI. If you know this already, ignore me.
PPI : points (or pixels) per inch. This is
the resolution of your source image. "Pixelation" is usually a function of PPI.
DPI : dots per inch. This is the number of dots your printer can cram into an inch. The source image size doesn't matter -- the printer prints at the same DPI, according to the size (in inches) ascribed to the source file.
Basically when you see "pixelation" on your image (for example... when you zoom in onscreen, or if you print), it's a function of the image resolution. As stated, the monitor shows images at 72 ppi. (pixels per inch). If you want to make the image larger, or if you plan to print, 72 ppi isn't enough...
For example, let's say I did an image that is 432x576 pixels. Now, that's a nice comfortable size on my smaller screen, but if I wanted to print that at monitor resolution... 72 dpi... it would be 6x8 and I would be able to see individual pixels, unless it was a very smooth image to begin with. If your brushes had hard edges or you used lines, it would be VERY obvious.
Now, one solution as DH suggested is to work at twice the size... that can work. I usually calculate it differently though. I say to myself, "What size do I want to print the final result at?" Let's say I want to print at 6x8 inches. I calculate that at say, 150 dpi. That tells me my source image should be 900x1200. If I expect to have hard edged lines, I might use 300 instead, or do some of the things DH suggested to smooth it.
Now, if that's too big to work with easily, or if you have an existing image you want to print without "jaggies", there is a way around it. It's NOT perfect since it can give you fuzzy edges... But, here's what you do. Let's use a 432x576 72 ppi image as an example.
Now, that pic is REALLY looking jaggy because it's totally aliased (sharp edges). Blurring or resizing can change that look.
Here's what it looks like after resampling the image to 50% of the original size, then back up again: http://www.pixelscapes.com/jen/misc/dpi/dpi72b.gif
And here's what it looks like after resampling THAT one 50% of its original size, then back up again: http://www.pixelscapes.com/jen/misc/dpi/dpi72c.gif
Now, it's true that the resampled images give an illusion of smoother lines... but, you lose a LOT of detail by doing that. Softening (or blurring) gives you a little more control. See this example, the original image softened slightly: http://www.pixelscapes.com/jen/misc/dpi/dpi72d.gif
Looks much better than the others, doesn't it?
Here's a combination of techniques: I softened and then resized 50%, then back up again. This one looks best of all. http://www.pixelscapes.com/jen/misc/dpi/dpi72e.gif
As you can see, these techniques can help avoid that hard edge on lines that gives that stair-step, pixelated, jaggy effect. However, softening or resizing an image can REALLY kill your detail, tiny textures or dots, and so on... The best thing to do is either 1) learn how to get your paint program to make strokes with softer edges, or 2) work at a much higher size than you need to, so all the details are bigger (so to speak) and therefore softening or resizing won't hurt them as much.
Sometimes it can be hard to work on a larger image, just in terms of size onscreen... if your program allows you to work while viewing the pic at a 1:2 zoom factor, that can really help.
Hmm, did this make sense? I didn't get much sleep last night.
-=- Jen / Pixelscapes