As some of you may be aware, for the past year or so, one of my side projects has been to put together a robust product review system for use here at WC! Well, the software is pretty much completed, and we are beginning to implement it.
Why a product review system?
For many reasons. First, it hasn't really been done yet. The art supply manufacturers will never do it, and neither will the retailers. The manufacturers won't do it because they risk exposing potential customers to other brands of, possibly, superior products. The retailers won't do it because they can't show favoritism (perceived or otherwise) among the brands they carry. Even if they did do it, they could only provide reviews on products they carry, which would not be inclusive of the entire industry.
In addition, I decided to do it because it was time to do it
. This is something that I wanted to do 4 years ago when I started this site, but we just didn't have the "critical mass" to pull it off. Now we do! With over 12,000 members now, plus 10 times that many that lurk here every month, I really feel like we can build something of enormous value to the community here.
What has been done so far
Initially, I chose the Oil Painting area as the first area to focus on, primarily because of its popularity, as well as the fact that I know we have some pretty knowledgeable folks there who could add tremendous value as we roll this thing out. We've made some substantial progress there, and are now beginning to move into other areas.
First, the demo/work-in-progress:
NOTE: FEEL FREE TO ADD TEST REVIEWS, BUT KNOW THAT ANY REVIEWS YOU ADD NOW WILL BE DELETED WHEN THE SYSTEM GOES LIVE!
Okay ... if you are interested in helping out, bear with me and read on ...
Some Important Background Information
The product review system itself is a VERY complicated piece of software. Among it's many features, it will allow you to:
- Compare a vendor to other vendors, on a category-by-category basis (in terms of averaged ratings across all product lines)
- Compare a product at the detail level with similar products. So you could compare the "pigment quality" of Grumbacher Pre-Tested Oils with other oil paints, for example.
- Browse categories, and sort by vendor name, product name, number of reviews, and average score
- Quickly find all the items by a particular vendor
- Perform keyword searchs across item names, descriptions, and vendors
- Find all the reviews by a certain community member (great feature, especially if you respect the opinions of a particular member, and want to see what else they've reviewed).
- Quickly start/join forum discussions on a particular product
- A whole lot more.
The entire system is based on what we call "rating types", or "rating questions". Each category (such as Oil Paints for example) can have up to 10 rating questions assigned to it. Now, I could take the easy way out and just use a single question (such as "Overall Rating") for each item/category, but that wouldn't add a lot of value. Amazon does this with its book ratings. But in my opinion, a quick 1-5 rating doesn't tell me much. As you all know, in the art business, materials cost money, and I'd much rather see an in-depth set of product ratings before I shell out my hard-earned money for a new $60 tube of imported Cobalt Blue.
Instead of the traditional "quick rating" scheme that most sites use, what we are doing is focusing on developing a very "niche" set of rating questions for as many categories as possible, so that a member can browse Studio Easels, for example, and compare the "Durability" of those easels, or perhaps the "Value for the Money", "Portability" for french easels, etc.
Note: Some categories may, indeed, end up with just a simple, one-question rating scheme, but for the "big" ones, I want to get as technical as possible, without getting ridiculously over-scientific in the process (if that makes any sense) - lol.
What do we need to do now?
First, we need to continue building up our database of vendors and products. See the specific FOCUS GROUP threads for more info on this.
Second, we need to focus on the rating questions for each category. What I'd like to do is work with everyone here and develop a list of rating questions for the big categories, and possible answers for them.
Some additional notes:
- There can be no more than a total of ten (10) rating questions (per category). However, 10 is a lot. If possible, try and group questions together. A list of 4-6 questions is more manageable from the reviewer's standpoint than 9 or 10.
- All questions must have exactly 5 possible answers, numerically ranging from 1 to 5, but with short phrases assigned to them. Example: Professional Quality: 1=Poor, 2=Adequate, 3=Average, 4=Good, 5=Excellent. This is just an example, the phrases could be anything (Durability: 1=Shoddily Built, 2=Not very stable, 3=Average Ruggedness, 4=Solidly Constructed, 5=Excellent Construction).
- Given the second point above, remember to avoid things that are not subjective enough. For example, we can't ask people to rate whether or not a product is student-grade. That is a factual piece of information - a yes or no. Rating questions must be something that can be graded or measured on a scale of 1-5!
- We are not implementing a "price" rating at this time, although we are looking into how to possibly accomplish this down the road. Pricing is simply too volatile, and is too difficult to keep current. There are other reasons which are too complicated to get into in this already lengthy post, so, in advance, no. You can, however, mention your views on the pricing of a particular item, in the text area of the reviews themselves.
- Once we finalize the list of rating questions, and begin to add reviews, we can't go back and change it. Due to the architecture of the software, once a rating "scheme" has been put into place, we have to live with it. So, I want to work with everyone here to ensure that we come up with a very good list the first time.
Some examples of rating "questions" to get us started in this discussion (these are the ones we have so far for Oil Paints):
- Color Variety: How would you rate the variety and variation of colors within this line?
- Consistency/Workability: How would you rate the consistency and workability of this paint? If too oily or too dry, give it a lower rating. If nicer and creamier, give it a higher rating.
- Color Quality: A simple measure of how much you like the colors in this line.
- Container/Tube Quality: How would you rate the construction quality of the container/tube? Does the tube develop holes easily? Caps hard to get off? Does the tube have a well-designed cap with a slot in it?
- Pigment Quality: How would you rate the "quality" of the pigment used in this line of paints? Good purity of color?
- Opaque Covering Ability: How would you rate the "covering" power of the opaque colors in this line?
- Transparency Rating: How would you rate the "transparency" qualities of the more transparent colors in this line?
- Value for the Money: How would you rate the value for your money when buying this product? Keep in mind that even poorly made products can have a high rating here, if they are priced right. The same can be said of expensive products if they are worth the price.
Thanks to everyone in the oil forum here for their hard work on developing this criteria. It isn't final yet, but as you can see, it is pretty far along.
Note: some of the rating questions above have applicability to other, similar categories as well (Acrylic Paints, for instance). Feel free to leverage the questions on the list above! No sense in re-inventing the wheel!
Ok ... now with all of that said:
I already begun loading up vendors and products. More are being added daily.
If you'd like to participate in the development of this exciting new tool, please read/post in one of the other category-specific threads in this forum. They will be marked as "Sticky", and will appear at the top. These will be our "focus group" threads for this project. As we reach conclusions in each of the focus group threads, they will be made "unsticky", and we'll add other areas to work on.
If your area of interest or knowledge isn't yet listed as a sticky thread, it will be at some point (if the category is deemed big enough). If you'd like to ask a general question, or throw out something for general discussion, just start up a normal thread in this forum.
I look forward to rolling this thing out finally - it has been a long wait!