2

The meta post advertizing site evaluation seems to give very vague guidelines on how exactly we should judge a give Q&A when voting. It merely says:

Let's try to look at this site through the eyes of someone who's never seen it before, and see how we stack up against the rest of the 'Net.

and

Run a few Google searches to see how easy they are to find and compare the answers we have with the information available on other sites.

This seems pretty general. However, from some discussions on podcasts, I have formed an impression - perhaps an incorrect one, that the desired methodology for evaluating a specific question should be:

  • Search the internet for answers to that question
  • Compare non-SE answers to the (presumably, the best) SE answer.
  • If SE answer(s) provide information materially better than the rest of internet, then vote "Excellent".
  • If SE info is about the same, vote "Satisfactory".
  • If the answer is worse than what can be found on internet (for example pure opinion, no sources, wrong), vote "Needs Improvement".

However, based on some patterns in the last Politics.SE evaluation, this is clearly not how everyone sees what the voting rules should be.

So, the high-level question is "What are the rules on how to decide how to vote on a specific question?".

Specific sub-questions are:

  1. If the Q&A contains a poor question but a great (better than the rest of Internet) answer, should the quality of the question itself affect the vote (e.g., assuming that the same answer to an improved question rates "Excellent", would the vote be expected to change to something lower for the same answer to a poorer question?)

  2. If the Q&A contains a great answer (better than the rest of Internet) alongside one or more poorer answers, should the quality of the worse answers affect the evaluation vote (e.g., assuming that the same answer if it was the sole answer rates "Excellent", would the vote be expected to change to something lower for the same Q&A that also have 2 awful answers?)

    • Sub-sub question: would the answer to this depend on the relative votes of a great answer vs. poor answers? E.g. rate "Needs improvement" if the great answer is at +1 and poor one at +5; and rate "Excellent" if vice versa?

  3. If the Q&A contains the same quanity of information as can be found - cumulitevely - on the internet, but carefully compiled from more than one internet source, does that qualify as "stacking up excellently against the rest of the Net"?

4
  • As a side note, while I deeply appreciate Yannis's view of this, and he may very well be correct, I am hoping ideally for an answer strongly based on "official" SE information - e.g., either existing blog post/podcast transcript/meta post by SE member, or if none such exist, an answer by Robert C. or anyone else on SE involved with site evaluations.
    – user4012
    Jun 13 '13 at 4:01
  • Some relevant info (with feedback from Community Managers) here: Strategies for handling beta community quality evaluations
    – yannis Mod
    Jun 13 '13 at 4:12
  • This is a subjective, qualitative self- assessment of your performance in this subject space. Our intention isn't to provide a formulaic procedure for which to derive your "grade." This is about stepping back from the analytics — questions/day, percent answered, voting, reputation — and saying simply "how are you doing?" I didn't post this as an 'answer' because the folks here are free to interpret (and answer) your questions as they see fit. But here's my latest article on the subject: What does the site self-evaluation measure?
    – Robert Cartaino
    Jun 13 '13 at 14:25
  • @RobertCartaino - There seems to be two components to self-evaluation that I'm struggling to fit together - the holistic "whole site trend" thing, and specific evaluation voting on specific questions. The latter one seems more quantitative, and the one I am curios about. The way it sounded in the podcast, the per-question evaluation is meant to somehow help guide holistic evaluation (e.g. help you notice patterns/trends), perhaps I didn't get it correctly?
    – user4012
    Jun 13 '13 at 15:02

You must log in to answer this question.

Browse other questions tagged .