15

There's a proposal on main Meta regarding accepted answers floating to the top in the list of answers. I'll selectively quote parts of CM Nicolas Chabanovsky's question here to get you up to speed with the specifics of the discussion:

Today we changed the way we sort answers on Stack Overflow. We no longer pin the accepted answer (with the green checkmark) to the top of the list of answers. By default, we now sort strictly by votes (descending order by highest score), and the accepted answer's order in the list is based on its score.

We can change the way the engine sorts answers in site settings. We would like to hear from you all if it is something you want to see on your site. (Please let me acknowledge in advance that we will not be able to run a test on each site.)

Please let us know what you think will work best for your site! If you can discuss this question with your community it would be awesome. We are going to collect feedback before the end of September 19th.

How does the Politics community feel about accepted answers floating to the top? Should we follow Stack Overflow's lead of sorting by answer score only?

Feel free to use the answer box here to share specific concerns about answer sorting on Politics. If you want to discuss specifics about the new way of answer sorting then it's probably best to reply to Nicolas' question on main Meta.

7
  • At what point do you think we should post an answer to the question on meta (if at all)? Sep 8 at 20:57
  • 2
    @EkadhSingh-ReinstateMonica I don't think we need to post an answer there to tell them our meta consensus. For that we can simply wait to see which will be the default option. If that option doesn't match our choice then we can add a [status-review] so the CM team is made aware. Of course you're welcome to post on the main meta post if you have other remarks that would be on-topic there.
    – JJJ Mod
    Sep 8 at 21:01
  • There are a couple of posts already suggesting this be site wide and even a suggestion to allow a site to opt out after the change. I think you could go and show your support there with votes.
    – Joe W
    Sep 9 at 0:28
  • Should this be featured? This will affect everybody on the site. Sep 10 at 16:01
  • 1
    @EkadhSingh-ReinstateMonica sure, added.
    – JJJ Mod
    Sep 10 at 16:25
  • I think there was a query that was missed and that was questions where the accepted answer matches the score of the top answer. The reason I mention this is because in those cases it is possible for the accepted answer to be at that top or not to be due to the random sorting of answers with the same score. I don't think it is a big deal but something to remember.
    – Joe W
    Sep 12 at 20:31
  • @JoeW yea, added that query for ties as well as one for cases when the accepted answer is also the (not strictly) highest scoring.
    – JJJ Mod
    Sep 12 at 21:30
21

I looked at the data a bit using this SEDE query. There are 704 questions where the accepted answer is outscored by another answer. In 486 questions the difference in score between the accepted answer and the highest scoring answer is at least 3.

Of those questions where the accepted answer is outscored, in 28 cases the accepted answer has a negative score.

As such, I also think it makes sense to score answers based on score only. While it doesn't happen that often, it prevents askers from prominently featuring a poor answer at the top.


The following query shows questions where the highest scoring answer was accepted and there's at least one other answer. This yields 3122 unique questions. The score difference between the accepted answer and the second highest scoring answer ranges from 0 to 186.

In 191 cases the accepted answer has the same score as another answer. Ties are also addressed in a on main meta which asks that the accepted answer is shown above other answers with the same score.


Let's address another concern that the non-accepted highest scoring answer (i.e. the answer that is not on top now, but will be after the change) is more controversial than the accepted answer. To measure controversy, I will simply use the downvote count.

Using this query, I found that there are 50 such* questions where the highest scoring answer has strictly more downvotes than the accepted answer. There are only 16 *such questions where the highest scoring answer has 4+ downvotes more than the accepted answer.

* such questions being those where the highest scoring answer has a strictly higher score than the accepted answer.

10
  • 7
    I am going to borrow those queries to ask this question on another site or two!
    – Joe W
    Sep 8 at 20:58
  • 3
    Frankly I would have liked a more flexible rule than the binary choice SE is giving us. It's usually the low- or negatively scored & accepted answers that are problematic. It's fairly plausible that 5-votes answer is better than an 8-vote one and was simply contributed later. But not so likely that a -10 (or even a -3) answer is better than those.... (I actually proposed something like this on meta SE last year or so, but it was shot down in flames.)
    – Fizz
    Sep 9 at 0:32
  • 2
    @Fizz yea, I think you'll have more luck suggesting that as an answer on Nicolas' main meta question (compared to your old main meta Q). One minor downside of such a choice is that it's slightly less transparent to new users (why is their accepted answer shown first when mine isn't?). I agree that setting a score threshold makes sense though and it'd be interesting to consider.
    – JJJ Mod
    Sep 9 at 0:39
  • looks like someone beat me to it meta.stackexchange.com/a/369577/278912 And more interestingly meta.stackexchange.com/questions/369568/…
    – Fizz
    Sep 9 at 0:45
  • Can you tell us in how many instances the highest voted answer predated the accepted answer as opposed to the other way around? We all know that early posted answers get voted on more, and that upvoted answers get upvoted more, that highest listed answers get seen more and voted on more, thus compounding these two unfortunate yet unavoidable undemocratic aspects of the 'voting' system on SE. yesterday
  • The only check and balance built into this system is the selected answer. Your reasoning here seems strange. The hundreds of late but excellent and better answers which have been selected by OPs will now vanish into obscurity and not get seen by the vast majority of users, just so the annoying 28 cases where the OP has apparently made a mistake get rectified. In the latter case there is virtually no problem anyhow. Th highest-voted answer is highly visible, effectively pinned as the second post. In contrast there will be huge damage done by selected late answers dropping out of view. yesterday
  • Thanks very much JJJ! Will peruse carefully. yesterday
  • @jjj Hang on there. Let's be careful and honest when we bandy about 'not true' and so forth. I very carefully said routinely. I put that word in specifically for that reason!!! That reason was to exclude from consideration as a protective mechanism the less than 0.001% (0.0001%?) of readers who follow a post on average. yesterday
16

I like the idea of going by score only as for the most part the accepted answer will have the highest score. However there will be times when an answer is accepted because it was what the OP wanted to hear even if it might not be a good answer.

4
  • Yes, but most often it is because an excellent answer came in late, often that wipes the floor with the first posted and so more highly upvoted answer. Under the new proposal, these answers will now all be unpinned and vanish into obscurity depriving readers of their benefit and compounding the undemocratic aspects of the voting system. At least at the moment, the highest voted answer appears directly under the accepted one. In the new system the hundreds of outstanding accepted answers will languish at the bottom of the page unseen. yesterday
  • @Araucaria-Nothereanymore. Just because an answer was posted first and accepted right away doesn't mean it is the best answer. Proof of that would be the fact that another answer later on outscores it.
    – Joe W
    yesterday
  • Most accepted answers that are outscored by the highest-voted answer were posted later, not earlier. yesterday
  • @Araucaria-Nothereanymore. And what exactly is the problem with that? If it got enough attention to get accepted it can also gain enough attention to get votes.
    – Joe W
    yesterday
8

I feel like there should be ('should' is a funny word) a hybrid model here which weights the accepted answer as higher scoring than it's actual votes, but does not automatically pin that response. For the reasons discussed, bias and low quality can appear in both accepted and high-scoring responses. SE's model is a little optimistic about human interaction, but to my mind it hits more than it misses.

I feel like a higher scoring answer that manages 125%+ of the votes on the accepted answer should be allowed to dislodge the OP's preference - especially on a site like Politics where askers aren't usually solving a practical problem and so identifying the answer that got them through their impasse doesn't happen so much.

1
  • 7
    Thanks, that's an interesting alternative. Unfortunately that's not something we can do on our own, but you're welcome to suggest it on Main Meta as well (assuming there's no answer that already suggests using a different weight for votes on the accepted answer). Over there it will get more views so it's more likely it will be considered by the CMs.
    – JJJ Mod
    Sep 9 at 19:18
2

I'm not sure that unpinning is really needed here. Answers on politics rarely become "outdated" like they do on SO and the other technical sites, especially not on the same time scales.

Yes, there are lots of cases where there are answers with more votes than the accepted answer, but I don't think that's actually evidence of needing unpinning, but rather is exactly why pinning was done in the first place. If the goal had always been to have the answer with the most votes at the top, there would never have been a reason to develop the pinning feature.

There are a couple of reasons (both of which are somewhat common on this site) why an accepted answer might have fewer votes than a non-accepted one, even if the accepted one really is better quality:

Lower quality answers posted sooner

It's common to get short, so-so answers posted rather quickly after a question is posted. Since SE tends to give more visibility to more recently-posted questions, these will typically attract significantly more upvotes than if the same answer had been posted at the same time as all of the others. And this tendency is magnified by the score sorting resulting in these answers appearing at the top. This effect doesn't actually reflect answer quality at all, but rather just how quickly the answer was posted. Especially when a lot of such answers get posted on a question, this also commonly means that much better quality answers posted a day or two later get few, if any, votes, due to being buried by the sort underneath all of the older ones that already have votes.

The dreaded HNQ effect

When questions hit the Hot Network Questions, they tend to a attract a lot of views while they're on HNQ, very many of which are from users who have no reputation on this site and, thus, can upvote, but cannot downvote. When this happens, answers tend to get a lot of "this agrees with my biases" upvotes, even if the answer isn't particularly good quality (or even if it's just outright wrong.) This is also a problem on Skeptics and other SE sites that tend to get a lot of controversial topics about which people have strong (but not always well-informed) opinions. This commonly leads to poor answers having high scores, as everyone coming from HNQ can upvote them, but only people with sufficient reputation on this site can downvote them.


In both of the above cases, pinning the accepted answer to the top can give more visibility to better answers than ones that may have more votes. In the case of answers that were posted later, but accepted while the question was still relatively active, pinning the accepted answer can also lead to a newer, better-quality answer eventually outscoring the older ones, which often wouldn't happen if it were buried underneath the existing answers.

I do agree that there are cases on this site where the OP selects an answer that is just what they want to hear rather than actually being the best answer, but it is my experience that both of the above cases happen far more frequently here than that.

It would be nice if SE had a better, more nuanced approach to sorting available, but, given only the options of pinning or not pinning, I personally lean toward leaving the pinning. Neither solution is perfect by any means, but, especially on sites where answers don't really become "outdated" often, I do think that pinning the accepted answer solves more problems than it creates.

13
  • 1
    SE is actually coming up with a method that measures recent votes instead of overall votes, so IMO this should take care of some of the problems mentioned in this post. Sep 20 at 16:34
  • @EkadhSingh-ReinstateMonica It would certainly help, but it wouldn't necessarily solve the problems. I'm not sure that it would help the HNQ issue much at all. It could help with the early vs. later answers, but it wouldn't solve that problem either. Part of the problem there is that score voting results in a feedback loop. The earlier answers will likely continue to receive more new upvotes simply because they are already more visible. Pinning an accepted answer, on the other hand, breaks that feedback loop (at least for one answer.)
    – reirab
    Sep 20 at 16:40
  • Fair, so I guess the only question left is whether or not that outweighs the abuse of having biased question askers picking an answer they like. Sep 20 at 16:47
  • 1
    @EkadhSingh-ReinstateMonica Indeed. At least in my experience here, these problems are more frequent than that one, but they're definitely all problems.
    – reirab
    Sep 20 at 16:53
  • Isn't it still better to rely on many HNQ voters compared to one asker? As seen in a query in my answer, 28 askers accepted an answer with a negative score when a (much) higher scoring answer is available. I'm not sure if downvotes are accessible separately in SEDE, otherwise I might attempt a query based on that. How often are answers with more downvotes (i.e. disapproval from site regulars) accepted when there is a less down voted answer with at least the same score?
    – JJJ Mod
    Sep 20 at 17:06
  • @JJJ At least to me, 28 doesn't really seem like all that many in the context of a site that's been around for this long. I forget exactly when this stack launched, but I've been here for almost 7 years. If they all happened since I've been here, that would work out to once every approximately 100 days. I agree that an answer with negative score shouldn't be pinned, but it seems like a much less common phenomenon than the ones I mentioned.
    – reirab
    Sep 20 at 18:59
  • As for one asker vs. many HNQ voters... maybe? Honestly, I hold a pretty low opinion of the quality of HNQ votes on even remotely controversial questions. When those hit HNQ, the votes tend to be more "this agrees with my biases" rather than "this is a useful, reasonable, well-researched answer." There certainly is a problem of people asking questions looking for a particular answer, but I tend to think, just from my observation, that that's less frequent than the HNQ problem. Not that it's uncommon, but it just doesn't seem to be quite as common as the HNQ problem.
    – reirab
    Sep 20 at 19:03
  • Yea, intuitively I'd say that the average asker suffers from the same flaws as the average HNQ voter. It's hard to quantify your problem because it requires comparing answers manually. I guess looking at down votes per answer as I described previously is still an interesting metric that can be automated. As for quantifying your objection, maybe go through a few months' worth of HNQ questions and extrapolate the findings?
    – JJJ Mod
    Sep 20 at 19:10
  • @JJJ Yeah, it's definitely difficult to quantify, since it relies largely on judgment of individual answers. I do agree that the same problem exists with askers, though it doesn't seem to me that it's quite as prevalent on average, likely due to asking a question involving the investment of significantly more effort than dropping an upvote on an HNQ answer on a site where you've earned 0 rep.
    – reirab
    Sep 20 at 19:17
  • I addressed this in my answer. Based on comparing the number of downvotes on the highest scoring answers with the number of downvotes on the accepted answer in cases where the former will replace the latter in the new sorting, I don't think this is a big concern. It seems to be just a handful of cases where the replacement answer is much more controversial than the accepted answer.
    – JJJ Mod
    Sep 20 at 20:30
  • @EkadhSingh-ReinstateMonica Ok, so after they've got rid of this egregious an undemocratic problem, maybe they could think about unpinning accepted questions, no? yesterday
  • @JJJ Part of the problem, though, is that it isn't really possible to tell how many of the accepted answers that now outscore the next-highest answer would have done so if not for having been pinned. If a later, but better answer gets accepted while a question is still active, it often ends up outscoring older answers that weren't as good due to the added visibility it gets. Whereas, that often doesn't happen if it languishes below other answers that were posted sooner.
    – reirab
    yesterday
  • @reirab true, though the of accepting is also recorded. As such it would be possible to see per answer how the score changed before and after accepting. It's a bit complicated because answers can get unaccepted and accepted again, perhaps multiple times, but the information is all there. In many cases no answer gets accepted so it's not something that worries me per se. The way I see it it's mostly a nice gesture from the asker to the answerer that gives them a bit of extra rep and shows appreciation.
    – JJJ Mod
    yesterday

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .