Should we poll the website users to corroborate/disprove the "bias" claimed by some users ? Or assess representativity?

Question

This site makes good efforts to maintain some neutrality, efforts which I think are successful.

However, sympathisers of the US republican party claim there is a pro-US democratic party bias on this website:

Are liberals incapable of seeing their own bias?

The liberal bias on this site is getting out of control

There would be a good way to either corroborate or disprove this, it is to ask for the political leaning of the users. I was thinking about asking several questions on large issues (liberal/controlled/planned economy; liberal/conservative society; environmentalism; secularism; equality before the law; ... )

Should we do something like this ?

I am aware that the conclusions of this poll would not mean much on the quality or bias of the answers written. It would indicate which biases we should be careful about. It would also indicate the representation of different political trends which could be useful. The poll could eventually be used again if someone claims there is a bias against his/her political opinion.

My personal empirical opinion on this is that it is indeed hard on this site to be a GOP sympathiser. But they are facing something different than a liberal bias against them. Comes to mind the fact that non-US users may be generally less close to the GOP, or that their party's positions are more easily challenged by factual studies.

Answer 1

Part of the problem is that this site is part of SE network; and as such it's a victim of SE's atrocious Hot Network Questions (HNQ) process, whereas a swarm of people (from all over SE but numerically, mostly from StackOverflow) with often very little knowledge of a site's subject area descend on a specific question deemed "hot" (as distinct from "good").

On SE in general (and this site as well), this often results in bad quality posts being upvoted way too high, because those visitor swarms contain thousands of new users who have enough reputation points to upvote due to 100 rep association bonus from SO - but neither enough rep (125 required) nor expertise nor interest to recognize and downvote bad posts.

Now, add that up to the fact that SO is heavily populated by more left leaning people, for a variety of demographic reasons, and you get an effect of a large left wing vote sweeping through some of the Politics.SE site's questions every day. Often, these happen to be on polarizing topics, which attract both interest AND poor content AND partisan interested voting all at the same time.

(as a side note, this effect isn't unique to Politics.SE - the same happens when politically polarizing questions arise on other soft-subject sites like Parenting, Workplace, Academia, main SE meta, etc... - Politics is just naturally more likely to have such posts due to our subject matter).

• Side note: since I was challenged on the scope of the issue for Politics, here's some data. As of right now, August 31 2017:

  • there are 5!!! questions on this site that are on HNQ at the same time. Usually it's at least 2-3 even on slow days.

  • Among the last 50 questions since August 24, there were 14!!! (33%) that were on HNQ.

As such, frankly, polling website "users" is beyond useless, since core site users who stick around productively constitute a small minority of all users who vote on HNQ posts and who drive a significant amount of the site's bias.

The only way to fix this problem is either fixing HNQ, which SE clearly has no interest in doing, or excluding Politics.SE from HNQ (which again they have no interest in doing AND would harm the site slightly by lowering traffic), or someone organizing swarms of right-leaning visitors who have enough rep to upvote (not likely to happen since StackOverflow is rather unique as a site).

Answer 2

This is most probably a bad idea. Most probably you could not learn anything valuable from such a poll because:

• Not everybody will take part
• Not everyone will answer truthfully
• The community changes all the time and may be different tomorrow from today

The attempt to learn something about a potential bias of the politics.stackexchange community by polling is futile. Don't waste your time!

Politics is indeed a highly subjective topic and the demography of this community might not reflect the demography of the general population (although there are no direct access barriers) but then again the Q&A system of the stackexchanges and in particular the voting relies somewhat on impartiality and objectivity of the voters (answers are sorted by votes, votes influence reputation which regulates privileges). This is probably a fundamental problem and I'm sure there are already countless debates about this on this meta (not familiar enough with this stackexchange to list canonical examples).

Finally, I think that there are quite some people who are actually able to separate their own political views from their voting behavior and vote based on merit and usefulness for a greater audience.

Answer 3

There would be a good way to either corroborate or disprove this, it is to ask for the political leaning of the users. I was thinking about asking several questions on large issues (liberal/controlled/planned economy; liberal/conservative society; environmentalism; secularism; equality before the law; ... )

Ignoring other problems, remember that this is an international site. By the standards of most European countries, the Democrats are a right wing party on many issues. They fit right in with people like Angela Merkel from Germany or David Cameron of the United Kingdom, members of the actual right wing parties in their country. But in the United States, the Democrats are the left wing party.

Beyond that, when it comes to Donald Trump, there are a number of people who voted for Trump because the alternative was worse. In particular, Trump polls badly with college graduates. But this site is marketed most heavily to programmers who either have college degrees or are mostly similar to those who do have college degrees. So even if we can properly measure right vs. left wing by US standards, it's not a great proxy for Trump.

Answer 4

First rule of SE Questions is that should be constructive and based on an actual problem you face. So if the bias is proved or disproved what is the constructive purpose you wish to achieve?

Then ask that question. "How can we best overcome X?" or "We may have an issue Y that is causing problem X, what can we do to assess if we need to make changes to address this?"

As it stands, while I believe there is a US Leftist bias here, I am not convinced that it is a problem with well asked questions. To me the problem only becomes an issue when they question is asking for a subjective evaluation if something is good or not. This is because good to a right leaning person is often the opposite of a left leaning person when it comes to politics.

So as long as we stop bad subjective questions like that I do not see the need to address the issue. But if you do, then figure out what you think the problem is and ask a question about that problem, not a solution to an issue that may or may not be a problem.

Answer 5

Overcoming Bias seems to be the topic. I would not take me out from being biased, or even look down on certain political direction. Fact is, that all buildings of truth are founded in the acceptance of basic truths, similar to dogmatic trust decisions. This implies, that all content follows an intended design.

When concerning a topic you reach at different conclusions than others, this might come from different basic settings. Really enriching is to reflect these settings; it happens when you speak with others on eye-level and it helps to understand and estimate other conclusions. So, I understand the purpose of meta-communities as place of reflection.

Representativeness: From prologue, no idea of an objective measure or assessment. Constraint is, that you can't force others to enter a reflective conversation. Best outcome is, when you are able to identify unwilling people, in case leave them alone, and instead work with others.