We had a meta question that discussed the use of GPT-3/AI to provide answers on this site and the answer appears to be that we should not allow it.
Use of ChatGPT to provide answers
Does this stance also apply to questions that use GPT-3/AI to help answer the question? Is it okay to use information/text from GTP-3/AI to provide an answer or should we encourage people to not use it at all?
I would personally argue against any involvement of any tool that generates any amount of content for you when constructing a question, an answer, or even a comment. I think the question of whether these tools are currently able to produce 100% accurate information is a horse that's on it's last breath, but even if in the future they provide completely accurate information I would still maintain this position.
My reasoning has nothing to do with correctness, but with copyright. Whether the information is truthful or not, when you supply content to Stack Exchange you are implicitly claiming that the content is your own and you grant a license to Stack Exchange to use it on this website, as well as others to use it so long as they provide correct attribution to you as the supposed author. By utilizing these tools you are in essence not only committing (in my opinion) some form of fraud, but there is as far as I know no way to trace the actual underlying rights of what particular human being was the creator of the generated content that was used to train the model. This can lead to some really messy situations where the content licensed to Stack Exchange by you is in fact protected by copyright by the person who originally wrote the book, or authored the thesis, or whatever original information was used to train these language models.
Whether the developers of these tools at that time have enough lawyers to defend their work is one thing, but allowing it here in any form I feel is a mistake. You cannot license content that is not your own work, and believing that by just modifying a few words here and there is enough to create a derivative that you can license to others legally is extremely suspect.
By popular demand, here are some direct quotes found by doing some searches on the Internet.
Quote 1:
To answer these questions and understand the legal landscape surrounding generative AI, [we] spoke to a range of experts, including lawyers, analysts, and employees at AI startups. Some said with confidence that these systems were certainly capable of infringing copyright and could face serious legal challenges in the near future.
Quote 2:
Ryan Khurana, chief of staff at generative AI company Wombo, says most companies selling these services are aware of these differences. "Intentionally using prompts that draw on copyrighted works to generate an output [...] violates the terms of service of every major player but enforcement is difficult," and companies are more interested in "coming up with ways to prevent using models in copyright violating ways [...] than limiting training data."
Quote 3:
"The Supreme Court doesn’t do fair use very often, so when they do, they usually do something major. I think they're going to do the same here," says Gervais. "And to say anything is settled law while waiting for the Supreme Court to change the law is risky."
Quote 4:
Generative AI systems might generate output media that infringes on existing copyrighted works. We think that this is an unlikely accidental outcome of well-constructed generative AI systems, though it remains possible due to overfitting or developers' intentions. In such cases, however, the proper solution is to entertain infringement suits for the outputs (with appropriate defenses, including fair use, available) as a court would for human-generated works.
Quote 5:
GPT-3 is using millions of documents (many of them copyrighted) to learn and build a model. When using GPT-3 the output is original text but how much of it can be said is derived from copyright holders? Nobody knows.
Quote 6 - This one is actually an argument from OpenAI itself in response to a legal request from US Government entity, so of course they argue for fair use. But they do highlight the absolute uncertainty about this.
Under current law, training AI systems [such as its GPT models] constitutes fair use, [but] given the lack of case law on point, OpenAI and other AI developers like us face substantial legal uncertainty and compliance costs.
I'm actually not going to link to the sources directly, if you want to find out where on the internet to find these quotes, you can start by going to your favorite search engine and typing a search query into the search box. Relying on something or someone other than yourself to tell you what to search for or where to find it I believe in this very specific context would be cheating, and completely against the spirit of this answer since you're not paying me for my time.
I recently flagged an answer on Stack Overflow that, in retrospect, I strongly suspect was written by GPT3.
It was basically saying, 2+2=5, about a less-used Python function, along with a claimed output printout showing that to be so.
Very authoritative-sounding, very clean and good looking post, by someone who had never written any answers before. The thing is, the name of the function is such that one interpretation was that it would behave the way the poster claimed it did.
Had I not used the function myself, I would have been fooled. As it is, I pasted the one liner in Python, confirmed that, of course, the output was as expected, not as claimed and flagged the answer.
Even if you know the subject, it would be easy to miss a nugget of complete nonsense that completely misinforms the readers of that answer later on, especially in a longer stretch of text where arguably GPT3 could be of assistance otherwise.
I think this question becomes easier to answer if we split up "is it okay to use information from GPT?" and "is it okay to use text from GPT?".
It's very clear that these services should not be used as sources of information. I'm sure there will be a day when the creators of these services start claiming their AI will only provide completely true information, but until then answers (or questions) that rely on them should be closed/downvoted/deleted/etc.
Over the last few weeks I've seen questions on a variety of SE sites whose entire premise was based on something that ChatGPT made up, like this one here on Politics or this one over on the History site. These questions get closed quickly and sometimes downvoted pretty heavily, so users across StackExchange seem to agree that questions based on claims from AI aren't worth answering.
However, if we treat these language models as just tools that generate natural sounding text, then they could be useful for answering questions in a way that still fits the spirit and purpose of StackExchange. Trilarion's answer to the other question gives an example of how someone could use ChatGPT to assist with writing an answer:
I could imagine reading a question, thinking about how I would answer it, then asking ChatGPT how they would answer it, then using that answer (the parts I actually agree with) to improve my own answer.
That seems like an acceptable way to use these tools: let it generate the basic outline of an answer, but rewrite it to fill in the facts and style yourself. If the person submitting the answer provides the actual information then they've done the hard and useful part of the work, and such AI-assisted answers should be upvoted/downvoted like any other answer.
Overall though, it certainly would be easier to just ban these things entirely: the benefits of making posts a bit easier to write don't outweigh the negatives of all of the completely useless questions and answers they're able to generate. Hopefully as the hype dies down and people get more familiar with what these services can and can't do we'll see better use of them.
