I would personally argue against any involvement of any tool that generates any amount of content for you when constructing a question, an answer, or even a comment. I think the question of whether these tools are currently able to produce 100% accurate information is a horse that's on it's last breath, but even if in the future they provide completely accurate information I would still maintain this position.
My reasoning has nothing to do with correctness, but with copyright. Whether the information is truthful or not, when you supply content to Stack Exchange you are implicitly claiming that the content is your own and you grant a license to Stack Exchange to use it on this website, as well as others to use it so long as they provide correct attribution to you as the supposed author. By utilizing these tools you are in essence not only committing (in my opinion) some form of fraud, but there is as far as I know no way to trace the actual underlying rights of what particular human being was the creator of the generated content that was used to train the model. This can lead to some really messy situations where the content licensed to Stack Exchange by you is in fact protected by copyright by the person who originally wrote the book, or authored the thesis, or whatever original information was used to train these language models.
Whether the developers of these tools at that time have enough lawyers to defend their work is one thing, but allowing it here in any form I feel is a mistake. You cannot license content that is not your own work, and believing that by just modifying a few words here and there is enough to create a derivative that you can license to others legally is extremely suspect.
By popular demand, here are some direct quotes found by doing some searches on the Internet.
To answer these questions and understand the legal landscape surrounding generative AI, [we] spoke to a range of experts, including lawyers, analysts, and employees at AI startups. Some said with confidence that these systems were certainly capable of infringing copyright and could face serious legal challenges in the near future.
Ryan Khurana, chief of staff at generative AI company Wombo, says most companies selling these services are aware of these differences. "Intentionally using prompts that draw on copyrighted works to generate an output [...] violates the terms of service of every major player but enforcement is difficult," and companies are more interested in "coming up with ways to prevent using models in copyright violating ways [...] than limiting training data."
"The Supreme Court doesn’t do fair use very often, so when they do, they usually do something major. I think they're going to do the same here," says Gervais. "And to say anything is settled law while waiting for the Supreme Court to change the law is risky."
Generative AI systems might generate output media that infringes on existing copyrighted works. We think that this is an unlikely accidental outcome of well-constructed generative AI systems, though it remains possible due to overfitting or developers' intentions. In such cases, however, the proper solution is to entertain infringement suits for the outputs (with appropriate defenses, including fair use, available) as a court would for human-generated works.
GPT-3 is using millions of documents (many of them copyrighted) to learn and build a model. When using GPT-3 the output is original text but how much of it can be said is derived from copyright holders? Nobody knows.
Quote 6 - This one is actually an argument from OpenAI itself in response to a legal request from US Government entity, so of course they argue for fair use. But they do highlight the absolute uncertainty about this.
Under current law, training AI systems [such as its GPT models] constitutes fair use, [but] given the lack of case law on point, OpenAI and other AI developers like us face substantial legal uncertainty and compliance costs.
I'm actually not going to link to the sources directly, if you want to find out where on the internet to find these quotes, you can start by going to your favorite search engine and typing a search query into the search box. Relying on something or someone other than yourself to tell you what to search for or where to find it I believe in this very specific context would be cheating, and completely against the spirit of this answer since you're not paying me for my time.