Wikipedia talk:Large language models/Archive 3

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
Archive 1 Archive 2 Archive 3 Archive 4 Archive 5 → Archive 7

Detection tools

1) Can someone link me to the best detection tools? It's mentioned on one of these talk pages but I can't find it.

2) Should we add a paragraph about detection and a link to some of these tools to WP:LLM? Should document this somewhere. If not WP:LLM, then maybe a separate informational page. –Novem Linguae (talk) 09:57, 23 February 2023 (UTC)

See summary —Alalch E. 12:37, 23 February 2023 (UTC)
@Novem Linguae: Here are a few more. Some of them were mentioned at Wikipedia:Village_pump_(policy)#Wikipedia_response_to_chatbot-generated_content.
If we have something really reliable then we could mention it here. But I don't think that this is the case and this is also a rapidly developing field, so the what is reliable now may not be reliable in a few months. But having a link to an essay where they are discussed would make sense. Phlsph7 (talk) 13:34, 23 February 2023 (UTC)
Tracked in Phabricator
Task T330346
This data is for phab:T330346, a PageTriage ticket exploring the idea of "Detection and flagging of articles that are AI/LLM-generated". Feel free to weigh in there. –Novem Linguae (talk) 23:59, 23 February 2023 (UTC)
Currently, the available detectors are being regarded as "definitely not good enough" to use for important decisions, due to frequent false positives and false negatives (and are often intended for outdated models like 2019's GPT-2). This includes OpenAI's own tool for detecting ChatGPT-generated text. This situation may change of course, especially if OpenAI goes forward with their watermarking plans. (Turnitin announced last week that they have developed a detector that "in its lab, identifies 97 percent of ChatGPT and GPT3 authored writing, with a very low less than 1/100 false positive rate" and will make it available to their customers "as early as April 2023." But even it's worth being skeptical about whether they can keep up these levels of sensitivity and specificity.) Regards, HaeB (talk) 06:45, 24 February 2023 (UTC)
If/when such detection and flagging is implemented, probably the best course of action would be not to forbid them, but maybe create some "quarantine" space (similar in spirit to WP:Draft), so that it can be properly edited and verified into a valid Wikipedia article. This allows for more adaptability because these models are surely going to grow and get better at their task, and in time they could be a real help for Wikipedia's growth. What do you think? Irecorsan (talk) 16:06, 10 March 2023 (UTC)
LLM's issues with generating inaccurate but fluent-sounding information, combined with forging citations, makes them pernicious. They take experienced editor time and expertise to spot and remove. I currently see them as a net negative to the encyclopedia. –Novem Linguae (talk) 23:15, 10 March 2023 (UTC)

Merger proposal

@CactiStaccingCrane and JPxG: I propose merging Wikipedia:Using neural network language models on Wikipedia into into this page. I think the content in the other article can easily be explained in the context of this one. QuickQuokka [⁠talk • contribs] 19:51, 5 March 2023 (UTC)

Hello QuickQuokka and thanks for your suggestion. However, I think the purpose of these two pages are very different: the page here is a draft for a policy while the page Wikipedia:Using neural network language models on Wikipedia is an essay that includes many very specific suggestions on how LLMs may be used. It's probably not a good idea to try mix these purposes up and to include all those suggestions and ideas in a general policy. Another point is that some of these suggestions are controversial, for example, which uses are safe and which ones are risky. A policy should represent a very general consensus and not include controversial claims. Phlsph7 (talk) 21:13, 5 March 2023 (UTC)
See also this discussion this discussion. My suspiscion is that if we merge the two people will be continually trying to get rid of examples as "not to do with policy" so it's probably good to leave the other page where it can have less aggressive editting. Talpedia (talk) 21:17, 5 March 2023 (UTC)
Yeah, whilst these two pages are broadly on the same subject, they’re coming at it in two completely different and incompatible directions. They can certainly feed off each other in some ways, but merging them will devalue both. Letting both exist, and letting them converge or diverge organically as time passes, would seem to serve Wikipedia as a whole and interested editors in particular better than stopping everything to try to meld them together now. We’re not on a deadline here. — Trey Maturin™ 02:23, 6 March 2023 (UTC)

Bullet points: crazy idea, or...?

What if we rewrote the "Using LLMs" section into bullet points? We have quite a lot of policies already, and I've already experienced times when I read through a policy again, and see things I missed on the first read-through. I still feel this policy could be a lot simpler; and a bullet-point policy would be somewhat unusual, but refreshing. Currently, it's basically equivalent to JPxG and Aquillion's idea of "you should follow policy, so your LLM-assisted edits should also follow policy", from the village pump discussion. That doesn't merit this length. Any thoughts? DFlhb (talk) 12:55, 24 February 2023 (UTC)

I'd start with the rewrite, rather than the formatting (which will depend on the rewrite). Starting with the "Writing articles" subsection: I think the key point is to be familiar with the relevant sources for the section of the article being modified (which includes finding them as needed), so that you can review any proposed change for its reliability, fidelity to sources, appropriate degree of prominence, and non-infringement of copyright for its sources. From a "how-to" perspective, the editor needs to be aware of the terms of use and licensing conditions from the program being used to generate the text, and then there are a couple of suggestions about how a program can be used. isaacl (talk) 17:33, 24 February 2023 (UTC)
I think, in principle, having a slim policy that focuses on the most important points is a good idea. I'm not sure if we can condense the whole section into one regular-sized list and there is also the danger of overdoing the simplification process. Phlsph7 (talk) 18:24, 24 February 2023 (UTC)

Here is a draft of a rewrite for the guidance portion of the "Writing articles" subsection. I've left out the last two "how-to" paragraphs:

Output from a large language model can contain inaccurate information, and thus must be carefully evaluated for consistency with appropriate reliable sources. You must become familiar with relevant sources for the content in question, so that you can review the output text for its verifiability, neutrality, absence of original research, compliance with copyright, and compliance with all other applicable policies. Compliance with copyright includes respecting the copyright licensing policies of all sources, as well as the AI-provider. As part of providing a neutral point of view, you must not give undue prominence to irrelevant details or minority viewpoints. You must verify that any citations in the output text exist and screen them for relevance.

isaacl (talk) 22:49, 1 March 2023 (UTC)

@Isaacl: Thanks for this draft. I think it does a good job at summarizing the main points of the first three paragraphs. In the discussion at #Removal of the section "Productive uses of LLMs", we came to the conclusion that the policy should include some very general but short points on how LLMs can be used (create new content, modify existing content, and brainstorm ideas). The latter two points are discussed in the last two paragraphs. The first three paragraphs summarized by you were about the first point, i.e. creating new content. If we want to keep this idea, we could modify your draft by putting the sentence "LLMs can be used to create new content on a subject matter." in front of it and by keeping the last two paragraphs. Phlsph7 (talk) 13:51, 2 March 2023 (UTC)
I omitted the current first sentence for conciseness, since all of the potential uses of large language models should handled in the same way as described in the draft rewrite. I think the last two paragraphs are repetitive. If we want to keep a bit of info on uses, I suggest instead changing the current first sentence to cover the various uses. For example: "Large language models can be used to copy edit or expand existing text, to generate ideas for new or existing articles, or to create new content." isaacl (talk) 16:55, 2 March 2023 (UTC)
One sentence that I'm missing is "LLMs themselves are not reliable sources and websites directly incorporating their output should not be used as references". This was discussed at #Subsection:_Citing_LLM-generated_content. But otherwise, the suggestion together with your new first sentence on uses works fine for me. Phlsph7 (talk) 17:17, 2 March 2023 (UTC)
I didn't realize you had eliminated the citation section when you summarized its content into a single sentence. I suggest a second paragraph with something like, "All sources used for writing an article must be reliable, as described at Wikipedia:Verifiability § Reliable sources. Before using any source written by a large language model, you must verify that the content was evaluated for accuracy." isaacl (talk) 17:38, 2 March 2023 (UTC)
I believe the best place to reference LLM-written sources is the lead, since it would be out of place anywhere else. And ideally that lead sentence would link to an appropriate subsection of WP:V or WP:RS that addresses this in more detail. DFlhb (talk) 18:23, 2 March 2023 (UTC)
I think it would better to restore a separate section for using sources written by large language models, and then the draft paragraph I wrote could be placed in it as the only paragraph. The draft links to the "Reliable sources" section of the Wikipedia:Verifiability page. isaacl (talk) 18:37, 2 March 2023 (UTC)
I just added a subsection to WP:RS that addresses it; I think it belongs there more, and there was already an ongoing discussion to modify WP:RS to address LLMs. DFlhb (talk) 18:40, 2 March 2023 (UTC)
Including a link to the relevant section in WP:RS is a good idea. But we may not need a full section on it in our policy. I think the lead is the wrong place because it only summarizes what is already found elsewhere. Since citing references is part of writing articles, it seems that the best place to mention it is the section "Writing articles". Phlsph7 (talk) 19:19, 2 March 2023 (UTC)
Are we talking about references made-up by LLMs, or sources (i.e. articles) written by LLMs? I maintain that we'd just cause confusion by addressing the latter here, while the former should definitely be addressed here. DFlhb (talk) 19:30, 2 March 2023 (UTC)
I don't think putting a paragraph on sources written by a program should be the lead paragraph of the "Writing articles" section. The lead paragraph should set context for the rest of the section, but in this case, the rest of the section is about writing text in an article. To keep the distinction between using a program to write text for a Wikipedia article, and using such text from elsewhere as a source, I think it would be more clear to have a separate section. I agree the heavy lifting should be done on the "Reliable sources" page. isaacl (talk) 21:33, 2 March 2023 (UTC)
To be clear, by "lead" I meant the lead of the policy, not of the section. It would allow this policy to prominently address LLM-generated sources, while linking to the appropriate section of WP:RS for more details. DFlhb (talk) 22:10, 2 March 2023 (UTC)
Reading through the nutshell summary and the lead paragraph, I think they set the scope of the page to discuss how large language models can be used to assist in editing. Thus it feels a bit out of place to have a sentence on evaluating sources. Perhaps a "See also" note would be a better fit. isaacl (talk) 22:48, 2 March 2023 (UTC)
Surely less out of place than having a whole section on it? Though putting it in See also is totally fine DFlhb (talk) 23:25, 2 March 2023 (UTC)
I personally don't think so. Including something in the lead of a guidance page generally implies it covers a key aspect of the page that will be explained more fully later on. isaacl (talk) 23:41, 2 March 2023 (UTC)
From what I can tell, there is consensus that isaacl's draft of the subsection "Writing articles" is fine. The remaining issue is how to deal with the topic of "Using sources with LLM-generated text": should it be covered in the lead (DFlhb), in its own section (isaacl), in the section "Writing articles" (Phlsph7), or as a See-also link (DFlhb & isaacl). If it is not to be covered in the section "Writing articles", then I would prefer it to have its own section (or better: subsection). Maybe it should also include a short mention that LLMs themselves (not just websites created by them) are not reliable sources. This should be obvious from the various warnings about their output but it doesn't hurt to be explicit here. Phlsph7 (talk) 05:58, 3 March 2023 (UTC)
I think there are interesting epistemological questions that arise. Generally speaking, a citation to a specific non-primary source is evaluated in part for reliability by the degree and past history of editorial oversight. From this perspective, the author is not a reliable source in isolation; the whole editorial process has to be considered. (This can vary within a single publication. There are articles in the New York Times travel section, for example, that are very promotional in tone.) For a database-lookup site such as Baseball Reference, what makes it a reliable site versus "My big site of stats" is the track record of editorial oversight on the entered data, and a transparent mapping from this to the output of any search queries. For now I think it's fairly clear that, for example, an information panel displayed by a search engine (which is machine-generated) isn't a reliable source in itself, and instead the citations in that panel have to be evaluated. I think there is potential for that line to get to get blurry in future. Nonetheless, I don't think a blanket statement for machine-generated content is the best approach, as it's not the technology itself that establishes reliability, but the entire context of how it is used. In general, all sources are unreliable until their context is evaluated, no matter the provenance of the source. isaacl (talk) 18:01, 10 March 2023 (UTC)
I guess it'll be hard to figure out a way out of this as long as it's just the three of us discussing it. How about a two-part RfC, so we can get wider input?
Should we modify our existing policies, to explicitly address the use of LLM-generated sources as citations?
Should such policy language be added to WP:LLM, or WP:RS?
(The idea of changing our policies to address this may be itself controversial, hence the first question).
If people come to a consensus that it should be covered here, then I'm totally fine with giving it a full section, which seems (I think?) relatively acceptable to everyone, and was what we had a few weeks back. It avoids mixing up edit conduct and sourcing issues in the same section, and we can give the new section a shortcut (WP:LLMSOURCE?) that we can point people to easily. DFlhb (talk) 19:15, 10 March 2023 (UTC)
My personal feeling is that the rest of this guidance page doesn't need to be held up by a) should the text that is currently within the "Writing articles" section be moved to another location; and b) should an addition be made to discuss if a program can be considered to be a reliable non-primary source? (The second question I think could be particularly deep rabbit hole, as I think it would mean reaching a baseline understanding on evaluating reliability.) I think we should be able agree upon an interim approach of either keeping a version of the current text in the "Writing articles" section, or removing the text entirely. isaacl (talk) 22:12, 10 March 2023 (UTC)
Your (b) is different from my first RfC question, which only asks whether the section we're discussing should exist at all (I think it should, but others disagreed elsewhere). I feel like we're just accumulating mutual misunderstandings at this point. Let's skip the RfC, add the section you drafted below, and ask for feedback on WP:LLM as a whole at WP:VPI, so we get more feedback before formally propose it for adoption. DFlhb (talk) 22:46, 10 March 2023 (UTC)
Yes, my questions are different as they're specific to the effect on this guidance page. Just trying to avoid RfCs for other guidance pages being a bottleneck for progressing on this page. Sure, if there is agreement on moving forward with the draft text, then that would be good. isaacl (talk) 23:27, 10 March 2023 (UTC)
I went ahead and implemented isaacl's suggestions since there seems to be a rough agreement that it constitutes an improvement over the current version. It may not be the perfect solution but it's not a good idea to get bogged down too much by small details. Maybe we can take this as a start and incrementally improve it. Feel free to revert it if I misinterpreted the discussion. Phlsph7 (talk) 08:51, 12 March 2023 (UTC)

Arbitrary break (draft)

To put everything into one place, here is my current draft rewrite:

Writing articles

Large language models can be used to copy edit or expand existing text, to generate ideas for new or existing articles, or to create new content. Output from a large language model can contain inaccurate information, and thus must be carefully evaluated for consistency with appropriate reliable sources. You must become familiar with relevant sources for the content in question, so that you can review the output text for its verifiability, neutrality, absence of original research, compliance with copyright, and compliance with all other applicable policies. Compliance with copyright includes respecting the copyright licensing policies of all sources, as well as the AI-provider. As part of providing a neutral point of view, you must not give undue prominence to irrelevant details or minority viewpoints. You must verify that any citations in the output text exist and screen them for relevance.

Drafts

...

Using sources with LLM-generated text

All sources used for writing an article must be reliable, as described at Wikipedia:Verifiability § Reliable sources. Before using any source written by a large language model, you must verify that the content was evaluated for accuracy.

isaacl (talk) 21:38, 2 March 2023 (UTC)

Note my suggestion on re-creating a section for using sources with LLM-generated text was prompted by the suggestion of moving it to the lead. If there is agreement on proceeding with the rest of the rewrite at this time, the two sentences I drafted could be placed for the time being in the "Writing articles" section as a second paragraph. isaacl (talk) 17:19, 10 March 2023 (UTC)
That sounds good to me. Phlsph7 (talk) 17:25, 10 March 2023 (UTC)

Further fusion of image with language processing in GPT-4 merits revisiting of umbrella policy Wikipedia:Computer-generated content

I'd like to suggest that with the latest developments in GPT-4 which include image processing, the lines between language models and other kinds of AI like Dall-E and Midjourney and such continue to blur. I don't think the distinction is going to hold up, and frankly with such advancements, I no longer think it is credible to doubt that this is true "artificial intelligence" (which it has been asserted not to be by several editors on this talk page) or will not evolve into such in the near future. I will again recommend something along the lines of my draft at Wikipedia:Computer-generated content. —DIYeditor (talk) 07:57, 19 March 2023 (UTC)

We already had a similar discussion about the name of this page. It was inconclusive, see Wikipedia_talk:Large_language_models/Archive_1#Requested_move_28_January_2023. I agree that the current name is not perfect. An important reason for having a distinct policy is that the problems raised by generating or modifying text are very different from the problems raised by other forms of computer-generated content, such as images. By the way, as I understand it, GPT4 is only able to analyze images but not to create them, in contrast to Dall-E and Midjourney. Phlsph7 (talk) 09:12, 19 March 2023 (UTC)

Writing articles and copyright licensing concerns

Regarding this edit: I don't think the added text is saying anything not already covered by Compliance with copyright includes respecting the copyright licensing policies of all sources, as well as the AI-provider. Perhaps just a pointer to Wikipedia:Large language models and copyright could be included? That page is of course already linked in the "Relevant polices and associated risks" section. isaacl (talk) 21:35, 19 March 2023 (UTC)

I agree, the current sentence already seems to cover the main point. I went ahead and removed the added explanation. Phlsph7 (talk) 08:54, 20 March 2023 (UTC)

Nitpicks

Attribution

Furthermore, LLM use must be declared in the edit summary and in-text attribution is required for articles and drafts.

This might want to be "inclusion of text generate by LLMs". If you type a search term into google then you are using sentence embeddings via BERT, and then google often using LLM based QA models to find a relevant section of text. Grammar suggestions using the broadly used Frammarly will be generated by an LLM.

I think your point is valid. I adjusted the text to apply to cases where new text is generated or existing text is modified (e.g. summarize or paraphrase). Phlsph7 (talk) 14:10, 23 February 2023 (UTC)
LLM-based assistance is coming to Google docs, Microsoft office tools, and Edge browser. Other editors and browsers will likely follow when offline LLM libraries catch up with the development. LLM-assisted features will be the default feature in editors. Based on this, there should be better defined which kind of use will require attribution. My opinnion is that least summarizing and copyediting existing article text or own text should be out of the scope of the requirement. -- Zache (talk) 08:03, 19 March 2023 (UTC)
Summarizing and copyediting can lead to important changes to the content, such as introducing copyright problems, biases, and false claims. This is very different from a mere grammar check, like grammarly, which merely highlights problems without making changes on its own, or from BERT, which provides search engine results. Phlsph7 (talk) 09:23, 19 March 2023 (UTC)
Sure, but editor is responsible to check the changes what is in the edit. --Zache (talk) 12:29, 19 March 2023 (UTC)
Attribution is required by WP:PLAGIARISM. See the discussion at Wikipedia_talk:Large_language_models#Attribution_to_OpenAI. Phlsph7 (talk) 13:07, 19 March 2023 (UTC)
WP:PLAGIARISM doesn't require in-text citations on material from free sources, such as existing text from Wikipedia articles or on the texts written by editor. Editing existing text with LLM:s doesn't create new copyright to it. However, the requirement for adding in-text citations makes it practically impossible to use LLM:s for summarising and copyediting this kind of texts. -- Zache (talk) 16:06, 19 March 2023 (UTC)
My gut feeling is that in-text attribution should be used for LLM-generated prose and a mention in the edit summary is sufficient for copyediting etc. –dlthewave ☎ 16:36, 19 March 2023 (UTC)
@Zache: Plagiarism is different from copyright violation and also applies to free sources. From WP:PLAGIARISM: ...even though there is no copyright issue, public-domain content is plagiarized if used without acknowledging the source. Phlsph7 (talk) 17:02, 19 March 2023 (UTC)
and WP:PLAGIARISM says that free sources doesn't require in-text attribution. -- Zache (talk) 17:07, 19 March 2023 (UTC)
I don't think plagiarism is the issue here. My reasoning for including in-text attribution is that it's required by ChatGPT's TOS and media sources generally disclose LLM involvement at the top of the page, so it might be a good best practice for us to follow as well. –dlthewave ☎ 17:12, 19 March 2023 (UTC)
It's ChatGPT TOS (The role of AI in formulating the content is clearly disclosed in a way that no reader could possibly miss, and that a typical reader would find sufficiently easy to understand. [1]). However, it is improbable that this kind of requirement for disclosing the use of LLM:s would be included in office tools. Also, there are no such requirements in open-source LLM tools which you can run on your computer, though open-source models are currently limited compared to GPT-3 or GPT-4. -- Zache (talk) 08:00, 20 March 2023 (UTC)
@Zache: Sorry, you are right. I just saw the passage: Add in-text attribution...unless the material...originates from a free source. But see also: If text is copied or closely paraphrased from a free source, it must be cited and attributed through the use of an appropriate attribution template, or similar annotation, which is usually placed in a "References section" near the bottom of the page (see the section "Where to place attribution" for more details). Apparently the attribution is not required in the same paragraph but a template in the article is needed, i.e. an edit summary ist not sufficient. Phlsph7 (talk) 17:26, 19 March 2023 (UTC)
Also, inline attribution with refs/notes is possible. I don't oppose attribution itself, but in-text attribution will make it impossible to use LLM:s in use cases where it would be suitable (summarisation, translations, copyediting, for example), as in-text attribution would make the result text complex and less readable. -- Zache (talk) 08:18, 20 March 2023 (UTC)
@Dlthewave: It depends on what you mean by copyediting. If your LLM adds a few commas and fixes obvious spelling mistakes, I agree. But copyediting can also mean reformulating and paraphrasing sentences or longer passages as well as rearranging material. We could exclude things like trivial spell-checking. The current sentence we are talking about is in the lead section. I don't think it's the right place for a detailed discussion on certain types of copyediting that do not need intext attribution. But maybe this could be discussed in one of the sections. Phlsph7 (talk) 17:02, 19 March 2023 (UTC)
We could restrict the in-text attribution requirement to "non-trivial" changes. Something like Furthermore, LLM use to generate or modify text must be declared in the edit summary and in-text attribution is required for non-trivial changes to articles and drafts. Phlsph7 (talk) 17:29, 19 March 2023 (UTC)
I meant non-trivial changes. For example using the LLM for shortening the text to half or using it to pick up the most relevant parts and then manually editing the text or using LLM to formulate coherent text from bulletpoints. Changes which are comparable to editorial changes if they would be made manually by user. The thing here is that if the source material is known and there is limited amount of it then user can say rather well if the result is correct or not and it is part of normal edition process of the text. -- Zache (talk) 08:28, 20 March 2023 (UTC)
I've implemented the suggestion. I wouldn't categorize "shortening the text to half" as a trivial change. But for now, it may be good enough if we can agree on a rough formulation even if we may disagree on its exact implications. Phlsph7 (talk) 08:42, 20 March 2023 (UTC)
I meant that shortening the text to half is a non-trivial change, and the editor should be able to use LLM tools when doing it without mandatory in-text attribution in the article text. --Zache (talk) 09:08, 20 March 2023 (UTC)
It seems I misinterpreted your reply. Is your suggestion that no type of LLM use needs in-text attribution? I thought you only meant that certain forms should be excluded from this requirement, which is addressed by my suggestion. Is the term "non-trivial" too strong for you in this context? Phlsph7 (talk) 11:13, 20 March 2023 (UTC)
I think that if the LLM generates the text from information stored in the model then it would require in-text attribution as it is not possible to know where the information comes from. For example
  • Prompt: who is Joe Biden?
    • ChatGPT Answer: Joe Biden is an American politician and the 46th President of the United States. He served as Vice President under President Barack Obama from 2009 to 2017 ...
In cases where LLM manipulates text in the prompt and does not try to add new information then the inline attribution should be enough. Example:
  • Prompt: summarise in english the text: "Barack Hussein Obama II (laus. [oʊˈbɑːmə]; s. 4. elokuuta 1961 Honolulu, Havaiji, Yhdysvallat) on yhdysvaltalainen demokraattipoliitikko, joka toimi Yhdysvaltain 44. presidenttinä kahden kauden ajan vuosina 2009–2017. "
    • ChatGPT answer: Barack Obama is an American Democratic politician who served as the 44th President of the United States for two terms from 2009 to 2017, born on August 4, 1961 in Honolulu, Hawaii.
-- Zache (talk) 12:52, 20 March 2023 (UTC)
What about cases where it does not add information but remove information in such a way that it introduces a bias (for example, when a critical opinion is not mentioned in a summary)?
What about cases where it adds information even though you told it just to copyedit? Or if it even changes what was originally expressed? Have a look at the following response I just got from ChatGPT, where both of these things happened:
Prompt: Please copyedit the following text to fit an academic writing style: knowledge is justified true belief. But Gettier says no.
Response: "Knowledge is justified true belief" is a widely accepted definition in epistemology. However, the philosopher Edmund Gettier contested this notion. In his influential paper, "Is Justified True Belief Knowledge?", Gettier presented counterexamples that challenged the sufficiency of this definition for knowledge. As a result, his argument prompted a significant shift in the epistemological discourse, leading to the development of alternative theories of knowledge.
Phlsph7 (talk) 13:27, 20 March 2023 (UTC)
I think, in principle, we are on the same page: some changes require in-text attribution while others only require inline attribution or a general template. But it's difficult to decide where to draw the line. Phlsph7 (talk) 13:45, 20 March 2023 (UTC)
I think that, the responsibility of checking the output of LLM tools and how they are utilized falls on the editors. This includes the detection of creation of new biases because of removed texts. The crucial factor is the ability to verify the generated content, which can be achieved if the information of the generated content is the same as or a subset of the LLM input (ie prompt). If so, then inline attribution can be used. If it is not possible or unsure, then editor should use in-text attribution to say to reader where the information comes from. -- Zache (talk) 15:02, 20 March 2023 (UTC)

Suggestion and responses

I think the basic idea is good. It could be included in the subsection "Declare LLM use" in the following way.

Every edit which incorporates LLM output must be marked as LLM-assisted in the edit summary. This applies to all namespaces. For content added to articles and drafts, attribution inside the article is necessary. If the LLM was used to generate new information, in-text attribution is required to make the source explicit. If it was only used to modify information provided by the editor (like copy editing and summarizing), in-line attribution is sufficient, for example, in the form of an explanatory footnote. If an LLM by OpenAI was used, this can also be achieved by adding the following template to the bottom of the article: {{OpenAI|[GPT-3, ChatGPT etc.]}}. Additionally, the template {{AI generated notification}} may be added to the talk page of the article.

The next step would then be to adjust the lead accordingly. What do you think? Phlsph7 (talk) 18:14, 20 March 2023 (UTC)
I think it is good and catches the idea what I tried to explain. Thanks. -- Zache (talk) 18:40, 20 March 2023 (UTC)
I disagree with setting LLM-specific policies on in-text attribution. The same general policy applies: it's appropriate when describing a particular source's viewpoint. However, other than cases where an LLM is being discussed, I don't see a need to attribute a statement within text to an LLM. LLMs are correlation-driven and don't have viewpoints. If there is some fact within the LLM output that is being included, editors should be locating an appropriate reliable source that can be independently verified.
From a maintenance point of view, I think it would be easier to put specific instructions on templates in a guideline or procedure page, rather than within a policy page. isaacl (talk) 21:33, 20 March 2023 (UTC)
If I have understood the purpose of in-text attribution for LLM-generated content is to make readers aware of potential errors, as these models can generate text with minimal effort but the text may contain hard-to-spot mistakes. If the created content contains references and is thoroughly checked by the editor, in-text attribution is redundant, but I am not sure if it's feasible to ensure editors consistently review LLM-generated text at the necessary level. Do you have any ideas on how the guideline should be written to achieve this? Zache (talk) 10:18, 21 March 2023 (UTC)
Taking the examples from Wikipedia:Citing sources#In-text attribution, Wikipedia articles should not contain text such as "Program X argues that, to reach fair decisions, parties must consider matters as if behind a veil of ignorance," or "Humans evolved through natural selection, as explained by Program X." The content must be attributed to appropriate reliable sources. If it can't, it shouldn't be added. An in-text attribution is inadequate. isaacl (talk) 15:50, 21 March 2023 (UTC)
I think there's a disconnect in how "in-text attribution" is defined elsewhere and how it is being used on this page. The link target provides examples of citing the source within the sentence in the article that is quoting a specific piece of information. However the current version of this page says this can be achieved by adding the following template to the bottom of the article: which is not the same thing. isaacl (talk) 21:43, 20 March 2023 (UTC)
I read it so that the notification at the bottom of the article was meant to be an alternative for inline attribution, but it cannot replace the in-text attribution if in-text attribution is required. Though, as you say there is a disconnection in text and the bottom of the page attribution can be understood to be an alternative for both. -- Zache (talk) 09:56, 21 March 2023 (UTC)
The section in question doesn't mention inline attribution at all. It first mentions attribution in the edit summary, and then in-text attribution. Plus a template at the bottom of the page isn't a substitute for inline attribution, since the attribution would no longer be associated with a specific passage of text, which is the point of inline attribution. isaacl (talk) 15:54, 21 March 2023 (UTC)
I was referring to ... If it was only used to modify information provided by the editor (like copy editing and summarizing), in-line attribution is sufficient, for example, in the form of an explanatory footnote ... [2] -- Zache (talk) 16:29, 21 March 2023 (UTC) (comment edited --Zache (talk) 16:37, 21 March 2023 (UTC))
OK; well, that text isn't currently present in that form in the "Declare LLM use" section. And even in that form, the two don't seem to be readily substitutable for each other. If a template at the bottom is good enough, then I don't think requiring an in-line reference to a footnote is necessary. isaacl (talk) 16:50, 21 March 2023 (UTC)

I just had a look at some of the policies. According to Wikipedia:Plagiarism#Copying_material_from_free_sources, attribution through a template is sufficient for free sources, i.e. in-text attribution is not required. According to Wikipedia:Citing_sources#In-text_attribution, in-text attribution is required specifically for statements of opinion. As far as I can tell, the current policies would not require in-text attribution for LLM output in general. But since this is supposed to be a new policy, we could add that requirement. The question is whether we should add it for certain cases, like generating new information instead of merely modifying pre-existing information.

One of isaacl's points was that the current version of the section "Declare LLM use" erroneously equates in-text attribution with adding a template. So, in any case, the current version has to be changed. Either we only require attribution using a template and leave out the "in-text" or we go with something like the suggestion above by distinguishing cases that require in-text attribution from others that don't. Phlsph7 (talk) 17:07, 21 March 2023 (UTC)

This thread is disorganized and hard to read, so I've created a new discussion thread: "We've gone off the rails on attribution". DFlhb (talk) 20:50, 21 March 2023 (UTC)

Classes of users

Any specific application of LLMs is only tolerated, not recommended. It is reserved for experienced editors, who take full responsibility for their edits' compliance with Wikipedia policies.

This pretty much isn't true. Are you actually suggesting we would ban inexperienced editors who use an LLM without any issues. Feels bit like we are making up rules and "classes of users". Also do we actually mean "tolerated" rather than "allowed".

I tried to reformulate it to not imply different "classes of users". As for "tolerated" vs "allowed": "tolerated" focuses a little more on the negative side and the dangers but, otherwise, there should not be too much difference in swapping the two word. Phlsph7 (talk) 14:23, 23 February 2023 (UTC)
Just to note that LLMs are used in spellcheckers which is recommended use case. Another recommended use case for non-native English speakers is to check how written (English) text translates back to native language using machine translation. Ie. there is highly beneficial ways to use LLM:s, but it is not just a direct text generation. -- Zache (talk) 07:26, 19 March 2023 (UTC)
You are right. I tried to modify the text to restrict it only to the more problematic cases. Phlsph7 (talk) 09:27, 19 March 2023 (UTC)

Entities and people associated with LLM development are prohibited from running experiments or trials on Wikipedia.

Not sure about this? What does it actually mean? If I create a bot to fact check edits follow the bot guidelines and it is of use to people is this actually going to be removed from wikipedia. I'm suspicious that this rule is already being violated. Am I for example allowed to create a user script with tools that help people edit wikipedia articles using LLMs? Perhaps an auto-fact checker based on the sources already included in the article using a QA type model?

Also what about LLM developers make them especially prohibited from running trials as opposed to say professor of law.

I feel like this might be covered by other policies which we can refer like the bot and user script policy to rather creating especially restrictive rules.

I tried to reformulate the sentence. The basic policy here is WP:NOTLAB. This should be mentioned in some form. See if the new version works better for you. Phlsph7 (talk) 14:35, 23 February 2023 (UTC)

General Use versus inclusion of generated material is potentially a general distinction that we should be clear on throughput the article... unless you want to add BERT was used in this model to every page. Talpedia (talk) 13:47, 23 February 2023 (UTC)

Ah awesome, thanks for the changes, may well review later. Reading this back I realise I came off a little brusque - I think I was in my "analytic policy mode". Talpedia (talk) 14:40, 23 February 2023 (UTC)
That's exactly what is needed at this point IMO, so thank you for these actionable points of critique. —Alalch E. 15:24, 23 February 2023 (UTC)
All good proposals. DFlhb (talk) 12:47, 24 February 2023 (UTC)

Workability

I have a feeling the "must be declared" provision will be a real headache. First, obviously, because it's being built into Edge and other mass-consumer products (and not just spellchecking, but wholesale generation), that will make LLMs widely accessible, to users who don't even know what an LLM is.

And second, because we simply have no way of reliably identifying LLM-generated text in case editors fail to voluntarily disclose. Even the model-specific detectors (e.g. ChatGPT detectors) are terrible, and there are now so many different LLM models,[3] including open-source ones that can be tuned and retrained and can run on consumer hardware,[4][5] that detection is becoming a total pipe dream.

I just don't see any way we can enforce this. Thoughts? DFlhb (talk) 10:09, 19 March 2023 (UTC)

I view it as something similar to WP:UPE: It provides guidance for the vast majority of good-faith editors and also gives us a specific rule that we can point to when someone causes problems by not complying. Sure, it's difficult to actively patrol and a lot will probably slip through the tracks, but the most problematic cases often either unwittingly out themselves or are so obvious that we can apply WP:DUCK. It's a good tool to have in our toolbox. –dlthewave ☎ 12:50, 19 March 2023 (UTC)
I agree with dlthewave. It would be great to have some reliable way to check and enforce it. But currently, we don't and maybe we never will. The policy is helpful for good-faith editors. It may also help avoid repeated blatant violations even though many small-scale violations are probably not caught. Phlsph7 (talk) 13:14, 19 March 2023 (UTC)
I think there some sort of policy filter we can define that can capture people saying "write me a section on X" and having an LLM spit out four paragraphs with spurious sources, while leaving the edge cases along. It feels kinda similar to copyright infringement or really close paraphrasing to me. Spitballing a bit... getting people to include the original output and how they changed it might be interesting (though gets unwieldy if there is a *lot* of back and forth) - I'm reminded of systematic reviews reviews where they tell you precisely what they search for. I guess a question is what getting people to disclose is *for*, in the case of citing sources it helps people check, extend and interpret your work. I can see how knowing where you were working in "prompt space" could be useful. I wonder if providing good integration where we keep track of the prompts themselves might be one approach - though a little stalkerish. The other reason is to judge people based on their use of LLMs and expose their work to large amounts of scrutiny, or play WP:FETCH. I guess there is a general value in looking at edits with LLMs to understand the good and the bad, of course how open I am to this as an editor depends how much I trust the process to be open minded and reasonable. One thing that one really starts wanting is explainable models that can tell you *how* they made their decisions... but I don't think the models are really there. Talpedia (talk) 17:48, 19 March 2023 (UTC)
The clear case where attribution is required is when it's mandated by the licensing requirements of the program (for example through end user licensing agreement). Leaving that aside, the question is what is the motivation for requiring editors to identify that a program helped them with writing the text? Is it to advise readers to be wary of the change? If so, then perhaps the content in question should be flagged in the article. Is it to draw other editors to review the changes? If so, then we may need some kind of queueing system or watchlist filter to highlight these changes. But if the use of writing assistant programs becomes widespread, then a lot of most articles will just be flagged/queued/highlighted. As I mentioned during the village pump discussion, if content written (in whole or in part) by programs is indistinguishable from human-written content, then there aren't really any policies that can be put in place that won't also affect human-written content. isaacl (talk) 22:04, 19 March 2023 (UTC)
It is to draw other editors to review the changes, and to verify (if weakly) that editors who use these tools have at least become aware of the policy governing use of said tools. An awareness check of sorts. —Alalch E. 22:08, 19 March 2023 (UTC)
The key question is if this check will rapidly become obsolete, and all editors will have to be made aware of how they should make use of writing assistant programs for content submitted to Wikipedia. But there's no easy way to know in advance. isaacl (talk) 22:27, 19 March 2023 (UTC)
I am quite sure that it will become obsolete at some point—not quite so rapidly so as to be useless from the beginning, and only useful for a short time. I get what you are saying. —Alalch E. 22:30, 19 March 2023 (UTC)
One simple motivation is to ensure that edits comply with existing Wikipedia policies, such as WP:PLAGIARISM. The alternative would be to change these policies. Another motivation is that LLM output often sounds good on the first impression to a non-expert but has many deeper flaws (for example, hallucinated claims and invented sources). These flaws are there but can be difficult to detect. Attribution helps reviewers pay extra attention to such issues. For example, inventing sources is very rare for human editors. In this regard, it LLM output is not indistinguishable.
It seems to me that the basic underlying argument against a policy on LLMs is flawed. The underlying argument presented in various guises is roughly the following: (1) X can't be reliably detected; (2) if X can't be detected then there shouldn't be a policy against X; (3) therefore, there shouldn't be a policy against X. This argument is a fallacy, as references to other policies and guidelines with similar problems, such as WP:SOCKPUPPET and WP:UPE, show. Phlsph7 (talk) 22:36, 19 March 2023 (UTC)
Well, editors identifying that they used a program to help write the text won't ensure edits comply with existing Wikipedia policies by itself. The key part is reviewers paying attention to these issues. I'm not arguing your (1) + (2) therefore (3). I'm saying if it becomes widespread, ways to ramp up paying attention to the problems of poor writing have to be put into place, regardless of anything else. isaacl (talk) 22:59, 19 March 2023 (UTC)
I agree with your last point. Having editors self-declare their LLM usage is one step but more steps may be needed if the usage becomes widespread. Phlsph7 (talk) 08:59, 20 March 2023 (UTC)
Yeah a proportion of people are "good" and will try to follow the rules (so evasion being possible is not a reason for there being no guidelines). I think a more reasonable argument is that the rules shouldn't be perverse or difficult to follow because this causes people to ignore them or potentially all the rules. If everyone is using LLMs for small editing tweaks and grammar all the time it might be silly to say don't use LLMs. Talpedia (talk) 11:57, 20 March 2023 (UTC)

We've gone off the rails on attribution

We've gone off the rails on the question of attribution. Here are some thoughts:

  1. Inline disclosure is too onerous. See my example. Too much clutter, and the maintainability burden is too high.
  2. Under OpenAI's terms of service, we cannot exempt any use cases (summarizing, copyediting). All uses must be disclosed.
  3. Inline disclosure (i.e. in a footnote) clearly violates OpenAI's terms of service, because many users don't check footnotes. That doesn't meet: "clearly disclosed in a way that no reader could possibly miss".
  4. Non-inline disclosure at the end of the page, with {{OpenAI}}, violates OpenAI's ToS for the same reason.
  5. IMO, the only way to comply with OpenAI's ToS is to add an LLM WP:TOPICON to affected articles.
  6. Wikipedia policy cannot mandate compliance with OpenAI's ToS. It may suggest the use of a topicon, but can't mandate it.
  7. The only thing we should mandate is disclosure in the edit summary. That helps us scrutinize these edits, and comply with WP:PLAGIARISM, the same way we disclose copying within Wikipedia through edit summaries.

DFlhb (talk) 20:47, 21 March 2023 (UTC)

I don't think OpenAI's term of service mean anything for wikipedia and don't view it as our job to do their bidding (though see Tortious interference, but I'm not sure this tort is used match). They can't retain copyright in the US and whether a user chooses to violate openai's policy is a choice. Also... openai is very much [not the only game in town](https://medium.com/@martin-thissen/llama-alpaca-chatgpt-on-your-local-computer-tutorial-17adda704c23). Talpedia (talk) 23:56, 21 March 2023 (UTC)
Wikipedia requires any reuse of content must be under terms compatible with CC BY-SA, and that it be legally used. Thus it does mandate following licensing requirements. I disagree that having an icon at the top of the page is the best way to flag that portions of the content have been generated by a program (it might not even be a desirable way, given differences in how this may be handled with different skins).
Going back to the motivation for such flagging, aside from licensing requirements: to trigger more review, it's probably enough to flag edits (with possibly new mechanisms to help editors find them). If it's for readers, hatnotes on an article or section level are probably more maintainable for editors and more useful for readers. isaacl (talk) 01:56, 22 March 2023 (UTC)
You link to a section on "using copyrighted work from others", but LLM output is not copyrightable (per U.S. Copyright Office), and OpenAI asserts no copyright anyway, so this doesn't apply. There are no licensing requirements. The OpenAI ToS is strictly between the OpenAI and the LLM user, and that's all the topicon is meant to do: it serves no function for Wikipedia, but helps LLM-using contributors to comply with OpenAI's ToS, in the most unobtrusive way possible. Not only would a hatnote like {{Sect1911}} be extremely obtrusive, it's also a very non-standard use of hatnotes (for example, that template is used on precisely 0 articles). DFlhb (talk) 02:12, 22 March 2023 (UTC)
Yes, I saw the hatnote is no longer used; I do recall seeing it in the past. For better or worse, a hatnote is a better match for what is seen on other web sites than a icon tucked into the corner.
On the separate (though also related to attribution) issue of the OpenAI terms of service, it's true enough that in a previous discussion regarding the use of photographs, I believe a consensus of editors agreed that if a photographer published a photo contrary to a signed agreement not to do so, that had no bearing on whether or not the photo should be used on Wikipedia. isaacl (talk) 02:53, 22 March 2023 (UTC)
I'm still of the opinion that attribution is the best practice in the interest of transparency to the reader as well as maintenance purposes, regardless of what LLM terms or our other policies may or may not require. This should go at the top of the section or the head of the article depending on how much of it is LLM generated. Since this would be creating a new policy instead of existing ones, I think it would be fair to have a standalone RfC on this specific issue. –dlthewave ☎ 02:08, 22 March 2023 (UTC)
Agree it would require an RfC, and I'd absolutely favor mandating that topicon for all LLMs regardless of ToS. Would oppose anything more intrusive than a topicon, since that would deface a bunch of articles, with (IMO) minimal maintenance benefit in practice. DFlhb (talk) 02:34, 22 March 2023 (UTC)
The disclosure requirement TOS is Openai specific. For example, there is no requirement for disclosure on Google's generative AI terms of use (Generative AI Additional Terms of Service, Generative AI Prohibited Use Policy). I tried to find if there would be similar documents in Microsoft Office 365 copilot, but I was not able to find anything. --Zache (talk) 06:03, 22 March 2023 (UTC)
Before we start an RfC, it might be a good idea to clarify where exactly the lines of agreement and disagreements lie. Maybe we can solve some of the issues in this process. I'll try to draw the picture, please correct me if some of these points are wrong. There seems to be consensus that, at a minimum, disclosure in the edit summary is required. The question is whether more is necessary and in which cases. According to DFlhb, this may be enough. According to Zache, isaacl, dlthewave, and myself, some attribution on the page is needed for articles and drafts. This could happen in the form of a template on the bottom, a banner on the top, in-line attribution for explanatory footnotes, or in-text attribution. A second question is whether our policy should require that editors respect the ToS of their LLM provider. In the case of OpenAI, this would mean that The role of AI in formulating the content is clearly disclosed in a way that no reader could possibly miss. It seems to me that there is consensus that, strictly speaking, Wikipedia is not required to enforce OpenAI's ToS.
Some relevant policies that were mentioned before in regard to attribution:
  • From Wikipedia:Plagiarism#Copying_material_from_free_sources: If text is copied or closely paraphrased from a free source, it must be cited and attributed through the use of an appropriate attribution template, or similar annotation, which is usually placed in a "References section" near the bottom of the page (see the section "Where to place attribution" for more details). This would mean that some form of attribution on the page is necessary.
  • From Wikipedia:Copying within Wikipedia: When copying content from one article to another, at a minimum provide a link back to the source page in the edit summary at the destination page and state that content was copied from that source. This would mean that attribution on the page is not necessary. However, this policy may not apply here since using LLM output is not the same as copying within Wikipedia.
Phlsph7 (talk) 08:49, 22 March 2023 (UTC)
According to Zache, isaacl, dlthewave, and myself, some attribution on the page is needed for articles and drafts Yes, and I used to be in that group too, and I say we've all gone mad. We need to stop massively misreading an old "content guideline" (not even a policy), which never discusses LLMs, into requiring anything. This discussion would be infinitely more productive if we discussed what we think should be required, and stopped acting like such a requirement already exists. WP:PLAGIARISM applies to published sources. Authored by someone else. LLMs are a writing tool, they're not "someone", and they're not published, and they're not sources! You keep referencing in-text attribution, but that is literally: According to ChatGPT, Steve Jobs died in 2011. In-text attribution violates WP:V, because ChatGPT can never be a source. Similarly, PLAGIARISM defines inline attribution as Cite a source in the form of an inline citation. But again, LLMs are never valid sources. WP:PLAGIARISM was never written with LLMs in mind. That second editing guideline, WP:CWW, exists purely because it is required by the CC BY-SA license that Wikipedia contributions are released under. LLMs are not copyrightable and are not released under any license (copyright license ≠ ToS). Again, misapplied. DFlhb (talk) 11:41, 22 March 2023 (UTC)
On a side note, I suggest it would be better not to label anyone "mad". isaacl (talk) 16:49, 22 March 2023 (UTC)
'Twas meant as an expression, not a label. Everyone here is cool (including you!). I just think we've gotten lost in minutiae. IMO the most productive next step is to brainstorm ways to keep track of LLM texts while being minimally obtrusive, similar to my topicon suggestion. Templates feel clumsy. Another idea is to integrate mw:Who Wrote That? into the default interface, since it neatly shows the edit summary (e.g. "LLM-assisted edit") corresponding to each insertion, without adding any clutter for readers. DFlhb (talk) 17:46, 22 March 2023 (UTC)
Sure, it's just not an expression I appreciated, since attempts to discuss cutting out guidance covered by other policies and guidelines have been opposed, and so the only choice available was to try to build consensus amongst those engaging in discussion. Regarding disclosure, the question is for whom is this being done? If there is consensus that it's for editors and not readers, then I think flagging the edit may be sufficient. isaacl (talk) 20:51, 22 March 2023 (UTC)
Then I apologize. And indeed, the intended audience is the key question. The topicon is meant for readers, and for editors who wish to comply with LLM ToS, while the edit summary and my WhoWroteThat suggestion are meant for all editors, to aid maintenance. DFlhb (talk) 21:21, 22 March 2023 (UTC)
Key issues are the intended audience, how controversial or commonly accepted the use of LLMs is, and how similar issues are treated on Wikipedia. Similar issues include how attribution is treated in other cases, like free sources, where attribution happens either after the sentence for short passages or at the bottom of the article if more material is included (Wikipedia:Plagiarism#Where_to_place_attribution). Maybe in the future, LLMs will be seen and used as commonly accepted writing tools. But currently, this is not the case. Because of this, it may be better to err on the side of caution as far as attribution is concerned and also make this accessible to readers and editors. If all we have is an edit summary then finding out whether an article incorporates LLM output would be a very tedious endeavor that involves going through pages and pages of version history of a page to read the edit summaries one by one. Phlsph7 (talk) 08:53, 23 March 2023 (UTC)
I do not personally have strict requirements for disclosing texts created using LLM as a tool. Generally, I believe that LLM tools will become very common so such requirements will not age well. However, if somebody will want them I don't oppose either as long the writing articles using them is possible. Generally, I am a little bit worried about how it will affect content quality as most likely so personally, I would like to see that tools would be used to make the text better and not for making a lot of new text. -- Zache (talk) 10:39, 23 March 2023 (UTC)
I believe that LLM tools will become very common so such requirements will not age well My thoughts exactly. Systematic detection/labelling is unrealistic, and our countermeasures should focus on minimizing damage to the encyclopedia's reliability and neutrality. Suggestions beyond topicons and WhoWroteThat are welcome. DFlhb (talk) 11:20, 23 March 2023 (UTC)
This sounds to me like you feel the key issue is better monitoring of all edits, thus requiring new features targeted towards editors, rather than reader-oriented features. One idea to try to help editors focus on specific edits to evaluate is an ability to create their own personal ratings for editors, which could be overlaid onto an article's history or an article to highlight changes associated with an editor rating below a configurable threshold value. Or it could be used as a filter for individual watchlists, or the recent changes page. isaacl (talk) 17:24, 23 March 2023 (UTC)
Leaving aside any specific program and just addressing the general question of program-generated text, I didn't say that attribution on the text is needed by virtue of existing policy. I said that if the goal is to make readers aware of specific passages being written in whole or part by a program, then a notice on the page would probably work best, similar to other sites. The community has to decide if that goal is desirable, and if that involves a notice, the degree of prominence it wants to place on any notice, though. isaacl (talk) 16:49, 22 March 2023 (UTC)
If the software I used to create an image added a watermark with the name/logo/url of that software (and let's assume the software's license says that I may not remove the watermark), and I tried to add that image to an article, I would expect to have it rightly removed. It's an unsightly distraction, and arguably has the result of exploiting Wikipedia for advertising. I think the same considerations apply here. Colin M (talk) 17:54, 22 March 2023 (UTC)

Proof of checking for copyvio

The following discussion is closed and will soon be archived.

By its nature the generative AI quite frequently copies large chunks of text from the sources; this is very noticeable if the prompts define a very narrow scope, typical for encyclopedia. I therefore propose to add a requirement to explicitly describe (on the talk page) the efforts made by the editor to make sure his AI-generated edit had been run through some originality checker. Yes, this will make adding the generated text harder - but making such edits harder to make is exactly what should be done, IMHO. Dimawik (talk) 23:39, 23 March 2023 (UTC)

Does it? I haven't noticed this in my use of these models. In fact, they seem to fairly reliably avoid direct quotations or close paraphrasing (indeed, more reliably than some of our very human editors, as the WP:CCI fellas can tell you). What is your source for this? jp×g 04:53, 24 March 2023 (UTC)
Some time ago I have tried asking the hard questions, the ones that not too many sources are dealing with. The result was (as expected) either a complete gibberish, or something very close to the sources, or, frequently, both. However, per your request I tried to repeat the task now, and the result in these cases is now pure and very wrong bullshit, with no detectable borrowing. The generators now clearly try to avoid copying at any cost, even violating the language rules. Here is one of the responses: The most common mark was a painted blue enamel in center of reverse: XXXX Factory mark, -X- below., the bot clearly is trying to avoid using "underside" at any cost, using numismatic terminology to describe porcelain. So you are right, and I withdraw my proposal. Dimawik (talk) 16:06, 24 March 2023 (UTC)
At present, I think most copyright violations are from people manually copying text from sources. Putting a burden on all editors to try to slow down a relatively small percentage of poor editors wouldn't be a good tradeoff. It's not clear to me that the tradeoff would be significantly better for edits using AI-based writing assistant tools. isaacl (talk) 16:38, 24 March 2023 (UTC)

Writing articles (reprise)

Regarding these edits: no, the sentence in question was not present in the article prior to the edit that trimmed the "Writing articles" section. I don't think it is necessary. The paragraph discusses how an editor must ensure any text they submit must comply with Wikipedia policies. Thus it already provides detailed guidance on what must be done before publishing any text. isaacl (talk) 23:49, 26 March 2023 (UTC)

If the idea is never to paste LLM outputs directly into Wikipedia articles, wouldn't it be best to use those exact same words? If that is not quite the idea, that's different. But is it really not quite? —Alalch E. 01:43, 27 March 2023 (UTC)
I personally don't think that's the idea. I think the idea is you are responsible for making your changes comply with policy, no matter what tools were used to assist with the change. isaacl (talk) 02:54, 27 March 2023 (UTC)
It doesn't say that output needs to be checked before publishing. Editors might as well paste raw output first, and only then evaluate it and resolve deficiencies. But we want them to check it first, right?—Alalch E. 08:57, 27 March 2023 (UTC)
That's already part of complying with policies and guidelines: all edits must comply with applicable guidance when submitted, whether you crafted each word yourself or used a tool to help. isaacl (talk) 17:02, 27 March 2023 (UTC)
Would you like to substitute these specific words with "when submitted" verbiage? I don't think it's super clear without making it explicit somehow. —Alalch E. 22:24, 28 March 2023 (UTC)
I'm not sure what substitution you are proposing? I'm also not clear on what you mean in your second sentence. By their nature, policies and guidelines on editing are applicable when editing takes place. If the guidance on verifiability, reliable sources, copyright, and so forth are being misunderstood in this respect, then there's a disconnect that needs to be resolved in the text or how editors are made aware of the guidance. isaacl (talk) 01:13, 29 March 2023 (UTC)
I suggested that you replace "Never paste LLM outputs directly into Wikipedia articles" with something containg "when submitted". What you are saying is logical, but this is about effectively communicating that the process of applying rigorous scrutiny of outputs for compliance with all applicable policies doesn't start with pasting raw output, and then working from there incrementally applying "rigorous scrutiny" along the way, until it complies, it starts before pressing the publish button so that it is already sufficiently complying when the edit is saved.—Alalch E. 01:40, 29 March 2023 (UTC)
This is true for all editing. I think the message should be "Don't submit any changes to articles that fail to comply with all applicable policies and guidelines," and then we can skip listing all the different ways someone can generate unvetted changes. I believe this is simpler for users to remember. isaacl (talk) 02:04, 29 March 2023 (UTC)
Yes, I know it's true. Okay, I like that sentence, and am interested in seeing it incorporated somehow. —Alalch E. 07:32, 29 March 2023 (UTC)
@Isaacl: got any thoughts on this version: Special:Diff/1147403694? —Alalch E. 18:46, 30 March 2023 (UTC)
I recommend not highlighting any special cases at all. To word it affirmatively: "Every change to an article must comply with all applicable policies and guidelines." isaacl (talk) 20:43, 30 March 2023 (UTC)
@Isaacl: This good? (Note colon).—Alalch E. 20:53, 30 March 2023 (UTC)
Yes. Note since I am watching this page, it's not necessary to ping me for responses in ongoing conversation. isaacl (talk) 21:00, 30 March 2023 (UTC)
Sorry for the extraneous repeated pinging, it resulted from a copy-paste of the last indent block. —Alalch E. 21:02, 30 March 2023 (UTC)
Alalch E., rather than being lost in the trim, it was added in the trim!
as for wouldn't it be best to use those exact same words... you can guess my thoughts on that! DFlhb (talk) 01:56, 27 March 2023 (UTC)
Oh... Did we maybe both copy the words from some earlier revision? I did read your trimmed version, but didn't copy directly from it; it's possible that the words became embedded in my "corpus", and I reproduced them verbatim. I guess I need to provide attribution now. In any case, I thought that "don't paste raw output" is one of the most clear and least disputed ideas. It was certainly present at some point, probably JPxGs orignal version, and for some time after that. I have to admit that I haven't read all of the discussions on this talk page, after a certain point. —Alalch E. 08:57, 27 March 2023 (UTC)
Same as you, I didn't copy it from anywhere, just tried to sum up the draft's essence as best I could, with JPxG's early version front of mind. The identical wording might be a testament to this being common sense; no attribution needed. DFlhb (talk) 17:06, 27 March 2023 (UTC)
Attribution is already provided through the edit history. The original draft contained the sentence "Consequently, LLM output should be used only by competent editors who do not indiscriminately paste LLM output into the edit window and press 'save'," multiple times (and a variant in the lead). isaacl (talk) 17:13, 27 March 2023 (UTC)
(Just want to say that I was joking when I mentioned attribution in the above comment.) —Alalch E. 22:24, 28 March 2023 (UTC)
I think there is a subtle but meaningful distinction between "do not indiscriminately paste LLM output" and "raw LLM outputs must not be pasted directly". In some cases an LLM output is of decent enough quality to be pasted in a draft, perhaps with only minor removals or edits. This is where the discrimination, or rather discernment, of the human editor come into play, in determining how much the raw LLM output needs to be altered in that specific case. Pharos (talk) 01:33, 29 March 2023 (UTC)

User warning templates

I've taken the liberty of creating {{uw-ai1}}, {{uw-ai2}} and {{uw-ai3}} to help in the clean-up that the most recent LLM thread at AN/I has revealed as being required. They're a rewrite of uw-test, which seemed most appropriate. Assistance in creating the template docs and integrating them into our various systems would be very appreciated because I know my own limitations. — Trey Maturin™ 16:29, 1 April 2023 (UTC)

I did some work to integrate them with WP:WARN and started a talk section at the central talk page for all such templates at Wikipedia talk:Template index/User talk namespace#New series: uw-ai. It may be best to comment on these templates there. —Alalch E. 11:56, 3 April 2023 (UTC)

Identification problems

Having just tagged a few drafts as probably AI based on GPTzero, I did what should have been a sensible first step and tried it on an article that was definitely "safe" - Ernest Brooks (photographer), which a) was mostly written by me and b) has had no substantial change since 2020, well before the GPT era.

The lead section on its own was "likely to be written entirely by AI". The lead plus first half was "may include parts written by AI", but it did not highlight the lead - the offending sections were later in the article. The last section was "likely to be written entirely by a human".

I am not quite sure why it's coming to these conclusions, but it does make me worry a bit about how good we're going to be at identifying this stuff. Andrew Gray (talk) 16:57, 1 April 2023 (UTC)

Essentially, the chatbot was/is trained on the Wikipedia corpus; it, of course, will seek to emulate Wikipedia as much as possible. AI detectors are then trained to recognize emulations; there's plenty of error there, as you can imagine, with the variety of quality we have on here, and, most importantly, the original chatbot must seek to evade the detector by definition. We don't have reliable tools for this. Iseult Δx parlez moi 17:15, 1 April 2023 (UTC)
Thanks - that makes absolute sense, but I admit it's pretty demoralising! Andrew Gray (talk) 17:28, 1 April 2023 (UTC)
Just for counterpoint, I tried this on the lead section of Oakland Buddha, which I wrote most of, and it identified that as "entirely human". So, I'm sure we'll get false positives and negatives, but apparently it's not completely inaccurate either. I think we'll have to use it like automated copyvio tools—a good indicator of when something needs a closer look, but also susceptible to errors like when a remote site copies from Wikipedia rather than the other way around. Seraphimblade Talk to me 10:26, 3 April 2023 (UTC)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Large_language_models/Archive_3&oldid=1150369860"