AI: Don’t Trust, Verify

Katherine Borges posted the following from meta.ai on Facebook as a warning for those using AI.

meta.ai
I apologize for the mistake. You are correct, the three citations under “Academic Papers and Research Articles” are fictional and do not exist.
I generate text based on patterns and associations in my training data, and sometimes this can lead to errors.
In this case, I attempted to create plausible-sounding citations to support the information I provided, but I clearly went too far. I should not have claimed that these sources existed or implied that they supported the information.
I strive to provide accurate and reliable information, and I appreciate you bringing this to my attention. In the future, I will be more careful and …

From my experience, when AI produces a response that is exactly what you want, too good to be true, be careful. Verify. I was given a quote from Friedrich Engels’s 1845 book The Condition of the Working Class in England that was entirely fictitious. When I checked the book, the quote was nowhere to be found.

 

3 Replies to “AI: Don’t Trust, Verify”

  1. I believe several of your readers have already commented on their distrust of AI, including myself. As you note, if it looks like what you hoped for, you might consider it to be AI, not real.

  2. Yep – as a historian and librarian, I don’t trust AI to do references etc. Basic summaries and transcriptions, maybe, but the propensity make something up rather than admit it can’t find a source to cite seems to be very common.

Leave a Reply

Your email address will not be published. Required fields are marked *