Dynamic Subspace

Tag: Hallucination

  • Generative AI for College Students Series: Watch Out for Fabricated Footnotes and Fake Citations

    an anthropomorphic cat as a professor wearing a suit and orange tie standing in front of a chalkboard in a classroom
    Image created with Stable Diffusion.

    Please keep in mind that new technology like Generative AI (Gen AI) shouldn’t simply make your thinking or work easier, much less take the place of the uniquely singular abilities of human beings to grow cognitively, think creatively, or evaluate critically. If you use Gen AI to simply avoid work, you are doing it wrong. Instead, using Gen AI in the spirit of Douglas Engelbart’s “augmenting human intelligence” and Donna Haraway’s configuration of the cyborg point the way to beneficial heightening of human possibility instead of harmful erasure of the cognitive distinctions of humanity. If you use Gen AI, use it wisely and use it well. This post is the eighth in this series.

    In the science fiction film Blade Runner, replicants—advanced androids indistinguishable from humans—question the nature of their existence. Similarly, students using generative AI tools to write papers may find themselves grappling with questions of authenticity, particularly when it comes to citations. While AI can generate well-formatted citations and quotes, these may be entirely fabricated, leading to academic dishonesty and intellectual confusion.

    The problem arises because AI tools do not “know” the sources they cite. They generate citations based on patterns in their training data, which may include errors, inaccuracies, or outright fabrications. For example, an AI might invent a book title, attribute a quote to a nonexistent author, or misrepresent the content of a real source. These fabrications can be subtle and difficult to detect, even for experienced scholars.

    Imagine a student writing a paper on the ethics of artificial intelligence. They prompt an AI tool to include a quote from a prominent philosopher. The AI responds with a quote that seems relevant and includes a properly formatted citation. However, when the student checks the source, they discover that the philosopher never wrote those words, or that the book cited does not exist. This scenario is not only frustrating but also undermines the integrity of the student’s work.

    This issue mirrors the theme of simulacra in science fiction—copies without originals. In Jean Baudrillard’s theory of simulacra, representations of reality become more important than reality itself. AI-generated citations are simulacra of academic integrity, creating a false appearance of legitimacy. Just as replicants in Blade Runner question their humanity, students must question the authenticity of AI-generated citations.

    To combat this problem, students must adopt a cautious approach to AI-generated citations. They should avoid prompting AI tools to generate citations outright and instead use AI to assist with finding credible sources. For example, a student could ask the AI to suggest relevant authors or topics, then locate and verify those sources independently. This approach ensures that the citations are accurate and legitimate.

    In conclusion, while AI tools can be powerful assistants, they are not substitutes for human judgment and critical thinking. The cyborg student must learn to use these tools selectively, always prioritizing accuracy and authenticity. By doing so, they can maintain the integrity of their academic work and avoid the dangers of fabricated footnotes.

  • Generative AI for College Students Series: Beware Hallucinations and Falsehoods

    an anthropomorphic cat as a professor in a business suit lecturing in front of a classroom with a chalkboard behind him
    Image created with Stable Diffusion.

    Please keep in mind that new technology like Generative AI (Gen AI) shouldn’t simply make your thinking or work easier, much less take the place of the uniquely singular abilities of human beings to grow cognitively, think creatively, or evaluate critically. If you use Gen AI to simply avoid work, you are doing it wrong. Instead, using Gen AI in the spirit of Douglas Engelbart’s “augmenting human intelligence” and Donna Haraway’s configuration of the cyborg point the way to beneficial heightening of human possibility instead of harmful erasure of the cognitive distinctions of humanity. If you use Gen AI, use it wisely and use it well. This post is the seventh in this series.

    In the realm of science fiction, the concept of cyborgs–beings that blend human and machine—often explores the tension between human intelligence and technological advancement. Today, as students increasingly rely on generative AI tools to assist with writing, they are, in a sense, becoming cyborgs of the academic world. These tools can produce coherent, polished text, but they also carry significant risks. One of the most concerning issues is the tendency of AI to “hallucinate,” or generate information that is entirely fabricated, made-up, or misleading. This phenomenon is not just a glitch; it’s a fundamental limitation of how these tools operate.

    Generative AI works by predicting and reassembling language patterns from vast datasets, often drawn from the internet. While the internet is a rich source of information, it is also a breeding ground for misinformation, conspiracy theories, and outright falsehoods. When AI tools process this data, they don’t distinguish between fact and fiction. They simply mimic the patterns they find. The result is responses that may look authoritative and well-written but are, in reality, partly or entirely inaccurate.

    Gen AI presents its responses without qualification or equivocation. The potentially wrong, made-up information in a given response is presented as if it were irrefutable.

    Consider a student working on a research paper about climate change. They prompt an AI tool to provide a summary of recent findings. The AI responds with a polished paragraph that includes specific statistics and citations. The problem? Some of those statistics might be fabricated, and the citations could refer to nonexistent studies. The student, unaware of the fabrication, incorporates this information into their paper, potentially spreading misinformation.

    This issue is reminiscent of the theme of unreliable narration in science fiction. In works like Philip K. Dick’s Ubik, reality itself is unstable, and characters must navigate a world where information is constantly shifting and misleading. Similarly, students using AI tools must navigate a landscape where the line between truth and fiction is increasingly blurred. The AI, like the narrator in Dick’s novel, presents a version of reality that may not correspond to actual facts.

    To avoid falling into this trap, students must approach AI-generated information with skepticism. They should verify any claims made by the AI by cross-referencing with credible sources. In other words, they must act as human fact-checkers, ensuring that the information they use is accurate. This process requires a critical eye and a willingness to question even the most plausible-sounding responses. Reading Gen AI responses with a healthy dose of skepticism and engaging in research written by authorities in those fields will help students verify those responses while gaining information literacy and research skills.

    The cyborg student, armed with both human critical thinking and the power of AI, must learn to use these tools responsibly. By doing so, they can harness the benefits of AI while avoiding the pitfalls of misinformation. The bottom line is Gen AI is a good tool that can work with text in various ways–especially text that you supply it with, but it shouldn’t be relied on as a knowledge base as it isn’t designed to be a reference in the same way as an encyclopedia, database, or textbook is.