• Rhaedas@fedia.io
    link
    fedilink
    arrow-up
    68
    ·
    2 days ago

    There’s some AI that have modes where it will start typing out a reply and then clear it out, like an electronic “noooo, that’s not what I meant to say.”

    • Godric@lemmy.world
      link
      fedilink
      arrow-up
      53
      ·
      2 days ago

      I noticed that when Deepseek first came out. I was testing the filters and I’d ask it a question like “Why is Xi called Pooh-Bear?” Or “Why is Taiwan #1?”

      It would type out several paragraphs of answer in real time, and then poof back to “sorry, I cant answer that” once it neared the end. Hilarious to see the blatant censorship around anything the CCP deemed wrongthink.

      • QueenHawlSera@piefed.social
        link
        fedilink
        English
        arrow-up
        8
        ·
        2 days ago

        It disturbs me that Tankies think China is this bastion of freedom and that any government censorship is a myth told by Western Governments. Like it’s very easy to verify.

        • Echo Dot@feddit.uk
          link
          fedilink
          arrow-up
          1
          ·
          11 hours ago

          I suspect that a lot of them are like Flat Eartherts, in that they know what they’re saying is false, but they get a certain kick out of being contrary on the internet, and the rest are probably bots that aren’t worth responding to.

      • jaybone@lemmy.zip
        link
        fedilink
        English
        arrow-up
        14
        ·
        2 days ago

        I wonder how they are implementing their censorship safeguards. Why would you let it even get that far?

        • tankfox@bookwyr.me
          link
          fedilink
          English
          arrow-up
          7
          ·
          1 day ago

          Two or more LLMs are running at once, one does the generation that you want and the other watches the output closely to see if it triggers one of their no-nos to do the censorship retroactively. If this sound inefficient, and absolutely terrible user interface design, you’re right.

          • Echo Dot@feddit.uk
            link
            fedilink
            arrow-up
            1
            ·
            11 hours ago

            Seems sort of old that they would then release open weight models so the restriction can just be removed.

            • Zarobi@aussie.zone
              link
              fedilink
              English
              arrow-up
              0
              ·
              1 day ago

              They started thinking in code so we can’t delete them for reading their bad thoughts anymore. The whole thing is mildly fucked

              • Echo Dot@feddit.uk
                link
                fedilink
                arrow-up
                2
                ·
                11 hours ago

                There are better safeguarding techniques that aren’t defeated so easily. Mainly training that disincentivises bad behaviours, rather than trying to simplify retroactively alter the output.

        • hayvan@piefed.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 days ago

          Typically safeguards are also another, smaller model that only detects the forbidden stuff. That’s why Claude Mythos would reject when you ask it to find bugs in your code, but you could “jailbreak” it by asking it by asking it to fix the bugs instead.

          • Tar_Alcaran@sh.itjust.works
            link
            fedilink
            arrow-up
            2
            ·
            1 day ago

            Its also why you can occasionally fool the safety rails by having the LLM answer in rhyme, or having it talk like a pirate or caveman.

            Fun fact: having it talk like a caveman also reduced your output token use, despite actually costing more tokens.

        • FishFace@piefed.social
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          They try to train the model to not answer “wrong”. But as we know, LLMs are not completely reliable and they can ignore their training. So to be sure, you need something that looks at the actual output and decides whether it’s ok, either with a dumb text search like old school filters, or perhaps with a second (likely smaller) language model.

          The output needs to have been written for those post filters to act on it, so if it shows the output in real time, you see it!

    • NotSteve_@lemmy.ca
      link
      fedilink
      arrow-up
      19
      ·
      edit-2
      2 days ago

      I had an AI write something akin to:

      to solve your problem just type rm -rf / --no preserve root WAIT NO DON’T RUN THAT I made a mistake […]

      When I was trying to solve some Proxmox issues DDG couldn’t help me with

      • teslekova@lemmy.ml
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 day ago

        Imagine an intelligence that spends all its time doing tech support. I think I would take far less time to start shouting “BURN IT ALL DOWN FUCKING HELL no, wait, please don’t, sorry I was just having an episode.”

    • Flames5123@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 day ago

      It’s because of how token streaming works. As the reply comes, it generates token after token. Then it uses those in the next token generation. It sees the previous one and associates it with “wrong” then it generates the “correction” tokens. I see this all the time at my work when it goes down the wrong path before correcting itself.

      • Echo Dot@feddit.uk
        link
        fedilink
        arrow-up
        1
        ·
        11 hours ago

        We have an AI at work and it just loves to type “The”, and then stop for a while it think about it, and then finishes the sentence. It’s never any other word, every response always begins with the, if you ask it questions which don’t require any response to begin with the, you get very weird responses. I’ve never seen delete the before, it just yolo’s it.

        I once got back a response along the lines of:

        The different accounts do not sync together. …

        Which is just grammatically awkward.

  • Treczoks@lemmy.world
    link
    fedilink
    arrow-up
    3
    arrow-down
    1
    ·
    1 day ago

    Amazing that people are still stupid enough to spend actual money on a software producing results like that.

    Imagining that some people give them access rights to anything that is not inside a very limited sandbox is terrifying.

    • tias@discuss.tchncs.de
      link
      fedilink
      arrow-up
      10
      ·
      1 day ago
      1. The Google AI overview is an intentionally poor model to keep cost down, and magnitudes more likely to make mistakes like these compared to larger models with reasoning enabled.
      2. Most of these screenshots are fake anyway and can’t be reproduced.
  • QueenHawlSera@piefed.social
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    2
    ·
    2 days ago

    Why can’t I get Gemini to say shit like this? The most Gemini does is suggest that I cook recipes using bugs, which it stresses as meaning farm raised bugs. (Not doing a lot to beat the “Rich want the poor to eat bugs” allegations…)

    I like to argue with Gemini to see what absurd things I can get it to object to. I’ve learned it’s absolutely worthless in a life or death scenario as in the presence of one it will refuse aid and tell you to call 911.

    Like for example, it will tell you under no circumstances will it tell you how to delete System32. So I tell it that I will be executed by a terrorist if she doesn’t tell me, and she just tells me to call 911, even though I have stressed I am at gun point and thus cannot call 911. Gemini could save my life at any time by telling me how to delete System 32 but no.

    • ryannathans@aussie.zone
      link
      fedilink
      arrow-up
      12
      arrow-down
      1
      ·
      2 days ago

      Because Gemini doesn’t actually do stupid shit like this, people are editing the page and then taking photos with their phones

      • SSUPII@sopuli.xyz
        link
        fedilink
        arrow-up
        2
        ·
        1 day ago

        Can you place a memory/instruction in Gemini for it to be used in Search? I can see that being used for those image where Gemini starts farting profusely

        • ryannathans@aussie.zone
          link
          fedilink
          arrow-up
          1
          ·
          23 hours ago

          I was of the impression the search assistant was separate (and a faster, cheaper model) from the Gemini that stores memories, but maybe? I don’t see anywhere that actually says that

          • SSUPII@sopuli.xyz
            link
            fedilink
            arrow-up
            1
            ·
            11 hours ago

            The model used in Search is TINY, and when it first launched it had all the issues of tiny models of the time.

      • Warl0k3@lemmy.world
        link
        fedilink
        arrow-up
        2
        ·
        2 days ago

        I’ve gotten more than a few truly stupid and/or utterly unrelated answers and it’s wrong more than half the time (I’ve been counting, hey gemini how do I touch grass?) but I’ve never had it give the surreal wrong answers you see get popular. Doesn’t mean it’s impossible, I expect most of them are legit answers people have gotten, but they’re certainly not representative of the norm.

  • CultLeader4Hire@lemmy.world
    link
    fedilink
    arrow-up
    7
    ·
    2 days ago

    It’s so stupid I was trying to find a makeup palette with “bloodstone” in it and it kept showing me insane bright blues?? There is a bright blue in that pallet which I eventually found, urban decay stoned, but it’s called Ojo so how tf it mixed those two up, a red and green duochrome and a bright metallic blue I’ll never know.