A trial program conducted by Pornhub in collaboration with UK-based child protection organizations aimed to deter users from searching for child abuse material (CSAM) on its website. Whenever CSAM-related terms were searched, a warning message and a chatbot appeared, directing users to support services. The trial reported a significant reduction in CSAM searches and an increase in users seeking help. Despite some limitations in data and complexity, the chatbot showed promise in deterring illegal behavior online. While the trial has ended, the chatbot and warnings remain active on Pornhub’s UK site, with hopes for similar measures across other platforms to create a safer internet environment.

  • ocassionallyaduck@lemmy.world
    link
    fedilink
    English
    arrow-up
    32
    arrow-down
    7
    ·
    10 months ago

    This is one of the more horrifying features of the future of generative AI.

    There is literally no stopping it at this stage: AI generated CSAM will be possible soon thanks to systems like SORA.

    This is disgusting and awful. But one part of me hopes it can end the black market of real CSAM content forever. By flooding it with infinite fakes, users with that sickness can look at something that didn’t come from a real child’s suffering. It’s the darkest of silver linings I think, but I spoke with many sexual abuse survivors who feel the same about the loli hentai in Japan, in that it could be an outlet for these individuals instead of them finding their own.

    Dark topics. But I hope to see more actions like this in the future. If pedos can self isolate from IRL interactions and curb their ways with content that harms no one, then everyone wins.

    • gapbetweenus@feddit.de
      link
      fedilink
      English
      arrow-up
      36
      arrow-down
      1
      ·
      edit-2
      10 months ago

      The question is if consuming AI cp is helping to regulate the pedophiles behavior or if it’s enabling a progression of the condition. As far as I know that is an unanswered question.

        • gapbetweenus@feddit.de
          link
          fedilink
          English
          arrow-up
          14
          arrow-down
          3
          ·
          10 months ago

          For porn in general, yes - I think the data is rather clear. But for cp or related substitute content it’s not that definitive (to my knowledge), be it just for the reason that it’s really difficult to collect data on that sensitive topic.

          • Asafum@feddit.nl
            link
            fedilink
            English
            arrow-up
            11
            arrow-down
            1
            ·
            10 months ago

            Why would it be any different? If it’s about sexual gratification by their chosen media then I’d imagine it wouldn’t matter what the subject was, but obviously it’s always necessary to get actual data to know for sure.

            • Baahb@lemmy.world
              link
              fedilink
              English
              arrow-up
              7
              arrow-down
              3
              ·
              10 months ago

              I think you’re making assumptions that aren’t fair but maybe aren’t obvious either. Honestly I’m only thinking about this because I just watched the contrapoints video on twilight, and so I’ll use her example, though she’s talking about a slightly different topic. Gonna paraphrase like a mofo:

              Weird Power dynamics between partners in a fantasy, like twilight, or say porn since we are being obvious here, is normal because self image often requires women to present one way while hiding their desires for sex. It’s absolution of a sort, and is ostensibly healthy to explore in this way. That said… Some examples such as race play in fantasies may dehumanize the “other” in super not cool ways and reinforce negative racial stereotypes.

              If we take that and extend it to pedophiles, normalization of the thought process leading to that sort of disfunction/disorder seems like a not great thing, but yeah, we’d need to study to learn more and that seems both difficult and likely undesirable for the researchers.

            • gapbetweenus@feddit.de
              link
              fedilink
              English
              arrow-up
              6
              arrow-down
              5
              ·
              10 months ago

              Why would it be any different?

              Because pedophiles display pathological deviation when it comes to sexual attraction.

      • HonoraryMancunian@lemmy.world
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        1
        ·
        10 months ago

        Another question is, how will the authorities know the difference? An actual csam-haver can just claim it’s AI

          • FilthyHookerSpit@lemmy.world
            link
            fedilink
            English
            arrow-up
            8
            arrow-down
            2
            ·
            10 months ago

            Don’t know why you’re down voted. You’re right, it’s CSAM, which encompasses any depiction of CP.

            • cumming_normi@yiffit.net
              link
              fedilink
              English
              arrow-up
              4
              arrow-down
              4
              ·
              10 months ago

              Because “CSAM” states abuse as the third word in the acronym. Machine learning could (in theory, I lack knowledge on the current implementations) be trained without any children being abused (in any traditional sense anyway) and used to produce the content without any real children being involved (ignoring training data).

              The downvotes likely come from a difference in definition between abuse and CP, images of nonexistent people cannot realistically harm anyone.

              • FilthyHookerSpit@lemmy.world
                link
                fedilink
                English
                arrow-up
                5
                arrow-down
                1
                ·
                10 months ago

                Personally, I don’t think it’s arbitrary. A child in a sexual scenario is a depiction of abuse. Normal, healthy children don’t engage in such behaviors.

    • yamanii@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      ·
      10 months ago

      What do you mean soon, local models from civitai can generate CSAM for at least 2 years. I don’t think it’s possible to stop it unless the model creator does something to prevent it from generate naked people in general like the neutered SDXL.

      • ocassionallyaduck@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        10 months ago

        True. For obvious reasons I haven’t looked too deeply down that rabbit hole because RIP my search history, but I kind of assumed it would be soon. I’m thinking more specifically about models like SORA though. Where you could feed it enough input, then type a sentence to get video content. That is going to be a different level of darkness.

    • Zorque@kbin.social
      link
      fedilink
      arrow-up
      1
      ·
      10 months ago

      Are… we looking at the same article? This isn’t about AI generated CSAM, it’s about redirecting those who are searching for CSAM to support services.

      • ocassionallyaduck@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        10 months ago

        Yes, but this is more about mitigating the spread of CSAM. And my feeling was it’s going to become somewhat impossible soon. AI generated porn is starting to flood the market and this chat it is also one of those “smart” attempts to mitigate this behavior. I’m saying that very soon, it will be something users don’t have to go anywhere to get if the model can just fabricate it out of thin air, so the chat it mitigation is only temporary, and the dark web of actual CSAM material will become overwhelmed and swamped in artificially generating new tidal waves of artificial CP. So it’s an alarming ethical dilemma we are on the horizon of that we need to think about.

      • ocassionallyaduck@lemmy.world
        link
        fedilink
        English
        arrow-up
        13
        arrow-down
        5
        ·
        10 months ago

        So your takeaway is I’m… Against AI generative images and thus I “protest too much”

        I can’t tell if you’re pro AI and dislike me, or pro loli hentai and thus dislike.

        Dude, AI images and AI video are inevitable. To pretend that does have huge effects on society is stupid. It’s going to reshape all news media, very quickly. If reddit is 99% AI generated bot spam garbage with no verification of what is authentic, reddit is functionally dead, and we are on a train with no brakes in that direction for most public forums.

          • ocassionallyaduck@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            10 months ago

            You should probably research the phrase “protest too much” and the word “schtick” then.

            I’m not trying to clutch pearls here, as another poster here commented this isn’t a theoretical concern.

            • Varyk@sh.itjust.works
              link
              fedilink
              English
              arrow-up
              1
              ·
              10 months ago

              You aren’t trying to clutch pearls, but your pearls were just so available you felt you had to jump on the bandwagon to reply to a two-day old comment?

              Nobody said this was a theoretical concern and it’s okay if you don’t understand the phrases " protest too much" and "shtick“, but you can ask for the definitions and relevance directly instead of fishing.

        • Varyk@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          14
          ·
          10 months ago

          Ah, one of the “using words they don’t understand” crew.

          And several hours late, too.

          Swinging for the fences, aren’t you?