stopthatgirl7@kbin.social to Technology@kbin.social · 1 year agoNow you can block OpenAI’s web crawlerwww.theverge.comexternal-linkmessage-square4fedilinkarrow-up132arrow-down10file-textcross-posted to: technews@radiation.partycampfire@lemmy.seedoubleyou.metheverge@zerobytes.monster
arrow-up132arrow-down1external-linkNow you can block OpenAI’s web crawlerwww.theverge.comstopthatgirl7@kbin.social to Technology@kbin.social · 1 year agomessage-square4fedilinkfile-textcross-posted to: technews@radiation.partycampfire@lemmy.seedoubleyou.metheverge@zerobytes.monster
minus-squareNaich@kbin.sociallinkfedilinkarrow-up10·edit-21 year agoIn /robots.txt - User-agent: GPTBot Disallow: / If you don’t want to read the article. Presumably they are doing this now because they have scraped everyone already.
minus-squarebionicjoey@lemmy.calinkfedilinkarrow-up1·1 year agoBetter to ask for forgiveness than to beg for permission
In /robots.txt -
User-agent: GPTBot Disallow: /If you don’t want to read the article. Presumably they are doing this now because they have scraped everyone already.
Better to ask for forgiveness than to beg for permission