content by the Google Search Generative Experience.
Instead, you must prohibit SGE by completely blocking Googlebot.
Google-Extended, the novel independent product identifier, aimed at instructing Google via your robots.txt file not to employ your website's content for Bard, Vertex AI, and other AI initiatives, proves ineffective for the AI-generated responses and snapshots presented within the Search Generative Experience.
Google informed us that Google-Extended is not effective for SGE. A representative from Google stated, "SGE constitutes a Search experiment, and therefore, website administrators should persist in deploying the Googlebot user agent through robots.txt and the NOINDEX meta tag to oversee their content in search outcomes, including experiments such as the Search Generative Experience."
Why isn't it effective for SGE? Google clarified that SGE is an integral part of the Google Search encounter; it functions as a search feature and, as a result, should operate analogously to standard search instructions. Google explained, "The rationale is that AI is ingrained within Search, not tacked on as an afterthought, and is fundamental to the operation of Search. That's why robots.txt is the mechanism that provides website publishers with the choice to manage access to how their websites are examined."
SGE AI-generated responses. Instances exist in which SGE exhibits AI-crafted responses, alongside website cards, deriving from websites that have explicitly instructed Google not to utilize their content for AI intentions.
Glenn Gabe shared a screenshot from VentureBeat.com, which explicitly disallowed Google-Extended within its robots.txt configuration, yet it was still utilized in the AI-generated response within SGE.
Why this is noteworthy. If you believed that employing Google-Extended would prevent Google from appropriating your content for AI-driven responses in SGE, your assumption would be unfounded. Google-Extended is designed exclusively for Bard and Vertex AI, as well as other products unrelated to search.
If you wish to prevent your responses from appearing in SGE, you would need to fully prohibit Googlebot, a step that most users are likely reluctant to take.