Ep172-Blocking Banner Ads From Getting Indexed
Episode 172 contains the notable Digital Marketing News and Updates from the week of July 31 - August 4, 2023.
1. OpenAI's GPTBot: The New Web Crawler That's Changing the Game - OpenAI has announced the launch of GPTBot, a new web crawler that uses large language models to crawl the web. GPTBot is designed to be more efficient and effective than traditional web crawlers, and it can be used to collect data for a variety of purposes, including research, development, and marketing.
1. OpenAI's GPTBot: The New Web Crawler That's Changing the Game - OpenAI has announced the launch of GPTBot, a new web crawler that uses large language models to crawl the web. GPTBot is designed to be more efficient and effective than traditional web crawlers, and it can be used to collect data for a variety of purposes, including research, development, and marketing.
You can use robots.txt to block GPTBot from accessing your website, or parts of it. To disallow GPTBot to access your site you can add GPTBot to your site’s robots.txt:
User-agent: GPTBot
Disallow: /
To allow GPTBot to access only parts of your site, you can add the GPTBot token to your site’s robots.txt like this:
User-agent: GPTBot
Allow: /directory-1/
Disallow: /directory-2/
GPTBot IP ranges. OpenAI also published the IP ranges that GPTBot uses. It only lists one, but I suspect they will add more over time.
2. Google Releases Responsive Search Ads Guide - Responsive search ads (RSAs) are a type of ad that allows you to create multiple headlines and descriptions, and Google will test different combinations to see which ones perform best. This can help you to create more effective ads that are more likely to get clicks.
To create an RSA, you will need to provide a minimum of 15 headlines and 4 descriptions. Google will then test different combinations of these headlines and descriptions, and the best-performing combinations will be used in your ads.
Now Google has released a Responsive Search Ads guide to help marketers improve their campaign performance.
The new RSA guide includes Google’s advice and information on how:
- To write ads for maximum conversions.
- RSAs use Google AI to generate ad copy.
- To evaluate a campaign’s effectiveness.
A Google spokesperson said: "Our goal is to simplify the work it takes to create high-quality ads that achieve your business objectives at scale. To do this, we’re hard at work applying the newest AI-powered innovations across asset generation, customizing assets for relevance, and both simplifying and improving tools for managing ads."
Read Google's Responsive Search Ads guide (PDF) for more information.
3. YouTube Shorts Adds Six New Features - YouTube Shorts, the short-form video platform from YouTube, has launched a number of new features to improve content creation. The new features rolled out on YouTube Shorts are:
- Collab – This new feature allows creators to film a Short alongside other YouTube or Short videos in a side-by-side format.
- Multiple layout options are available. Q&A stickers – Content creators can use this new feature to ask viewers questions. The audience can then leave their replies in the comments section, enabling creators to give them a shoutout as they can see who left the response.
- Mobile-first vertical live – This new feature is what will allow live creators to get discovered in the Shorts feed. YouTube is hoping this will enable creators to connect live with a new audience and build communities in a more modern way.
- Creation suggestions – YouTube can now automatically bundle the audio and effects from a Short you may be remixing. The platform can surface the same audio time stamp from the Short you just watched, and add the same effect as a creation suggestion.
- Content creators also have the option to mix and match.
- Shorts playlist – Creators can now save Shorts to playlists directly onto YouTube.
Transform horizontal videos to Shorts – This new feature is being trialled in the coming weeks. Users will now be able to choose a video to remix and adjust the layout, zoom and crop to turn it into a Short.
4. YouTube's New AI-Generated Video Summaries: A Quick Way to Get the Cliff Notes - YouTube is testing a new feature that uses artificial intelligence to generate summaries of videos. The summaries are displayed in a small box below the video player, and they provide a brief overview of the video's content.
“We’re starting to test AI auto-generated summaries on YouTube, so that it’s easier for you to read a quick summary about a video and decide whether it’s the right fit for you. To begin with, you may see these summaries on watch and search pages. While we hope these summaries are helpful and give you a quick overview of what a video is about, they do not replace video descriptions (which are written by creators!).”
The summaries are generated using a technique called natural language processing, which allows computers to understand and process human language. The summaries are not perfect, but they can be a helpful way to get a quick overview of a video before you decide to watch it.
Remember, generative AI produces a lot of misleading, untrue, and confused content.
5. Linking to High-Authority Sites Won't Help Your SEO - Google Search Advocate John Mueller recently clarified that linking to popular sites like Wikipedia does not impact search rankings, dispelling a myth that has persisted among SEO practitioners. He stated that linking to high-authority sites does not help SEO, and that the focus should be on providing useful links that benefit users.
Mueller further illustrated his point with a humorous example, saying, “Here’s my affiliate site about handbags – and here’s a link to CNN & Wikipedia, please take me seriously now, k?” His statement underscores the misconception that linking to high-authority sites is akin to borrowing their credibility. Instead he advised: “Does this link provide additional, unique value to users? Then, link naturally. Is this link irrelevant to my users? Then don’t link to it.”
6. Keyword Stuffing: How Many Times Should You Use a Keyword on a Page? - Keyword stuffing is a practice of repeating the same keywords or phrases too many times on a page in an attempt to manipulate search engine rankings. However, Google has become very good at detecting keyword stuffing, and pages that engage in this practice are often penalized in search results.
In a recent Twitter thread, Google's John Mueller said that there is no need to worry about keyword stuffing if you only use a keyword 10-20 times on a page. However, if you unnecessarily overuse a keyword, then you may be engaging in keyword stuffing and could be penalized by Google. The question you should ask yourself : “Will the reader be ok with having to read the keyword multiple times?”
7. Google Releases Appeal Form For Ads Trademark Policy Violations - Google announced a change to its Google Ads trademark policy and will require advertisers to file an appeal form if their ad is flagged for trademark infringement. Previously, advertisers could simply click a button to appeal a flagged ad, but the new process requires them to provide more information about their ad and why they believe it is not infringing on a trademark.
The change is intended to simplify and speed up the appeals process for advertisers, as well as to provide more clarity for trademark holders about why their trademarks are being used in Google Ads. However, some advertisers have complained that the new process is too cumbersome and time-consuming.
8. Blocking Banner Ads From Getting Indexed - Someone on Reddit asked Google’s John Muller if it is possible to prevent Banner Ads from getting Indexed. The person was worried that GoogleBot will consider banner ads as duplicate content since the same banner will appear on all the pages on the site.
John simply answered that: “You cannot noindex a part of a page like that.” And John is **somewhat** correct. However, if you have good knowledge & understanding of Semantic HTML then you will know that to avoid potential confusion and issues, you need to put your banner ads outside of the “main” tag or if it appears within “main” then wrap the ad within the “aside” tag.
While Google in general can identify what content is advertising and which content is main content, but don’t leave it to chance, use semantic HTML markup to make it clear for Google. It’s an SEO best practice to make the page structure exceedingly clear for Google.