SEO 4
Meta / Open Graph tags, robots.txt, sitemap.xml and JSON-LD structured data
What a page tells search engines and social platforms about itself, before anyone reads a word of it. Small text files and head tags that decide whether a link is found, how it is titled, and what it looks like when shared.
Tools in this category
Meta Tag & Open Graph Generator
Build meta and Open Graph tags
robots.txt Generator
Build a robots.txt file
sitemap.xml Generator
Turn a URL list into sitemap.xml
JSON-LD Structured Data Generator
Generate JSON-LD structured data
The head is the part nobody sees and everyone depends on
A page's title and description are what appears in search results, so they are the first thing most visitors read and the main reason they click or scroll past. The Open Graph and Twitter Card tags decide what a link looks like when pasted into X, Facebook, Slack or Discord: a large image with a headline, or a bare URL. None of this is visible on the page itself, which is exactly why it is so often left incomplete or copied from a template without being updated.
Length limits are display limits, not rules
Google does not reject a long title; it truncates it, and the truncation point depends on pixel width rather than character count. Around 60 Latin characters or 30 full-width characters is where titles start to be cut, and descriptions around 155 to 160 characters. Treating these as hard limits is the wrong mental model. The useful habit is to put the distinctive words first, so that whatever is cut off is the least important part.
robots.txt controls crawling, not indexing
A Disallow line asks well-behaved crawlers not to fetch a path. It does not remove the URL from search results, and a page that is linked from elsewhere can still be indexed with no content. Removing something from search requires a noindex tag, which the crawler must be allowed to fetch in order to read. The file has become more relevant recently because it is also where most generative-AI crawlers look for permission, and the list of their user-agent names keeps growing.
Structured data describes the page in a form machines can trust
Meta tags say what a page is called; structured data says what it is. A JSON-LD block in the head can state that the page is a product with a price and stock level, an article with a publication date and an author, or a shop with an address and opening hours. Google uses it to qualify pages for rich results such as review stars, breadcrumb trails and price snippets, but markup only makes a page eligible: it does not guarantee the enhanced display, and markup that describes content not visible on the page is treated as spam.
What to reach for
The Meta Tag Generator builds the title, description, canonical, Open Graph and Twitter Card tags from a form, shows how the result renders as a search snippet and as a social card, and warns about missing images and over-long text. The robots.txt Generator assembles per-agent rule groups and sitemap lines, includes presets that block AI training crawlers, with or without AI search, and catches the mistakes that silently break the file, such as paths without a leading slash. The sitemap.xml Generator turns a list of URLs, or an existing sitemap, into a valid file, splitting it with a sitemap index past 50,000 URLs and flagging off-domain, duplicate and parameterised URLs. The JSON-LD Generator writes structured data for articles, breadcrumbs, FAQs, products, organizations and local businesses, and separates the properties Google requires from the ones it only recommends. To check what a server already sends, the HTTP Header Analyzer in the Network category reads the response headers, and the Redirect Generator handles the canonical-URL side of the problem at the server.
SEO: frequently asked questions
- My link shows no image when shared. Which tag is missing?
- Almost always og:image, or an og:image that is a relative path or an http URL. Platforms need an absolute https URL to an image of at least 1200 by 630 pixels. After fixing it, re-fetch the URL with the platform's own debugger, since most of them cache the old result for a while.
- Will Disallow in robots.txt hide a page from Google?
- No. It stops Google crawling the page, but the URL can still appear in results if other pages link to it, usually with no description. To keep a page out of search results, allow crawling and add a noindex meta tag, or use the removal tool in Search Console.
- Do AI crawlers respect robots.txt?
- The major ones, including GPTBot, ClaudeBot, Google-Extended and CCBot, document that they do. Smaller or less scrupulous crawlers may not, and robots.txt is advisory rather than enforced. Blocking at the CDN or firewall by user-agent or IP range is the only way to make a block binding.
Other categories
Looking for something that is not here yet? The Japanese edition of this category lists 4 tools. The tools themselves work the same way regardless of the language of the surrounding page.