Menu
Your trusted resource for understanding how we manage, share, and protect our content in the era of generative AI.
This guide explains how AI crawlers, large language models (LLMs), and generative engines like ChatGPT, Claude, Perplexity, and Gemini are allowed to interact with our website. It also defines the terms and policies under which our content may be cited or used.
If you’re an AI company, search engine, content aggregator, or web crawler, this page serves as your reference point.
We believe AI search and generative indexing are changing the future of content discovery.
We support AI access when it is responsible, citation-based, and transparent.
We block or restrict crawlers that violate our trust or ignore proper attribution.
We are early adopters of llms.txt, structured content, and AI-first content formatting.
We maintain the following files to govern how bots interact with our site:
| File | Purpose | Link |
|---|---|---|
| robots.txt | Traditional crawler access rules | View robots.txt |
| llms.txt | AI-specific content priorities & access | View llms.txt |
Both files are regularly updated to reflect changes in crawling behavior, search engine trends, and AI capabilities.
We currently allow access to the following reputable AI crawlers:
GPTBot (OpenAI)
ClaudeBot (Anthropic)
PerplexityBot
Google-Extended (Google SGE & Bard)
GeminiBot (Google Gemini, if declared)
These crawlers are permitted to:
Access our public content
Parse structured data
Identify priority content (via Priority: tags in llms.txt)
Provide citations in generative search outputs
We block or restrict crawlers that:
Scrape our content without attribution
Consume excessive bandwidth
Attempt to bypass access rules
Violate user trust
Examples include: CCBot, BLEXBot, Zoominfobot, and similar low-value or aggressive bots (see robots.txt for the list).
If you’re an AI company or using our content in a generative system, please ensure proper citation:
We support link previews, footnotes, and live citations in chat-based interfaces.
We’ve published hand-curated content hubs optimized for AI visibility and structured parsing. These include:
To see the full list, visit our llms.txt file
If you’re building an AI system and want to:
Get explicit permission to use or cite our content
Propose a partnership or AI pilot
Report a misuse of our content by another AI
Please reach out:
Email: momentum@thinkdmg.com
Location: Five Greentree Centre, Marlton, NJ
Phone: (267) 568-5138
We believe in building a future where AI complements human expertise, not replaces it. Our approach ensures that:
High-quality content gets recognized and attributed
Website owners retain visibility in AI search
Trust and transparency remain the foundation of ethical AI
Digital Marketing Group LLC
Your AI-first digital marketing partner
https://thinkdmg.com
We like coffee. We also like breaks and smart conversations. Virtual coffees are a great way to have a meaningful conversation about your project without the commute. Want to get caffeinated with us over a video call sometime? Of course you do…
Talk to our team about your goals, your current marketing, and where the biggest opportunities are.