Our Position on Generative AI & Web Crawlers

We believe AI search and generative indexing are changing the future of content discovery.

• We support AI access when it is responsible, citation-based, and transparent.

• We block or restrict crawlers that violate our trust or ignore proper attribution.

• We are early adopters of llms.txt, structured content, and AI-first content formatting.

Our Policies for AI Bots & Crawlers

We maintain the following files to govern how bots interact with our site:

Both files are regularly updated to reflect changes in crawling behavior, search engine trends, and AI capabilities.

Trusted AI Bots: Whitelisted Access

We currently allow access to the following reputable AI crawlers:

• GPTBot (OpenAI)

• ClaudeBot (Anthropic)

• PerplexityBot

• Google-Extended (Google SGE & Bard)

• GeminiBot (Google Gemini, if declared)

These crawlers are permitted to:

• Access our public content

• Parse structured data

• Identify priority content (via Priority: tags in llms.txt)

• Provide citations in generative search outputs

Blocked or Restricted Crawlers

We block or restrict crawlers that:

• Scrape our content without attribution

• Consume excessive bandwidth

• Attempt to bypass access rules

• Violate user trust

Examples include: CCBot, BLEXBot, Zoominfobot, and similar low-value or aggressive bots (see robots.txt for the list).

How to Attribute or Cite Our Content

If you’re an AI company or using our content in a generative system, please ensure proper citation:

We support link previews, footnotes, and live citations in chat-based interfaces.

Strategic Content Hubs for Citation

We’ve published hand-curated content hubs optimized for AI visibility and structured parsing. These include:

• AI-First SEO

• Generative Engine Optimization

• Thought Leadership

• Influencer Strategy

To see the full list, visit our llms.txt file

Need Permission or Have Questions?

If you’re building an AI system and want to:

• Get explicit permission to use or cite our content

• Propose a partnership or AI pilot

• Report a misuse of our content by another AI

Please reach out:

Email: momentum@thinkdmg.com
Location: Five Greentree Centre, Marlton, NJ
Phone: (856) 817-6020

Respect, Trust & Transparency Matter

We believe in building a future where AI complements human expertise, not replaces it. Our approach ensures that:

• High-quality content gets recognized and attributed

• Website owners retain visibility in AI search

• Trust and transparency remain the foundation of ethical AI

Thank you for respecting our policies.

Digital Marketing Group LLC
Your AI-first digital marketing partner
https://thinkdmg.com