• About Us
  • Contact Us
  • Terms & Conditions
  • Privacy Policy
Technology Hive
  • Home
  • Technology
  • Artificial Intelligence (AI)
  • Cyber Security
  • Machine Learning
  • More
    • Deep Learning
    • AI in Healthcare
    • AI Regulations & Policies
    • Business
    • Cloud Computing
    • Ethics & Society
No Result
View All Result
  • Home
  • Technology
  • Artificial Intelligence (AI)
  • Cyber Security
  • Machine Learning
  • More
    • Deep Learning
    • AI in Healthcare
    • AI Regulations & Policies
    • Business
    • Cloud Computing
    • Ethics & Society
No Result
View All Result
Technology Hive
No Result
View All Result
Home Technology

AI firms blindsided by beefed up robots.txt instructions

Linda Torries – Tech Writer & Digital Trends Analyst by Linda Torries – Tech Writer & Digital Trends Analyst
September 11, 2025
in Technology
0
AI firms blindsided by beefed up robots.txt instructions
0
SHARES
0
VIEWS
Share on FacebookShare on Twitter

Introduction to Really Simple Licensing

Leading Internet companies and publishers, including Reddit, Yahoo, Quora, Medium, The Daily Beast, Fastly, and more, are introducing a new standard to end AI crawlers from scraping content without permission or compensation. This new standard is called "Really Simple Licensing" (RSL) and it evolves robots.txt instructions by adding an automated licensing layer. The RSL standard is designed to block bots that don’t fairly compensate creators for content.

What is Really Simple Licensing?

The RSL standard is an open, decentralized protocol that makes clear to AI crawlers and agents the terms for licensing, usage, and compensation of any content used to train AI. It is free for any publisher to use and is based on the "Really Simple Syndication" (RSS) standard. RSL terms can be applied to protect any digital content, including webpages, books, videos, and datasets.

Key Features of RSL

The new standard supports a range of licensing, usage, and royalty models, including:

  • Free: allowing AI crawlers to use content for free
  • Attribution: requiring AI crawlers to give credit to the original creator
  • Subscription: requiring AI crawlers to pay a subscription fee to use content
  • Pay-per-crawl: publishers get compensated every time an AI application crawls their content
  • Pay-per-inference: publishers get compensated every time an AI application uses their content to generate a response

Creation of RSL

The RSL standard was created by the RSL Collective, which was founded by Doug Leeds, former CEO of Ask.com, and Eckart Walther, a former Yahoo vice president of products and co-creator of the RSS standard. The idea to use the RSS "playbook" to roll out the RSL standard arose after Leeds invited Walther to speak to University of California, Berkeley students at the end of last year.

Impact of RSL

The introduction of RSL is expected to have a significant impact on the way AI crawlers interact with online content. By providing a clear and automated way for publishers to specify the terms of use for their content, RSL aims to prevent AI crawlers from scraping content without permission or compensation. This could help to ensure that creators are fairly compensated for their work and that the online ecosystem is more sustainable.

Conclusion

The Really Simple Licensing standard is a significant development in the fight against AI crawlers scraping content without permission or compensation. By providing a clear and automated way for publishers to specify the terms of use for their content, RSL aims to create a more sustainable online ecosystem. With the support of leading Internet companies and publishers, RSL has the potential to make a major impact on the way AI interacts with online content.

FAQs

  • What is Really Simple Licensing?
    Really Simple Licensing (RSL) is an open, decentralized protocol that makes clear to AI crawlers and agents the terms for licensing, usage, and compensation of any content used to train AI.
  • Who created RSL?
    The RSL standard was created by the RSL Collective, which was founded by Doug Leeds, former CEO of Ask.com, and Eckart Walther, a former Yahoo vice president of products and co-creator of the RSS standard.
  • What types of licensing models does RSL support?
    RSL supports a range of licensing, usage, and royalty models, including free, attribution, subscription, pay-per-crawl, and pay-per-inference.
  • Is RSL free to use?
    Yes, RSL is free for any publisher to use.
  • What types of content can be protected with RSL?
    RSL terms can be applied to protect any digital content, including webpages, books, videos, and datasets.
Previous Post

VMware starts down the AI route, but it’s not core business

Next Post

Why Developers are Switching to Google ADK

Linda Torries – Tech Writer & Digital Trends Analyst

Linda Torries – Tech Writer & Digital Trends Analyst

Linda Torries is a skilled technology writer with a passion for exploring the latest innovations in the digital world. With years of experience in tech journalism, she has written insightful articles on topics such as artificial intelligence, cybersecurity, software development, and consumer electronics. Her writing style is clear, engaging, and informative, making complex tech concepts accessible to a wide audience. Linda stays ahead of industry trends, providing readers with up-to-date analysis and expert opinions on emerging technologies. When she's not writing, she enjoys testing new gadgets, reviewing apps, and sharing practical tech tips to help users navigate the fast-paced digital landscape.

Related Posts

Expert Panel to Decide AGI Arrival in Microsoft-OpenAI Deal
Technology

Expert Panel to Decide AGI Arrival in Microsoft-OpenAI Deal

by Linda Torries – Tech Writer & Digital Trends Analyst
October 30, 2025
Closed-Loop CNC Machining with IIoT Feedback Integration
Technology

Closed-Loop CNC Machining with IIoT Feedback Integration

by Linda Torries – Tech Writer & Digital Trends Analyst
October 30, 2025
1 million users discuss suicide with ChatGPT weekly
Technology

1 million users discuss suicide with ChatGPT weekly

by Linda Torries – Tech Writer & Digital Trends Analyst
October 30, 2025
Tree-GRPO Reduces AI Training Expenses by Half and Enhances Performance
Technology

Tree-GRPO Reduces AI Training Expenses by Half and Enhances Performance

by Linda Torries – Tech Writer & Digital Trends Analyst
October 30, 2025
Meta denies torrenting porn to train AI, says downloads were for “personal use”
Technology

Meta denies torrenting porn to train AI, says downloads were for “personal use”

by Linda Torries – Tech Writer & Digital Trends Analyst
October 29, 2025
Next Post
Why Developers are Switching to Google ADK

Why Developers are Switching to Google ADK

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Latest Articles

New MLPerf Inference v4.1 Benchmark Results Highlight Rapid Hardware and Software Innovations in Generative AI Systems

New MLPerf Inference v4.1 Benchmark Results Highlight Rapid Hardware and Software Innovations in Generative AI Systems

March 4, 2025
Agentic AI Workflow Design Patterns

Agentic AI Workflow Design Patterns

May 4, 2025
Comet Browser: The AI-Powered Browser That Just Went Free

Comet Browser: The AI-Powered Browser That Just Went Free

October 6, 2025

Browse by Category

  • AI in Healthcare
  • AI Regulations & Policies
  • Artificial Intelligence (AI)
  • Business
  • Cloud Computing
  • Cyber Security
  • Deep Learning
  • Ethics & Society
  • Machine Learning
  • Technology
Technology Hive

Welcome to Technology Hive, your go-to source for the latest insights, trends, and innovations in technology and artificial intelligence. We are a dynamic digital magazine dedicated to exploring the ever-evolving landscape of AI, emerging technologies, and their impact on industries and everyday life.

Categories

  • AI in Healthcare
  • AI Regulations & Policies
  • Artificial Intelligence (AI)
  • Business
  • Cloud Computing
  • Cyber Security
  • Deep Learning
  • Ethics & Society
  • Machine Learning
  • Technology

Recent Posts

  • Thailand becomes one of the first in Asia to get the Sora app
  • Clinician-Centered Agentic AI Solutions
  • Expert Panel to Decide AGI Arrival in Microsoft-OpenAI Deal
  • Samsung Semiconductor Recovery Explained
  • Closed-Loop CNC Machining with IIoT Feedback Integration

Our Newsletter

Subscribe Us To Receive Our Latest News Directly In Your Inbox!

We don’t spam! Read our privacy policy for more info.

Check your inbox or spam folder to confirm your subscription.

© Copyright 2025. All Right Reserved By Technology Hive.

No Result
View All Result
  • Home
  • Technology
  • Artificial Intelligence (AI)
  • Cyber Security
  • Machine Learning
  • AI in Healthcare
  • AI Regulations & Policies
  • Business
  • Cloud Computing
  • Ethics & Society
  • Deep Learning

© Copyright 2025. All Right Reserved By Technology Hive.

Are you sure want to unlock this post?
Unlock left : 0
Are you sure want to cancel subscription?