AI bots
-
Mitigating the impact of AI bots
Over the past few years, repositories have been encountering a growing number of bots trying to access their resources. These bots, or crawlers, navigate the internet, gathering data and indexing information for search engines and large language models, as well as for other purposes. While some of these bots are rather innocuous, others are sufficiently
-
COAR Launches AI Bots and Repositories Task Force
There are a growing number of aggressive crawlers interacting with repositories. While many crawlers are rather innocuous, others are sufficiently aggressive that they are increasingly causing service disruptions in repositories. A survey undertaken by COAR in April 2025 found that 90% of respondents indicated their repository is encountering aggressive crawlers (referred here colloquially as “AI
Categories: AI bots
-
Open repositories are being profoundly impacted by AI bots and other crawlers: Report from a COAR Survey
Executive Summary Every day, multiple bots access the repository at all hours 24/7. We estimate performance degradation due to bot activity about once or twice a day, and at least once a week the system crashes entirely requiring an intervention – typically a service restart. Survey respondent There are a growing number of AI bots
Categories: AI bots
Categories
- AI bots
- Annual Meetings
- Artificial Intelligence
- COAR Notify
- Good Practices for Repositories
- Governance
- Knowledge Commons
- Metadata and Vocabularies
- Multilingualism
- Next Generation Repositories
- Our Collective Voice
- Paper Pledge
- Publish-Review-Curate
- Regional Initiatives
- Repositories and Publishing
- Research Data Management
- Resources
- Training and Capacity Building
- Uncategorized