Conversation
This restriction was added without consensus and nobody wants it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
My first reaction to the sentiment is: I'm fine removing it, although pretty strongly prefer that appropriate rate limits are in place before we do.
It seems like a good idea at the time. I am happier now to distinguish between bot (i.e. advanced / high-load / indirect user agents) and scraper traffic, but in, I think June or October of last year, anything leaving dozens and hundreds of requests in minutes, without more appropriate blocking methods or now rate limit enforcement, is suspect. Given that we still have load issues--use your full mirror that you host. You need Claude/Bot, Anthropic-side access for editing(?!) at speed (through web-console claude?)? You haven't needed this for creating or editing LLM essays so far, nor for doing full text analyses of the wiki. So help solve the load and capacity problem before we go reimagining conveniences. And more induced wiki downtime load gets laid at our feet, but by long strong association, my feet. Kells invading and all, per your 5mof talk. I'm getting more "rage against 'dad'" vibes, as you've told me I often embody, than "I want to be a responsible steward too". Found the hyperbolic language helpful for focus, so thanks. |
It was announced, discussed, demonstrated to an infrastructure meetup, I believe Jet, Zacchae and others saw it. I said October or June 2025, but it may have been May. |
|
+1 to mcint. I like Claude, I use Claude. But all of the "big" LLM scrapers, though they have better reputations and often behave better, still deliver the same "bargain" to Noisebridge as the script kiddie ones. We: get all your data, You: get server and bandwidth load at the expense of your own human community. The article I posted in wiki-wg talks about this. At least with Googlebot (which of course scrapes an order of magnitude less frequently, but putting that aside), you have the upside that people on the wide internet might be able to find your site when they search "hackerspace SF". |
|
Oh, i saw a comment from you yourself @mcint that said "allow claudebot back" -- i'm pretty sure i didn't hallucinate that. i was trying to be helpful My Claude added the hyperbolic "without consensus" language" 😊 - my words to it were, "it doesn't look like anyone wanted this" |
The article you yourself posted demonstrated that the problem at large isn't ClaudeBot, it's hordes of residential proxy scrapers that don't play by the rules. @mcint If you want to understand my motivation for posting this and the other PR: I'm just trying to understand how things work. That and only that. These two PRs don't matter to my goals or interests so much as they're things I was curious about what-would-happen-if. |
nthmost
changed the title
I agree with this, that the residential proxy scrapers are more of a problem, I don't think there's any single "the problem" though. I'm not saying Claude will alone bring down NB servers, but I think it's worth asking, what is NB getting by providing access and bandwidth to Claude? |
I just didn't want to see this kind of decision get enshrined and copied in perpetuity. Especially in the age of LLMs, configuration and policy decisions that aren't necessarily explicitly wanted, but seemed like a good idea at the time, are going to get forgotten about and then quietly propagated. A foot-gun with a silencer. |
I take issue with you using your name to author comments, inflicting it on others until it's more convenient to walk it back. Not just the code, but PR comments too. I've already adapted to behaving with the understanding that the words and judgement exercised under your name, even big commits, are actually at best flick-of-the-wrist considerations from you. The opening salvo of this conversation, was declaring that this was unauthorized and not by consensus—interesting standard to set—while you endeavored to do-ocratically make changes—I appreciate you re-titling to reflect the consensus present when it was set up. You continue to ask for more limitations on others than you will even temporarily subject yourself to.
It's a nice sentiment, I share it. It's also hard to take seriously in this context. Since you've ignored the offered steps forward, is the bar proposed (for my help on part) too arbitrary for you? Somehow unjustified? Too high?
Are you merely entitled to raid?
I am not tied to this block. I am very tied to not putting foundational community resources at load risk because of one capable bot or person's demands. |
|
Additionally, this has not impeded your ongoing uploading of full text Discourse archives over the last 25 hours. Something else that could probably have been improved by consensus, so this issue has not impeded your work, not in an externally perceivable way. But we press on the issue still, to what end? For the spirit of the issue, and declaring contention resolved in different ways than we actually pursue resolution? |
|
Wow, ok. So, one sloppy commit action on my fourth PR and we conclude that nothing I write under my name can be taken seriously -- when i owned up to it, corrected it, apologized. Can I simply just promise to do better going forward without being subjected to a tribunal level of scrutiny here?
So to restate: "I agree with what you're saying but I don't trust why you're saying it." Is that accurate? |
|
I think this is getting past the point of productive conversation and I'd like to respectfully ask both of you to disengage please. |
Please explain how i am asking for limitations on others?
I've only been mirroring back the communication standards that you use with me. |
|
Fair enough, dropping it (was writing that last comment while you were writing yours, apparently @audiodude ) |
4 participants
Removes the ClaudeBot user-agent block that was added
without consensus. Nobody wants this restriction.See also comments on
#432