#foswiki 2016-05-16,Mon

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***ChanServ sets mode: +o CDot [06:53]
................................................................ (idle for 5h17mn)
ChanServ sets mode: +o gac410 [12:10]
..... (idle for 22mn)
ChanServ sets mode: +o Lynnwood [12:32]
....... (idle for 32mn)
jomohi, no release-meeting today? or i missed its time once again? [13:04]
***ChanServ sets mode: +o Lynnwood__ [13:05]
vrurghi jomo , gac410 was too busy last days I guess. No time to prepare. [13:11]
gac410Oh crap. I was off by a week. I just checked the calendar. My apologies
Time files when you are having fun, Or painting a house as it were.
So jomo, you didn't miss it. I did. :(
[13:12]
jomo:)
nobody other interested anyway - so, maybe in May.30 :)
[13:18]
gac410not a lot of development activity lately, Except for vrurg anyway.
I guess real life interferes. It certainly is for me.
[13:21]
vrurgMine week wasn't that active after all too. [13:22]
jomook, so CU again in 2 weeks :) :) [13:27]
gac410drop in any time :) [13:27]
..... (idle for 20mn)
***ChanServ sets mode: +o Lynnwood [13:47]
.......... (idle for 47mn)
ChanServ sets mode: +o Lynnwood__ [14:34]
.......... (idle for 45mn)
geetar has left
geetar has left
[15:19]
....... (idle for 33mn)
ChanServ sets mode: +o Lynnwood [15:54]
.... (idle for 19mn)
flipflipHi. What's up with https://foswiki.org/? save is veeeeery slow and upload times out all the time. Using "build.pl upload". [16:13]
gac410Sometimes the bots really hammer the server. Things slow down. [16:13]
flipflipSo just wait and try later? [16:14]
gac410You could try uploading to trunk.foswiki.org Same accounts, etc. [16:14]
uebera||Currently, it's http://ahrefs.com/robot/ which is causing the load. [16:19]
flipflipSame database, too, it seems?
*database of topics, attachments, etc.
It just now managed to upload my plugin .zip on normal foswiki.org. Let's see.
[16:20]
gac410Yes, trunk.f.o symlinks most of the pub and data directories from f.o, except for System, Main & Sandbox.
btw you were asking about 'perltidy' and the commit failures. you were not running into perltidy, but into
the topic timestamp / meta checks.
(I hate that particular check. ... really annoying.) I have a script that finds and fixes all the meta of modified topics, to match the current date.
I suppose I ought to commit it to the buildcontrib
[16:21]
flipflipYes, I know. Both, actually, but perltidy complained about stuff that I haven't touched. I couldn't get rid of the timestamp errors. So I've temprarily disabled the hooks. [16:23]
gac410perltidy only checks the .pl and .pm files. The hook checks the meta data. [16:23]
flipflipYes, I know. It checked the whole file (Attach.pm) but complained about lines that I haven't touched. And yes, the other errors were from .txt files.
I eventually gave up. Though I think it could have mean't the META:FILEATTACHMENT timestamps rather than the META:TOPICINFO one. Should I touch all of them when chaning a System topic?
*meant
So the bot can bring down foswiki.org with Crawl-delay: 30?
[16:24]
gac410Some of the bots ignore crawl delay. Really annoying. Have to play whack-a-mole trying to ban them
soon to pop up on other IPs with different agent strings.
[16:27]
flipflipYeah, I know, it's annoying. This one though claims to honour the Crawl-delay. Anyway.. I'll try trunk.foswiki.org now [16:28]
gac410And they also ignore nofollow. Have some that click every column sort of ever table in every direction. Which is a killer. I added sort=off on every table as I came across them in System a while back. [16:29]
uebera||The bot definitely did not honour the delay. That's why now it's known as a bad bot. [16:29]
gac410Sometimes bots hijack other's agent strings too. Not necessarily an honorable profession ;) [16:30]
uebera||Maybe we need some feedback mechanism on Twitter, e.g., "@foswiki: 'AAAbot' (http://bbb), thank you for hammering us (xxx accesses from yyy ips within the last zzz minutes). Please come again. NOT." [16:34]
gac410We should go back to the javascript based sorting. had it on old server but it conflicted with new EditRowPlugin, which does it's on js based client sorting.
But only on pages with an EDITTABLE macro, The other extension. JQTableData plugin??? (can't remember) did it on all pages
[16:40]
.... (idle for 16mn)
uebera||: more bots? I'm having trouble saving a new task.
Ah... finally
[16:57]
GithubBot[distro] gac410 pushed 1 new commit to master: https://git.io/vrnL3
distro/master 07f6e32 George Clark: Item14072: Add script to fix up timestamps...
[17:01]
***GithubBot has left [17:01]
FoswikiBothttps://foswiki.org/Tasks/Item14072 [ Item14072: Add utility script that fixes up topic Meta so that commit hooks don't complain. ] [17:01]
flipflipDisallow: /*?*
That's what I have in my robots.txt. Not sure if it does anything, though. But the idea is to not make them follow anything with query paratmers.
[17:01]
gac410flipflip: from your distro checkout, or any extension checkout, run core/tools/fix_topic_timestamps.pl ... that should make sure meta is all correct. [17:02]
flipflipThat works. Thanks. So it indeed was the META:FILEATTACHMENT timestamps. I don't quite see why I should touch those when changing some text. [17:06]
gac410(I agree with you ... but other devs wanted the attachment and topic meta to all have the same timestamps.
wildcard matching in robots.txt must be something new. I guess a few bots honor it, but not all
[17:06]
flipflipOK, no problem. It's just not very obvious. I'll check if I can improve the hook's output message. It does say "TOPICINFO", but it doesn't say "FILEATTACHMENT". That would have helped. [17:08]
gac410I think it does dump the lines that it doesn't like. [17:09]
flipflipIt says: ERROR: core/data/System/FileAttachment.txt meta-data is incorrect; cannot check in: TOPICINFO: date must be within 259200 seconds of 1463418308 date must be within 259200 seconds of 1463418308 date must be within 259200 seconds of 1463418308
(on four lines)
looking at pre-commit script now
a trivial change will improve the output. That needs a new Tasks.Item I guess.
So it will say: FILEATTACHMENT 'Sample.txt': date must be within 259200 seconds of 1463418781
[17:09]
gac410If you want, hijack Item14072 ... [17:14]
flipflipinstead of just "date must be within..." [17:14]
gac410It's similar enough ; [17:14]
flipflipwill do [17:14]
gac410I'm usually lazy with new tasks, esp. developer changes that don't need release note treatment [17:15]
.... (idle for 15mn)
GithubBot[distro] phkehl opened pull request #10: Item14072: improve pre-commit error messages (master...Item14072) https://git.io/vrnOg [17:30]
***GithubBot has left [17:30]
.......................................... (idle for 3h29mn)
geetar has left [20:59]
....... (idle for 30mn)
GithubBot[distro] gac410 pushed 2 new commits to master: https://git.io/vrnxS
distro/master a9e85ee Philippe Kehl: Item14072: improve pre-commit error messages...
distro/master 0df6b9e George Clark: Item14072: Merge branch 'phkehl-Item14072'
[21:29]
***GithubBot has left [21:29]
FoswikiBothttps://foswiki.org/Tasks/Item14072 [ Item14072: Add utility script that fixes up topic Meta so that commit hooks don't complain. ] [21:29]
GithubBot[distro] gac410 fast-forwarded Release02x01 from ac7dfe0 to c558ce9: https://git.io/vrnpf [21:31]
***GithubBot has left [21:31]
.... (idle for 15mn)
ChanServ sets mode: +o Lynnwood__ [21:46]
...... (idle for 28mn)
ChanServ sets mode: +o Lynnwood [22:14]
................ (idle for 1h17mn)
GithubBot[distro] gac410 pushed 3 new commits to Release02x01: https://git.io/vrcYN
distro/Release02x01 e03af84 Ray Plante: Item14069: SecurityTests: unit test to detect js injection into attach page
distro/Release02x01 3fbedc8 George Clark: Item14069: Relocate & tidy SecurityTests...
distro/Release02x01 2679175 Ray Plante: Item14069: fix: entityEncode vulnerable user input
[23:31]
***GithubBot has left [23:31]
FoswikiBothttps://foswiki.org/Tasks/Item14069 [23:31]
GithubBot[distro] gac410 pushed 1 new commit to master: https://git.io/vrcOe
distro/master 8211771 George Clark: Merge branch 'Release02x01'
[23:32]
***GithubBot has left [23:32]
GithubBot[distro] gac410 closed pull request #9: Item14069: fix XSS (javascript injection) vulnerability in attach (master...Item14069jsinject) https://git.io/vrqMn [23:33]
***GithubBot has left [23:33]
gac410Oh crap. I just re-ran the push message from github for Michael's bad merge. Saw the failed push on github webhook log and hit retransmit. [23:36]
rouilj1gac410 have you considered something like: http://spambot.sourceforge.net/
oh crud. spambot has nothing to do with rerunning the push.
[23:36]
gac410Didn't actually push to the Repo's I just re-ran the webhook which updated all the tasks again. [23:37]
rouilj1ah ok, so you have to re-edit the tasks. [23:43]
gac410Yes Need to review all the commit messages that were processed by the exit, and remove the duplicate commits
What a pain I clicked "replay" and then looked in horror when I realized what I did. No real damage, but a good bit of tedious work.
[23:44]
rouilj1yup.
Also http://opensource.adnovum.ch/mod_qos/ is another thing you can use to limit the access by bots.
more complex than spambot to configure but looks like it provides finer grained access rules (including using browser id srtings).
[23:47]
gac410y, I'll leave that to uebera|| or gmc to consider. I just shot my foot ... and my evening. :( [23:51]
rouilj1fair enough. /me passes bandage [23:54]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)