#foswiki 2014-07-15,Tue

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***orwell.freenode.net sets mode: +oo pharvey SvenDowideit
orwell.freenode.net sets mode: +o ChanServ
[03:13]
......... (idle for 43mn)
gac410What is going on with FoswikiBot ... I don't see any errors on the server. [03:59]
..... (idle for 23mn)
***gac410 has left [04:22]
...................... (idle for 1h49mn)
ChanServ sets mode: +o pharvey [06:11]
.................................... (idle for 2h59mn)
foswiki_irc4Hi, I'm trying out the foswiki VM from Download/DownloadVirtualMachineImage, but can't access bin/configure. What user/password should I use? I've tried admin/foswiki but that doesn't work. Note that the VM is running on my machine, I'm accessing from my desktop firefox browser. Thanks. [09:10]
......... (idle for 43mn)
***orwell.freenode.net sets mode: +ooo pharvey ChanServ SvenDowideit
orwell.freenode.net sets mode: +o Babar
[09:53]
...... (idle for 27mn)
ChanServ sets mode: +o Lynnwood [10:23]
............................... (idle for 2h33mn)
ChanServ sets mode: +o gac410 [12:56]
gac410foswiki_irc4: Did you download the latest 1.1.9 version of the vm? [12:57]
TarboxHey, guys. How's LdapContrib going? I'm ready to contribute if it's not. [13:00]
***dgretch has left [13:01]
gac410Tarbox what do you mean? I know people use it, MichaelDaum is the primary maintainer [13:02]
TarboxSomeone from TWiki posted a huge amount of updates and Michael was a little concerned about the time it would take to migrate it to Foswiki.
IIRC
[13:02]
gac410foswiki_irc4: Did you follow the steps on http://foswiki.org/Support/VirtualMachineImages All the passwords used are there
Tarbox: I think I recall comments that the updates had issues.
[13:03]
TarboxThat's where I think I can help. [13:04]
gac410http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2014-06-11,Wed&sel=4#l0 [13:04]
Tarboxdrat
hah I was even there that day.
Well the current LdapContrib is having issues. Groups lose members sometimes. refreshldap=1 fixes it.
[13:05]
gac410Tarbox you had a question that day about isInGroup and Expand
As far as LdapContrib goes, I don't know anything. Expand option on groups - I added that one I vaguely recall.
[13:07]
TarboxYeah. One line fix to the code turned something that was taking 40ish milliseconds down to microseconds. [13:08]
gac410Ooo... is it general enough to contribute it back? [13:08]
TarboxWell, I've only confirmed it's "safe" for LdapUserMapping and TopicUserMapping.
Hence the question on that day.
[13:08]
gac410Well the only testing I've done is TopicUserMapping, and the unit tests. [13:09]
Tarboxexpand is passed on to the underlying mapping, the mapping performs the expand, returns the result, and then the calling function performs a second expand, which does nothing but call "isGroup" once for each member in the group (that the mapping already expanded into just users)
If the rule is that mappings have to respect the expand, it's safe to just remove that line.
[13:09]
gac410The option was added so that the Group maintenance API could suppress the expansion. hm... When the expand happens is it recursive groups contain groups contain groups ?
I wish I could figure out why foswikibot is thrashing.
All I see on the server is: WARN 2014/07/15 07:23:24 Lost connection to server chat.eu.freenode.net. no reason why.
Tarbox, as long as the expand is *complete*, then the isGroup test is excessive. It's too long ago for me to recall though.
[13:10]
TarboxUserMapping::isInGroup is recursive on itself when expand is set. [13:14]
gac410Hm. yes that does come back to me. [13:15]
TarboxIt also calls TopicUserMapping::eachGroupMember with an expand option set. or LdapUserMapping of course.
LdapUserMapping::eachGroupMember is recursive on itself.
[13:15]
gac410yeah. I'm recalling it. It was a bit of a challenge to get expand propagated everywhere. through some convoluted calls.
If you want to check it in, open a task and I'll take a look and run a pass through the tests. It sounds like a reasonable fix.
[13:16]
TarboxI have never done that. Are there instructions? [13:18]
gac410"done that" ?? checkin a fix, open a task , or run unit tests :) [13:19]
Tarboxyes [13:19]
gac410Ah. I'd have to get you core developer access to svn, and you'd need to build a development envrionment. This will be Sooooo much easier if we could get the git migration completed. [13:20]
TarboxIt's one line... [13:20]
gac410If you could, just open a task. and I'll make the change. [13:21]
Tarboxright. Instructions for that? [13:21]
gac410Foswiki.org/Tasks "Create a new task button" .... fill in what you can and we'll clean it up. [13:22]
TarboxThanks. [13:22]
gac410Thank you. Every little bit helps for sure! [13:22]
TarboxI'm ready for a big bit on LdapContrib. I don't even know where to start debugging. [13:22]
gac410Ah... Log in first. [13:22]
TarboxEven if there's just a list of bugs that TWiki claimed to fix, I can do something.
I hear DB_File mentioned once.
[13:23]
gac410Do you have access to the TWiki website? [13:23]
Tarboxif it requires something mroe than browsing, no. [13:23]
gac410Some of their site requires login. I think you can look at tasks, but the trac tool for reviewing changes is locked down. [13:24]
TarboxIs that just a matter of signing up? [13:25]
gac410No idea. I've been away from twiki for a long time. You might have to request permission. [13:26]
JulianLevensTarbox, someone else is also very interested in LdapContrib enhancements [13:30]
TarboxIt could be me. I complain a lot sometimes. [13:31]
JulianLevensNope, see http://irclogs.foswiki.org/bin/irclogger_log/foswiki?date=2014-07-10,Thu&sel=9#l5 [13:31]
Tarboxwoo [13:31]
JulianLevensMaybe you could work together?
I think this guy has done quite a lot of work on this
[13:32]
TarboxYou know, related issue, not as urgent as LdapContrib, I've been using 1.2.0 caching and I get deadlocks with both the perl db and full grown mysql
I think of it since he mentioned mysql.
[13:33]
gac410yes he did the checkins on twiki as well. At least he closed the enhancement tasks in May. [13:33]
TarboxAh, well, that makes sense. [13:34]
JulianLevensPing TerjeAndersen [13:34]
gac410Tarbox, That one should have an urgent task ... Cache issues on trunk IMO are critical - should block 1.2 [13:34]
TarboxIt's so vague though, what od I even say? I can't get much farther than "It don't do right." [13:35]
gac410hm. What happens with the deadlock? crash, hang ??? [13:35]
Tarboxcrash if it's the baby one... god I need to look up the name of that.
deadlock error returned to the browser if it's mysql
DBI::SQLite
crashes hard. mysql just returns an error.
[13:36]
gac410I'd recommend opening a task, mark it urgent, if you have any info from the crash - where the crash occurrs, anything would be helpful.
Hey gmc ... good morni... er. never mind.
[13:38]
TarboxIt appears to be a collision. Occurs randomly while under load. [13:38]
gac410Do you have users who do things like open multiple browser windows and run simultaneous attach uploads to the same topic? [13:39]
TarboxEdit access is highly restricted, but I can easily see my users opening multiple tabs.
So easily that I'd have to consider that a functional requirement.
[13:39]
gac410Yeah, I'm just guessing of things that might cause collisions. not saying it shouldn't be supported. [13:40]
TarboxSorry, that came out poorly. I'm writing up this task and it changes the way I think. [13:40]
gac410And it doesn't have to be edit doh... I keep forgetting that cache is updated on view. [13:41]
Tarboxand rest
if it matters
[13:41]
gac410yes. there was a bad bug with rest that I fixed. [13:41]
Tarboxwhen? [13:42]
gac410It used the same Web/Topic as view, Let me look. [13:42]
Tarboxhttp://foswiki.org/Tasks/Item12964 [13:43]
gac410Foswikirev:17738 [13:43]
FoswikiBothttp://trac.foswiki.org/changeset/17738 [ Changeset 17738 – Foswiki ] [13:43]
gac410Tarbox, Excellent! Thanks for the fix. I'll review it and run the unit tests.
The cache issue was Item12915:
Hello foswikibot :P
[13:45]
***ChanServ sets mode: +o Lynnwood [13:46]
gac410http://foswiki.org/Tasks/Item12915 [13:46]
Tarboxthe view/rest bug is interesting, but I don't think it was causing my deadlocks. Everything we have is either always view or always rest in the normal course of events. [13:52]
gac410Ah Tarbox, I think that the code can't be commented out, but it could be conditional. [13:52]
TarboxWell, that's why I was asking about it the other day.
I knew it was making assumptions about the implementation, but I didn't know if that was part of the contract.
[13:53]
gac410hm... maybe not. It already is conditional
no you are right, It's not needed.
If expand is true, then no groups should have been returned. I'll remove it and see if the tests all pass.
[13:53]
Okay, no change in unit test results with or without that code. I agree it seems bogus. And you can squarely blame me. It was not leftover code. I added it.
Redundant redundancy. Belts and suspenders I guess :)
[14:02]
TarboxIt's reasonable not to want to be pantsed. [14:05]
gac410Just checked in the fix. Thanks. [14:05]
GithubBot[foswiki] FoswikiBot pushed 1 new commit to master: http://git.io/HC3mRQ
foswiki/master 52fec0f GeorgeClark: Item12964: Remove useless scan of groups...
[14:08]
***GithubBot has left [14:08]
gac410There are a number of very recent fixes to the TWiki renderer that claim to improve performance on long pages containing tables & lists. I was thinking about merging them, but CDot probably should review as they are changes to table code. [14:09]
TarboxDo you still want me to open a task about page caching? [14:10]
gac410Yes please. If you are seeing random deadlocks, it should be reported. [14:11]
Tarboxoh hell, I didn't know I could debug this. [14:11]
***dgretch has left [14:14]
................................. (idle for 2h40mn)
GithubBot[foswiki] FoswikiBot pushed 1 new commit to master: http://git.io/9By63w
foswiki/master f4eefaf GeorgeClark: Item12888: Tests fail if config changes def webnames...
[16:54]
***GithubBot has left [16:54]
............ (idle for 58mn)
ChanServ sets mode: +o MichaelDaum [17:52]
TarboxMichaelDaum, if you need any gruntwork on LdapContrib I'm available. [17:52]
MichaelDaumHi Tarbox
MichaelDaum reading logs ... sec
I reviewed TWiki's LdapContrib the other day ...
[17:53]
TarboxSomeone linked me that. You did not seem excited. [17:54]
MichaelDaumit has got about an additional 1.3k lines of code [17:54]
Tarbox@_@ [17:54]
MichaelDaumyes I do have concerns about some of these changes
anyway
whatever you try to do, Tarbox, please first create a bug item that properly explains what you are trying to do ... before you get started ... not after
then we can discuss what is best
most importantly: you need to get me on the train so that I understand the issue
okay Tarbox?
[17:55]
Tarboxsure [17:58]
MichaelDaumgreat. looking forward. [17:59]
TarboxWhat's happening right now is every once in a while a group gets truncated.
I tell it to reload and it works again.
Now, I do have some custom reload code, but I also have debug telling me every time my code is run, and there doesn't seem to be a connection between reloads and losing group members.
[18:00]
gac410MichaelDaum: since Cache is your baby :) Tarbox reported earlier he's seeing deadlocks on occasion in 1.2 [18:01]
TarboxI'm still trying to get something other than "It don't do right" for that.
Trying to figure out how to turn on deadlock logging for innodb.
Then I'm planning on just spamming the server with requests until something dies.
[18:01]
MichaelDaumhm, I've been running page caching using sqlite for quite some time without any problems [18:02]
gac410Hm. Quick google: SHOW INNODB STATUS This will show you the most recent deadlock, including the two competing queries [18:02]
TarboxAmusingly, I run that and it doesn't list a deadlock.
I suspect my status got flushed or something.
s'been a while.
So I figure if I'm going to force it, I may as well do it right.
[18:02]
gac410ping gmc_ are you around? [18:04]
MichaelDaumtrunk.f.o has been running page caching for ages now [18:05]
TarboxWell I'm not claiming I haven't done something stupid. [18:05]
MichaelDaumwhat's the kind of circumstances that trigger the error for you, Tarbox ? [18:05]
gac410Unfortunately the activity is pretty low. Tarbox, you said with sqllite, you were seeing a crash when the deadlock occurs? [18:05]
TarboxPages would come back with some sort of apache error. I'm afraid I can't remember anymore.
It was a sort of "OMG THROW THE SWITCH THROW THE SWITCH" moment for me.
[18:06]
MichaelDaumdon't understand [18:06]
TarboxI'm remembering now.
If I used sqlite sometimes pages got timeouts.
40 seconds, coming from fcgid
If I use mysql I got deadlock errors.
which, on the bright side, did not take 40 seconds.
[18:07]
MichaelDaumhow many users do you have on the site? what kind of pages are you testing?
how many updates do you see at the moment of the deadlock?
[18:07]
TarboxWhen it was live we have several hundred active users at a time, but even with that big number we only got a page request once every few sseconds.
No updates.
Or at least, no edits, if that's what you mean.
[18:08]
MichaelDaumso multiple requests, all fetching pages from the cache? [18:08]
Tarboxyes [18:08]
MichaelDaumthese all to a select each on a page variation
can't see how simultaneous selects could deadlock a DB
in any case: good that you hammer the code
but for a real bug, it needs a bit more info how to repro.
[18:11]
TarboxYeah that's what I was saying earlier to gac410. :P [18:13]
MichaelDaum;/ [18:13]
Tarboxworking on it. [18:13]
MichaelDaumthanks
have to run again. let me know via f.o/Tasks. see you.
[18:13]
Tarboxyou too [18:14]
....... (idle for 31mn)
How on earth does a table get 3000 rows without an UPDATE_TIME? [18:45]
.................. (idle for 1h29mn)
Well I cannot duplicate the deadlock error but I can create all the deadlocks I want. Is that worth it? [20:14]
gac410What is it deadlocking on? [20:14]
Tarbox*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 4405 n bits 384 index `foswiki_cache_deps_index` of table `Foswiki`.`foswiki_cache_deps` trx id 31BC1 lock_mode X waiting
Record lock, heap no 235 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
0: len 12; hex 4d61696e2e576562486f6d65; asc Main.WebHome;;
1: len 19; hex 4d61696e2e576562507265666572656e636573; asc Main.WebPreferences;;
2: len 6; hex 00000007c38d; asc ;;
looks like the most interesting part? I have several pages of info on this.
[20:15]
gac410I'd suggest opening a task, and documenting what you can. Hm. it's deadlocking on the index? [20:16]
TarboxYeah.
I cheated though.
Since our wiki is mostly read only but it was erroring out on views, I made a wild guess and set it to pretend that the cache was always dirty.
Throw some spam at it and bam deadlocks.
[20:17]
gac410Well the "first view" for a user / page combo will write a new cache entry. So I could see view causing deadlocks, [20:18]
TarboxI have a small number of users being shared by a larger population.
like multiple wikiguests, depending on circumstances.
[20:19]
gac410ah... so multiple endpoints hitting the same cache entries. [20:20]
Tarboxyup
And caching is either on or off.
So turning it on opens the floodgates on cache misses.
[20:20]
gac410Another good point to note in the task. [20:20]
TarboxYou wanted me to mark this urgent, right? [20:22]
gac410do you know if the cache is using record or page locking [20:22]
TarboxI'm using the default settings for everything. Is that a foswiki setting or a mysql setting? [20:23]
gac410Yes I think it should be urgent, We can downgrade once it's understood more
Not sure, it might depend on how the table was created / type of db (innodb?) etc.
[20:23]
TarboxIs this engine or extension? [20:23]
gac410Engine
Just talking about it here, It may be due to commit strategy, are we doing too much work from the start to the commit. I'm not really familiar with our cache code so no idea.
[20:23]
TarboxWell I think an "aw shucks" commit strategy would work. If something's locked throw your work away.
It'll all get cached eventually.
[20:30]
How do I set my email on this in a way that won't be harvested? >_> [20:40]
gac410email on what? [20:40]
Tarboxthe task [20:41]
gac410You don't need to set your email. It uses the email in your registration info [20:41]
TarboxHow do I look at that?
I've a habit of using fake emails.
[20:41]
gac410hang on
you are using a hotmail domain ... does that help?
[20:42]
TarboxYes.
It's the wrong one.
from my current POV at least.
[20:43]
gac410You can change your email at http://foswiki.org/System/ChangeEmailAddress
That even shows your current email.
[20:43]
Tarboxthanks. [20:44]
gac410Nobody can see your email except for the few of us in the admin group. [20:45]
ping gmc are you around. Just wondering if you have any ideas on what's happening with Foswikibot [20:51]
TarboxNow I just need to figure out how to duplicate the ldap thing. [21:00]
........ (idle for 36mn)
gac410Tarbox: If you enable TRACE in Foswiki/PageCache.pm and Foswiki/PageCache/DBI.pm Does that provide any additional information around the deadlock? [21:36]
TarboxI'm going to have to try that tomorrow. [21:37]
.......................... (idle for 2h6mn)
foswiki_irc7Hi, I'm having trouble with "403 - Forbidden: Access is denied" being shown when a user tries to edit a document that is locked by access control. Can anyone help? [23:43]
gac410Is the user supposed to be able to edit?
And can anyone else edit?
[23:44]
foswiki_irc7Hi, no, the user is not allowed to edit the page. When I use a browser on the server, the correct Foswiki "Access Denied" screen is shown. However, if I do the same operation on another workstation, the browser returns the "403 - Forbidden: Access dened" error. [23:46]
gac410Ah.. okay. hm... So the wiki is operational and that user on that workstation can edit other topics?
This sounds like it could be apache configuration.
[23:47]
foswiki_irc7Yes, no problem with creating new topics or editing existing ones that are available for edit. Server is running Microsoft Internet Information Services Manager 7 [23:49]
gac410ah... IIS. We don't have many using that one for sure. I guess I can skip any apache questions I had. :( [23:50]
foswiki_irc7I agree that it's probably a permissions thing, but I've had a good look at the setup and can't see the problem. [23:52]
gac410So a user at a remote workstation is fully functional.... then you add "Set ALLOWTOPICCHANGE = SomeUser" in the topic, and the edit fails with a 403... [23:52]
foswiki_irc7Actually I've been using the Workflow plugin to lock the page, but I will try the "Set ALLOWTOPICCHANGE = SomeUser" approach to answer your question, and will report back. [23:54]
gac410Hm. I don't remember what WorkFlowPlugin does to protect the topic.
WorkFlowPlugin does throw a 403 error when blocking an edit: Message should read: You are not permitted to edit this topic. You have been denied access by Workflow Plugin'
It might be that IIS handles that 403 return differently from Apache? No idea though, just guessing.
[23:55]
foswiki_irc7I've just created a new topic, locked editing using the "Set ALLOWTOPICCHANGE = SomeUser" technique, and the user on the remote workstation gets the same "403 - Forbidden: Access is denied" error. [23:58]
pharveyis it an IIS 403, or a Foswiki "access denied" page
(sorry if this was already answered)
[23:59]
gac410What I really don't understand ... if you do the same thing running the browser on the you get the Foswiki access denied, and not the IIS 403 [23:59]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)