#foswiki 2012-01-23,Mon

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
gac410NelsonC: Bad news. I installed ubuntu 11.10, installed the minimum dependencies, untar'd foswiki, renamed the sample httpd config file, restarted apache. configured foswiki, email to gmail worked fine.
Packages I installed: apache2 rcs libnet-smtp-ssl-perl libalgorithm-diff-perl libapache-htpasswd-perl libarchive-tar-perl libarchive-zip-perl libcgi-session-perl libdigest-sha-perl libdigest-sha1-perl libhtml-parser-perl libwww-perl liblocale-maketext-lexicon-perl liburi-perl libhtml-parser-perl
[00:19]
dj_segfaultSo the bad news is you installed Ubuntu 11.10? I sympathize. [00:23]
gac410no... I was unable to recreate his issue. [00:24]
dj_segfaultI know, just joking. I'm staying with 10.04LTS until they can show me actual usable improvements. [00:24]
gac410On his system, Net::SMPT::SSL was unable to negotiate a mechanism for Authen::SASL and email fails. [00:24]
dj_segfaultI was trolling yesterday and caught some of that. [00:25]
pharveyI'd stick with 10.04 but I've been unable to get foswiki + mongodb + utf8 working properly on perl 5.10 [00:25]
gac410So maybe he is missing a package, or it's related to 64bit. [00:25]
dj_segfaultI tried several times to set up SMTP auth on my home server and never got it working right. Right now I can only send from my intranet, which sucks. [00:26]
pharveypharvey wonders if he did an cpan install of perl modules [00:26]
gac41011.10 has perl 5.12.4
NelsonC ... I also ran the automatic update after first boot. nothing else, Did you do anything differently?
[00:26]
dj_segfaultI see mongodb in Synaptic. What doesn't work (feel free to tell me to shut up and sit down if it's a long story and you know for a fact it doesn't work) [00:28]
gac410i REALLY don't like the ubuntu api. It was bad enough with gnome. Now whatever it is they have - took me 20 minutes to find xterm, and I have yet to find a gui package manager. [00:28]
dj_segfaultThey don't use Synaptic anymore? [00:29]
gac410make that GUI not API ... Synaptic is not installed by default. [00:30]
pharveygac410, it's the Unity debacle I've been grumping about. Nobody uses a real PC with large screens anymore - it's all about the tablets ;-)
dj_segfault: I get unicode character corruption in Foswiki, partly because Foswiki isn't unicode-clean, and MongoDB *is* - anyway, I suspect there's some core module in 5.10 that's got a bug that contributes to the problem, whatever the case, perl 5.12 'fixes' it
[00:31]
dj_segfaultM'kay. I can't imagine it would be hard to get perl 5.12 onto Ubuntu 10.04LTS, but it may be more work than it's worth. [00:33]
pharveyI'm sure once we get Foswiki core doing unicode properly, my problem will go away; I've spent days trying to isolate modules and figure out reliable ways to reproduce the problem, but it just boils down to the fact we need to get Foswiki core doing unicode properly.
Well, that's what I've done in prod - but it bothers me, because the rest of the system is written against perl 5.10, and it shows (lots of warnings all over the place, luckily no breakages yet).
[00:33]
dj_segfaultYou could use 10.04LTS as your desktop and run the server in a VM with a newer version. That way you have the Gnome2 UI we all mostly love, and the server gets what it wants. [00:34]
pharveyI really shouldn't have to play these games, so I went to xfce... also I like perl 5.14 (I'm on debian testing on the desktop :-) [00:35]
dj_segfaultIf you have the disk space, it makes a certain sense to do your work in a VM so you can test in different environments [00:37]
pharveyOf course. That's what I do
there's a difference between my desktop needs and server needs, I think I've been conflating the two in my discussion here.
(most) of our services are split into test+prod VMs, sometimes it's just easier to reverse the roles of two VMs when upgrading something difficult (like Foswiki), so the back-out plan is much simpler
[00:38]
........ (idle for 39mn)
***flexibeast has left [01:19]
NelsonCgac410: Let me see which of those packages I installed (I took notes). Is there a way to check via command line? No GUI. [01:23]
gac410NelsonC: dpkg --list will list installed packages. grep for the names? [01:30]
NelsonCok [01:30]
gac410CDot - I can't save comments into WorkflowPlugin controlled topics. new 1.1.4 install, svn version of CommentPlugin, WorkflowPlugin, and your new Meta.pm
If i revert the Meta.pm change, then comments save fine using the new WFP
[01:38]
NelsonCgac410: I had surprisingly few of those packages. Are any of those missing mandatory? http://pastebin.com/L9WmvRVQ [01:41]
GithubBot[foswiki] foswiki pushed 6 new commits to master: http://git.io/7VlDOQ
[foswiki/master] Item11464: DownloadZipPlugin initial release - OliverKrueger
[foswiki/master] Item11464: borked import - OliverKrueger
[foswiki/master] Item11464: borked import - OliverKrueger
[01:47]
***GithubBot has left [01:47]
FoswikiBothttp://foswiki.org/Tasks/Item11464 [ Item11464: DownloadZipPlugin initial release ] [01:47]
gac410NelsonC - no probably most are optional. I ran algorithm-diff is needed for compare to work. The archive tools are used for installing plugins, etc. cgi-session - I think is mandatory. libwww-perl that's needed for configure to download files.
If you don' thave the libdigest - those might be needed for auth to work.
[01:55]
NelsonCThat's odd. I've been following the instructions...where should I have looked to know to install these? [01:56]
gac410They are listed in the InstallationGuide. It lists the perl modules required. [01:57]
NelsonCOk let me go through that with a fine tooth comb [01:57]
gac410Most are listed as "Optional" but ... probably better to install.
Going through the list. If Archive;:Tar and/or Archive::Zip are missing, configure will try to use the shell tar or zip tools if needed.
CGI::Sessions I wouldn't consider them optional, but foswiki will run without sessions.
[01:58]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/zwkHSA
[foswiki/master] Item11464: DownloadZipPlugin initial release - OliverKrueger
[02:00]
***GithubBot has left [02:00]
gac410The Digests are needed for passwords, and probably also used by Authen::SASL I'm guessing.
Never did find Encode::compat for ubuntu, but probably not needed for english language systems.
HTML::Entities and Parser. required if you want Wysiwyg.
LWP ... that's needed if foswiki is going to do any external access. libwww
URI - not sure what it's used for.
[02:00]
NelsonCShould I test after installing each one to see if things get fixed or otherwise do this in a structured way? [02:03]
foswiki_irc9Hi Kip, I have another question for you please. Can you please tell me why below statement does not work [02:04]
gac410My best guess would be to try the Digest modules. and then test [02:04]
foswiki_irc9%CALC{$LISTSIZE($percntSEARCH{"parent.name = 'ParentTopicTest1'" nonoise="on" type="query" format="$topic, "}$percnt)}%
The value of LISTSIZE should be 4 it returns 2
[02:04]
gac410foswiki_irc9: Are you trying to count the children of a topic? [02:06]
foswiki_irc9Yes, but in the actuality I am just trying use a dynamically generated list size so I can use it in a Foreach statement [02:07]
gac410I thought there was a way with formatted search to just return the number of hits, but I'm not getting anywhere.
%SEARCH{"blah" format="" footer="$ntopics" nonoise="on" }%
[02:16]
pharveyfoswiki_irc9: ALC doesn't know how to evaluate $percntDELAYED macros, so $percntSEARCH won't work
foswiki_irc9: try %SEARCH{"parent.name='ParentTopicTest1'" type="query" nonoise="on" format="" footer="$ntopics"}% to get the no. matching topics
[02:17]
gac410yup - that's what I said ... sorta [02:18]
pharveyah, sorry, multitasking, and multifailing here :-) [02:20]
NelsonCgac410: Installed the Digest::SHA and SHA1 modules and test email again, same error [02:24]
foswiki_irc9Yes, It did return 4 Thanks! I will try to place this in my Foreach statement now. :) [02:25]
gac410hm. I was really hoping that was it. My only thought now is something 64bit related? [02:26]
NelsonC;_;
If you're running a VM dedicated to Foswiki, what is your preferred OS?
[02:26]
gac410I doubt it could be any of the others. [02:27]
NelsonCOne other thing: after installing those packages, is there anything I have to reload (apache, etc)?
I just did the "sudo install..." and then tried the email test again
[02:27]
gac410NelsonC: I doubt a reload will help. [02:28]
pharveyfoswiki_irc9: btw, ForEachPlugin plays funny games with Foswiki's macro expansion behaviour, see http://foswiki.org/Support/Faq39 [02:29]
gac410Hm. Authen::SASL supports two backends. pure perl, and cyrus
try installing libauthen-sasl-xs or something similar?
[02:29]
NelsonCI got an "unable to locate" error for that
I see a libauthen-sasl-perl and libauthen-sasl-cyrus-perl (using Google)
[02:32]
gac410do you have libsasl2-2 and libsasl2-modules installed? [02:36]
NelsonCYes they're both there [02:37]
gac410I'm running out of ideas. I've got a nearly default install of ubuntu 11.10 - only the packages listed I installed, plus the auto update of a few hundred packages after install. [02:43]
NelsonCI remember selecting the "auto update security patches" option but I haven't tried to run any update [02:44]
gac410Maybe that's the difference. I let it do a full upgrade.
The update manager reports "your system is up to date" on my install.
[02:44]
NelsonCaccountsservice apport command-not-found command-not-found-data
initramfs-tools initramfs-tools-bin initscripts language-selector-common
libaccountsservice0 libcups2 libpam-smbpass libwbclient0 python-apport
python-gobject python-gobject-cairo python-problem-report samba samba-common
samba-common-bin samba-doc smbclient sysv-rc sysvinit-utils tzdata
tzdata-java upstart winbind
those will be updated
[02:46]
gac410well nothing sounds obvious. :-(
About the only other thing I can think of - I installed ubuntu-11.10-desktop-i386.iso
I don't have a real 64bit system to test on.
[02:47]
NelsonCHmm. I mean you're welcome to SSH into mine but I don't want to impose. [02:51]
gac410Interesting - found a bug report. http://forums.devshed.com/perl-programming-6/smtp-ssl-not-working-on-32-bit-systems-635440.html
Same problem, but reversed. Failing on 32 bit works on 64. Suggestion is that it is a module built on 64bit failing on 32bit.
Suggestion was to install the modules using CPAN so they will locally compile.
Might try uninstalling libauthen-sasl-perl, and install Authen::SASL using CPAN
[02:52]
NelsonCok I'll wokr on that. I recall seeing a CPAN tutorial on the foswiki page somewhere. [02:56]
gac410as root. perl -MCPAN -e shell It should ask a bunch of questions defaults usually are right.
Then from the cpan shell, install Authen::CPAN
er. install Authen::SASL
[02:59]
NelsonCOh that's right. I remember trying this earlier before giving up and doing it with packages. I just get zillions of errors like this http://pastebin.com/8mGNRNJZ [03:01]
gac410install libwww-perl
LWP is a fundamental perl module needed for ftp, http, etc.
[03:02]
NelsonCthat sounds about right [03:03]
gac410Probably need that for foswiki as well.
If you want configure to install plugins, and/or want %INCLUDE to include external urls (though that is not recommended for security)
[03:03]
NelsonCCannot install libwww-perl, don't know what it is.
" "
[03:04]
gac410apt-get install libwww-perl ??? [03:04]
pharveyI highly recommend installing dh-make-perl, you get a tool called cpan2deb. Then cpan2deb Authen::SASL, and install the .deb it generates. [03:05]
NelsonCoh I'm sorry I thought I was supposed to do "install libwww-perl" in the cpan> [03:05]
pharveyrather than doing cpan install [03:05]
gac410sorry NelsonC - pharvey probably has a better suggestion. That way apt will know about all your local mods. pharvey - that will compile Authen::SASL locally? [03:06]
pharveyyes
it will complain about missing build dependencies, that's the only annoying part
whereas cpan will temporarily install build-deps for you automagically
to be honest, it probably doesn't matter, I just prefer using debs ;-)
[03:06]
NelsonCalrighty; I'll try that route! [03:08]
gac410And the package updater won't automatically replace it with the ubuntu version at some future date? [03:08]
pharveyoh, it will, if the distro comes up with a later version, yes
usually I consider that a feature, but in this case, maybe not...
you can always use apt pinning to prevent apt from touching that package
[03:09]
gac410If it quietly breaks email 6 months from now, that might not be easy to debug. [03:09]
NelsonCI'm looking forward to that head-scratcher [03:10]
gac410in any event, if compiling Authen::SASL locally, that will be really good to know. From the other google hits about 32bit Authen::SASL makes you wonder if they "fixed" 32 bit, and somehow broke 64
if compiling ... locally ^^fixes it^^ that will be good to know
Maybe even worth adding to the known issues even though it's not our bug
[03:10]
pharveyNelsonC: alternatively you can tell apt to 'hold' (don't update) an already installed package, something like: echo libauthen-sasl-perl hold | dpkg --set-selections [03:13]
NelsonCAt least I get appropriate errors when trying a test email with libauthen-sasl-perl removed http://pastebin.com/SPzt7jnB
Wow, 120MB for dh-make-perl?
[03:14]
gac410I wish I could have recreated your issue locally - I might have been able to figure out why Authen::SASL can't find an available mechanism to use. [03:16]
pharveyNelsonC, that includes a lot of build-deps for making XS stuff IIRC.
you probably want to install build-essential too.
Or maybe that's already depended upon indirectly.
[03:19]
NelsonCI'll check when it's done
Yeah, that was included
[03:22]
gac410NelsonC: I found one other google hit, suggests that libnet-smtpauth-perl is needed for authenticated smtp. I don't have it installed, I use libauth-sasl
http://web.archiveorange.com/archive/v/LLXmL3yBdvkg7Zy5oVD9
though again it's claiming works on 64bit fails on 32bit. yeesh
This link has a test program to test ssl email with gmail http://forums.devshed.com/perl-programming-6/smtp-ssl-not-working-on-32-bit-systems-635440.html
[03:26]
NelsonCpharvey: having trouble with cpan2deb http://pastebin.com/rvrm2bHW [03:31]
gac410NelsonC: I'd give libnet-smtpauth-perl a try. [03:31]
NelsonCgac410 great let me check that [03:31]
pharveywhat version of dh-make-perl do you have? apt-cache show dh-make-perl [03:32]
NelsonC0.73-1 [03:33]
pharveyweird :( I'm using 0.74 FWIW.
pharvey thinks he's going to need CPAN:Devel::Cycle :(
[03:39]
NelsonCweird. I did " sudo apt-get install dh-make-perl" [03:39]
pharveyah, well I'm on Debian testing, which I guess is slightly newer. [03:42]
NelsonCUbuntu server 64 11.10 [03:42]
gac410I installed that libnet-smtpauth-perl - at least it didn't break email. [03:43]
pharveyI wonder if you could use 12.04's dh-make-perl on 11.10
https://launchpad.net/ubuntu/precise/+package/dh-make-perl
[03:44]
gac410cpan is sounding much better ;-) [03:44]
pharveyyeah [03:44]
NelsonCbut is that why it failed when I tried to run cpan2deb? The version? [03:44]
pharveywell, FWIW, you can cpan2deb DhMakePerl, but that has its own problems :-)
I honestly don't know. That sounds crazy, that the official dh-make-perl is just broken out of the box.
bloody open source! :)
[03:44]
gac410why on earth has ubuntu moved the close button to the upper left of the window where the menu usually is. That would be like auto maker swapping brake and gas. [03:45]
NelsonCSo those errors imply that it's fubar'd? [03:46]
pharveymy Ubuntu 10.04 servers are running dh-make-perl 0.68
NelsonC, I really have no idea, from the output you posted. You might see this kind of thing if you're running a version of DhMakePerl that's too old for your system perl, but I mean, this is the official package your distro is shipping
and 0.73 isn't that old
and 0.68 works fine with perl 5.12 as I've been using it myself
I'd try running it as a different user, maybe your ~/.cpan is fubar
[03:47]
NelsonCOk. How, eh, might I do that? [03:50]
pharveysudo -u differetuser or, if you don't have another user account handy, perhaps just try temporarily renaming ~/.cpan out of the way
and don't run this with sudo
you can build .debs without being root
[03:50]
NelsonCok. Yeah I tried without sudo first and got different errors so I assumed it was a permissions thing [03:51]
pharveyit could be your ~/.cpan is half-owned by root [03:52]
NelsonChttp://pastebin.com/AX14VJav [03:52]
pharveycan you chown -R nelson:nelson ~/.cpan ?
pharvey thinks he's using 2/3rds less ram to run FoswikiSuite now :-)
SvenDowideit: why didn't Foswikirev:13690 do what I thought it would to the nightly build output?
[03:53]
FoswikiBothttp://trac.foswiki.org/changeset/13690 [ Changeset 13690 – Foswiki ] [03:56]
gac410pharvey: that's a huge reduction in memory. Does that translate to the resources used for normal processing, or is that more related to the tests themselves. [04:06]
pharveygac410, actually, I was comparing apples & oranges. The 'maximum resident memory set size' given by /usr/bin/time -v seems to be a different number to the instantaneous RES memory toward the end of FoswikiSuite (Eg. in TopicUserMapping tests).
Maximum resident set size (kbytes): 2963984/2733872, 54:31/43:41 (before/after numbers)
so I've made an improvement, but not as drastic I thought a few minutes ago :)
toward the end of TopicUserMapping tests, I see top showing ~850MB virt and ~650MB RES for TestRunner
these changes are purely as a result of proper management of Foswiki singleton/session object, in our unit test framework & unit tests themselves
so the improvements here won't translate to anything meaningful for end users of Foswiki, just devs might be happy to see FoswikiSuite banging their PC a little less harder :)
[04:08]
gac410the faster they run the more often we'll be able to run them. I don't run the full suite very often - it just takes too long.
in any event, it's very good work you are doing. Well worth the effort.
Interesting. On ubuntu, installing a module with CPAN detects the system installed version and won't reinstall. On gentoo, cpan installs into a different location.
[04:18]
.... (idle for 17mn)
pharveythat's weird. On Debian, cpan installs into /usr/local/share, whereas official packages are in /usr/share [04:37]
gac410I already shut down so I can't show the exact message. But basically CPAN said Authen::SASL was already up to date and would not be updated.
So far nothing I do breaks the gmail connection. Something is very wrong on his system.
[04:38]
...... (idle for 28mn)
***gac410 has left [05:07]
NelsonCI'm sorry, I got disconnected
pharvey: I will try going through CPAN since I'm not sure how to troubleshoot dh-make-perl
[05:13]
pharveysounds reasonable [05:14]
NelsonCHow do I check the IRC log to see if I missed anything?
Also, thank you again for your suggestions
[05:15]
pharveythe #foswiki channel title contains a link
http://irclogs.foswiki.org/bin/irclogger_log/foswiki
hrm, my singleton cleanup branch has 96 commits... probably too noisy for svn
[05:21]
SvenDowideitpharvey, when are you gunna merge? :p [05:24]
NelsonCthx [05:25]
SvenDowideitand no, its not too noisy - just giggly :D
i added the unicode, store2 and plack github forks to ohloh
and um, i think its confused :)
[05:25]
pharveySvenDowideit, when I'm happy with trunk-svn, which should be in a few days.. [05:26]
SvenDowideitpharvey, too sweet
Estimated project cost: $203,660,323
[05:26]
pharveyso, you don't think people will scream at 100 commits in UnitTestContrib? Hehe. [05:26]
SvenDowideiti won't :) [05:26]
pharveywe're millionaires! [05:26]
SvenDowideittis a point, we should sell the codebase and share evenly between all commiters to the project [05:27]
pharveyselling to ohloh.org? [05:27]
SvenDowideitsuites me :)
for me its more an incentive to get a few more people committing _now_
so they don't miss out on all that funnyrunnymunnyhunny
[05:27]
pharveyhaha!
I could probably cherry-pick the 100 commits down to ~10, for every (affected) test I have separate session/readTopic/perlcritic commits
but that will probably take at least an hour of time
[05:28]
SvenDowideitwhereas if you just push
it will annoy a few people for a few minutes
[05:29]
pharveysold [05:29]
SvenDowideitso the question is, will it annoy people for a total of more than 60mins :D
you could commit it to github, tell people that you'll push to f.o unless they want to rebase it for you
SvenDowideit thinks crownsourcing trouble is fun
[05:29]
pharveyoh, good idea [05:30]
SvenDowideitdoes anyone here have a google appliance indexing their wiki? [05:31]
pharveyhttps://github.com/csirac2/UnitTestContrib/tree/Item11431-rebased [05:31]
SvenDowideitdon't look at me :) I'm happy to be spammed by your good work :p [05:31]
pharveyFoswikiSuite is 10mins quicker on my 4GB/quad 3.0GHz PC now [05:32]
SvenDowideitnice
um, any difference on a time -v ./view BigWeb/BigTopicSearch ?
[05:32]
pharveysince when? .. not as a result of this work, it's just fixing our dodgy tests :P [05:33]
SvenDowideitah, cool - so you've reduced the number of sessions we make per test? [05:33]
pharveyI've removed all Foswiki->new & finish
and also doing $topicObj->finish() in most places that I noticed
and also removed Foswiki::Meta, except where actually testing Foswiki::Meta
[05:34]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Meta [ (Foswiki login) PerlDoc ] [05:35]
pharveyerrr, Meta->new/load with readTopic [05:35]
SvenDowideitSvenDowideit marvels that would make things faster [05:35]
pharveyby leaking less memory, I think. [05:36]
SvenDowideithttps://www.ohloh.net/p/Foswiki is nicer than it used to be
30 day commit activity for eg tells me something i actually care to know!
[05:36]
pharveyperltidies? :)
ah, new committers, and everything.
[05:37]
SvenDowideit2 new committers joined the project
Andrew Jones Jan 20
KipLubliner Jan 17
12 committers made 447 commits
1955 files modified
30327 lines added
15749 lines removed
[05:37]
pharveyyeah, this month is special, I perltidy'd core & UnitTestContrib :P [05:37]
SvenDowideitwe need to start excluding jquery etc
so that the numbers are better
right now they are crazy
[05:38]
pharveyI'm concerned it's counting each repo cumulatively [05:38]
SvenDowideitit is
but i'm asking ohloh if they can just fix that
[05:38]
pharveyif we used git submodules, instead of copy & paste, we wouldn't have to exclude jquery etc. [05:39]
SvenDowideitgiggle [05:40]
pharveyoh right, we use svn :)
so, shall I invite objection on foswiki-svn? What would you do?
I guess I could make it one giant big fat commit
super-squash 5000
[05:40]
SvenDowideitSvenDowideit - i don't know - i expect i'll squash the store2 work if it ever lands
but thats more to simplify the diff so could review it
[05:41]
pharveyyou have no choice - you've not been rebasing [05:42]
SvenDowideiti always have choice :p [05:42]
pharveypff [05:42]
SvenDowideitjust hard work pointless ones vs simpe quick and easy
ok, yup, claytons choice, written up by hobson
[05:42]
pharveyok, I guess I can put a link to my github branch in the squash-commit, that way if somebody wants to understand what I did, they can look at github. [05:43]
SvenDowideityup, thats true [05:43]
pharveysold mk II [05:43]
SvenDowideitcos long term, no-one cares that you are a XXXX [05:43]
pharveydid you ever profile TopicUserMapper tests? [05:44]
SvenDowideitdunno tbh [05:44]
pharveydominated by: (apart from readLine): Foswiki::Prefs::Parser::parse & Foswiki::Plugin::registerHandlers [05:46]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Prefs::Parser [ (Foswiki login) PerlDoc ] http://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Plugin [ (Foswiki login) PerlDoc ] [05:46]
pharveyah, Prefs::Parse is dominated by my $text = $topicObject->text();
so I guess it's 80% rcs overhead
[05:46]
SvenDowideitSvenDowideit mumbles [05:47]
pharveypharvey wonders if running those tests with RcsLite would help
oh, Foswiki::Plugins::InterwikiPlugin::initPlugin is huge. I guess that's more rcs overhead...
[05:47]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Plugins::InterwikiPlugin [ (Foswiki login) PerlDoc ] [05:48]
pharveyDCOMMIT HAS BEGIN0RATED
42 files changed, 3304 insertions(+), 2183 deletions(-)
on code that nobody cares about
[05:59]
SvenDowideitSvenDowideit screams&shouts :p
and has to go wake girls soon
[06:00]
.... (idle for 16mn)
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/pWTI7w
[foswiki/master] Item11431: Clean UnitTests of Foswiki->new/finish - PaulHarvey
[06:16]
***GithubBot has left [06:16]
FoswikiBothttp://foswiki.org/Tasks/Item11431 [ Item11431: Foswiki::Plugins::SESSION inconsitencies ] [06:16]
.......... (idle for 47mn)
GithubBot[foswiki] foswiki pushed 1 new commit to Release01x01: http://git.io/Z3arRw
[foswiki/Release01x01] Item11349: Save some TopicUserMappingContrib runtime - PaulHarvey
[07:03]
***GithubBot has left [07:03]
FoswikiBothttp://foswiki.org/Tasks/Item11349 [ Item11349: Fix memory leaks ] [07:03]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/Iz2Zmg
[foswiki/master] Item11349: Save some TopicUserMappingContrib runtime - PaulHarvey
[07:04]
***GithubBot has left [07:04]
GithubBot[foswiki] foswiki pushed 1 new commit to Release01x01: http://git.io/I5AjPQ
[foswiki/Release01x01] Item11431: Clean UnitTests of Foswiki->new/finish - PaulHarvey
[07:16]
***GithubBot has left [07:16]
FoswikiBothttp://foswiki.org/Tasks/Item11431 [ Item11431: Foswiki::Plugins::SESSION inconsitencies ] [07:16]
NelsonCgac410: I hope you see this tomorrow. Installing Authen::SASL via cpan> worked! http://pastebin.com/vgXFxrU8
I was never able to get dh-make-perl working. I had to sudo "perl -MCPAN -e shell" to get it to work, but, by gum that did it. Email is working! Thanks again :)
[07:18]
............................................. (idle for 3h43mn)
MichaelLorenzenI've got a suggestion for the foswiki.org page: There is a new link to the support web on the to
... top claim about the friendly community (which it is!). There should be links to extension web (and maybe from the other claims to relevant topics also).
[11:02]
........ (idle for 37mn)
pharveyMichaelLorenzen: thanks, I've added some links :-) [11:40]
.... (idle for 18mn)
CDotMichaelLorenzen: I just uploaded WorkflowPlugin to http://foswiki.org/Extensions/Testing/WorkflowPlugin if you would like to try it
once we are happy it has fixed your problem, I will release it.
[11:58]
AlexanderStHi there, is there anyone with knowledge about the template login manager? I´m just hacking case insensitivity back into the ldap contrib. It´s working with the apache login, but i have to put authUser = lc(authUser); into loadSession within the LoginManager.pm. Is there another solution besides javascripting the login template? [12:04]
CDotAlexanderSt: I doubt it. The template login manager assumes case is significant. There would seem to be two possible approaches; (1) make case insensitivity a feature of the template login manager based on some switch (maybe an EXPERT configure option) or (2) derive a new login manager from template that is case insensitive. The pro of the first approach is that we only have one login manager to maintain; the con is yet more configure options :-(
or, of course, (3) use a javascript approach with a custom template for LDAP - which on reflection, is the best approach IMHO.
[12:07]
AlexanderSthmmm, but if i choose the third way i have to check if the ldap login should be case insensitive anyways...
i would like to provide only one configure option "Case Sensitivity = off" within the ldap options to disable the normale case sensitive login behaviour.
it should work for template and apache login as well
[12:12]
pharveyUpdated 1.1.4 known-issues http://foswiki.org/Support/KnownIssuesOfFoswiki01x01#Release_Foswiki_451.1.4 [12:16]
***kip3f has left [12:16]
CDotAlexanderSt: indeed; but that check can be done in the LDAP template; it doesn't need to pollute the core code
e.g. %IF{"{LDAPContrib}{LowercaseUsernames}='on'" then=".......
MichaelLorenzen: ping
[12:24]
AlexanderStok i see, but isn´t it more consequent to write an LdapTemplateLogin acording to the LdapApacheLogin.pm?
in that case i prevent any javascript issue
[12:28]
CDotyeah, you could do that, but then you'd have to maintain it :-(
(or more accurately, "someone" would have to maintain it)
[12:32]
AlexanderSt...i talk to michael :-) [12:33]
MichaelLorenzenCDot: I'm back [12:33]
CDotsince he wrote the ldap support, that would be a good idea
(11:58:23) CDot: MichaelLorenzen: I just uploaded WorkflowPlugin to http://foswiki.org/Extensions/Testing/WorkflowPlugin if you would like to try it
(11:59:13) CDot: once we are happy it has fixed your problem, I will release it.
[12:33]
MichaelLorenzenCDot: Ok, I'll try
pharvey: wrt foswiki.org: very nice, maybe add the links to the graphics before the claims also?
[12:34]
SvenDowideitAlexanderSt, er, best approach is not to do it in the template login
but to do it in ldapconrib, before it is sent to the ldap server
that way you only do it once, in one place, and the rest of the code doesn't care about the change
also means you don't need js
SvenDowideit goes to bed ;0
[12:46]
..... (idle for 20mn)
MichaelLorenzenCDot: I succesfully tested with your uploaded WorkflowPlugin.
BTW: I could not install with installer (extension testing is added), was this by purpose?
s/installer/configure/g ^
[13:07]
CDotMichaelLorenzen: dunno; I never tried to install from Testing. But if the fix is good, I will release the WFP. Many thanks (and thanks for being persistent in getting us to analyse and fix this; it turned out to be quite a critical bug, albeit rather obscure!) [13:19]
MichaelLorenzenWell, in fact it there was a time when I thougt, I had some obscure misconfiguration or something. I was pleased when you could reproduce the error. I'm very pleased by the support I received from you and from the foswiki community. [13:23]
FlorianPi want to execute a Makefile from a plugin. I write set FOSWIKI_ROOT=/var/www/wikitest.intern.ffg.at/private/foswiki and then make? [13:25]
gac410Hey CDot .. good morning. ... [13:37]
CDotgac410: mornin'. Just added "powered by perl" logo to the foswiki.org page. Gotta show support! [13:38]
gac410I installed a new clean 1.1.4, installed the latest Workflow and Comment Plugins. Everything worked great. But then I applied your meta change from RCS and WFP would not authorize save by CP. Is there more than just the Meta.pm needed? [13:38]
CDotno, only Meta [13:39]
gac410Hm... That's what I was afraid of. :-( [13:39]
CDotyou are logged in, right? the topic exists?
CDot recommends enabling the debug lines in ControlledTopic.pm "_isModifiable" to understand why the save was denied
[13:39]
gac410Yes, Same WFP example provided by MichaelLorenzen - The State changes worked one -> two -> 3 [13:40]
CDotstate changes go through a subtly different path.... [13:40]
gac410Okay - It's in my ubuntu vm so will take me a bit to get going on it.
I'll debug it a bit and let you know.
[13:41]
WoHoo! NelsonC reported that installing Authen::SASL via CPAN on Ubuntu 11.10 64bit fixed broken gmail authentication. Cool.
Maybe we ought to add that to the known issues, or the Installation page. that was a beast.
MichaelLorenzen: You said "BTW: I could not install with installer (extension testing is added), was this by purpose? " Could you explain that?
[13:47]
CDotgac410: I assumed he meant from the =configure= page [13:52]
gac410That should work fine - but need to edit expert setting to set a 2nd repo. It's a bit complex unfortunately. [13:52]
FlorianPI downloaded an extension from trunk an d now i need to run the makefile. How can i do this? [13:53]
gac410FlorianP: You have a trunk svn checkout? or have BuildContrib extension installed? [13:53]
FlorianPgac410: a svn checkout (i think)
gac410: I copied it with sudo svn co ........
[13:54]
gac410okay good. So first you need to pseudo-install developer. ..... cd yourcheckout/core ./pseudo-install developer
The Need to set ENV variables. FOSWIKI_HOME=/var/www/foswiki/trunk/core FOSWIKI_LIBS=/var/www/foswiki/trunk/core/lib:/var/www/foswiki/trunk/core/lib/CPAN/lib
(using your install path ... not mine :-) )
[13:55]
FlorianPgac410: like:set FOSWIKI_ROOT=/var/www/wikitest.intern.ffg.at/private/foswiki [13:56]
gac410I never can remember syntax. export FOSWIKI_ROOT /path/to/svn/checkout Cant' remeber if you need equal-sign. And usually a SVN checkout has a core directory. [13:58]
MichaelLorenzengac410: The new version of WorkflowPlugin was not listed in configure ( I have added the 2nd repro for testing - I can see TinyMCEPlugin 1.2.4 for example).
WorkflowPlugin is not listed on http://foswiki.org/Extensions/Testing/WebHome - maybe that's the reason?
[13:58]
gac410Ah. yes, Maybe CDot or Michael didn't release to Testing. Need to alter the build to upload to testing. [13:59]
CDotI did release to Testing [13:59]
FlorianPgac410: I am a little bit confused. I did this once before and there i copied the files from svn in the directories of my installation set FOSWIKI_ROOT und then executed make [14:00]
gac410CDot - Topic doesn't have a package form for some reason. [14:01]
CDotc-rap [14:02]
gac410CDot - Build should have recovered the package form from Extensions, and applied it to Testing.
FlorianP: If you have a svn checkout you should have /path/to/svn/checkout/YourPlugin ... and /path/to/svn/checkout/core/(lib, working, data, ... )
[14:02]
CDotthe build.pl upload instructions are not very clear; I might have configured it wrong [14:04]
gac410whan you run build.pl upload, it asks if directories are correct. You say no. It asks for the upload location, and then the "find the form" location.
If it finds the topic in the first location, it doesn't download it from the 2nd location.
[14:05]
FlorianPgac410: yes i have trunk/solr/(lib, pub, tools, ....) an i copied trunk/solr/lib to /lib and trunk/solr/pub to /pub [14:06]
gac410FlorianP: Do you have trunk/core/lib, trunk/core/pub, etc. [14:06]
FlorianPgac410: and no there is nothing with core.. [14:06]
gac410Oh. hm... I have no idea. [14:07]
FlorianPgac410: I copied the files from trunk with this command: sudo svn co http://svn.foswiki.org/trunk/SolrPlugin/ trunk/solr [14:08]
gac410tbh I have no idea. I think in your realy foswki install, you need BuildContrib installed. Set the FOSWIKI_HOME and FOSWIKI_ROOT to point to real install and then...
cd to path/to/trunk/solr/lib/foswiki/Plugins/solr
and then in the SolrPlugin directory, run ./build.pl release
[14:09]
CDotthe ConfigureTests are blowing up; but it doesn't seem to be my fault, AFAICT [14:10]
gac410hm. trunk?
I just rebased - they pass fine here on release branch
[14:11]
CDot13791 - pharvey
trunk
[14:13]
gac410checking out now ... git is slow flipping from release to trunk.
All tests passed (22)
1..576
[14:13]
CDotyou sure? reading the code, i can't see how the configuretests could pass..... [14:15]
FlorianPgac410:
prf@wikitest:/var/www/wikitest.intern.ffg.at/private/foswiki/trunk/solr/lib/Foswiki/Plugins/SolrPlugin$ ./build.pl release Use of uninitialized value in split at ./build.pl line 2. Can't locate Foswiki/Contrib/Build.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at ./build.pl line 3. BEGI
[14:16]
gac410FlorianP: Is BuildContrib installed and FOSWIKI_ROOT and FOSWIKI_HOME set
CDot - what fails?
[14:17]
CDotRunning ConfigureTests
ConfigureTests::test_Util_createArchive_perlZip
Assertion (this is not a topic object) failed!
at /home/foswiki/trunk/core/lib/Assert.pm line 80
Assert::ASSERT(undef, 'this is not a topic object') called at /home/foswiki/trunk/core/lib/Foswiki/Meta.pm line 1086
[14:17]
gac410What line in ConfigureTests.pm
That test passes fine here.
[14:19]
FlorianPgac410: yes [14:20]
CDotI also get: ConfigureTests::test_Package_sub_install
*** Assertion failed!
at /home/foswiki/trunk/core/lib/Assert.pm line 80
Assert::ASSERT('') called at /home/foswiki/trunk/core/lib/Foswiki/Meta.pm line 1251
Foswiki::Meta::get('Foswiki::Meta=HASH(0xab0d538)', 'FILEATTACHMENT', 'file3.att') called at /home/foswiki/trunk/core/lib/Foswiki/Configure/Package.pm line 703
Foswiki::Configure::Package::_installAttachments('Foswiki::Configure::Package=HASH(0xaa80b20)', '/home/foswiki/trunk/core/working/tmp/test_ConfigureTests/test...', 'Sandbox/Subweb/TestTopic43', 'Testsandboxweb1234/Subweb/TestTopic43', 'Foswiki::Meta=HASH(0xab0d538)') called at /home/foswiki/trunk/core/lib/Foswiki/Configure/Package.pm line 627
[14:20]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Meta [ (Foswiki login) PerlDoc ]
http://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Configure::Package [ (Foswiki login) PerlDoc ]
[14:20]
CDot(after fixing the other error, which is deffo an error)
gac410: you sure you are running with asserts enabled?
[14:21]
gac410gac410 looks [14:22]
FlorianPgac410: Sry i have to leave, Anyway thanks for your help [14:22]
CDotCDot definitely has the latest code from svn [14:22]
gac410nope - that was it. Fails now. Strange, I thought that TestRunner enabled asserts.
wow - that fails in the test setup.
[14:23]
CDoty. suspect pharvey is running witout asserts too
the ConfigureTests fail due to an incorrect assert in Meta.pm (preumably Paul added it to try to debug?)
no, it was sven; in march 2011
[14:24]
gac410Why on earth failing now.
ah pharvey changed meta to Func::readTopic . in this case web and topic are undef - was trying to create an empty meta I assume.
[14:29]
CDotyep
the correction is to remove the assert, and return undef if _indices is undefined (which is a valid and legal state)
[14:31]
gac410that gets by that problem and breaks another test.
ASSERT( defined( $this->{_indices} ) ) if DEBUG; Line 1252 in Meta.pm
[14:34]
BabarCDot: tsss. You call that a fix???
return unless exists $this-<{_indices};
[14:48]
CDotpfffffft..... pedant [14:49]
Babarif it exists and is empty, then somebody screwed up some more [14:49]
CDotand shouldn't that be -> not -< ?
I guess
CDot was just trying to get the tests to pass, and sophistry is always the first thing out of the window in those cases ;-)
CDot has gone for a ride before it gets too dark
[14:49]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/COrzEw
[foswiki/master] Item11466: fixes for borked unit tests; looks like 'someone' is running unit tests with ASSERT disabled. Tsk tsk. - CrawfordCurrie
[15:03]
***GithubBot has left [15:03]
FoswikiBothttp://foswiki.org/Tasks/Item11466 [ Item11466: Borked unit tests ] [15:03]
BabarCDot: sure. I guess multitasking isn't good on a monday... [15:11]
........................... (idle for 2h13mn)
gac410We don't have a good place to list the .deb packages needed to install Foswiki from tarball on a ubuntu system. And our list of "optional" CPAN dependencies is a bit optimistic that some are really optional.
I think our list of dependencies in InstallationGuide should add the name of the .deb or .rpm under the dependency. CGI::Session<br/>libcgi-session-perl for example.
and we need someplace for notes like ... Authen::SASL is broken on ubuntu 10.11 x64 - need to remove and reinstall using CPAN.
[17:24]
Plazmaso i've got some css on my foswiki page and it works as i expect.. but it's pushing the default sidebar downward.. any idea what it could be?
i'm pretty sure it's something in my css that's forcing it down, but i'm not super versed into it and am out of ideas
wiki isn't public either :/
[17:28]
gac410Do you have any unclosed div or spans? Sometimes that can mess up the page. [17:29]
Plazmawell , nevermind
it fixed itself
[17:29]
gac410cached? [17:29]
Plazmaand yea i'm using noting but <div> ..
no.. i added a <br /> at the end of a list
weird
[17:29]
gac410CDot: I give up. WFP and CP work fine now on another new install with your modified Meta.pm So whatever was going on last night, seems to not recur on another install.
gac410 joins Plazma with another "it fixed itself" experience. :-(
[17:40]
CDotgac410: I was thinking.... [17:42]
gac410I try to avoid that [17:43]
CDotwibnif configure didn't sho you options that weren't relevant [17:43]
gac410like ... [17:44]
CDotlike {Htpasswd} options which aren't useful unless an Htpasswd password mgr is selected [17:46]
gac410yes definitely. I disliked even adding them for that reason.
Email has some similar issues - Don't show mailprogram if Net:: selected, and don't show user / pass / hostnames if Mailprogram selected.
Maybe some sort of selector block or twisty that only is open depending upon a selected value.
[17:46]
CDoton phone [17:51]
.... (idle for 15mn)
Plazmaanyway to customize "topic does not exist page" I mostly want to put something funny on top, and then have the options to create it below [18:06]
CDotoff phone
gac410: my though was "why not use JS"
since we depend on it anyway; the HTML generator could auto-create appropriate JS to hide/show dependent stuff depending on values entered *real time*
which would be quite nifty :-)
[18:06]
gac410yeah. Also realtime would be a nice summary or maybe just a count of changes by the submit button.
Hm. Would you classify CGI::Session, or the Digest:SHA / Digest:SHA1 modules as optional? Most of the time? Some of the time? In really unusual installations?
Unless I don't understand the requirement, I'd say most installations need session support. and probably password support as well.
Same as the optional HTML* for Wysiwyg and CompareRevisionsAddOn. Pretty unusual to not want them.
[18:15]
CDotanother example; we have a slew of RCS options that are useless if you are not using RcsWrap
anyways, it's just an idea - I'm not going to implement anything. Just thought it was worth raising, see if you thought I am crazy.
[18:26]
gac410I think it's a very good idea. [18:28]
.... (idle for 15mn)
The only file in core using Digest::SHA1 is core/lib/CPAN/lib/Cache/FileBackend.pm ... everything else uses the newer Digest::SHA
wtf - WHY have we removed the CPAN requirements from the installation guide and systems requirements documentation.
[18:43]
CDotHideIrrelevantConfigureOptions
http://foswiki.org/Development/HideIrrelevantConfigureOptions
[18:45]
gac410<sarcasm> I know. We can simplify installation by ignoring the system requirements.... Expose the requirements by letting the admin discover stuff that doesn't work. </sarcasm> [18:45]
PlazmaCDot: anyway to customize "topic not found" page ? Mostly I want to enter a graphic/image on top + a custom message, THEN below it have the 'create topic.." stuff [18:46]
CDotone prob with CPAN modules is the dep list changes depending on what you select as options
so only =configure= has a "true" list (and even that's a bit dodgy)
[18:46]
gac410Right. but options like Wysiwyg and CGI Sessions are somewhat mainstream.
But to delete the list, optional or not, from the InstallationGuide, and SystemRequirements pages ???
[18:47]
CDotPlazma: define your own skin, then add your own messages.myskin.tmpl, which includes messages.tmpl. Then redefine just the no_such_topic template. [18:51]
Plazmahmm
can't i edit the default one?
[18:53]
gac410Plazma: Sure - if you want to edit it again after every release update.
By "define your own skin" it's as simple as creating messages.plazma.tmpl which includes messages.tmpl and redefines no_such_topic. One line plus the redefined template.
Then you change SKIN = pattern to be SKIN = plazma,pattern in your SitePreferences
No offense to however did it, this is nutz. Winding my way through multiple documents, to *finally* find Support.HowToInstallCpanModules which says " CGI::Session - Needed for Foswiki's session feature which is really a must have"
If we are going to hide the dependencies, then we ought to ship them in our own lib/CPAN/lib directory Foswiki should be *feature complete* out of the box, or it must list the required deps for the default feature set.
[18:54]
CDotgac410: was it me? if it was, I may have removed it because it was woefully misleading (or simply wrong) [19:01]
gac410gac410 must feel like ranting today. I've been trying to find a location to describe the "Find" by NelsonC - that Net::SMTP::SASL is broken on Ubuntu x64
CDot - I have no idea. I'm floundering around - I cant find where they were removed. I suspect the FoswikCamp AJ effort at restructuring the install document.
[19:01]
CDotI have no problem with restoring the list *BUT* there needs to be some way to keep it in synch with the "master" list
CDot sees the time and runs to have a shower before going out
l8r
[19:02]
gac410Trying to help NelsonC - he wanted to know why I installed certain libs. The list was in the 1.1.4 docs.
l84
l8r
[19:03]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/0U1BnA
[foswiki/master] Item11458: simplify and streamline handling of password file; it now must exist for Foswiki to run, and will be created by =configure= if not. This lets us do enhanced checking in =configure= while reducing the runtime burden. - CrawfordCurrie
[19:17]
***GithubBot has left [19:17]
FoswikiBothttp://foswiki.org/Tasks/Item11458 [ Item11458: TopicUserMapping blocks registration if passwords are not writable - FAIL!! ] [19:17]
.... (idle for 16mn)
Plazmagac410: ohhh ok.. thanks!
you too CD
you too CDot
[19:33]
................. (idle for 1h21mn)
BabarBabar wonders if perl optimizes things like "return 0 if ( -e $path && -f $path && -w $path );" by itself
that is, this could be written: return 0 if ( -e $path && -f _ && -w _ ); to access the filesystem only once
CDot: +lib/Foswiki/Configure/Checkers/Htpasswd/LockFileName.pm 0444 but I don't see any file by that name in the commit. Did you forget it?
[20:54]
gac410I hope we don't ship a lock file by default - I think it should only exist when locked?
ah... never mind Doh.
[20:57]
We have two DEPENDENCIES files lib/DEPENDENCIES and lib/Foswiki/Contrib/core/DEPENDENCIES that are somewhat overlapping, sorted differently, and different minimum revisions on some.
We ship the version in lib.
[21:05]
***SvenDowideit sets mode: +oooo ArthurClemens Colas ColasHome gac410
SvenDowideit sets mode: +oooo gmc Lynnwood MartinCleaver pharvey
[21:07]
...... (idle for 25mn)
ChanServ sets mode: +o OliverKrueger [21:32]
pharveygac410: doesn't TestRunner enable asserts?!
pharvey esplodes
pharvey goes to work
[21:33]
gac410pharvey - sort of. It enables asserts ... but not all asserts. I never have understood that [21:34]
***gac410 has left [21:46]
...... (idle for 25mn)
pharveybut it says "enabling ASSERTs" when it starts... [22:11]
gac410I know ... it does. I wonder if it's timing - in any event, you get *more* asserts when running with asserts enabled in LocalLib.cfg
I've been burned by this before as well.
[22:12]
Babarbut I think the error I reported is genuine. CDot added the file to the MANIFEST, but not to SVN.
the other one was more something to benchmark. So... let's do it :)
[22:16]
gac410Agreed Babar - that's why I said never mind. Strange though I have that file -but it says "Not checked in"
I'm going to add some *brief* system requirements by distribution to the SystemRequirements page - behind twisties ... for RPM based, and Deb based.
I think I have the Ubuntu details somewhat correct - need someone to verify RedHat / RPM based.
[22:17]
Babarthat's easy...
oops sorry
for ASSERTS...
it enables ASSERT, but it's too late often when it does
everything that has already been compiled won't get changed
so, as most ASSERT are done in BEGIN blocks, these won't work
[22:25]
gac410Ah. Maybe we need to just change the message. instead of saying"Enabling Asserts" Check and recommend that Asserts be enabled in LocalLib.cfg [22:26]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/sS-D1g
[foswiki/master] Item11467: Restore Perl deps to SystemRequirements - GeorgeClark
[22:33]
***GithubBot has left [22:33]
FoswikiBothttp://foswiki.org/Tasks/Item11467 [ Item11467: Need better documentation and handling of Perl dependencies ] [22:33]
tsnfooAnybody here using MongoDBPlugin? How stable and/or usable is it? [22:37]
Babartsnfoo: pharvey uses it daily, and SvenDowideit wrote it :) [22:37]
tsnfooBabar: yeah, I've been watching the commit logs for a while, but wasn't sure if they are actually using it in production, and how usable it is for anyone is is not one of the devs ;-) [22:38]
gac410it's live on a public site iirc. [22:38]
pharveytsnfoo, trin.org.au has been using it in production for a year
it requires trunk, not compatible with 1.1
[22:38]
tsnfoopharvey: oh, trunk you say? [22:39]
pharveyindeed
check http://foswiki.org/Tasks/MongoDBPlugin before putting it into production :-)
[22:39]
tsnfooTrying to decide which is more likely to result in my co-workers killing me: a slow 1.1.4, or an unstable trunk... [22:40]
pharveyHehe. Well, we have over 200,000 topics, and we need to SEARCH over some very large webs [22:40]
gac410what are you currently running [22:40]
tsnfooYeah, that lists scares me. It is longish, and has some oldies on it.
Running 1.1.2. Preparing a 1.1.4 VM.
[22:41]
pharveysome of them probably need to be closed - but some are waiting on 'store2' work to eliminate other problems :/ [22:41]
gac410Do you see 1.1.4 as slower than 1.1.2? [22:41]
tsnfooAbout the same.
Topic views average 4-6 seconds. Anything with even a simple query takes 8-12. Searching webs takes 10-30.
Complex searches have a 10-15 segfault rate.
[22:41]
gac410hm. That seems high. Are you in a vm for production?
segfaults? really?
[22:42]
tsnfooYeah. RHEL 6.2, 4gigs RAM, 2 virtual cores. [22:43]
pharveyhttp://wiki.trin.org.au/Lauries/ is a SEARCH-powered grid in a web of 25k topics. [22:43]
tsnfooYep, 500's. [22:43]
Babarmaybe I put way too much iterations in my benchmark.. [22:43]
gac410500's are not necessarily segfault. Are you running fastcgi / fcgi [22:43]
tsnfoomod_fcgid [22:43]
gac410Okay - the 500's are most likely the fcgid timeout - Foswiki.org runs into that. [22:44]
tsnfooYes; time-outs I believe.
We have some wiki apps that are pseudo-relational, pulling data from multiple topics, to push topics though WorkFlowPlugin.
That's where we see the timeouts.
[22:44]
pharveytsnfoo, are they OP_ref queries? Foo/Bar/Cat [22:45]
tsnfooIf there's no load, the topics load in 305 seconds. Get more than 3 or 4 users, and the load average spikes, and you start to see timeouts. [22:46]
gac410Check your setting for FcgidIOTimeout. I don't recall the default but Foswiki.org increased it to 180
never mind. if 305 is normal, you must already have increased it ;-)
[22:46]
tsnfooYeah, I have upped that a bit
pharvey: there are some queries like that
[22:46]
pharveyit's worth pointing out that our site is run on two single-core VMs. The mongod server has 8GB ram, Foswiki 4GB [22:47]
tsnfooBut even the simple ones are incredibly slow. And search takes forever. [22:47]
pharveyhow large are your webs? Or is it that you're getting lots of hits? [22:47]
gac410What type of I/O for your VM? where I worked, we had to convert off of VM and NAS. (An older single core desktop was faster than the IT provided VM) [22:47]
tsnfooLemme look really quickly... [22:47]
pharveygac410, that was true in our last DC, but the new setup is enormously fast... hdparm pretends it's getting 200MB/sec :)
faster than my PC
[22:48]
gac410excellent. [22:48]
pharveybut it's fake i/o - it bogs down if you start hitting blocks which aren't cached (?) but it still seems like 50MB/sec there. [22:49]
gac410VM's have to be set up my someone really knowledgeable in the vmware (or whatever). Ours was overcommitted, constantly hitting balloon memory IT insisted that was good for vmware. [22:50]
Babarpharvey: I get 3640 MB/s for cached, 80 MB/s for buffered. On my laptop :p [22:50]
tsnfooMost of the webs involved had 3-5k topics
VMWare hosts, iSCSI to EMC SAN
[22:50]
pharveyRight now our prod server is getting Timing buffered disk reads: 208 MB in 3.00 seconds = 69.31 MB/sec
but the thing about mongo is it prevents any calls to rlog/rcs
so mostly I'm worrying about network traffic to the mongo server :)
tsnfoo, what's youre fcgid max processes?
[22:51]
tsnfooI must be doing something terribly wrong.
tsnfoo loading up his conf
pharvey: 150
But the thing isn't even usable for < 10 users.
[22:53]
pharveythen I'd limit it to 10
too many simultaneous processes will just make it thrash itself to oblivion
[22:54]
tsnfooYeah, every damn view request spikes to 100% of a core [22:55]
gac410foswiki.org is set to 50, but I never see more than 5 or so. [22:55]
pharveyours is limited to 5, and timeout is 30s
I try to keep average request times to 1s (goal) but it's more like 2s
[22:55]
gac410tnsfoo are you using RCSLite and PurePerl search (or better yet NativeSearch) [22:56]
pharveyadditionally, I've been mounting all filesystems with noatime, I wonder if that helps [22:56]
tsnfooWould you guys mind taking a look at my fcgid conf if I paste bin it? [22:57]
pharveysure [22:57]
gac4101.1.2 , 1.1.3 for sure has a bug on the WebTopicCreator page - Need to restrict the Template search for topics named *Template [22:57]
tsnfooProbably the worst fcgid conf ever -> http://pastebin.com/27K6X4v4 [22:59]
pharveytsnfoo: I'm curious about your memory situation - do you feel that the OS is keeping most of the *.txt files in buffers/cache - this is critical. Perhaps reducing the number of procs would help achieve that [22:59]
tsnfoopharvey: I'm not sure how to test that. It doesn't feel like anything is kept in a cache. [23:00]
BabarItem11458 ... [23:00]
FoswikiBothttp://foswiki.org/Tasks/Item11458 [ Item11458: TopicUserMapping blocks registration if passwords are not writable - FAIL!! ] [23:00]
pharveydo you have a way to check i/o stats for your VM?
how much system or i/o wait time do you see?
FWIW I have DefaultMaxClassProcessCount 5
how many processes do you see normally? what's your load average? I feel as if you might have too many processes and that's squeezing your disk cache out of RAM, causing heavy i/o
[23:01]
BabarBabar is shocked by his benchmark results... [23:03]
pharveyBabar, discuss [23:04]
gac410please [23:04]
tsnfoopharvey: I can log into the VCenter Client - I assume VMWare tells me some of the I/O stuff. [23:04]
pharveymy zabbix monitoring is telling me that our average i/o wait time is 2%; 12% system (networking with mongod, I reckon) and 35% user [23:06]
Babarpharvey: check the like the bot pasted [23:07]
pharveyBabar, that's a useful benchmark, and helps educate others why we've coded a certain way.. would be cool to see them in unit tests, even if as DISABLED tests
Babar, I read a perl optimization article somewhere that said using shift rather than my ($foo) = @_ can stop perl from optimizing sometimes...
something to do with modifying the stack? Or something. Could be lies :-)
[23:09]
Babaryeah, I wondered if I should use $_[0], but I thought that if I do the same in both, it shouldn't impact
I think it's the opposite. Some people claim that my ($foo) = @_; tells perl this function only takes one parameter, so it can discard the rest, whereas shift doesn't
[23:11]
pharveyhttp://makepp.sourceforge.net/2.0/perl_performance.html under "Don't modify the stack" [23:12]
Babarbut I've read all and its opposite, so...
ah, so they agree with me :)
[23:12]
pharveyand then http://stackoverflow.com/questions/1328411/is-shift-evil-for-processing-perl-subroutine-parameters has a benchmark showing something different [23:15]
tsnfoo: the only non-default mod_fcgid conf setting I have is DefaultMaxClassProcessCount 5 - I suggest you try setting smaller limits (say 10) in your other directives as well [23:20]
tsnfoopharvey: I'll try that, and see if I can figure out how to measure disk access and other IO
I"ll I've been using until now is top ;-)
[23:20]
pharveydoes that show wait times?
x.x%wa in
the top
top of top, haha!
[23:20]
tsnfooyeah [23:21]
pharveypharvey needs coffee [23:21]
tsnfooLOL
Indeed
I need beer.
[23:21]
pharveywhat wait times do you see?
usually that correlates well with disk i/o
[23:21]
tsnfooLet me hit it and take a look [23:21]
SvenDowideitgac410, i had something important to tell you a few days ago when you were asleep. can you tell me what it was? :D [23:28]
..... (idle for 20mn)
gac410SvenDowideit: Right. sure. hm. I hit the lottery? [23:48]
Babar - HtPasswdUser has a really complex one: if ( ( !-e $path ) || ( -e $path && -r $path && !-d $path && -w $path ) ) { Or is that the one you are talking about. [23:55]
BabarI think that was part of the one I was talking about [23:57]
gac410Hm. It's tested as part of every session load of TopicUserMapping. Seems like we would only need to test that on pages that will actually change the password.
So you can make if faster, but it would be even better if we only called it when needed. Configure/Package has a chained file test too. but only used when installing/removing packages.
[23:57]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)