#foswiki 2012-07-05,Thu

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***grainne has left [00:34]
................................................................................... (idle for 6h53mn)
ChanServ sets mode: +o MichaelDaum [07:27]
MichaelDaumMichaelDaum found out why a simple view is iterating over ALL revisions of a topic ... Foswiki::UI::View::revisionsAround ... thats called by a %REVISIONS% in pattern skin [07:34]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::UI::View [07:34]
pharveygood catch [07:44]
MichaelDaumnext one: getRevisionHistory
this returns a revision iterator
which basically is a list of numbers backwards from maxRev to 1
this list is created by calling _numRevisions
guess what this function is doing?
_numRevisions is called 62 times on a normal topic (System.WebHome)
on 23 topics
these are 23 calls _numRevisions to different topics and 39 are repetitions
(on a simple view)
this all results into 1059 calls to _readTo (in RcsLite)
these 1059 calls run over 7 files in my wiki
most of these calls are done by the core to find out the max rev ... also recorded in META:TOPICINFO
however
[07:45]
pharveyno wonder the getRevisionHistory mongo listener makes a difference then :) [07:57]
MichaelDaummost of these code paths don't care about the max rev being cached into META:TOPICINFO
they all trigger a full iteration over the ,v file
no wonder that RcsWrap kills the system
it is called much too often as the audit shows
so RcsLite is only so much faster than RcsWrap because the core makes so many calls into the RCS system
or put the other way around RcsWrap could be of a similar speed when called more sensible
best case
[07:57]
pharveyindeed. But we did talk about possible improvements to RcsLite, eg. reading only the first few lines of a ,v instead of the whole thing, just to get numRevisions [08:02]
MichaelDaumthe advantage of RcsLite over RcsWrap will definitely diminish
it seems the RcsLite code had that in mind by inventing this _readTo ... (only read up to the next char required)
however _readTo is wrapped into a while(1) loop in _ensureProcess() which crawls the ,v exhaustively
_ensureProcessed seems to be invented to make sure all history data has been gathered in case it is required...
_ensureProcessed is called pretty much all over RcsLite in different code paths
so this api can't use any information of the kind "I only need the info up to the max rev point in the ,v file"
it is a kind of "hit him on the head" just to make sure he is dead: an overkill.
the code is correct ... but ... crap
sometimes there's no iterative improvement from "correct" to "performing"
[08:02]
***Babar sets mode: +oooo AndreU Colas gac410 pharvey [08:20]
....... (idle for 30mn)
MichaelDaumoh boy
why doesn't the store handler for an meta object point back to the Foswiki::Meta?
[08:50]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Meta [08:51]
MichaelDaumif it did, would it be able to properly use TOPICINFO
it doesn't have the topicObject. instead it stores the web and topic string
so there's no easy find(TOPICINFO) for getLastRevisionID in there
again the api is done in a way it _cant_ be optimized
[08:51]
jastanyone familiar with a problem where, if special characters are allowed in topic names, new topic files are created with *really* weird characters in the filename? e.g. 䴩 (consuming two extra characters from the name) instead of ä [08:55]
Babaryes,it's a feature :) [08:55]
jastcan it be turned off? :} [08:55]
Babarwhat encoding do you have?
what OS?
[08:56]
jastutf-8, debian squeeze [08:56]
Babarthen yes, that's a normal feature. Why do you care? [08:58]
jastbecause in some cases we now have both names... and GenPDFPrincePlugin likes to pick the other one
so some images won't render properly etc.
where does this special encoding come from, anyway?
[08:59]
CDotMichaelDaum: what "store handler"? Are you referring to the RCS store implementation?
the Foswiki::Store API "talks" meta-objects, so there's no excuse for the store internals.
[09:05]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Store [09:05]
CDotoic - you are referring specifically to getLastRevisionID
^LastRevisionID^NextRevision
agreed, that's an obvious optimisation to the VC stores, without touching the Foswiki::Store API. Go for it!
[09:06]
MichaelDaumthis _getTOPICINFO ... in VC::Handler ... thats SMELL isnt it. it implements a META:TOPICINFO parser of its own, i.e. not using Foswiki::Meta (which it doesnt have a grip on down there)
first optimization is to remove %REVINFOS% from pattern
second is to teach getLastRevisionID META:TOPICINFO
note there's a %REVINFOS% (plural) and a %REVINFO% singular
only the latter is documented
third: there are three calls to %REVINFO that all go all the way down to _numRevisions ... on the same topic
(in pattern skin)
[09:11]
jastdoes this look familiar to anyone: There was a compile error: "LDAP_SUCCESS" is not exported by the Net::LDAP::Constant module [09:22]
JulianLevensjast; yes I've seen that one before. I raised something about that on f.o [09:35]
MichaelDaumokay responses are back to <500ms for system.webhome
(1.68s w/o fastcgi)
[09:35]
JulianLevensjast: http://www.foswiki.org/Tasks/Item8967
jast: I hacked LdapContrib.pm as follows
# use Net::LDAP::Constant qw(LDAP_SUCCESS LDAP_SIZELIMIT_EXCEEDED LDAP_CONTROL_PAGED);
sub LDAP_SUCCESS { return 0; }
sub LDAP_SIZELIMIT_EXCEEDED { return 4; }
sub LDAP_CONTROL_PAGED { return '1.2.840.113556.1.4.319'; }
Not pretty and a maintenance burden, but at least I can use configure again. Net::LDAP::Constant plays with the symbol table and that confuses configure
[09:37]
jastJulianLevens: funny thing is it happens within the "use Net::LDAP;" [09:43]
MichaelDaum(67ms with fastcgi + page cache on the second hit) [09:43]
JulianLevensjast: That's sounds like it's a different problem, especially as LDAP_SUCCESS is so basic to the whole thing. Are the related Net::LDAP modules all up to date? [09:48]
.... (idle for 19mn)
jastJulianLevens: there are no newer ones in Debian, anyway... and if I do perl -MNet::LDAP etc., everything works fine
Babar: oh, and I also care because the 'special' filenames are displayed like that in searches
[10:07]
JulianLevensjast: where are you actually getting the compile error, i.e. which module has the "use Net::LDAP" that's failing? [10:13]
CDotMichaelDaum: IIRC it implements a very simple parser that never looks beyond the first line of the file.
CDot waits to see what MichaelDaum checks in
[10:14]
jastJulianLevens: LdapContrib [10:17]
JulianLevensjast: Sorry, I've ran out of ideas and I don't really have the time anymore today. [10:30]
***ChanServ sets mode: +o MichaelDaum [10:40]
.......... (idle for 48mn)
ModAcOstMichaelDaum: there are still issues with that CaseSensitiveLogin feature. When enabled "AdminUser" will become "adminuser" and things fall appart (eg. solr will crash because it can't read DocumentForms unless adminuser is added to AdminGroup). [11:28]
MichaelDaumModAcOst, have you got $Foswiki::cfg{Register}{AllowLoginName} = 1; ? [11:30]
ModAcOstWill have to check [11:30]
MichaelDaumalso: have a check on this patch for Users.pm http://pastebin.com/c3tMyZ1c
line numbers probably dont match
[11:31]
ModAcOstok [11:32]
MichaelDaumthe problem was that a wrong user mapper was used for a wiki name sometimes
the getMapping thing
[11:34]
ModAcOstok, will have to eat first, or I won't understand it anyway... [11:35]
MichaelDaumAdminUser should never be lowercased. the ldap user mapper isn't responsible for it. [11:37]
.................. (idle for 1h26mn)
foswiki_irc8Hello, I have a question: Is there any possibility to use the FlowchartPlugin with the WYSIWYG-Editor? Because I don't want to use macros for each new flowchart due to time reasons. [13:03]
.... (idle for 19mn)
MichaelDaumjast, have a look at http://ivaynberg.github.com/select2/ [13:22]
foswiki_irc8MichaelDaum: do you maybe have a solution for me? [13:22]
MichaelDaumfoswiki_irc8, nope sorry. [13:23]
foswiki_irc8Ok. [13:25]
.... (idle for 17mn)
gac410foswiki_irc8: If you are asking if there is a Wysiwyg way to "draw" the flowchart, and have it saved as macros, I'm not aware of anything. [13:42]
.... (idle for 19mn)
jastMichaelDaum: so basically we'd just need to make a new form field type for that, right?
MichaelDaum: in any case you might want to pick up the useHidden part since otherwise textboxlist still doesn't play nice with mandatoriness
[14:01]
...... (idle for 26mn)
MichaelDaumjast, select2 is a hybrid of a classical combobox and a tagging kind of thing. having a combobox formfield would be great.
see also the comboxxx widgets at http://www.jeasyui.com/
[14:28]
foswiki_irc8gac410: Thank you for this hint.
Good bye.
[14:42]
***foswiki_irc8 has left [14:42]
.... (idle for 19mn)
jastMichaelDaum_: so we still have to answer the question of how we can pass parameters to formfield types from the form...
in a way that doesn't smell
[15:01]
.... (idle for 15mn)
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/w5EGHQ
[foswiki/master] Item11983: prevent excessive calls into the revision system - MichaelDaum
[15:16]
***GithubBot has left [15:16]
FoswikiBothttp://foswiki.org/Tasks/Item11983 [ Item11983: prevent excessive calls into the revision system ] [15:16]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/Q_2G_g
[foswiki/master] Item11983: add a proper META:TOPICINFO record not to slow down the system looking up the ,v file - MichaelDaum
[15:30]
***GithubBot has left [15:30]
............... (idle for 1h11mn)
divaDebHi guys. Looking forhelp wrt authentication - Our old twiki sites have one form of authentication and it looks like this:
<Directory />
Options +FollowSymLinks
AllowOverride None
AuthName Protected
AuthType Basic
AuthGroupFile /sites/users/groups
AuthUserFile /sites/users/users
Require group this-twiki.domain.net
Require valid-user
</Directory>
On foswiki, we've enabled ApacheLogin using bin/configure
But this .htaccess interferes with the previous directive when normal users log into the site.
Does anyone have any ideas what I could do to just have the Directory directive honored w/o .htacess ?
I can disable (move configure) after the website is complete.
[16:41]
gac410AllowOverride None will prevent .htaccess files from being used by apache. [16:44]
fooloveis there a plugin of sorts to convert foswiki content to migrate it over to archer [16:44]
divaDebAllowOverNone doesn't prevent that access. foswiki honors it.
@gac410 - note that that is in the directive already.
[16:45]
gac410.htaccess files and foswiki are totally unrelated. .htaccess instructs *apache*, foswiki knows nothing of .htaccess.
divaDeb: right, I see the directive, which means .htaccces file in that directory is ignored.
So I don't understand how it can interfere.
What do you mean by interferes with users. (please keep the conversation public).
[16:47]
divaDebSorry, I'm new to IRC.
When a new user tries to login in to the site, the first thing that comes up is a query for AdminConfigure
If the password is given,
then the 2nd username/password query is given - regarding users and groups - if the user answers appropriately, then access to the site is granted
[16:50]
gac410The directive you posted above shoud impact just acces to the root directory. Could you pastebin (http://pastebin.com) your complete apahe configuration
In your bin/configure, what type of authentication is set?
[16:52]
foolovegac410 is there a foswiki/twiki to html convertor? [16:54]
divaDebApacheLogin [16:54]
gac410divaDeb: How did you build your apache configuration? Sample files, Foswiki:Support.ApacheConfigGenerator [16:54]
FoswikiBothttp://foswiki.org/Support.ApacheConfigGenerator [ ApacheConfigGenerator ] [16:54]
divaDebI used the ApacheConfigGenerator, but this is a virtual site, so had to integrate it into that [16:55]
gac410foolove: I thing PublishContrib? Not sure
What is AdminConfigure ?
[16:55]
divaDebI'd rather not publicly paste in my config w/o taking out some company sensitive info first. just fyi [16:57]
foolovegac410 will have a look [16:57]
divaDebAdminConfigure appears to be a default name for access to bin/configure - that at least is my understanding?
Thanks, gac410. Your questions give me food for thought.
[16:57]
gac410divaDeb: AdminConfigure must be something you've set up locally We don't have anything like that by default.
With Apache, generally you either use apache configuration files - like generated by ApacheConfigGenerator, or you use in-directory .htaccess files. Typically not both. Either technique though is specific to apache, and foswiki doesn't have any code that's aware of them.
[17:11]
divaDebPerhaps how I interpret what I am seeing incorrectly, but it appears that LocalSite.cfg has login settings that can over-ride, or at minimum will direct how apache does authentication [17:16]
gac410no, they tell Foswiki whether or not to look for auth. information from the apache environment. Apache uses configuration or .htaccess files to determine what access to allow. When foswiki gets control, that has already happened, and LocalSite.cfg says whether or not to use that info. [17:19]
divaDebThanks, that clears up some of the muddle.
Is there a way to just tell FW to not look for any auth information at all? I just want to user the user and group files at this point in time.
wait, I see "none" in bin/configure. Will give that a try.
(missed it before !)
[17:19]
gac410hm. I think so, yes. Though in that configuration, all access will be treated as the guest user.
If you are migrating from TWiki, you should be able to configure similar Auth and Login settings and have Foswiki work the same way.
The LoginManager, PasswordManager and UserMappingManager all work together.
[17:22]
divaDebI wish I could. Unfortunately, the 3.x Twiki is a very bastardized version from source - very atypical, all manual config. So what I'm doing is trying to make sure that the powers that be have the same kind of access as before. 2nd step will be to get LDAP authentication working after the wikis have all be converted to FW. [17:25]
gac410LoginManager controls whether Foswiki "trusts Apache", or asks for it's own userid/password.
PasswordManager controls where passwords and email addresses are stored. Used by TemplateLogin, as well as by UserRegistration
UserMappingManager controls how the login id is mapped to the WikiName, as well as Group membership.
TWiki 3.x ... wow that is quite old. It's well before my time.
I think SvenDowideit recently did a migration from an old version. Though he is probably sleeping now.
[17:26]
divaDebI've got the fw working, it's just the authentication that is tweaking me. You've given me some ideas, I'll run with that for a while and see what I can do with it.
Thanks!
I'm converting about 20 twikis. So much fun.
[17:28]
gac410yw. I think Sven encountered some issues where old T* topic names could include a slash That is now interepreted has a hierarchical web separator. [17:30]
divaDebGood to know. [17:31]
.... (idle for 17mn)
Gah, I'm finding that guest access is sucky access. hah.
Does guest access == no edit capabilities?
[17:48]
gac410No, there is a way to allow edit as well. er. Though I think that the access controls - ALLOWWEBCHANGE or ALLOWTOPICCHANGE needs to be consistent. [17:51]
divaDebokay, will look into those. thanx [17:52]
gac410ex. if a web is DENYWEBCHANGE = TWikiGuest or ALLOWWEBCHANGE = SomeOtherUsers, then disabling authentication won't be helpful ... I dont' think. [17:52]
divaDebok [17:53]
gac410I've not run a wiki configured without authentication, so I'm not really sure how it will work. [17:54]
divaDeb: Just ran a quick test. With LoginManager set to none, I can edit topics that do not have restrictions applied. But if the topic or web has ACL's defined, then they are still enforced.
So it's a catch22, you cannot log in to satisify the ACLs, so you are blocked.
[18:02]
divaDebwell, to be fair, it's not totally w/o authentication - it's using a form of htpasswd with users and groups defined.
Got around the catch22 - editing is working.
[18:02]
gac410Right, but you instructed foswiki to not look at it. [18:03]
divaDebyes, that's true.
But, it is enforcing the user and groups so authentication is working as I need it to.
[18:03]
gac410If you tell Foswiki to use the Login Manager Apache, then it will look at the apache provided login name. And then the UserMapper = TopicUserMapping will attempt to match it against the WikiGroups and Main/ user topics [18:04]
..... (idle for 23mn)
divaDebCan anyone tell me why the CentOS (and I assume the Redhat) RPMs install Foswiki by default to /var/lib/foswiki instead of something more sane, like - /var/www/foswiki ?
(Probably should ask whoever pkg'd up the RPMs for CentOS, I suppose.)
[18:27]
.... (idle for 16mn)
gac410divaDeb: I suspect it has to do with keeping the lib directory outside of the web server path.
SvenDowideit builds the .deb and .rpm packages.
I've not looked at them myself, but the .deb's are the same.
or at least similar I guess.
[18:44]
......................................................... (idle for 4h44mn)
pharveyfoolove: if you are exporting Foswiki into another software, and it understands HTML, might be worth running the bin/view scripts on the command line, something like: ./view -topic SomeWeb.SomeTopic -skin plain > SomeWeb.SomeTopic.html
I've never used PublishPlugin but that's probably a better option :)
[23:29]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)