#foswiki 2016-11-02,Wed

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
foswiki_irc0I see..I should probably take it out then. One of the reasons we are upgrading is to get better performance [00:00]
gac410What version of perl on your new server? [00:00]
foswiki_irc05.18.3
5.18.2
not 5.18.3
[00:01]
gac410Not terrible, but for best performance 5.20 or 5.22. Unicode is pretty bad on older Perls
5.24 is the lates.
[00:02]
foswiki_irc0Can I simply upgrade perl version? [00:02]
gac410What OS are you running?
(For Foswiki 1.1.5, Definitely NOT. It will crash on new perl. )
OS / distribution that is.
[00:02]
foswiki_irc0Sure, hopefully we will cutover to foswiki 2.1 soon
we have ubuntu 14.04 LTS
[00:03]
gac410Ah okay. Well an upgrade to 16.04 isn't too painful. Did it on my laptop recently. 16.04 LTS is on Perl 5.22.1
On Foswiki.org, we used plenv for a locally compiled perl. But now that 16.04 is out, we would not have bothered.
[00:05]
foswiki_irc0Alright, I will check with our IT support to see if we can upgrade to 16.04 [00:06]
gac410If they want an explanation. Foswiki has switched to use Unicode internally. Perl regular expression performance is much slower on Unicode text (It doesn't matter if there are actually double-byte characters)
But with latest perl it's back to where older perl was on ascii text.
vrurg ... when you have time for me to be distracting to you ... I'll merge master. But that involves restructured code in Config::Bootstrap. which probably needs more Moo Foo that I have available currently :D
[00:13]
.............. (idle for 1h8mn)
vrurggac410: It's gonna be even worse than that because bootstrap is part of Foswiki::Config code as well as readConfig() method. [01:25]
FoswikiBothttps://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Config [01:25]
vrurgAnd I'd be happy to migrate save() into Foswiki::Config too but that's currently beyond my possibilities. [01:26]
gac410Right. on master I refactored Bootstrap out into a separate module. [01:27]
vrurgWhat's actually the reason behind it? I thought that it natively belongs to Config. [01:29]
gac410We talked about this a while ago. Bootstrap is *only* used for that first connection when no config is saved. So useless code. Not a huge saving .. but. And, as a separate module it's easier to split it out and test it without mucking with the configure code.
https://github.com/foswiki/distro/blob/master/core/lib/Foswiki/Configure/Bootstrap.pm
It was a very very simple restructure on master. And BootstrapTests can load and drive it without any risk to the rest of the configuration.
On another note. Ugliness in Net.pm (not yours, master/rel) For some reason, it's failing for me - looks like the STARTTLS probe is doing hostname verification even when supposedly disabled.
No changes to code, so it's either in perl modules or openssl :(
[01:30]
vrurgReally, I now recall the memory concerns. Well, as far as all bootstrapping is just a call to a bootstrapFunction() – it's only a matter of minor patches. [01:36]
gac410okay good. you scared me. [01:38]
vrurgI scared myself. ;) [01:38]
gac410And.... even worse, trying to debug the AutoConfigureEmail wizard, .. tools/configure wizard code somehow doesn't print any results. web works, but not cli :(
again, master / rel. ... not you
[01:38]
vrurgBTW, it's a funny situation I now have with the config: reading is in Foswiki::Config while saving is in ConfigureContrib.
Something is capturing the stdout/stderr?
[01:39]
gac410ConfigurePlugin .. but yeah, the plugin has all the update code.
Yes. part of the autoconfig. Capures it and sends it to the web in a popup for the debug. But the shell doesn't print it :(
Capture is correct. but why it isn't output. yet another mystery. This thing is determined to kick my butt.
[01:40]
vrurgI'm concerned about saving because I'd like to have a replacement of LocaLib.cfg and the best would be to have it transparently incorportated into Foswiki::Config. But this would require all code be under the same roof. [01:41]
gac410well the purpose for LocalLib was to be able to set itself before any foswiki code runs. No need for it, if you have to run foswiki to update it. Chicken/egg. [01:42]
vrurgGood luck on this! I was recently fighting similar issue while developing the Dependencies. ;)
About LLC – I have a trick in mind. We can have some keys on Config decared as 'early stage keys' which means they're not save to LSC, but handled specially. How exactly would depend on an extension implementing it – but only if the new extensions model gets accepted.
It's too long story for this time of day. But could be a great solution because it would allow Foswiki to configure LLC with ConfigurePlugin.
[01:42]
gac410If you can run configure without LLC existing, I'm not sure we really *need* LLC.
(er. that came out wrong. On systems that would be non-functional without LLC)
[01:46]
vrurgOr there could be a new extension which would make it possible to incorporate configuration into a topic, so user wouldn't ever leave comfortable Foswiki environment. [01:47]
gac410Gahhhh NONONON no system paths or OS specific info in a topic. Danger Danger. [01:47]
vrurgOk, too late, need to go.
cu!
[01:48]
gac410Thats one thing we gradually stripped out of TWiki ...
CU. Have a good night.
One day in the past, SMTP hostnames and passwords, etc were all overridable preference settings. What a huge security issue.
[01:48]
...... (idle for 29mn)
well crap.. Seems to be changes in IO::Socket::SSL. Fell back to an older version, and discovery works fine. [02:19]
vrurgI'm here by occasion. I guess I know what is it. There is an option to be set. Either try checking agains my branch – I could have included the fix. Or remind me tomorrow and I'll try to find where I used to deal with it. [02:27]
gac410the SSL verify option is indeed set. But seems to be ignored in the recent modules.
I'll take a look at your branch .. Maybe another option is needed.
strange though that sending works if you save the config. It's the probes by the wizard that don't work.
[02:27]
Ah... vrurg, I doubt you have the fix. The whole reason I noticed the issue is I tested the mail wizard on your branch, and then moved to master to discover it was failing there too. And finally with perlbrew, fell back to an old IO::Socket::SSL and got it working. [02:35]
Anyway, IO::Socket::SSL 2.019 Works IO::Socket::SSL 2.024 fails. [02:41]
....................................................... (idle for 4h32mn)
***ChanServ sets mode: +o MichaelDaum [07:13]
................... (idle for 1h32mn)
ChanServ sets mode: +o CDot [08:45]
................. (idle for 1h24mn)
ChanServ sets mode: +o Lynnwood [10:09]
............ (idle for 57mn)
GithubBot[DatabasePlugin] fschlich pushed 1 new commit to master: https://git.io/vXsaw
DatabasePlugin/master 2a12b98 Florian Schlichting: Item14207: tell MySQL that our strings are utf8...
[11:06]
***GithubBot has left [11:06]
FoswikiBothttps://foswiki.org/Tasks/Item14207 [ Item14207: tell MySQL that our strings are utf8 ] [11:06]
.... (idle for 18mn)
GithubBot[DatabasePlugin] fschlich pushed 1 new commit to master: https://git.io/vXswY
DatabasePlugin/master 9346b9f Florian Schlichting: Item14207: release as DatabasePlugin 2.2
[11:24]
***GithubBot has left [11:24]
.......... (idle for 49mn)
ChanServ sets mode: +o Lynnwood [12:13]
...... (idle for 27mn)
ChanServ sets mode: +o gac410 [12:40]
............. (idle for 1h3mn)
vrurggac410: Briefly checked the wizard code. Setting of SSL_verify_mode is not enough to suppress validation. To me it was necessary to set 'verify_hostname => 0'. Only then it worked as expected. [13:43]
GithubBot[DatabasePlugin] fschlich pushed 1 new commit to master: https://git.io/vXsNO
DatabasePlugin/master 88e0a81 Florian Schlichting: Item14207: use a non-SVN revision
[13:51]
***GithubBot has left [13:51]
FoswikiBothttps://foswiki.org/Tasks/Item14207 [ Item14207: tell MySQL that our strings are utf8 ] [13:51]
gac410vrurg ... thanks. Strange that STARTTLS needs it, while plain old TLS does not. I'll add it in. Thanks for looking. [14:00]
vrurgWelcome! It is also required by plain SSL – it's the way my scripts are connecting to a cisco cucm. [14:02]
gac410Interesting. For me, the wizard succeeds with the TLS connection on 465 with just verify_mode set. [14:03]
GithubBot[distro] fschlich pushed 1 new commit to master: https://git.io/vXsx9
distro/master 718231f Florian Schlichting: Item14198: do not crash when the URI does not contain a userinfo part...
[14:09]
***GithubBot has left [14:09]
FoswikiBothttps://foswiki.org/Tasks/Item14198 [ Item14198: PublishPlugin fails in Foswiki 2.1.2 while trying to render zones ] [14:09]
GithubBot[PublishPlugin] fschlich pushed 1 new commit to master: https://git.io/vXsp3
PublishPlugin/master 7202e86 Florian Schlichting: Item14198: _renderZones() has moved in Foswiki-2
[14:12]
***GithubBot has left [14:12]
gac410fsfs: What version of URI are you running with. I suspect an out-of-date module. [14:15]
fsfshmm, 1.64, Debian Jessie [14:18]
gac410userinfo has notes in the "changes" log going back to 2003. - release 1.25
Anyway ... if you want a fix in a release, please commit to Release02x01 branch. We then merge into master. But can never merge from master into Release02x01
[14:18]
fsfsI think the userinfo method is only defined for certain types of URIs, not for URI::_generic which is what filesystem paths apparently are [14:21]
gac410Ah... [14:21]
fsfsso yes, Lynwood would probably like to see something released soonish; for me it doesn't matter but more people testing wouldn't hurt
gac410: should I cherry-pick the commit to Relase02x01?
[14:22]
gac410Yes please, cherry-pick that fix into Release02x01. The advantage of the upwards merge into master, is that the commit id doesn't change. you can do things like git branch --contains "commit-d" and it will find the merges, but not the cherry-picks. [14:24]
fsfsBTW I was wondering why perltidy hasn't shouted at me today yet, is it possible I lost a post-commit hook or something? [14:24]
gac410As long as you ran pseudo-install for an extension in the github foswiki repo, it should be there.
pseudo-install re-adds them
You can look in the .git/hooks directory - 2 of the hooks should be symlinked into the tools path
[14:25]
fsfsok... [14:27]
gac410eg. pre-commit -> /var/www/foswiki/distro/core/tools/develop/githooks/pre-commit Thats the one that checks tidy [14:28]
fsfshmm, they existed but were dangling for some reason :-/
pseudo-install.pl recreated them properly now
[14:29]
gac410I was going to suggest that maybe the tidy hook has trained you to enter all code tidy in the first place :D (Hasn't worked for me) [14:30]
fsfs:-D [14:30]
GithubBot[distro] fschlich pushed 1 new commit to Release02x01: https://git.io/vXGee
distro/Release02x01 219f796 Florian Schlichting: Item14198: do not crash when the URI does not contain a userinfo part...
[14:30]
***GithubBot has left [14:30]
fsfsgac410: say, I've got an issue with a local extension repository. Previously extensions could specify any location in their PACKAGES_URL, but nowadays foswiki.org is hardcoded in tools/extender.pl
plus, the value is not really a URL any more, but the name of a repository as defined in LocalSite.cfg
[14:32]
gac410ugh. That somehow got munged up along the way with the redesigining of the extensions installer Needs a task for sure. I've never tried it with a private repo. [14:34]
fsfsI'll report a task [14:34]
........ (idle for 35mn)
Item14208 [15:09]
FoswikiBothttps://foswiki.org/Tasks/Item14208 [ Item14208: extender.pl hardcodes Foswiki.org extension repository ] [15:09]
gac410Thanks fsfs ... Can't promise when I'll get to it, but I'll try to figure something out. [15:10]
fsfsthanks - if tools/configure can be made to work with both keys and URLs that would of course be perfect, but I could live with that very small patch as well [15:11]
gac410It can be made to work with a key by adding -set {key}='value' ... to whatever wizard is being run.
If there is no -save, then the config won't be updated. But I suppose you want -save when installing extensions so the merge of the spec files gets saved.
Anyway, I'll review it.
[15:12]
..... (idle for 21mn)
***ChanServ sets mode: +o CDot [15:34]
.......... (idle for 48mn)
foswiki_irc0_Hello! I am trying to setup an Apache authentication with Kerberos. Almost everything works so far, except one little thing: When logging in with Firefox from Windows, the usernames are in UPPERCASE. The Usermapping to the WikiUsers doesn't work then. Can I tell Foswiki somehow to ignore the case or convert all usernames to lowercase?
I found the LdapContrib extension which seems to have such options, but for now I don't want to use this. Is there really no way of doing this? I cannot believe to be the first one having this problem.
I searched the whole day for options up to now. I would assume such an option in mod_auth_gssapi, but there apparently is none :(
[16:22]
Well... I guess I could hack this into TopicUserMapping.pm [16:34]
LynnwoodGreetings all! I have a situation come up where the password file (data/.htpasswd) got screwed up. Basically a large part of it got lost.
This was a site that I recently updated. Has anyone seen this happen? Any ideas why it would happen? My only hunch is that it has to do with character encoding.
And on that subject, what is recommended for how to deal with htpasswd file on site that has been converted to uft8?
Can the htpasswd file be updated or should we use the setting for specifying the encoding for that file and specify older encoding?
[16:44]
......................... (idle for 2h0mn)
CDotLynnwood: the passwd file is managed by Apache::Passwd, AFAIK. If it has problems with encoding, likely they are upstream :-( [18:47]
LynnwoodCDot thanks for that lead. I can't seem to replicate issue. [18:47]
CDotDoes it look like it stumbled on a wide character? [18:48]
LynnwoodI have scoured the file to make sure no non-uft-8 characters are there. [18:48]
CDotlook for high-bit chars as well [18:48]
LynnwoodThat's about the only idea I could come up with. I've never seen this happen.
ok, thanks.
[18:49]
CDotthere are tools you can use to validate the encoding used in the file, I think
CDot seems to remember one or more command-line tools for doing that
file -i passwd
[18:49]
.......................... (idle for 2h6mn)
gac410Lynnwood: Large part of htpasswd lost? On 2.1.2? Or some older version. We indeed had a severe bug but that was a long time ago ... and fixed ages ago.
actually it would end up with an empty file, not a partial. I really doubt that's what you ran into though.
[20:57]
Lynnwoodhey gac410 - this was on 2.1.2 [20:58]
gac410no 1.1.something iirc. [20:59]
Lynnwoodyea, this happened after recent upgrade which is why i suspected encoding. [20:59]
gac410We kept losing the file on foswiki.org. It was a race condition between fcgi handlers iirc. Crawford was the one to find the problem. It was ugly for sure.
we do not have any mechanism to fix the encoding on .htpasswd. It has to be a manual process. There is an expert parameter in the config to allow .htpasswd to have a different encoding, but strongly not recommended.
[20:59]
Lynnwoodit was very strange. It cut off maybe half the file. It happened at same time as someone either registering or changing password, although strangely, the truncation did not happen in immediate vacinity of where the change occurred. [21:00]
gac410strange. [21:01]
Lynnwoody, i looked at that but wantd to avoid it. [21:01]
gac410Another tool youl can use is the "isutf8" command line tool... comes in with misctools or some sort of "miscellaneous" package. [21:02]
LynnwoodThe earlier version of the site was ISO-8859-1.
Yes, I noticed that in the documentation.
[21:02]
gac410and the "perl say ..." command documented in the UTF8MigrationConsiderations [21:02]
Lynnwoodright. i found that today.
I think some password or email had a character that was not properly ascii.
[21:02]
gac410If the code had trouble reading the htpasswd, I'd expect that you'd have a crash in the log - [21:03]
Lynnwoodhmmm i should go back and try to check that. [21:04]
gac410Where you can get into trouble, is the file is cached. And if for some reason the read died corrupting the cache, another txn could write out the partial file. We fixed it by always re-reading the file before a save, rather than trust the cache.
And if the read failed as part of an update, I would expect a crash and not complete to write out the file.
The problem characters for an iso-8859-1 file would be the "high ascii" - accented characters u-umlat, etc.
[21:05]
Lynnwoodhelpful info
the log you mention - would that be apache log or htpasswd?
[21:16]
Well this is strange. I do find in apache log a couple of entries like this: [21:21]
gac410if it's a crash, it would be in the apache log. Probably should check the foswiki error.log too. [21:22]
Lynnwood[Wed Nov 02 13:50:18.035198 2016] [cgi:error] [pid 21965] [client xxxx:xxx] AH01215: cannot change user passwords using Foswiki::BaseUserMapping at /path/to/foswiki/lib/Foswiki/Users/BaseUserMapping.pm line 496., referer: https://this.site.com/System/ResetPassword [21:23]
FoswikiBothttps://trunk.foswiki.org/System/PerlDoc?module=Foswiki::BaseUserMapping [21:23]
Lynnwoodhowever. the site is configured to use TopicUserMapping [21:23]
gac410That looks like someone was trying to set or change a password on AdminUser, RegistrationAgent, etc. Those are covered by the base mapper.
So I don't think that would cause issues.
[21:23]
Lynnwoodok [21:24]
gac410Just recreated it here. logged in as "admin" and tried to change password. Got that message. [21:25]
Lynnwoodhmmm
i don't think there is an admin password
[21:25]
gac410well it could happen for any ID managed by base mapper. WikiGuest, AdminUser, RegistrationAgent, UnknownUser, ... [21:26]
Lynnwoodok. so if the user thought they were logged in but were not... and tried to change password. [21:27]
gac410Not sure. Playing with the screen, I have to be in the admin group, otherwise it needs the "old password". Admin users can set a new password for any ID
And of course base users don't have passwords, except for AdminUser, but that's "special"
[21:30]
LynnwoodI search foswiki error.log and only entries I find for "password" are like the one above, or some where the password file was not found (because I was trying to fix it) [21:35]
gac410I'd expect that a utf8 character failure would result in a real crash - in the apache error log. Do you use fcgi? There are two recommended settings - Cache passwords, and Detect modification.
Both should be enabled.
[21:36]
Lynnwoodyea, as yet I haven't enabled fcgi - there was a rush for getting it updated and i've had other priorities than going back and enabling fcgi. [21:37]
gac410Having cache enabled and detect modification disabled, would be bad. Oh.. okay, for plain old cgi, cache is useless, and detect modification also not needed. [21:37]
LynnwoodI had to re-write all the javascript for a bunch of apps (there was lots) because they wanted to remove any in-line js. So I changed everything to store js in files and load.
fcgi was one less thing for me to work around.
[21:39]
gac410cool. Did you check them in? That's something we really need to do for all of core. [21:39]
LynnwoodThese were mostly all custom apps. The only places where I did it on standard pages (such as "onchange" parameters) was where I absolutely had to.
i didn't modify distributed files.
[21:40]
gac410oh okay [21:42]
Lynnwoodactually NatSkin did not have much that was not already in files. [21:42]
gac410yeah michael has been pushing toward supporting content security policy denying inline javascript [21:42]
Lynnwoodbut my apps were full of it. I'd gotten use to rendering javascript on the fly and adding to script with ADDTOZONE
me also
later
[21:42]
............. (idle for 1h2mn)
vrurgvrurg wonders if [[#AnAnchor][Link]] must use full URL including query params
Doesn't work when used with PerlDoc.
[22:45]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)