#foswiki 2015-10-02,Fri

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
GithubBot[distro] gac410 pushed 8 new commits to Release01x01: http://git.io/vczYh
distro/Release01x01 3c99bb4 George Clark: Item13775: Workaround for CGI::Param in list context...
distro/Release01x01 e0657cd George Clark: Item13776: Escape braces in regular expressions.
distro/Release01x01 21c5380 George Clark: Item13777: back-port URLPARAM encode and SEARCH decode...
[00:22]
***GithubBot has left [00:22]
FoswikiBothttp://foswiki.org/Tasks/Item13775 [ Item13775: CGI::Param called in list context - Basic fixes for Foswiki 1.1 patch ]
http://foswiki.org/Tasks/Item13776 [ Item13776: Perl 5.20 - 5.23 compatibiltiy fixes for Fosiwiki 1.1 ]
http://foswiki.org/Tasks/Item13777 [ Item13777: Backport URLPARAM encode and SEARCH decode changes from Foswiki 2.0 ]
[00:22]
..... (idle for 21mn)
gac410MichaelDaum. I've updated Release01x01 in git to hopefully be compatible with new JQueryPlugin, as well as fixes for braces, CGI -any, param in list context, etc.
You said you had a bunch fixes you apply for 1.1.9 systems. Could you see how what we now have in github compares to your changes?
Then we need to decide. Patch contribs, or should we build a 1.1.10 with this small number of critical bug fixes.
Due to the size of these I'm thinking about building 1.1.10.
That would also pick up all the translator's efforts.
[00:43]
.............. (idle for 1h5mn)
GithubBot[distro] gac410 pushed 2 new commits to Release01x01: http://git.io/vczEB
distro/Release01x01 e4eff8d George Clark: Item13776: More braces need escaping
distro/Release01x01 de2d949 George Clark: Item13779: Sync Build tools over to Release01x01 branch...
[01:50]
***GithubBot has left [01:50]
FoswikiBothttp://foswiki.org/Tasks/Item13776 [ Item13776: Perl 5.20 - 5.23 compatibiltiy fixes for Fosiwiki 1.1 ]
http://foswiki.org/Tasks/Item13779 [ Item13779: Sync BuildContrib over to Release01x01 branch ]
[01:50]
..... (idle for 21mn)
gac410MichaelDaum. I also uploaded JQueryPlugin from Foswiki 2.0.2 It's still got compatibilty issues in the Ajax helper, it doesn't have the corrupted. foswiki.jquery.js [02:11]
......... (idle for 40mn)
GithubBot[distro] gac410 pushed 1 new commit to Release01x01: http://git.io/vczK3
distro/Release01x01 a1daad6 George Clark: Item13776: More braces need escaping
[02:51]
***GithubBot has left [02:51]
FoswikiBothttp://foswiki.org/Tasks/Item13776 [ Item13776: Perl 5.20 - 5.23 compatibiltiy fixes for Fosiwiki 1.1 ] [02:51]
..................... (idle for 1h44mn)
***gac410 has left [04:35]
............ (idle for 58mn)
ChanServ sets mode: +o CDot [05:33]
....... (idle for 34mn)
ChanServ sets mode: +o MichaelDaum [06:07]
....... (idle for 32mn)
LavrCongratulations all for the 2.0.2 release. Now we are back in the best league. Thanks all :-) [06:39]
........... (idle for 50mn)
CDot:-) [07:29]
.............. (idle for 1h9mn)
MichaelDaumfinally we can botch it again adding new features [08:38]
........ (idle for 37mn)
jastyay! [09:15]
...................................... (idle for 3h6mn)
***ChanServ sets mode: +o gac410 [12:21]
gac410Hi MichaelDaum Did you see my notes from the logs? [12:22]
MichaelDaumy
blogged the new release; updated wikimatrix and wikipedia
[12:23]
gac410Oh... I hope you omitted some of the details! [12:24]
MichaelDaumsome plugins are now incompatible with <2.0.2 ... for minor reasons ... which still need to be sorted out somehow [12:24]
gac410y. I've been testing some of that. It's why I uploaded jQuery last night - ran into the corrupt utf8 code [12:25]
***ChanServ sets mode: +o Lynnwood [12:29]
gac410well at least the were not particularly awful exposures.
So anyway. I
I've been testing 1.1.9 trying to find / fix all the unescaped braces, and then testing the 2.0 plugins on 1.1.9
I'm having a heck of a time with TinyMCE.
I cannot get the new one to work on a fresh 1.1.9 install.
[12:33]
MichaelDaumtinymce needs a desperate update for 2.1 [12:35]
...... (idle for 28mn)
GithubBot[TablesContrib] gac410 pushed 1 new commit to master: http://git.io/vc2zL
TablesContrib/master 1a7cb35 George Clark: Item13782: Update to Foswiki 2.0.2
[13:03]
***GithubBot has left [13:03]
FoswikiBothttp://foswiki.org/Tasks/Item13782 [ Item13782: TablesContrib needs update to 2.0.2 release. ] [13:03]
17SADP1LG[distro] gac410 pushed 1 new commit to Release01x01: http://git.io/vc2zs
distro/Release01x01 b9518ac George Clark: Item13776: Change in stat call parameters
[13:03]
***17SADP1LG has left [13:03]
FoswikiBothttp://foswiki.org/Tasks/Item13776 [ Item13776: Perl 5.20 - 5.23 compatibiltiy fixes for Fosiwiki 1.1 ] [13:03]
6JTAB5WE0[distro] gac410 pushed 1 new commit to master: http://git.io/vc2zG
distro/master 20c1503 George Clark: Item13781: Update extension dependencies
[13:03]
***6JTAB5WE0 has left [13:03]
FoswikiBothttp://foswiki.org/Tasks/Item13781 [ Item13781: Dependencies of default extensions are out of date ] [13:03]
GithubBot[distro] gac410 pushed 1 new commit to Release01x01: http://git.io/vc2gf
distro/Release01x01 b2081d4 George Clark: Item13776: Perl hash ordering breaks email encoding
[13:06]
***GithubBot has left [13:06]
..... (idle for 20mn)
gac410MichaelDaum: the LABEL= option we added to config.spec breaks SELECT type fields in 1.1 configure.
It ends up being picked up in the last choice.
Looks like CommentPlugin and AutoViewTemplatePlugin will be broken on 1.1 configure.
[13:26]
MichaelDaumy [13:28]
gac410Probably easiest is to just remove the LABEL= on those two.
The fixes to make 1.1.9 work with recent perl are quite extensive. I think it's way too much for a Patch contrib.
[13:29]
..... (idle for 20mn)
GithubBot[distro] gac410 pushed 2 new commits to master: http://git.io/vc2yv
distro/master 1647af6 George Clark: Item13780: Remove LABEL= - incompatible with Foswiki 1.1 configure...
distro/master 49b661e George Clark: Item13781: Bump versions for new DEPENDENCIES files....
[13:49]
***GithubBot has left [13:49]
FoswikiBothttp://foswiki.org/Tasks/Item13780 [ Item13780: Configure Spec LABEL= option is not compatible with Foswiki 1.1 ]
http://foswiki.org/Tasks/Item13781 [ Item13781: Dependencies of default extensions are out of date ]
[13:49]
....... (idle for 31mn)
jmk0wait, what? perl introduced new stuff that breaks foswiki 1.1.9? [14:20]
gac410yes
perl 5.18 starts the issues. 5.20 worse. 5.22, disaster 5.23 deepens
[14:20]
jmk0what release of perl?
i can probably search the logs
[14:20]
gac410They've been deprecating things. Most wide reaching is { } braces must be escaped in a regular expression if not used as meta. so any regex that searches from SOMEMACRO{ generates warnings.
Even more serious are changes to CGI. -any pragma is deprecates, and causes warnings or crashes. and calling param in a list context too.
[14:22]
jmk0calling param in a list context? [14:23]
gac410Perl also randomizes hash ordering, which I just found breaks Render.pm - causes a crash.
yes.
need to call multi_param() if it's possible to be multivalued.
[14:23]
jmk0the hash ordering I knew about.
i don't know what the param in a list context, multi_param() stuff is about... something i should look at to familiarize myself?
[14:24]
gac410There is a workaround. You can set a global to disable the check ... for now.
The Release01x01 branch in git now has most of this patched.
[14:25]
jmk0I have to sort hash keys in my plugin to make sure to get consistent results from session to session
ok so multi_param is a CGI thing
[14:26]
gac410The multi_param is a security related change. If you have URL foo=blah;foo=blah2, param('foo') can return an array, if called in a list context. [14:26]
jmk0nothing like core changes to make everyone's day :) [14:27]
gac410It's a bit unpredictable, so on some software, it might cause issues. we know of no exploits in Foswiki, but took the "due diligence" approach to find every call to param and make it appropriately param or multi_param if multi-values are acceptable
1.1.9, I did the global "don't check" override.
Foswiki overlays CGI with Request:: so we duplicated all the CGI implementation into Foswiki::Request, and apply all the same checks.
[14:28]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Request [14:30]
jmk0thanks foswiki bot :D [14:31]
gac410So anyway, running 1.1.9 on Perl 5.23 is not for the fearless :) [14:31]
jmk0param() returning an array (reference?), that's the prior behavior? presumably it doesn't do that now, if i understand you correctly
what about 2.0.x? still an issue or no?
[14:31]
gac410No. 2.0 completely addressed the changes, and works with old and new versions of CGI, without resorting to disabling the checking.
in recent CGI. say you call SomeFunction( $cgi::param('someparam'), 'other stuff'); that will fail with an error. param() called in list context.
So in recent CGI, If you want multiple parameters, and routine can handle it, call $cgi::multi_param() if not, call scalar $cgi::param()
the issue is that in this sort of call SomeFunction( ... Passing multiple parameters, will cause other parameters to be shifted right, to accommodate the array.
[14:32]
jmk0definitely don't want that
guess you have to do it in two steps now? i'm assuming multi_param returns an array and not an array reference
[14:37]
gac410multi_param returns an array.
so by calling SomeFunction( scalar $cgi:param(), other, stuff ) passing multiple value URL param cannot shift other, stuff, to the right.
[14:37]
It's an annoying change, added in CGI version 4.08. Any site that updates CGI and running 1.1.9 will have logs flooded with error messages. Or if ASSERT is enabled. Crash
4.21 also *removes* the -any pragma, so any "use CGI qw(-any); or other forms of entering that pragma will crash. Foswiki 1.1.9 used it in several places.
[14:46]
LavrInstead of making a heck of an effort making 1.1.9 work on very new Perl versions - I would focus on making small tools to help upgraders converting to UTF-8 [14:48]
jmk0what was the -any pragma? I'm familiar with "normal" exports and groups but not things like that [14:48]
LavrAn existing 1.1.9 will not get a new Perl version unless the admin is suicidal [14:48]
gac410The issue is more CGI. The changes are "vulnerability" fixes, so distros are picking them up. [14:49]
LavrAnd if you move to a new server - we want people to upgrade to Foswiki 2.0.2 now that we have a stable release. The hurdle to jump is conversion to UTF8. [14:49]
gac410jmk0: the -any pragma was a useless CGI html generation feature. With CGI you could call CGI::table() to generate a table tag. Fine. but the -any pragma allowed you to call CGI::fizbin() and create a <fizibin> tag. [14:50]
jmk0shh
er. ahh
[14:51]
gac410Lavr, right. The problem is that a conversion to 2.0 is non-trivial, especially if a site has private extensions that need updating. If a server gets a new CGI due to security updates, it's a big issue,
I've got Release01x01 branch running fine on latest CPAN and perl now. I'd like to generate a couple of PatchContribs. to address. the CGI change, and to address our URLENCODE change.
[14:51]
LavrNew CGI can happen yes. But not a new Perl generation. No major distro updates Perl to a new generation. Only patch releases of Perl. [14:52]
gac410The easiest way to build a patch, is to actually get Foswiki running with the patch. And then generate the Diffs. And I couldn't get 1.1.9 to run on my system any more. So meh... I fixed it. [14:54]
Lavrgac410 which distro is your "normal" distro? [14:54]
gac410well my laptop is running on Ubuntu 15.4, which is where I do all my dev Perl 5.20.2 is the distro version.
My production servers are Gentoo... and they stay relatively current as well.
I use perlbrew to test 5.8.8 -> 5.23.3 But getting old CPAN is more difficult than getting old perl.
[14:55]
jmk0if there's an issue, what are the symptoms going to be? We're using perl 5.18.2 and CGI 4.21 [14:57]
gac410with 1.1.9? [14:57]
jmk0yes [14:57]
gac410hm Are your logs getting lots of errors? [14:58]
jmk0not that i can see
which logs, though, apache or wiki? not that i see many in either
[14:58]
Lavrwell. it is good someone is testing bleeding edge distros. But any business admin would normally go with the long term supported versions of Ubunto or another enterprise distro. And they are still a generation or two behind on Perl versions [14:58]
jmk0Use of uninitialized value within %Foswiki::Render::ESCAPED in substitution iterator at /var/www/foswiki/lib/Foswiki/Render.pm line 1007. [14:58]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Render::ESCAPED [14:58]
jmk0sorry, looks like we ARE getting errors [14:59]
gac410jmk0 that's a perl hash ordering bug. I just fixed it in git. [14:59]
jmk0or not :) [14:59]
gac410in the Release01x01 branch [14:59]
jmk0[Fri Oct 2 09:48:00 2015] -e: Use of uninitialized value $name in string eq at /var/www/foswiki/lib/Foswiki/Form.pm line 576. [14:59]
gac410http://git.io/vc2gf fixed the ESCAPES error [15:00]
jmk0most of the errors seem to be that form one [15:00]
gac410I'm not aware of that one if it's a bug.
You must have patched the multi_param and cgi -any errors. The last one completely breaks foswiki
Lavr, I have test VM's with Ubuntu 14.04LTS and 12.04LTS for testing as well.
[15:01]
jmk0is there a difference between /regex/ and m/regex/ ? [15:02]
gac410no. [15:02]
jmk0ok [15:03]
gac410It just makes the "match" aspect explicit rather than the default.
m/x/ match s/x/y/ substitute /x/ match but implied.
[15:03]
jmk0fwiw there are over 6000 instances of that Form.pm error message in our error log
vs only 1300 of the ESCAPED one :)
[15:03]
gac410The ESCAPED one is random. User will get a crash screen though. It depends on where the backslash ends up in the array ;)
er.. .in the string built from the array.
[15:05]
jmk0sounds like there's at least a fix for it
i don't see any "use CGI (-any)" or anything like it in lib/...
[15:05]
gac410search for '-any' [15:08]
jmk0i did. find lib/ -name \*.pm | xargs grep -- -any [15:08]
gac410you must have patched it.
Your form.pm crash. There are no fixes in that area. I'd guess you have a bad form defn somewhere maybe?
Do you have a traceback? who calls getField( with the undef ... I'd guess the problem is up higher.
[15:09]
jmk0no, just that error message
curiously enough, i find a ton of them all at the same time. Though for whatever reason, apache's error.log doesn't always have a time stamp with every message
[15:11]
gac410This is the -any pragma patch: https://github.com/foswiki/distro/commit/7da696e0af28aab79e51be2239bcaac740a021c2 [15:13]
jmk0yeah, Comment.pm at least has been patched on our system, didn't look at all of them [15:15]
gac410Lavr, the reason I test on latest perlbrew is to head off future issues. That and jomo kicks me when he finds issues unless stay up with it.
jmk0: you must have patched. the -any pragma change is not forgiving. It just crashes.
[15:15]
jmk0those Form errors happened in two batches, one of 2142 instances (all at once) and another of 4284 (no time stamp) [15:16]
gac410There was one perl change in 5.23.3 so far that needed a patch. stat(@array) is no longer legal.
Maybe an oddball search? no idea.
And nowhere in the batch was a traceback, not even on the initial one?
[15:17]
jmk0stat(@array) ? My old 2000 edition of the perl book doesn't even hint that's a legal thing
nope, no traceback
[15:18]
gac410Right. It's not legal, but in common use. PlainFile logger used it in one place.
jomo caught the issue
And suggested the fix.
-sub _stat { stat(@_); }
+sub _stat { @_ ? stat( $_[0] ) : stat() }
[15:19]
jmk0stat called without any arguments does what? current working directory? neither the book nor perldoc seem to explain that [15:23]
gac410tbh I'm not sure. :( I took his patch and it seemed to fix a crash I was encountering, so I was happy. [15:25]
jmk0lol [15:25]
gac410I suspect in this case, that routine would never be called with undef. It's attempting to determine if a log needs rotation. But it sure did kill foswiki logger on perl 5.23.3
jomo's task Item13749
[15:26]
FoswikiBothttp://foswiki.org/Tasks/Item13749 [ Item13749: Perl's "stat()" will not work with with array arg and other Perl 5.23 compatibilty issues. ] [15:28]
gac410Lavr: Other than the encoded link issue, what are your concerns for the utf8 migration .. where do we need work? [15:30]
LavrThere are two things with the conversion. First - we have two tools. I am not sure how the bulk convert thing is doing the job. Did not try it. It is a painful tool because it requires that you get both a 1.1.9 and a 2.0.2 installation going. Not very reasonable.
And then we have the charset converter that I think is doing a pretty good job if you tell it that the input data is windos
windows
So being a little more clear in which tool to use and lock in on ONE tool would be a good step
[15:35]
gac410yeah, the bulk_copy has some issues. Not only the two installs, but can't handle bad charsets, and omits any file that Store doesn't understand.
CharsetConverter is in-place, so it leaves any cruft in place.
CDot wants bulk_copy to be the choice because, only it can convert from RCS to PlainFile, and/or any future DB based store.
But I think CharsetConverter is a better tool.
[15:36]
LavrThe other thing is that even with the tool there will be issues. If we assume the tool does a really good and OK job at converting attachments to UTF8 there are still all these manually pasted in URLs that uses the iso centric %XX chars. [15:38]
gac410actually *any* URL inserted by Foswiki::Attach() will have the %XX characters. regardless of utf-8 or iso. [15:39]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Attach [15:39]
LavrI was thinking that we could make some semiautomatic tool that walks through all topics and each time it sees a URL that points locally it can suggest a convertion that you can interactively say YES or NO to
Command line tool.
[15:39]
gac410It could be close to automatic. Needs to find each [[]] and <> link that references a pub location. decode, test if file exists. if so, move on. If doesn't exist, try to convert to utf-8. if file exists, update URL and move on. If file doesn't exist, now you need manual intervention [15:41]
LavrIf I look at my own wiki I have plenty of attachments where I have manually removed huge ,v files to save space. If the bulk thing thinks that is croft and ignores these files it will really do harm [15:41]
gac410No. Store still sees them The issue is hidden attachments Store ignores any file with leading dot or underscore, and also ignores subdirectories. [15:42]
LavrI do not remember having any dot files or underscore files or sub directories that are not the result of a sub web. [15:43]
gac410Some of the extensions use _blah filenames to hide caches and other stuff [15:44]
***ChanServ sets mode: +o CDot [15:45]
gac410Trying to bulk_copy System web is death-to-Foswiki :D
Howdy CDot
[15:45]
CDotDeath, doom, gloom, destruction. All excellent subjects for Hollywood films. [15:45]
gac410:) [15:46]
LavrI think the current best advice to upgraders is to NOT convert from RCS to plain file. After all - it does not really give any huge advantage I would think. [15:46]
gac410well it is a huge performance boost for topics with large history, or large binary attachments even with smallish historys [15:46]
CDotLavr: au contraire. If you have a deep history e.g. in a history topic, it makes a significant difference.
otherwise I wouldn't have burnt the candle at both ends to develop it :-)
[15:47]
LavrBut only for those few topics with huge history. Not that the wiki runs with double the speed or formatted searches are twice as fast. Nothing that makes me scream - I need it. Take my money now! [15:48]
gac410y. I really wanted to convert foswiki.org, mainly due to our huge Extension attachment histories. Some of them were becoming impossible to upload. [15:48]
LavrIn any case. As an upgrader I think it will be nice to be able to make a two step approach. First convert to UTF8 and see all is well. And then you can consider the RCS to Plain [15:49]
gac410Something is still better on 2.0 with those big binary attachments. I was able to upload JQueryPlugin attachments on f.o fcgi, without timeouts. which is really surprising. [15:49]
LavrAnd I would assume such conversion could be done from a 2.0.2 to a copy of itself [15:50]
gac410Yes bulk_copy works that way. [15:50]
LavrWhich would be 10000 times more easy to setup than fighting to get a 1.1.9 working on your new server just to run convertions [15:50]
gac410Especially if new server has perl from this decade. [15:50]
LavrYes that is what I was thinking
The problem for the upgrader is downtime. The conversion from ISO to UTF has to be done with the production server down or in read only.
Unless you want to manually convert 100s of changes topics later
So an efficient convertion you know you can do over a working day in a weekend is essential
So I would advice an upgrader to take these steps..
[15:51]
gac410anyway, to prioritize. I think we need in order. 1) Patch 1.1.9 for backwards compatibility to new JQueryPlugin. 2) Patch 1.1.9 for the CGI changes, as that is more likely & disabling. [15:53]
Lavr1. Get new hardware now that you have the chance. And a new up to date distro with LTS support [15:53]
gac410Then link fixer for CharsetConverter
Finally consider a 1.1.10 with the perl compat fixes.
[15:53]
Lavr2. Install a 2.0.2 on it and get it running with all the plugins you depend on
3. Install your tailorings - -logos etc etc
[15:54]
gac410Lavr, you can run CharsetConverter on 1.1.9, and get it running with a utf-8 store. There are limitations because core is not unicode, but it basically works.
I think MichaelDaum runs all his 1.1.9 systems with uft-8 store.
[15:55]
LavrI would not count on that idea
The 1.1.9 is production. You do not want to touch it.
[15:55]
gac410That was also how we did Foswiki.org. So we could run trunk in parallel [15:56]
LavrI think it is essential that the admin can spend all the time he wants on the new server while the old keeps running.
And then on a Saturday you take the production server down. Pull over all the webs to the new server. Run the Charset Converter maintaining RCS as store, and get that running.
And then you have the new server up running end of Saturday. And you have Sunday to troubleshoot.
[15:57]
gac410It really depends on whether you want to symlink new foswiki into existing 1.1.9 store.
Although I suppose you can still do that by configuring 2.0 for iso-8859-1 store.
[15:58]
LavrI think the best future proof advice to give is - when you upgrade from 1.1 to 2.0 - convert to UTF. Anything else is just pain and more pain. [16:00]
MichaelDaumYes. Been using utf8 on 1.1.9 for a long time [16:00]
LavrAnd that did not give any broken URLs with %XX in links all over? [16:01]
MichaelDaumas well as all 1.2.0 during the development cylce ... before and after CDot implementing unicode properly [16:01]
LavrThat is impossible! [16:01]
gac410MichaelDaum: how do you deal with attachment links encoding URLs as ..... exactly
If you have an attachment with an umlaut, 1.1.9 will encode it as the iso- character %xx, 2.0.0 will encode it as the %xx%xx utf8 char
[16:01]
MichaelDaumI always avoided unicode wikiwords or urls or attachment names like the devil the holy water
topics have allways been created using transliterations
[16:03]
gac410If you can control your users, yes that's a solution. [16:03]
LavrSo in other words, your users never added attachments? [16:03]
MichaelDaumLavr, ? [16:03]
LavrOr created topics where they decided on the names? [16:03]
MichaelDaumthey were auto-corrected [16:03]
gac410Do you have some js that corrects filenames during upload? [16:04]
MichaelDaumfor those clients with even stricter needs there is a plugin TopicNameValidationPlugin [16:04]
gac410So somehow you prevent a user from attaching "André.jpg" [16:05]
MichaelDaumI think that never was a problem [16:06]
LavrWell. We are discussing how we should guide a normal Foswiki that has been in production for years upgraded to 2.0 and the reality is that they are full of %xx links and attachments and topics with any sort of chars [16:06]
MichaelDaumand no. people did not have André jpegs on their hard discs [16:07]
gac410With 1.1.9, the filename and link will both be iso-8859 encoded. Except the link will actually use %xx for the high characters. So converter cannot fix it.
No documents with an umlaut in the name? Even in a German company?
both lavr and andreli were banjaxed by attachment filenames. But counts were low enough to permit manual correction.
[16:07]
MichaelDaumjust tested a frühstüch.jpeg on a client's 1.1.9er ... no problems what so ever ... sorry [16:10]
LavrI am lucky I am in an English speaking company. If I had been in a Danish company there would have been 10000 topic names and file names with Danish chars. [16:10]
gac410Did you insert a link to the attachment? Using mainline foswiki Attach, which RFC encodes the URL? [16:11]
LavrThe problem comes when you have created it on a site in iso. And then convert the site to UTF. [16:11]
CDotremind me, someone. Why doesn't the CharsetConvertorContrib rename attachments to the UTF8-encoded name? [16:12]
gac410It does. It's the links that are an issue because they are plain ascii with all high characters changed to %xx entities [16:12]
CDotoh right, you mean the links generated when an attachment name is embedded in a topic [16:13]
gac410so iso umlaut becomes utf8 umlaut in the pub/... location. But the link points to the %xx iso character.
right.
[16:13]
CDotdo people really use that? I never have
and if MichaelDaum doesn't either, I would imagine he hasn't encountered this scenario
[16:13]
gac410I see it used .. .heck I often use it myself. Click create link. done. [16:14]
MichaelDaum/pub/Sandbox/AttachmentTest/frää.jpg properly encoded as /pub/Sandbox/AttachmentTest/fr%c3%a4%c3%a4.jpg ... [16:14]
CDoty, but Michael re-engineered all the UI in NatSkin [16:14]
gac410I just don't use umlauts very often. [16:14]
CDotso maybe he doesn't support that embedding business
CDot is guessing
[16:14]
gac410Okay MichaelDaum so sounds like core is seeing the utf-8 bytes when it inserts the link, so no problem. [16:15]
MichaelDaumy [16:15]
gac410I guess it's only going to be existing links that were inserted while store / charset was iso-88 [16:16]
MichaelDaumah ok
a situation I never was in
[16:16]
LavrThe problem is normal users copy and paste links in the most bizarre ways. I have 1000s of links to topics and images which are the entire URL with http://blabla
And they work!
[16:16]
MichaelDaumLavr, problems only start when your SiteCharSet is _not_ utf8 when people paste in unicode chars.
on a 1.1.9er
[16:17]
LavrWell not really [16:17]
gac410They would probably be okay, unless they are copied from a link generated by the Attach dialog [16:17]
LavrBecause those chars looked like garbage from the moment they pasted them in
And then they edit again and fix it in most cases.
[16:17]
MichaelDaumI rember having seen these kind of horrors long time ago
on twiki cairo
[16:18]
LavrOh yes. [16:18]
gac410Anyway, I think we need to figure out what possible links point to the current wiki pub. http: <a href= <img <script <... and if it points to a pub location, or has macros that point to a pub location, need to decode / encode [16:19]
MichaelDaumeach save making those strings worse and worse [16:19]
LavrWhen I did the test convertion on my production site I had around 70 or so topics with UTF chars. I spent 3-4 hours manually renaming attachments and links to them to pure ASCII as preparation. The plain text simple garnage chars I do not care about. They just convert over as garbage [16:20]
gac410Ah... . Is there any way to "subtract" from a SKIN setting? ie, if I don't want NatEdit, set SKIN = -natedit to remove natedit from the setting? [16:20]
MichaelDaumset SKIN = pattern [16:20]
LavrBut if I had had 1000 of these it would have taken a month so it is not a reasonable way to ask admins to upgrad
e
[16:20]
gac410Ah... but only if I know the original skin setting. [16:20]
MichaelDaumy [16:20]
gac410ie, might have SKIN = myskin, fooskin, natedit, pattern ...
Anyway, it was just a feature I was pondering and missing :D
[16:21]
MichaelDaumyou will have to know that by heart [16:21]
LavrSo Michael. My idea to help the upgrader was to have a simple command line tool you can run AFTER you have run the Charset conversion that tries to repair %XX URLS. And it could be interactive so it asks 1 by 1 if you want to convert to a suggested value or manually type a replacement string. Then the repair could be done in 10-30 minutes on a very bad site [16:23]
gac410If it can be at all automated, I'd prefer to do it as part of the conversion. [16:24]
MichaelDaumLavr, good idea [16:24]
LavrIf the link can be matched with an already converted attachment then it can be automatic [16:24]
gac410Also have to remove <verbatim> blocks. [16:25]
MichaelDaumguys have to leave you alone with these thoughts and head into my weekend [16:25]
gac410Ah and that's another feature question. Why don't we make TakeOutBlocks / PutBackBlocks accessible via Func. We have lots of extensions [16:25]
MichaelDaumgac410, +1 [16:26]
gac410that violate the API or roll their own. [16:26]
MichaelDaumy
bye gotta run
[16:26]
jmk0i haven't been following closely but i thought takeout/putback WERE accessible [16:26]
gac410gac410 goes to make a feature proposal.... Have a nice weekend
jmk0: nope. Not part of the published API.
[16:26]
jmk0ah, i've been using Foswiki::takeOutBlocks
and putback
[16:27]
gac410yes. Technically you can't use them, that violates the API. [16:27]
jmk0i see
welp :)
[16:27]
gac410Foswiki methods are considered private. But everyone does it [16:27]
jmk0yeah, those should be accessible to extensions imjo
-j
[16:28]
LavrI do not use the private Foswiki functions. I am clean like an angel. More holy than the Pope. :-) [16:29]
jmk0that doesn't sound too hard ;-) [16:29]
LavrRight :-)))
clean like a Hells Angel
[16:29]
gac410There is no real risk for stuff like that, but Func:: is assured to be stable. Foswiki:: is free to change, as long as Func keeps it compatible. [16:30]
LavrMy next big move at work is to get some new hardware before I upgrade. Requires funding. Alternative is to put to new 10000 rpm harddrives in my current server and upgrade it. I have those in a drawer
I know endusers are more forgiving to CHANGE (ARGH!!!) when they also see a speed increase so I prefer to get some kick ass hardware upgrade
[16:35]
gac410SSD [16:36]
vrurgAnybody to advise on adding a new repository on github for a new contrib? And giving access to an existing but abandoned one? Docs aren’t of much help here, unfortunately. [16:36]
gac410Hi vrurg ... I can add access to abandoned, or create new repos. What do you need? [16:37]
LavrI SSD in my mirror backup server. And yes it is faster. But I have so much RAM in my production server that most files are cached anyway so the difference is mainly when you search over many webs [16:37]
gac410Only "account owners" can create new repos, so we have not delegated that authority too widely. [16:38]
vrurggac410 – DatabaseContrib/DBIQueryPlugin. I have them in my personal profile but would be better to integrate. [16:39]
gac410okay. hang on. [16:39]
vrurgThat’s understandable. I’d only suggest adding a paragraph about how to find people with proper permissions into starting new extension documentation.
Thanx!
[16:40]
gac410okay. DBIQueryPlugin is added to FoswikiDevelopers team
I'll create a new DatabaseContrib If you could give it a few hours to let our cronjobs update it after I create it, that would be great.
Do you have a short description for DatabaseContrib ?
[16:41]
vrurgYep. Strange I haven’t included it into my personal repo… wait a min.
Provides subroutines useful in writing plugins that access a SQL database.
[16:43]
gac410okay .. hang on
Okay DatabaseContrib created. Next time the cronjob runs, it will auto-install the hooks needed to send emails, irc notices, and update tasks with commits.
[16:44]
vrurgThanks! :) hopefully it’s gonna be useful not for me only. [16:47]
gac410That would be great. Next run of hooks update will be about in an hour. if I'm reading my crontab correctly
btw if you have energy, translate.foswiki.org can use all the help it can get :)
oh... I see you are already there. the foswiki-translations@lists.sourceforge.net is a good list to subscribe to. Very low volume. We only ping it ahead of releases to encourage final translations.
We are trying to use a "continuous translation" model, with relatively frequent string updates, so translators don't have a massive catch-up before each release
[16:48]
vrurgAll my energy is in documentation/test suites for the moment. :) Translations web requires some time to understand the way it works, but I need to understand the development procedures of foswiki in first place. [16:52]
gac410The big difference is git. Other than that, most of the basic procedures are similar to what t* was doing [16:53]
vrurgBut I’ll get into it asap. [16:53]
Lavrgit was a big learning curve for me and I am still not using it as a power user [16:54]
gac410Our plan was to eventually create some sort of request form that could automate creating a new repository, but we ran out of gas / energy along the way. [16:54]
vrurgWell, git is not a problem. Not that I’m a big professional with it but one way or another it’s been used in several projects. Yet, I never followed proper development procedures with TWiki either. Never had enough time, unfortunately. [16:55]
gac410One thing we ask is that if you use a repository in the foswiki account, that you also create a Task (like with TWiki) and identify each commit wiith at ItemNNNN: in the commit message [16:56]
LavrI have given up committing core changes on my git repo from which I run pseudo-install because I now have 50 files or so that are changed to get it running close to how my production site looks like. So I have a virgin git clone of the distro that I copy my changes to and commit from there. An extra step but it works for me for now [16:56]
gac410Lavr once you approach power user status, you can consider creating a "LocalDataContrib" and put all your local files in there. That way you can pseudo-install your additions into any checkout.
No need to push it anywhere ... It's just an extension where you ran a "git init"
[16:57]
vrurgRight, making a task every time a typo is being changed is too much of a burden. So, I’d rather commit into my personal most of the time, than put together a pull request and initiate a task for it. [16:59]
gac410I use webname overrides to make it easier. Usersweb instead of Main, Litterbox instead of Sandbox. even keeps it cleaner [16:59]
Lavrvrurg - as a plugin developer you just create a task for the job and checkin the number of times you need to. 100 if needed. [17:00]
gac410No need to do separate tasks per. Just a single "Create DatabaseContrib ..." task can cover all your commits until you get to release. [17:00]
vrurgOk. [17:00]
gac410One thing though that helps us all. Once you "Release", close the task and open a new one. It can be tough to figure out what's changed in some cases. [17:01]
vrurgGood enough. But then it gets me back to the documentation – it definetely lacks step-by-step description for the whole procedure with cross references to more detailed descriptions of how do some particulars staff. [17:02]
gac410Also, feel free to tag, so when you release v1.0 of your extension. issue a "git tag 1.0" (or whatever tag you want) and then git push --tags
Yeah. We really need help bringing the docs forward. Unfortunately it was mostly written by devs who've been using git for a long time, and things get overlooked.
[17:02]
LavrGeorge - if foswiki.org on 2.0.2? [17:03]
gac410And we still keep finding stale SVN docs that need revision.
Yes. 2.0.2
I did that before we announced.
[17:04]
vrurgIn my case it’s gonna be a release out of the box. Both modules are done and working – ported from TWiki and polished. There will be not starting cycle. Yet, I’m not sure if I’m gonna commit it until plguin’s documentation is refurbished. [17:04]
LavrHmm. Then we have an old bug that returned. Low priority but it was resolved.
Look at http://foswiki.org/Tasks/Item13616
I put a < in front of the title
And look at the form
[17:04]
gac410yeah.... damn I see it. [17:05]
Lavr<tr style='vertical-align:top'> [17:05]
vrurgOk, thanks everybody but it’s really time to finish the doc. :) [17:05]
gac410vrurg: whatever works for you. it sounds good.
we're pretty laid back here. Not a lot of bs hopefully. :D
[17:05]
LavrGeorge http://foswiki.org/Tasks/Item13618 was the bug item
And it is still open.
[17:06]
gac410The unicode core is a BIG change for any extension that interfaces with the outside world vrurg So for 2.0, you may run into issues there.
Lavr, yeah nobody ever picked it up to fix it.
[17:07]
LavrI had the feeling someone did or said he did on IRC. And maybe the fix just never got checked in [17:08]
gac410I don't recall that. DataForm stuff generally ends up with MichaelDaum. I know enough to create holes in feet. [17:09]
LavrIt is not the end of the world. It is a pretty strange thing to start a text field with < and the result is just a cosmetic artefact. [17:09]
gac410Maybe a change from your procedures... But I've been using "ReleasedIn" as a way to say "please fix it in this release" and Target n/a vs. patch is still the document or not flag.
I'll run reports for any ReleasedIn for the current release, regardless of status, to see if there is pending work that should be resolved.
[17:10]
LavrI cannot remember how we used it 3 years ago :-) [17:12]
gac410I *think* it was more backwards looking, set when actually fixed in a release. But I felt I really needed some sort of staging list. regardless of whether we indended to document or not. [17:13]
LavrAnd one symptom I see on my own behavior with NatEdit is that I save without filling out the form. And then I have to edit again to fill the form. Somehow the tabs destroys my mental workflow. [17:14]
gac410Y, I occasionally do that too. [17:14]
LavrWhich is also why - when I upgrade I will taylor the NatEdit to show the form instead of the text if an app is mainly form based.
My requirements database as an example has 99% of info in formfiels and the text area is only used for notes.
[17:14]
gac410I use http://foswiki.org/Tasks/TasksByRelease?release=2.0.2&checkins=All frequently. and http://foswiki.org/Tasks/TasksByRelease?release=2.0.2&checkins=Out
jeeze. CDot, relocating Configure::Util into Configure::Package makes PatchFoswikiContrib complicated. :( It's referenced directly in the PREUNINSTALL and POSTINSTALL exits.
[17:16]
LavrYou should upgrade the MultiTopicEditPlugin to the latest version on f.o. [17:20]
gac410MultiTopicSavePlugin? installed 1.10, available 1.10 [17:23]
Lavrhttp://foswiki.org/System/MultiTopicSavePlugin says 1.8
But InstalledPlugins says 1.10
[17:24]
gac410InstalledPlugins shows 1.10. Strange. I ran into that before. For some reason the Plugin topic doesn't get updated.
Not sure why it happens. When I tried to debug it I could not recreate it
[17:25]
LavrIf we ever want to make metrics topics in Tasks look at the MultiSearchPlugin I did. It can create tables of stats in one search in 2-5 seconds that would normally take 30-60 seconds to do with nested searches [17:27]
gac410I just do not understand what's going on. I just updated MultiTopicSavePlugin... Said it worked. But topic still says 1.8
It bumped the rev, showed that it was updated today.
[17:29]
LavrNot in http://foswiki.org/System/WebChanges [17:31]
gac410Refresh your cache. [17:31]
LavrIs the Cache sick? [17:32]
gac410(link at bottom of page>
Some updates still fail to invalidate the cache. Probably more because of how the installer works.
[17:32]
CDotinteresting; I saw 1.10 of MultiTopicSavePlugin. Then I refreshed the cache on WebCHanges, and now I see 1.8 of MultiTopicSavePlugin
is the cache eating changes?
[17:33]
gac410No idea. I just verified by checking the .txt file. MultiTopicSavePlugin.txt says 1.8
I just installed it locally as a new install, and it is 1.10
For some reason, the installer is not saving the topic file ... and I have no idea why
[17:35]
CDotpermissions? [17:36]
gac410Maybe the changes so that it runs without a session? [17:36]
CDoty, maybe
how did I see 1.10, then? Was that a manual interrupt by you?
[17:36]
***ChanServ has quit IRC (*.net *.split) [17:37]
LavrI still see 1.8 in System web [17:37]
gac410No. I have no idea why... I just checked timestampls .txt and .txt,v are both Oct 2, 17:27
.txt and .txt,v both contain Version 1.8, and no other variations in the history
[17:37]
***asimov.freenode.net sets mode: +o ChanServ [17:40]
gac410It's got to be an issue in Package.pm. Clearly the topic is being saved, and recorded in .changes, and in the ,v history. But it's not the right text.
I looked at this back on 2.0, and could not figure it out. It was in the middle of the upgrade and I was pulling my hair on a bunch of stuff and neglected to open a task. :(
World was burning around me with utf8 issues, backlevel extension topic was least of my worries :D
[17:40]
CDotgac410: are the other files of the plugin updated? Is it just the .txt? [17:47]
gac410I'm not really sure. I *think* it's just the plugin topic itself. [17:48]
CDotCDot thinks that's really unlikely [17:48]
gac410yeah I agree. Though in that case we'd be leaving lots of stuff un-updated, and I'd expect things to be broken. [17:49]
CDotwait a mo
InstalledPlugins says 1.10. It reads from the .pm/
[17:49]
gac410right. [17:50]
CDotso, where did you get the archive? [17:50]
gac410From Extensions. I ran the web installer. [17:50]
CDotOK, Extensions claims to be 1.10 [17:51]
gac410Should have downloaded it.. Unless somehow it's reusing an older copy, but then I'd expect the pM to be out of date.
Installed in on a clean system, and it is completely consistent 1.10
[17:51]
CDotthe topic in the tgz says 1.10
what are other difference s in the topic?
Lavr: ?
n.m. I see the change history
ok, so, the correct .pm's are getting installed, and the correct .txt is in the package
[17:52]
gac410yes. It all looked good on a clean system. [17:53]
CDotis it !noci? [17:54]
gac410Even with !noci, it will still checkin because history exists. [17:54]
CDotthere's a Sandbox file for it too
is that updated?
[17:54]
gac410existing history trumps !noci (or it shoud) [17:54]
LavrNo !noci in MANIFEST [17:55]
CDotno Sandbox/MultiTopicSavePlugin.txt installed. Missing from MANIFEST? [17:56]
Lavrdata/System/MultiTopicSavePlugin.txt 0644 Plugin doc page [17:56]
CDotSandbox [17:56]
gac410There are Sandbox files. [17:56]
LavrOtherwise it would not have been packaged [17:56]
gac410MultiToipcSaveTest MultiTopicSaveTestTarget ... A bunch of files listed in Sandbox.WebChanges [17:57]
CDotit's in the MANIFEST, but it wasn't installed.
Unless the data/ was mucked up somehow?
[17:57]
gac410hm I wonder... Is it due to the same file in multiple locations? [17:58]
CDotmy ( $tweb, $ttopic ) = _getMappedWebTopic($file);
gac410: do you have an installation report? Does it claim to have written these files?
if so, what does it claim to have done?
[17:58]
gac410I didn't run it with DEBUG enabled. [18:00]
CDot$reporter->ERROR("Target $file is not writable");
no errors obvious?
[18:00]
vrurgBTW, any objection to `use v5.10’ in the code? [18:00]
CDotvrurg: no
it's your code, your dependencies
[18:01]
gac410CDot, Just ran with debug, on my system. [18:02]
CDotI can't wait..... [18:02]
Lavrdrum roll [18:02]
vrurgRight. But perhaps there’re reasons to do so which I might be missing. [18:02]
gac410> Installed:  data/System/MultiTopicSavePlugin.txt as /var/www/data/Foswiki-2.0.2/data/System/MultiTopicSavePlugin.txt
let me edit the topic, to get some history and try again.
[18:03]
CDothmm, how did it do that, then? [18:03]
gac410>Installed says it just copied the file. (no history, so why check it in.) [18:04]
CDotthe code path for the meta->saveAs does a next, so that message is not generated (or am I missing something?)
lines 968 and line 992
[18:04]
gac410Okay, ran it again, with a modified System/...txt file. It Installed, so didn't do a checkin, and it overlayed the file. Ah... I'm running from CLI to get a debug log. [18:06]
CDotonly way I can see for that message to be generated is if the horrible condition on line 908 fails [18:07]
gac410And CLI will never checkin, since it doesn't have a session any more. [18:07]
CDotah yes [18:07]
gac410Oh... I'm running plainfile store, that might behave differently. "diff" shows my change but viewing topic does not.
Okay... From the web: Checked in: data/System/MultiTopicSavePlugin.txt as System.MultiTopicSavePlugin
and you don't want to heere this, but it created a new rev, and removed my edit, just like it should
So the differences I can think of. Foswiki.org is running fcgid, and it's using RCS store
[18:08]
***ChanServ sets mode: +o Lynnwood [18:25]
.................... (idle for 1h37mn)
GithubBot[distro] gac410 pushed 1 new commit to master: http://git.io/vcVjC
distro/master 8093c4b George Clark: Item13759: PatchFoswikContrib 2.0 compatibiltiy...
[20:02]
***GithubBot has left [20:02]
FoswikiBothttp://foswiki.org/Tasks/Item13759 [ Item13759: PatchFoswikiContrib not compatible with Foswiki 2.x ] [20:02]
........... (idle for 52mn)
foswiki_irc4Hello folks, I am completely new here, and I have tried to install Foswiki on a shared hosting apache environment and I am not getting anywhere after about an hour so reading the how to's and attempts. I think where I am being held up is that Foswiki needs perl to run, and I am completely unsure how to activate perl on my shared hosting account, any suggestinos or further reading is greatly appreciated. Thanks so much. [20:54]
gac410There are many shared hosts with many differnt platforms. who are you using? [20:55]
foswiki_irc4Thanks, A2 hosting with multi admin panels for each domain. [20:55]
gac410I think A2 uses cPanel? [20:56]
foswiki_irc4Yes they do, I can run one cpanel for each domain.
So in otherwords I have dedicated cpanels
[20:56]
gac410We've had folks here asking about A2 before. I have never seen the panels, so I have no idea how it all works. But they do support perl I believe.
You probably need to ask them for support or documentation.
[20:57]
foswiki_irc4Yes they do support perl
thanks gac410
I will hit them up in live chat
If I may ask, does Foswiki run off perl modules?
[20:57]
gac410Foswik 2.0.2 has one hint in it for cPanel. You probably need to rename bin/LocalLib.cfg.txt to bin/LocalLib.cfg and then un-comment the line near the top that says something like "use cPanelSomethingOrOther ... I don't remember exaxtly [20:58]
foswiki_irc4I read that too [20:59]
gac410Yes Foswiki requires perl And a number of perl CPAN library modules. See Foswiki:System.SystemRequirements (Click on one of the "show" links to reveal the requirements for an OS [20:59]
FoswikiBothttp://foswiki.org/System.SystemRequirements [ SystemRequirements ] [20:59]
foswiki_irc4Thanks gac. have a great day [20:59]
gac410you too. [20:59]
foswiki_irc4gac, if you are still there, here is what my installation looks like. http://tpwtestuser.com/
Am I almost there?
[21:02]
gac410All I get is a forbidden ... I tried URL ..com/foswiki/bin/view and get an "internal error" which probably says you are missing dependencies.
cPanel has some way to install CPAN modules, so you need to install the required modules shown on that SystemRequirements page.
And edit the bin/LocalLib.cfg so that it can find the modules.
[21:03]
foswiki_irc4Thanks gac, out of here till I know what I am doing, thanks for your time! [21:05]
gac410Click the "show" link under Debian OS. The left-most column in the table is the perl modules. So install them using cPanel. [21:05]
foswiki_irc4Thanks! [21:06]
gac410So stuff like CGI CGI::Session, Algorithm::Diff etc.
Crypt::PasswdMD5
Without at least CGI, CGI::Session and Crypt::PasswdMD5, you'll be stuck with "internal server error"
[21:06]
foswiki_irc4Thanks again gac. I believe I am now on the right track, it all comes down to missing dependencies with my perl modules. [21:08]
gac410I think so, yes.
We do have a way to install a *subset* of the dependencies, but if you can install them with cPanel you will be much better off.
[21:08]
foswiki_irc4First time working with perl, I have installed MediaWiki, DokuWiki,Tikiwiki and others, they were a slam dunk, perl is new to me. [21:10]
....................... (idle for 1h50mn)
GithubBot[DatabaseContrib] vrurg created master (+1 new commit): http://git.io/vcwp8
DatabaseContrib/master bc27c01 Vadim Belman: Item13783: Initial commit.
[23:00]
***GithubBot has left [23:00]
FoswikiBothttp://foswiki.org/Tasks/Item13783 [ Item13783: DatabaseContrib -- new contrib ] [23:00]
GithubBot[DBIQueryPlugin] vrurg pushed 1 new commit to master: http://git.io/vcwjB
DBIQueryPlugin/master fbfa223 Vadim Belman: Item13784: Finished porting from TWiki.
[23:11]
***GithubBot has left [23:11]
FoswikiBothttp://foswiki.org/Tasks/Item13784 [ Item13784: DBIQueryPlugin -- finishing port from TWiki ] [23:11]
GithubBot[DBIQueryPlugin] vrurg tagged v1.05 at 2e47069: http://git.io/vcrvk [23:17]
***GithubBot has left [23:17]
GithubBot[DatabaseContrib] vrurg tagged v1.01 at ba0d5d6: http://git.io/vcrvz [23:19]
***GithubBot has left [23:19]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)