#foswiki 2013-10-29,Tue

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
gac410Hm Perl 5.18.1 and beyond breaks SEARCH by topic modified. No idea why :( [00:08]
pharveyargh. We'll have to look at perldelta
for clues
[00:14]
gac410Wondering if it is the changes to hash ordering, Fn_SEARCH::test_orderTopic fails, and I don't think its an inssue in the testcase
I think this needs to block 1.1.9, :(
Results returned from %SEARCH is inconsistent. order=modified and others,
[00:16]
pharveysome interesting stuff in the perl 5.18 delta
lexical $_ and smartmatch ~~ are being downgraded to experimental
Archive::Extract deprecated
[00:25]
gac410Smartmatch ... that's a good move, I'm sure we don't use it, [00:26]
pharveydo our tests rely on consistent hash ordering? [00:27]
gac410Not this particular test. That's what worries me.
It creates a set of topics with known attributes, and then expands a bunch of %SEARCH macros with specific ordering.
[00:28]
pharveyI see [00:29]
gac410$format just lists the topic names, and the names flip around at times.
I doubt we use lexical $_ either.
Geeze, This is bad. On 5.19.1, reloading WebChanges 10-20 times, Once in a while it fails Could not perform search. Error was: Use of uninitialized value within %Foswiki::Render::ESCAPED in substitution iterator at /var/www/foswiki/trunk/core/lib/Foswiki/Render.pm
ESCAPED is a hash - but it's a constant.
well, not a declared constant, but it's initialized when declared and never modified.
Hm... that last one, only fails on 5.19.5 ... Perl 5.18.1, I can't recreate it.
Back to search order, I don't observe any issues repeatedly issuing the same macro in a Sandbox topic. :(
[00:29]
pharveyso it's different between Foswiki->new()s?
I can't dig at the moment, but my first thought would be to add some logging around the resultset iterator stuff
and/or disable the MetaCache or MetaInfoCache or InfoCache or whatever it's called
[00:44]
gac410no, same Foswiki new(), Test repeatedly calls expandMacros
Similar search over same set of topics.
Is the Meta / InfoCache configurable?
[00:45]
pharveyno, there's an if statement somewhere which decides to check the infocache and I usually just add an if (0 && .... to make it never go down there
sorry, perhaps that's useless advice :}
I might be sending you down an irrelevant rabbit hole
[00:48]
gac410np ... I'm just poking around.
Lots to do for 5.18 compat. Need to change tests that compare html to use html_equals to deal with attribute order in tags.
Mostly it's test bugs, but I hate it when it's unclear.
[00:48]
pharveyoh, gac410. In trunk it seems to be Foswiki::MetaCache
just make the hasCached return false
[00:50]
gac410yeah. I'm on release11 for now though [00:50]
pharveyah
same thing on Release01x01
[00:50]
gac410Okay, So the Search::InfoCache is something different? [00:51]
pharveyI must be remembering some dodgy mongo code :)
MetaCache avoids parsing topics more than once... I think...
InfoCache avoids running the same search more than once? :/
[00:51]
gac410Still fails with meta->hasCache returns zero. so Not the meta cache. [00:53]
pharveyit does seem the bug is more likely in InfoCache than MetaCache [00:53]
gac410Yeah. [00:53]
pharveymight have to add some prints around that resultset ordering stuff [00:56]
gac410Yeah I was trying that, but not seeing anything. Not sure yet wtf I'm doing. [00:59]
pharveyI guess it would be good to clarify if the initial sort is good, but subsequent ones are bad (indicating a cache problem) or not (indicating something broken with the way our store measures time-modified)
sorry for stating the obvious :P
[01:02]
gac410Yeah, Just confirmed that it's the results set from InfoCache that is wrong, so it's deeper
I don't think it's time modified, because sometimes order=modified is right, but order=modified, reverse=on is wrong, against the same topics.
[01:03]
pharveyouch [01:05]
gac410Damn ... #be very careful with this test and the one above - the change in order between QueryTopicTwo,QueryTopic is due to them having the same date, so its sorting by topicname
And that's what I'm seeing. Those two topics flip positions randomly.
[01:10]
pharveyoooh. [01:10]
gac410I think I'm going to open a separate task for this one. I think it's more than just a test artifact. [01:13]
..... (idle for 20mn)
Okay, I don't see how that code actually sorts by (modified : topicname ) I think it's just previously used to get the topics in name order and that no longer happens,.
The raw list of topics comes back in a different order for each %SORT expansion.
[01:33]
........ (idle for 35mn)
pharveyI assume the VarFORMAT tests work ok? [02:09]
gac410Yes, at least they have not failed yet. Though this stuff is all inconsistent.
Anything that has a hash gets different results for each run.
[02:11]
pharveythat's a new... feature :/ [02:13]
gac410I assume that this is a blocker, SEARCH should return the same order, repeatably [02:13]
pharveyperldelta makes this out to be a big change
or at least that's how I've read it
[02:13]
gac410yup. p5p list was pretty active about it a while ago. I dropped off though while I was away. Couldn't keep up. [02:14]
Well VC/Handler::getTopicNames returns a sorted alpha list, But InfoCache loads them alphabetical by height, changing every run. yeesh. [02:21]
***ChanServ sets mode: +o SvenDowideit [02:25]
gac410Anyway, I opened http://foswiki.org/Tasks/Item12618
Well, I'm burned out for tonight. Somewhere we must be passing the topic list via a hash, as I can't explain any other reason the order would change between calls to expandMacros in a single test.
Store returns a sorted array of topic names.
[02:27]
pharveydon't burn out gac410 :) [02:33]
gac410Nah I won't just tired. Long day today. [02:34]
pharveyhope tomorrow is better [02:38]
........ (idle for 37mn)
***ChanServ sets mode: +o SvenDowideit [03:15]
........... (idle for 54mn)
gac410Order of zones has definitely changed. with perl 5.18+ [04:09]
I think this one will be a blocker too.
g'night all
[04:17]
***gac410 has left [04:17]
...................... (idle for 1h45mn)
ChanServ sets mode: +o CDot [06:02]
............................................................................. (idle for 6h20mn)
GuilainCSorry gac410, don't understand your yesterday msg : (23:45:41 - gac410 : GuilainC: where is the viewplainViewTemplate), i was away from my computer; MichaelDaum_ have you take a look about my trouble (yesterday evening on my jquery trouble) ?
I've serveral meeting this afternoon, still here during the next 30 min, and i will get back in 6 hours. any help will be always appreciate
[12:22]
***ChanServ sets mode: +o gac410 [12:35]
..................................... (idle for 3h3mn)
gac410CDot: The failing zone tests with perl 5.18+, As I recall, zone render order is quite critical. In looking at the test fails, it looks like the zones really do output in a different order. [15:38]
MichaelDaumgac410, of course the final result of %RENDERZONE can vary in the boundaries of all given ordering constraints. [15:40]
CDotzone ordering is imposed by MichaelDaum's zone rendering code. So if the order is differnt, that's because there are no interdependencies [15:40]
MichaelDaumthe CSP is underspecified / has multiple solutions
which the system choses from randomly
[15:41]
gac410Okay, so we need to look at each test, decide if a constraint was violated, and either change the test or ...
ug.
[15:42]
MichaelDaumthe only way to force one specific linear order is to add more constraints, e.g. alphabetic order of files [15:42]
gac410I think the Cache tests also fail due to render order, It's more than just tag changes. Blocks of javascript move around. :(
Some are simple. zone test_7 and test_8 both fail. Expected:'text1text2' But got:'text2text1'
[15:43]
MichaelDaumthose shouldnt test the <head> then
the PageCache is botched on 1.1.x anyway
[15:44]
gac410Okay, I'll ignore cache. Could you look at Zone tests test_7 and test_8? They are very simple, and are reversed with perl 5.18,.x [15:46]
MichaelDaumwill do [15:46]
gac410Actually they are random. Running ZoneTests::test_7 it works about 50% of the time.
test_8 is identical, except test_7 uses a \n between the zones.
[15:46]
......... (idle for 41mn)
MichaelDaumtest_7 is wrong as there's no ordering constraint at all [16:28]
gac410Great, that means 8 is wrong too [16:28]
MichaelDaumthey have the same fix: adding a requires="id1" ... checking this in [16:29]
gac410okay, I' [16:29]
MichaelDaumMichaelDaum checking those tests on trunj [16:29]
gac410ZoneTests::test_HEAD_merged_with_SCRIPT and ZoneTests::test_HEAD_split_from_SCRIPT fail too, probably the same issue
And ZoneTests::test_legacy_tag_param_compatibility
I'll check them later today ... have some stuff to do around the house
[16:30]
Thanks for checking for me MichaelDaum
This hash ordering changes in perl 5.18 is painful
Translation status. Danish, Dutch and French have 8 words remaining, The rest are complete.
[16:37]
MichaelDaumPretty cool.
wrt test_HEAD. the misc7 snipped has got no constraints at all. so yes same problem as test_7 and test_8
"you want it to be at a certain position? you have to say so."
[16:42]
...... (idle for 26mn)
gawd who has written these tests [17:12]
gac410Not me ;) [17:12]
.... (idle for 17mn)
Cool. Kenneth just translated Danish, so we are down to Dutch and French ... If you can help, please let us know! Thanks
CDot, regarding %SEARCH, I've dug into the test failures. The problem is the input topic list into InfoCache is not sorted, and any tests that expect *secondary* order by topic name randomly fail.
The comments suggest that for whatever sort key, duplicates will be sorted by topic name, and with perl 5.18, that's not happening any more.,
[17:29]
CDotumph. That's Sven's code, and I've never really got the gist of it [17:31]
gac410Store eachTopic returns a sorted list, I've verified that, but somehow between there, and the InfoCache, the order changes. :( [17:32]
CDotcomments where? [17:32]
gac410In the unit test,.
#be very careful with this test and the one above - the change in order between QueryTopicTwo,QueryTopic is due to them having the same date, so its sorting by topicname
[17:32]
CDotthe infochache is inherently unsorted, isn't it? I thought it was keyed on topic name
ok, that's a Sven comment - you can tell by the lack of a space after the #
[17:32]
gac410:)
SvenDowideit: Are you areound?
Can we drag you back ... without kicking & screaming :D
[17:33]
CDotmiddle of the night for him, innit? [17:33]
gac410No idea. I guess so :( Hopefully he reads the logs. He still keeps a presence in the chat.
The infocache is passed a reference to a topic list array @{ $this->{list} } and the list *should* be sorted, no idea why it isn't. It should be built from store's eachTopic, which has an explicit sort and returns a list iterator.
But print statements shows that store returns sorted, and infocache processes unsorted, and my head explodes
I suspect this is the root cause of the other ordering tests, group names, etc. And I somehow think that this is significant
I opened http://foswiki.org/Tasks/Item12618 to track this issue. I'm still debating if it should block 1.1.9
If we hit 100% on the languages, and no result, I'll probably defer it with a warning in the rel notes that perl 5.18+ can cause changes in search result order.
It was annoying that perl 5.14, or 5.16 broke 1.1.8 hard
[17:34]
......... (idle for 44mn)
CDotvery annoying. But hash ordering sensitivity is something we knew about, and for the most part coded around
writing tests that are insensitive to hash ordering is very hard, so tends to be skipped :-(
FWIW I do not believe it should block.
[18:24]
gac410CDot: okay thanks. I just hate mysteries. Maybe Sven will have idea on how/where the infocache topics list is passing through a hash [18:34]
CDotI think it *is* a hash. But I don't really understand how that affects the ordering.
the inforcache is just a way of avoiding repeatedly going back to the store for each search hit on the same topic
so should not affect the search result ordering, AFAICT
[18:35]
gac410Right, it seems to happen somewhere between the store topic list which *is* sorted, and the infocache input which has lost it's order
the TopicUserMappingAsGuestTests::verify_secretGroupIsHiddenFromGROUPINFO_* tests should be easy to fix, just grep for the "should be hidden" group,
If I print the topic list prior to filtering in InfoCache, it has a consistent order prior to perl 5.18, and changes for each call to InfoCache 5.18+
But the "error" if it is one, only happens when multiple topics in the list have an identical primary sort key
[18:37]
.... (idle for 17mn)
GuilainCgac410: as a french guy, following your last message if i can help to something (translation & co), please feel free to contact me, in order to explain what to do (in private chan ?) it will be a pleasure to help. [18:57]
..... (idle for 24mn)
gac410GuilainC: It's pretty easy, no need for private channel. If you register an account on http://translate.foswiki.org/ I can then give you permission to check changes into pootle.
Note that the correct project is http://translate.foswiki.org/projects/foswiki/ (There is also an "Everything Foswiki" which is defunct.
There are only 8 strings needing translation. Our regular translator for french is away for a while.
[19:21]
GuilainCprocessing....
foswiki.org login is different of translate.foswiki.org ?
[19:24]
gac410Once you are logged in, and have permission, the "8 words remaining" link will take you to the first translation, and there will be an "accept" button to apply any suggestion.
Yes, completely different system :(
[19:25]
GuilainCduly noted.. :( /me registering
my login same as foswiki.org : GuilainCabannes
[19:26]
gac410er... It's a "Submit" button, not apply. As I'm mono-lingual I don't do much translation Okay hang on a moment.
You should now be authorized for French. You *might* need to log out and back in, I'm not really sure.,
[19:28]
GuilainCoki, i'm setting up the system
what's policy on email publication ?
[19:31]
gac410er... I don't understand [19:31]
GuilainCis publish over the web as it ?
oki, sorry for my english :)
[19:31]
gac410If you are asking about foswiki.org, we don't share emails. [19:32]
GuilainCis the email using for sign translation is spread over the web (aka inserted inside file web accessible), i'm speaking translate.foswiki.org
in the profil panel : firstname, surname, e-mail
[19:32]
gac410Oh... hm... Ah... Let me check [19:33]
GuilainCit's find [19:34]
gac410If it is, it's not intentional. I don't see the email of users when not logged in. [19:34]
GuilainCit's just for saving my privacy.... and in theses cases, do someting like myemail_AT_domain.come
oups it's fine, i've acces, try to do my job.
[19:34]
gac410The only place I see my foswiki email (I use unique email per web project just for that reason :) ) is when I send to the foswiki mailing lists. [19:35]
GuilainCGuilainC trying to understand pootle... (how that's work !?!) [19:36]
gac410I think, if you follow that "8 words need translation" link, and then "submit" your first change, it will auto step through the remaining 8 [19:37]
GuilainCit's what i think too, so the word to translate is from the #55 up to #63 ? but what is there already a translation? [19:38]
gac410Ah, it might be a suggestion, that just needs to be accepted, or it might have been a minor change that is still valid [19:39]
GuilainCok... if i made something wrong, everything could be change :) let's go ! [19:39]
processing... for information it give me the words to check one after one only in the list of the 8 words to do... but what i see is the words around the selected words to check/modify but not have to modify the others around (sorry not clear)
gac410: i suppose i've to submit my file ?
[19:45]
gac410I'm probably the worst person to ask how pootle works :( Anyway looks like you are at 100%
No... no need to submit
That gets done automatically ever 2 hours. The server cron task looks for new translations and commits them into svn.
All auto-magic :)
[19:47]
GuilainCgreat :)
i've understand, i can upload the core.po file if i've modified it on my computer and not inline
[19:48]
gac410Thank you very much for helping on this. One more language to do and we'll be ready to release [19:49]
GuilainCand do you know what's the others projects (terminology & so on ?)
so no knowledge in Deutsh...
[19:49]
gac410No idea about uploading manually, I run a shell tool - xgettext which merges strings from the foswiki source into core.po, and the individual language,.po files. [19:50]
GuilainCoki geeks method ;)
:D
[19:50]
gac410I commit to SVN, and cron merges from SVN into pootle, and changes show up as "pending". Once updated, cron pulls them back. [19:50]
GuilainCgac410: are you IT admin ? [19:51]
gac410The end result is nobody actually touches any .po files. All we do is put strings that should be translated into MAKETEXT macros, or calls to locale::maketext in the perl code.
Well, I've worn many hats. Currently retired, I've done some admin, many years of routed network design & sales, and many years of mainframe database stuff before that :)
[19:51]
GuilainCstill to do something for the whole topics in system, having help in french for example should be appreciate by some of my co-worker
oki i see...
who manage the pootle plateform ?
[19:53]
gac410Yes, That's a big task. I think there is a contrib somewhere that can load alternate topics by language. [19:54]
GuilainC(for example i see no french language for NatSkin ? MichaelDaum_ ? [19:55]
gac410The guy who manages our server has been away. I do some little stuff Ah... MichaelDaum are you listening :) [19:55]
GuilainCyes there is a plugin like that... i think to use if for making my personnal foswiki website with multi language [19:55]
gac410Traditionally Foswiki "extensions" have never been translated Micha started to add translation for NatSkin. Not sure how far he went, or if the server backend - cron jobs, etc. support it. [19:56]
GuilainCah, bad for the admin of server, same, if i can help for small task.. if I know the OS...
is see, in french we say "everthing has to be cabling"... not sure to be understanding in other language
[19:56]
gac410I think I understood. :) [19:57]
GuilainCok sorry, i'm debian user... i've done bsd just one time, when i was young... (teenager to be precise) [19:59]
gac410I'm in the same boat. Mainly Gentoo and occasional debian. [19:59]
GuilainCand perl & cpan... i need help :) so i can't help for the server.. [19:59]
gac410I have to apologize. The "commit" of translations into SVN identified the translation author by email address. [20:00]
GuilainChave you think to do a kind of "backup" server with different os, & co, it's increasing cost of maintenance, true but in case of something going wrong on one context of one server the other is often not affected [20:01]
gac410yes indeed. "mono-culture" can be quite dangerous [20:01]
GuilainCah gac410. I've already try to change into my pootle profil but it doesn't accept something different than an email
what's boring with e-mail, is, there is always new robots which detect e-mail, and spam you...
[20:02]
gac410Yeah. I can change email addresses if you want. It does need to be a valid email or password reset and the like won't work.
If you have a gmail or yahoo or other free email account, you could change to that.
I can't edit it out of the SVN log, but I could update the file with a revised email address.
[20:03]
GuilainCit will be a pleasure if you can, but don't worry...you can just add something as NOSPAM in the domain (making error for sender not, the receiver :) !) pootle accept that
as myemail@NOSPAMcabannes.info
normaly human understand, robots not...
[20:07]
gac410Okay will do. [20:08]
GuilainCmany thanks [20:08]
gac410Okay done. [20:10]
GuilainCoki, as yesterday you tell me, you don't play with jquery || ajax ? [20:11]
gac410Well, I do a very little once in a while, mainly to fix obvious bugs. [20:12]
GuilainCso i've to check MichaelDaum_ ... i'm trying to submit form with foswiki save method with ajax...
i've to go, see you !
[20:13]
gac410GuilainC: See you, thanks again. BTW MichaelDaum and MichaelDaum_ are the same person, the _ says he is "away from keyboard" [20:26]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)