#foswiki 2011-09-06,Tue

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***pharvey has joined #foswiki [00:00]
pharveygac410, do you know how to fix trac.foswiki.org ? I guess we need to update a port/package thing, but I'm not very BSD savvy :) [00:09]
gac410Whoops - looks like trac.foswiki.org is borked - Cannot load Python bindings for PostgreSQL
pharvey: No idea - earlier Babar reported issues with the upgrade for pootle as well.
See his notes at 6:31:34. I know nothing about ports or bsd.
er. That is my local time - his notest from about 2 hours ago.
notest / notes
[00:09]
pharveygac410, ah I see. Thanks
I'll use github for diffs till then :)
somehow I prefer the trac diffs
[00:14]
***GithubBot has joined #foswiki [00:16]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/uQdJYw
[foswiki/master] Item9563: Add test for preference variables - GeorgeClark
[00:16]
***GithubBot has left [00:16]
FoswikiBothttp://foswiki.org/Tasks/Item9563 [ Item9563: pushTopicContext does not re-read preferences in afterSaveHandler ] [00:16]
***MartinCleaver has joined #foswiki
MartinCleaver has quit IRC (Changing host)
MartinCleaver has joined #foswiki
[00:24]
GithubBot has joined #foswiki [00:31]
GithubBot[foswiki] foswiki pushed 1 new commit to Release01x01: http://git.io/_uZP-Q
[foswiki/Release01x01] Item9563: Add test for preference variables - GeorgeClark
[00:31]
***GithubBot has left [00:31]
......... (idle for 41mn)
MartinCleaver has quit IRC (Quit: MartinCleaver) [01:12]
..... (idle for 24mn)
foswiki_irc8 has joined #foswiki
foswiki_irc8 has quit IRC (Client Quit)
[01:36]
GithubBot has joined #foswiki [01:46]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/w89lGw
[foswiki/master] Item10188: more File::Spec - PaulHarvey
[01:46]
***GithubBot has left [01:46]
FoswikiBothttp://foswiki.org/Tasks/Item10188 [ Item10188: Make pseudo-install aware of git ] [01:46]
.... (idle for 15mn)
***MartinCleaver has joined #foswiki [02:01]
MartinCleaver has quit IRC (Quit: MartinCleaver) [02:07]
........................ (idle for 1h55mn)
GithubBot has joined #foswiki [04:02]
GithubBot[foswiki] foswiki pushed 2 new commits to master: http://git.io/Kol8iQ
[foswiki/master] Item10188: .gitignore core's installed/symlinked files too - PaulHarvey
[foswiki/master] Item10188: Remove debug statement - PaulHarvey
[04:02]
***GithubBot has left [04:02]
FoswikiBothttp://foswiki.org/Tasks/Item10188 [ Item10188: Make pseudo-install aware of git ] [04:02]
........... (idle for 50mn)
***gac410 has left [04:52]
.... (idle for 18mn)
Bamieater_ has joined #foswiki
Bamieater has quit IRC (Ping timeout: 240 seconds)
[05:10]
..... (idle for 20mn)
GithubBot has joined #foswiki [05:31]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/MEt9TA
[foswiki/master] Item10188: Fix removing rules from .gitignore when uninstalling - PaulHarvey
[05:31]
***GithubBot has left [05:31]
FoswikiBothttp://foswiki.org/Tasks/Item10188 [ Item10188: Make pseudo-install aware of git ] [05:31]
pharveyBabar: I pseudo-install now properly maintains .gitignore for each extension, and also keeps core/.gitignore up-to-date with installed and uninstalled extensions :)
so now git submodule status accurately shows me which extensions have been modified
and git status ... no longer shows pages and pages of untracked files
SvenDowideit: at some point, let me know if the new pseudo-install works on a 'everything' git checkout
wrt .gitignore
SvenDowideit: also updated pseudo-install's doc, with a link to a Development topic with initial content (mostly bullet point notes). So I hope this somewhat addresses your concerns about repo-per-extension
[05:43]
.... (idle for 18mn)
***terceiro has quit IRC (Quit: Saindo) [06:03]
.... (idle for 16mn)
CDot has joined #foswiki [06:19]
CDotCDot notes the (tm)wiki BackupRestorePlugin and wonders in what way "backup and restore" functionality eases the upgrade process. I guess it has some benefit if you don't have admin access to the server; but then, if you don't have admin access, how can you execute on the upgrade? [06:25]
pharveySomebody should really blog about the plugin backup/restore george did with the installer. [06:26]
***MichaelDaum has joined #foswiki
ChanServ sets mode: +o MichaelDaum
[06:26]
MichaelDaummorning guys. anybody admin-ing http://trac.foswiki.org/ ? [06:28]
pharveyMichaelDaum: Babar said something in the logs, about updating python 2.5 -> 2.7 later today (?) [06:28]
CDotpharvey: aye, we need to blog more, for sure. All of us. [06:29]
MichaelDaumWikiRingBot, seen Arthur [06:29]
WikiRingBotWikiRingBot has last seen ArthurClemen 31 Aug 2011 - 11:11 GMT on #foswiki
WikiRingBot has last seen ArthurClemens 5 Sep 2011 - 21:40 GMT on #foswiki
[06:29]
***pharvey has quit IRC (Quit: ChatZilla 0.9.87 [Firefox 3.6.21/20110830204742])
GithubBot has joined #foswiki
[06:29]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/19a6Zg
[foswiki/master] Item10188: Improve handling of comments in .gitignore - PaulHarvey
[06:30]
***GithubBot has left [06:30]
FoswikiBothttp://foswiki.org/Tasks/Item10188 [ Item10188: Make pseudo-install aware of git ] [06:30]
***denisr has joined #foswiki [06:34]
MichaelDaumto all: don't add YAJP (yet another jquery plugin) to JQueryPlugin. instead: make it a contrib of its own. [06:39]
.... (idle for 18mn)
***ModAcOst has joined #foswiki [06:57]
BabarMichaelDaum: yes, I was trying to upgrade f.o, and python is making troubles, so trac might be down [07:05]
MichaelDaumBabar, k. just wanted to make sure the problem is known. [07:06]
Babaryes, sure. I fought many hours last night... but some modules want python 2.5, and one mandatory for trac requires python >= 2.6 [07:07]
ModAcOstGood morning everyone. Is there anything build into foswiki I can use to extract meta data from the topic text? I want to use beforeSaveHandler to check if some meta changed. [07:10]
Babarpharvey: thanks btw, all I wanted to write was to track generated files and remove them when uninstalling. It seems you did that for me :) [07:12]
***GithubBot has joined #foswiki [07:17]
GithubBot[foswiki] foswiki pushed 1 new commit to Release01x01: http://git.io/6PNxRw
[foswiki/Release01x01] Item11111: reverting - ArthurClemens
[07:17]
***GithubBot has left [07:17]
FoswikiBothttp://foswiki.org/Tasks/Item11111 [ Item11111: Add facebox plugin ] [07:17]
***mark_doe has quit IRC (Ping timeout: 240 seconds) [07:18]
Colas has joined #foswiki [07:23]
.... (idle for 17mn)
verboEse is now known as VerboEse|Off [07:40]
MichaelDaumModAcOst, yes, you will need to implement both beforeSave and afterSaveHandler. see ClassificationPlugin which does exactly that to find out if it needs to recompute metrics of a taxonomy. [07:45]
ModAcOstWill do, thanks for the tip [07:45]
***pharvey has joined #foswiki [07:47]
pharveyBabar: you mean for a purpose other than .gitignore? (tracking generated files) [07:48]
Babaryes, to clean up upon deinstallation
as they were generated by pseudo-install they should be nuked by it too
[07:49]
pharveydoesn't pseudo-install already do that?
oh, it only unlinks from core
you mean from the extension too?
[07:49]
Babarit unlinks generated files?
since when?
you coded that? :)
[07:56]
***VerboEse|Off is now known as verboEse [08:02]
pharveyBabar: what else would uninstall do? :P [08:07]
***MartinRowe has joined #foswiki [08:07]
pharveyIt leaves the generated files in each extension directory
it just unlinks them from core
[08:08]
Babarwhy?
ah right
because it removes the link to the extension too soon
therefore, it cannot uninstall the files, as they're already gone
fyi, I'm upgrading to python 2.7, at least trying to. So pootle and trac will be broken until I'm done
[08:13]
pharveycool
Babar: if we want the generated files removed from the extension dir as well as core, I that should be possible
[08:19]
***Alexander__ has joined #foswiki [08:22]
pharveyit's just, I use git clean for that, so never really bothered :) [08:22]
Babaryeah, that's what I wanted to code, but as you coded most of it already...
yeah, me too. But you have to gitignore the LocalSite.cfg, otherwise you get surprises :)
[08:22]
***Babar sets mode: +oooo CDot Colas pharvey SvenDowideit [08:22]
pharveylib/LocalSite.cfg is actually a symlink for me :)
but yes, I think gitignore is important.
[08:23]
Alexander__Hi there! I have a question regarding the WysiwygPlugin: Sometimes we have a problem if we save a topic using the internet explorer. It seems that the topic doesn´t get parsed into wikitext. Is there any known issue? It´s absolutely strange because it seems to apear randomly :( [08:30]
It seems that the secret_id get lost. but why does the id get lost only sometimes and not allways?!? [08:35]
..... (idle for 20mn)
***ArthurClemens has joined #foswiki [08:55]
SvenDowideityippeee, osx lion is shite, but at least virtualbox works, and has rdp cooked in
now to test kinosearch and see if it gives better results (better as measured by the user's opinion) cos solr is giving more false results for them than correct ones, and they're pissy at the lack of docco
not that i'm thrilled about the docco for kino, but in the end, its about 'them' getting the search resluts they expect
[09:00]
MichaelDaumSvenDowideit, who is them? [09:04]
SvenDowideithas anyone here worked out how to make osx lion useful (other than formatting the disk and installing a real os? [09:04]
MichaelDaumwhich false results are these? [09:04]
SvenDowideiti'm not allowed to say their name - as they paid me to sign an nda [09:04]
MichaelDaumk. if they need solr help let me know. [09:04]
SvenDowideitin their case, they enter a domain specific keyword, and get 60% results based on other spellings
and somewhere near the end, they get the exact matches
[09:04]
MichaelDaumdid you switch of phonetic matches from the dismax handler? [09:05]
SvenDowideitso they expect 2 results, and instead they get 120 wrong ones
wheres the docco that tells me how / where?
aka - no
[09:05]
MichaelDaumthe dismax handler is best docu'ed by the solr project itself [09:05]
SvenDowideitpeople in my part of the world assume that if its not default or well docced, that its not ready for use [09:06]
***CDot has quit IRC (Quit: Leaving.) [09:06]
SvenDowideitthe dismax handler is - but should i be customising what is indexed, or how foswiki quieries?
SvenDowideit isn't an expert in solr, and probly won't become one
[09:06]
pharveyAlexander__: which version of IE? Can you please raise a task? Any JS errors? What version of TinyMCEPlugin are you using? Could you try the testing build of TinyMCEPlugin at http://foswiki.org/Extensions/Testing/TinyMCEPlugin ? [09:07]
MichaelDaumquick fix for you: find solrindex.xml ... look up requestHandler name="dismax". it has got a bunch of fields it looks up to boost results, i.e. a phonetic one. try disabling it. then restart solr. [09:08]
SvenDowideitthe first q i was asked is 'what do i type into the search box to tell it to search for literal results
ok, so we have no control of 'how it searches' from the browser?
[09:08]
MichaelDaumsearch box as it is on System.SolrSearch uses dismax. its behavior is configurable using solrindex.xml ... which in itself comes with lots of comments to help you on your question [09:09]
SvenDowideitits the user that wants to control the behaiviour at the time they write the search [09:10]
MichaelDaumnext tip: try using solr search on the cmdline: cd bin; ./rest /SolrPlugin/search q="whatever" [09:10]
Alexander__pharvey: it seems to be the ie7, 8, 9. We don´t use the TinyMCE... [09:10]
MichaelDaumnext tip try %SOLRSEACH{"query" type="standard/dismax" ...}% [09:10]
pharveyYou're using CKEditor? Which version WysiwygPlugin? [09:10]
SvenDowideitaha, thats a useful tip [09:10]
Alexander__perhaps i found the reason. I don´t use the saveCallback like in tinymceplugin [09:11]
SvenDowideitso i don't need to kill the dismax handler, i cna define a custom named handler that ends up using a different setting set? [09:11]
Alexander__so the secret_id could get lost during the save action
but why should the ie delete the save_id ?
yes we are using CKEditor.
[09:11]
pharveyI'm not sure. Besides improving UTF8, I'm desperately trying to find time to improve WYSIWYG (including making CKEditor work without hacks).
But I have never noticed it failing for TinyMCE
[09:13]
SvenDowideitpharvey: how about - i'll disable mongodb until you fix utf8 :p [09:14]
MichaelDaumSvenDowideit, definitely a possibility [09:14]
pharveySvenDowideit: you'll disable mongo? on our site? Our 500 timeout errors aren't very informative :P [09:14]
Alexander__hmm, i will try to add the savecallback to the CKEditor [09:15]
MichaelDaumthose field boosts coming with SolrPlugin by default are just an educated guess, rather than well chosen. [09:15]
SvenDowideitMichaelDaum: excellent - that means 'they' can tweak a little over time
thats the problem - educated > my knowledge
[09:15]
MichaelDaumhey it is "just" xml >:] [09:15]
SvenDowideityes, but the values / weightings are not xml they're heuristics [09:16]
MichaelDaumy [09:16]
SvenDowideitie, format has buggeral to do with the important thing
which is to return what the user expects
[09:16]
pharveywe should investigate to see what drupal's settings are. They seem quite successful [09:16]
MichaelDaumsolr is highly configurable how it behaves ... too much tweakery sometimes...so with power comes responsibility ... (not to screw it up too much) [09:16]
pharvey(for solr) [09:17]
MichaelDaumpharvey, I already looked at it
and went the other way: snarfing in the best they have
[09:17]
pharveykool [09:17]
SvenDowideitwell, i've had no positive feedback from my solr deployments yet, always (ok, so i've only got 3 corps using it in production) complaints about way more noise than good results :/ [09:17]
MichaelDaumand they do some quite wrong actually ...
SvenDowideit, let me know how I can help.
[09:17]
***McAldo has joined #foswiki [09:18]
pharveyunlike Foswiki, which does everything perfectly :)
pharvey ducks
[09:18]
SvenDowideitMichaelDaum: docco [09:18]
McAldohello [09:18]
MichaelDaumSvenDowideit, a bit of money to put into my mouth, cus I've got children as well. [09:18]
SvenDowideittechy money spenders expect docco before they put money down [09:19]
MichaelDaumpharvey, the drupals got lots of crap in their repos ... even more than foswiki ... which is also due to their overall size being vast [09:19]
SvenDowideitok, me goes back to the install fest to see what happens 'when' [09:19]
MichaelDaumas a sideeffect they dont care as much about solr in drupal which could improve as well
drupaleans tend to celebrate themselves more than fossies
so you get more whining from foswiki about good code ... whereas drupal even praising some quite bad code
[09:20]
pharveytrue. We don't have enough zealots :) [09:21]
MichaelDaumSvenDowideit, about kino vs solr: not a fair comparison actualy given you'd like ward off http://www.cmswire.com/cms/information-management/10-features-that-will-make-your-sharepoint-search-shine-012528.php
... which frankly is YAPSA (yet another pointless sharepoint article)
[09:24]
pharvey"Ability to sort" -> LOL [09:25]
MichaelDaumwhat is _not_ pointless is the users needs on the intranet that sharepoint tries to address [09:25]
SvenDowideityup
technically, sharepoint could actually kill foswiki
if someone would write a frontend wrapper that exposed the foswiki functionality that it has
[09:25]
MichaelDaumcoming from solr, this YAPSA is hot marketing steam: write about sharepoint tips&tricks no matter what [09:26]
SvenDowideitbut righ now, i'd rather poke my eyes out [09:27]
pharveyisn't re-re-writing store & meta worse than poking your eyes out? :-) [09:27]
SvenDowideithowever, store2.0 is going to give us '1. Content Findability Over Multiple Sources' [09:27]
MichaelDaumsimilar to google+ tips&tricks / road to success / how to get the most out of it [09:27]
SvenDowideitthe search code i was working on was to expost '2. Search Content On Other Sites (Federation)'
ArthurClemens: already added '3. Highlighted Search Results' once
SvenDowideit fell asleep scrolling down
[09:28]
pharveysee 8. "Ability to sort" - hah! [09:28]
MichaelDaumpharvey, funny ey? me reading "Not many commercial search engines offer the ability to sort" ... like WHOT? [09:29]
pharveyeven the most stupid spreadsheet software can sort [09:32]
MichaelDaumnother lesson learnt: how to kick off a hype. today: the microsoft way. [09:32]
pharveypharvey wonders if the original visicalc didn't have sort [09:33]
SvenDowideitSvenDowideit wonders if anyone else thinks it'll be interesting to have a XLS_Store
where one web in your wiki is a read write spreadsheet
[09:33]
MichaelDaumand btw technically foswiki could actually kill most other wiki engines. no joke.
that is foswiki as is NOW
it is only us not getting out of the techie pudding
[09:36]
SvenDowideity, agreed, but we're still in stealth mode [09:37]
MichaelDaumdid you see twiki dropped out of the gartner magic quadrant for social software: http://files1.jivesoftware.com/images/resources/analyst-coverage/gartner-mq-ssitw-2011-22748.png [09:38]
pharveya profoundly interesting point over other alternatives is the separation of concerns between search, query, store. [09:38]
SvenDowideitnope, i'll go look for a giggle :)
SvenDowideit challenges micha to become the marketer that he should/wants to b
[09:38]
pharveyI really look forward to making SolrPlugin a kind of peer with MongoDBPlugin, assuming we can clean up search formatter [09:39]
MichaelDaumSvenDowideit, I've been working on a marketing campaign that I'd like to discuss on the GA/FoswikiCamp [09:40]
SvenDowideitSvenDowideit is still trying to find the key selling point' for solr - the faceted thing is UI, not index/query
and as you point out, we separate search, store and ui
atm, i (admittedly ignorantly) think we can add a faceted UI to any index / query
and probably should do so specifically for SEARCH
[09:40]
MichaelDaumkey selling point for solr: faceted search meets structured wikis [09:42]
SvenDowideitso if i make a faceted UI for SEARCH (type=query) then solr is irrelevant? [09:42]
MichaelDaumor put the other way around: why use a plain keyword search on data tha companies put lots of effort to attach metadata to [09:42]
SvenDowideitat this point, from a user pov, foswiki solr search appears to just be a pointy-clicky way to creating AND queryies
and its not currently particularly meta aware - i don't think is can solr search for topics that have a particular form field with a particular value that (nest a few more ands)
[09:43]
MichaelDaumthink of solr as a dedicated text processing server ... computing facets is by far non-trivial [09:44]
SvenDowideitatm, i think of facets as anding
which i presume is incorrect
[09:45]
MichaelDaumadding filters is just the query part. facets are in the result set, not the query. [09:45]
SvenDowideitand so i guess you need to work out howto explain it better from an end user elevator pov
ie, http://wiki.apache.org/solr/SolrFacetingOverview implies its just a clicky AND
but as the categories (from what i cna work out) are determined by the admin, not the end user / content generator)
its not as useful as an in-wiki buldable set of selectable categories to then drill down?
[09:45]
pharveyIn one way faceted search is a bit like a nested search. Solr can present results that aren't just matches, but grouped & metric'd, so you can write a UI that allows a user to efficiently drill down or explore. [09:48]
SvenDowideitSvenDowideit needs a better ceo level summary (i think) [09:48]
pharveyso, we don't necessarily need solr to do that, but it's solr's bread & butter, and it does it in a very scalable way [09:49]
SvenDowideithow does the expert influence the grouping that is presented to the less expert [09:49]
MichaelDaumpharvey, right. facets are used to drill down or shift between alternative hit sets in a well informed way thus preventing dead alleys in your exploration [09:49]
SvenDowideitgiven that the expert does not have non-wiki access to the server? [09:49]
pharvey... and then you get bogged down into how many ways you want to index the data :)
SvenDowideit: well, what do you expect from a project designed by a bunch of java-heads :P seriously though, solr is by far the best choice for natural language search (assuming we can tweak the config well enough)
[09:49]
MichaelDaumfaceted search lowers the cognitive burdon to come up with a good combination of keywords drilling down manually. facets rather trigger your basic hunting instincts by giving you hints.
hints about what is in the hit set
[09:51]
SvenDowideitexcept that the query language is not there atm
either we need a foswiki demo site that shoes it off well
ie, index the foswiki.org content and show people how fantastic is is for finding the task / feature / docco they want
or, some other foswiki app
[09:51]
pharveyit's a lot more expressive than kino... [09:53]
SvenDowideity, thats one reason i stopped looking at kino
the requests i get are 'why the f**k does it not work like google'
as in, when the user starts to search, they expect to be able to type in the search
and then refine :/
[09:53]
pharveyI'm using a bunch of services at work that seem to use solr in a manner consistent with google :/ although phrase matching isn't quite the same [09:54]
SvenDowideity? excellent - that gives hope that we can fix it to match user expectation [09:55]
pharveymind you - their data is less ad-hoc than hours. They can afford to fine-tune their config on these one-of-a-kind installations.
than ours*
[09:55]
SvenDowideitso the index isn't wiki compatible?
its more a cms like index that needs a tuneing god?
[09:56]
pharveyhttp://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
"Most Solr users define custom Analyzers for their text field types consisting of zero or more Char Filter Factories, one Tokenizer Factory, and zero or more Token Filter Factories; but it is also possible to configure a field type to use a concrete Analyzer implementation "
[09:57]
SvenDowideitSvenDowideit wonders how much the form field hints would help us generate better solr index hints [09:58]
pharveylots - for example, no need to fuzzy search something that can only have 8 possible values, or is only ever a number
but then again, I don't really know what I'm talking about :)
[09:58]
Babar===> py27-PyLucene-3.0.3_1 is marked as broken: does not build.
just my luck... can't install pootle!
[09:59]
pharveyso... time to make trac a redirect to github? :) [10:00]
Babarok, I'll remove lucene from pootle then [10:00]
MichaelDaumsome more on this years magic quadrant on social software http://www.gartner.com/technology/streamReprints.do?id=1-173SL7W&ct=110826&st=sb [10:00]
Babarnah, pootle, not trac :)
ok, translate is back, without lucene
trac should follow shortly
[10:00]
pharveyMichaelDaum, interesting to read about X-Wiki on there. Had a long discussion with somebody trying out Foswiki, they're now using X-Wiki (does that make me an anti-sales-person? :-)
I actually pointed to X-Wiki because they're java types, and they're working on annotating/marking up documents (so their annotating thing would fit quite well).
[10:05]
MichaelDaumxwiki deserves lots of respect. same mindtouch: they are new in the survey. congrats. [10:09]
Babarthey're java folks and they didn't take confluence? :) [10:10]
SvenDowideitmmm, ok, another attempt to see if i grok the use case for solr [10:12]
MichaelDauminteresting gartner puts jive at the top, whereas I only once had to challange foswiki against it
it was more clearspace vs twiki those days
[10:12]
SvenDowideitMichaelDaum: the faceted search thing is oriented towards people that don't quite know what their searching for
rather than the users that were involved in the wiki content creation and are used to using search to quickly find the doc that they know is there?
[10:12]
Babarok, trac is back with python 2.7 [10:13]
SvenDowideitont of the corps uses wiki for their support KB, and their support engineers used to use search to quickly find one of several thousand docs they wrote [10:13]
MichaelDaumSvenDowideit, cant subscribe to such statement [10:13]
SvenDowideitand for them solr is giving every topic other than the one they want [10:14]
Babarpharvey: the folks at my former job ditched my twiki after I left and put a confluence. But they're using jira too, so it might make some sense [10:14]
MichaelDaumfindability is findability: the more precise the better [10:14]
SvenDowideitusing SEARCH, the topic they want is in the to5
with solr, its on page 30-something
[10:15]
pharveyyeah, there are benefits in certain situations. Certainly, they don't think about "wiki-apps"... want a task tracker? write a new plugin :) Want to re-use that tracker's data somehow? Pain... [10:15]
SvenDowideitor something like that [10:15]
MichaelDaumSvenDowideit, I really cant comment on that specifc case as I dont see the data nor the query nor your solr config. [10:15]
SvenDowideitSvenDowideit has to get a test server working here in zuri :(
solr config is as released atm
[10:15]
MichaelDaumwhat exactly are your searching for? what are you getting? did you disable phonetics? [10:16]
SvenDowideitthe query is for a word like ATM (er, actually, it might have been ATM!)
not yet - i'm still setting up a server to try things out on
leaving the country kinda made it all difficult :)
SvenDowideit has to find the data too :/
[10:16]
MichaelDaumSvenDowideit, try the query admin&nalyser of solr to see how your query is tokenized. [10:17]
SvenDowideitmildly facetious answer: 'what wiki topic do i use to get to that' :) [10:18]
MichaelDaumsee the solr docu for the exact url on your sever. mine is http://localhost:8983/solr/localhost/admin/analysis.jsp [10:18]
SvenDowideitmine, its innaccessible, due to 2 vpn's and security [10:18]
MichaelDaumtry ssh+tunnel [10:18]
SvenDowideitlol [10:18]
MichaelDaumforwarding some port [10:18]
SvenDowideitssh? thats firewalled
which is why i'm setting up a test server here
[10:18]
MichaelDaumanyways.
this analysis.jsp page will give you hints how a specific handler + field is processed: from the query perspective as well as for data stored in some document
[10:19]
SvenDowideitthe admin web ui is horrid for lynx/w3m
so more setup
[10:19]
pharveyMichaelDaum: got a public example - https://wiki.trin.org.au/System/SolrSearch?sort=score+desc&display=list&rows=20&origtopic=System.WebHome&search=LITTORINIDAE
the first few hits do not contain that word
(LITTORINIDAE)
but I have default phoenetic stuff turned on
[10:20]
MichaelDaumpharvey, it lists them because the phonetics got too much boost in your hit set. so fuzziness. [10:22]
SvenDowideiteven when its off? [10:22]
pharveyindeed. But you'd probably expect the topics containing that exact spelling should rank higher [10:22]
MichaelDaumI totally agree [10:23]
pharveyso I think there's some (hopefully) simple tuning left for SolrPlugin to make that happen [10:24]
MichaelDaumby default phonetics participate in the search result with a factor of 0.4 ... see conf/solrindex.xml
make it a 0.04 and restart solr
no reindexing needed
[10:24]
pharveyA person who knows they want littorinidae will see more accurate hits at https://wiki.trin.org.au/System/WebSearch?tab=search&search=LITTORINIDAE&scope=text&web=all
ok, gimme a sec
[10:24]
MichaelDaum... after a year wainting for perl to grep for it [10:25]
pharveynah, that's a mongo search [10:26]
SvenDowideit:p [10:26]
pharveybruteforce still has to check ACLs
~6 mins on bruteforce IIRC
so the fact it doesn't give you a 500, is a big milestone for me :)
[10:26]
SvenDowideitSvenDowideit recons it'll be really simple to add text indexing to mongodb [10:26]
pharveybut solr is the right tool for the job here, we just have to tweak it [10:26]
SvenDowideitbut i'd rather use a real indexer
i guess itsmostly the solr search hadnler that annoys me
[10:27]
MichaelDaumbesides: the search interface in this wiki.trin.org.au install is quite broken
I can see why you guys run nuts on this install
this is *really* useless
[10:30]
***pharvey has quit IRC (Ping timeout: 276 seconds)
pharvey has joined #foswiki
[10:31]
SvenDowideitwe're pretty reliant on whats released - as there's no docco to take a new solr user further [10:32]
Babarplease people, let me know if you see anything wrong with foswiki.org. Thanks. [10:32]
MichaelDaumSvenDowideit, this solr deployment is all broken.
there _are_ lots of pending fixes ... but this install shows errors I never have seen before
[10:32]
SvenDowideitthat result set looks similar to what the other users i've seen are getting
tho the others at least still get the faceted ui as they use patternsknin
[10:33]
pharveyMichaelDaum: we had some students re-design our site skin, and ... it was never quite completed :/ Secondly, this SolrPlugin is quite hacked, and I'm not sure why [10:34]
SvenDowideitbut they do want their websearch topic back :/ [10:34]
MichaelDaumpharvey, I see :/ [10:34]
pharveyoh, I remember: the facets have been hidden behind a modal dialogue (!) [10:35]
MichaelDaumok guys. I promis to check in solr fixes as soon as I can. pharvey, let me know if I can help you out on your trin.org.au wiki. [10:37]
pharveyMichaelDaum, I'll try get Temi to share his notes on the SolrPlugin macro(s) - I'm not sure he got very far, perhaps just gathering the parameters they understand
and I will try to find out why we needed to hack SolrPlugin
pharvey checks to see if we already pushed it to github
nope
[10:40]
MichaelDaummost probably these hacks can be ditched
better start with a fresh solrplugin
[10:41]
SvenDowideitoh, MichaelDaum welcome to the discuss mailing list :D [10:42]
MichaelDaumMichaelDaum didnt find a way to change his email addr on the ml [10:42]
SvenDowideitya, its dumb like that [10:43]
MichaelDaumthere _is_ one but oh well found the link only after subscribing yet again [10:43]
SvenDowideityay, looks like we'll be in stuttgart next w/e
lol
and after tha pam and the girls will actually be allowed to stay here
[10:43]
MichaelDaumhttp://www.web-devcon.de/ is in hamburg 17+18 Oct [10:44]
SvenDowideiti think i'm in Lenzerheide [10:44]
pharveyoh wait: https://github.com/csirac2/SolrPlugin/commits/temi-20110510 [10:44]
SvenDowideitonna mtb - hoipefully not stacking it
oh, btw, we need to shift off github
linus has decided to DOS it :}
[10:44]
pharveySvenDowideit, whaddya mean? And isn't gitorious slightly less awesome than github?
pharvey doesn't like any of the DIY git web-frontends that were around ~ 12 months ago :(
[10:45]
MichaelDaumpharvey, which branch are you using? [10:46]
pharveywe seem to be running the temi-20110510 branch [10:47]
SvenDowideitpharvey: y, they all suck more than github [10:47]
MichaelDaumMichaelDaum downloads all of the bunch to see whats in it [10:47]
SvenDowideitfoswiki 2.0 plus gitsourcecodestore (tm) will er, probly never exist
even though i did the first 20% just to show
[10:47]
pharveygitsourcecodestore? [10:48]
SvenDowideit:)
to differentiate it from git_topic_store
git_source_store should give a more trac like result - as the 'topics' are not tml
SvenDowideit wonders how to actualise that in the brace new 2.0 idea
[10:48]
pharveyMichaelDaum: I really can't follow the commits properly to see what they're doing. I think I'll have to ask Temi to spend some time sorting this out (working to a vanilla SolrPlugin). [10:50]
SvenDowideitSvenDowideit loves that he can apt-get install foswiki-helper-solrplugin
and wait for 400MB to download and has a working foswiki that has solr working right after saving from configure
[10:52]
MichaelDaumuh there are lots of not-so-good "ideas" in that solr repo
like this one https://github.com/csirac2/SolrPlugin/commit/e2ac7d655cf1cb28b062f078a98fd01edd3dc795
[10:53]
pharveyinteresting :) [10:54]
MichaelDaumwow whats going on here https://github.com/csirac2/SolrPlugin/commit/5599bc18361b6e6685afc20067a95c810bae1a93 [10:55]
pharveythat's the main commit, which I need to understand [10:55]
MichaelDaumcan't we coordinate & discuss the needs for things like that on foswiki.org/Task ? [10:56]
pharveyYes, of course. It is my fault Temi went down this path. He did this work when I was too busy to help him solve some problems, and when I finally got time, he was too busy to help me untangle what he'd done.
I should have some time next week. I suspect that we can remove all these changes, and just customise via templates (I hope)
I seem to recall he had some difficulty setting dismax query via an ajax search-as-you type, or something
[10:58]
SvenDowideithow i wish we had 1. a commandline configure setter and 2. dependencies for enabling plugins [11:01]
MichaelDaumpharvey, the bulk addition is two new rest handlers restSOLRFORMAT and restSOLRSMARTTERMS [11:02]
pharveythose aren't yours? [11:02]
MichaelDaumcan't see what changed in the Indexer: https://github.com/csirac2/SolrPlugin/commit/a9bca64937e4beb8ab74188c0b288aef1c50658b#lib/Foswiki/Plugins/SolrPlugin/Index.pm [11:02]
pharveyoh, that's a trivial change... amplified by perltidy
we have a task open already
avoids error when form def topic is missing or contains no form def
Item10720
[11:03]
FoswikiBothttp://foswiki.org/Tasks/Item10720 [ Item10720: Solr indexing sometimes fails on topics with bad form definitions ] [11:03]
MichaelDaumis there any net value that I need to take care of besides perltidy
ah kool
[11:03]
SvenDowideitgiggle - solr search for 'tcp' in the system web returns 2 results
VarWTIKIWEB (which does not contain it?)
and RelNotes01x00 wich might
[11:04]
MichaelDaummerged Item10720 [11:05]
SvenDowideitchanging the phonetic setting to 0.04 does not change the order - VarTWIKIWEB is still first :/ [11:07]
MichaelDaummust be a pth tweak [11:08]
SvenDowideitactually, i'm wondering if 'sort by relevance' isn't happening
and instead, its sorting by date?
[11:09]
MichaelDaumwhats this https://github.com/csirac2/SolrPlugin/commit/122de50d34406cf1ab4082e080859009060645d0#L4L41 [11:09]
SvenDowideitSvenDowideit finds that kino does the right thing out of the box and marvels [11:11]
pharveyMichaelDaum: I'll have to add that to my list of questions :-)
MichaelDaum: I don't want to waste your time too much; perhaps it would be better if I have a meeting with Temi and talk about getting his nifty new features merged somehow, or maybe find a way to do without
SvenDowideit: used phonetic^0.01, this looks better now: https://wiki.trin.org.au/System/SolrSearch?sort=score+desc&display=list&rows=20&origtopic=System.WebHome&search=LITTORINIDAE
MichaelDaum: I would definitely recommend adding some docco about the phonetic boost. Shall I create a task?
[11:11]
MichaelDaumpharvey, yes please. [11:14]
pharvey(with some candidate text) [11:14]
MichaelDaumI will disable phonetic boost all together, but give it a better default... 0.01 seems okay you say? [11:15]
pharveyyep [11:15]
***Alexander__ has quit IRC (Ping timeout: 258 seconds) [11:15]
pharveymaybe SvenDowideit can agree
can confirm*
[11:15]
SvenDowideitpharvey: wow :/
0.01 is better than 0.04
which suggests that corect spelling has a very low rating indeed :(
lets see how i feel about 0.0
er, this is mad
phonetic set to 0, and it still gives me a result that does not appear to contain the literal queried for
[11:16]
pharveymaybe 0 is a bad value. Can't you just remove the phonetic boost entirely? [11:18]
SvenDowideittryin [11:19]
pharveyMichaelDaum: Item11112 - trying to craft some words for this setting [11:19]
FoswikiBothttp://foswiki.org/Tasks/Item11112 [ Item11112: phonetic boost is too high ] [11:19]
***Alexander__ has joined #foswiki [11:19]
SvenDowideitseems not - solr fails to start? [11:20]
MichaelDaumpharvey, all of restSOLRFORMAT shouldn't be necessary. things like this are normally done using templates only [11:20]
pharveyk [11:20]
MichaelDaumthere are sets of configs used in restSOLRFORMAT to extract specific results. there might be some ajax code that relies on these endpoints, that might need a rewrite once you upgrade. [11:22]
SvenDowideitmmm, weirder, it starts but won't talk to foswiki
no idea where to debu
ls
[11:22]
MichaelDaumSvenDowideit, presumably port taken by something else [11:22]
pharveytomcat logs? [11:22]
SvenDowideitnot using tomcat [11:22]
MichaelDaumlsof [11:22]
SvenDowideitjetty works very well from my debs [11:22]
pharveyjetty then
pharvey hates tomcat, it's like an XML apache :-)
[11:23]
MichaelDaumMichaelDaum too ... it deletes apps with no sign of a backup once it thinks the app has to go [11:24]
SvenDowideitmmm, lets put it this way
when i put phonetic back into the cfg, it works
when i remove it, it doesn't
[11:25]
pharveylame :/
SvenDowideit: BTW if you get time, let me know if pseudo-install's new .gitignore works on your git checkout
[11:26]
SvenDowideitpharvey: /me would nbeed to know what it .... er....does [11:30]
pharveywell, I updated the doc, which has a link to a Foswiki:Development.GitAndPseudoInstall :) [11:31]
FoswikiBothttp://foswiki.org/Development.GitAndPseudoInstall [ GitAndPseudoInstall ] [11:31]
pharveythat doc isn't finished, but it has most of the notes one needs, I think..
it automatically maintains a .gitignore in each extension repo, for any automatically generated files
er, in each extension's root
hopefully it's not sensitive to repo-per-extension vs big-fat-repo
you complained that I had no doc yet, and this pitiful effort is an attempt to address that :)
[11:31]
SvenDowideitgrin
now all i need to do, is to git clean and re-pseudo-nstall
which means i need to commit some stuff thats not ready >:}
oh crud.
the reason i was re-viisting solr was cos we wanted to install it / kino on another site
and now i finally remember that its running widgetskin
which definitly won't like the hacks that solrplugin uses
click install kinosearch
like 'duh'
[11:36]
pharveykino actually works? I honestly couldn't make it do anything useful last I tried ~ year ago [11:40]
SvenDowideitit certainly gave me the results i was expecting just then
it uses the same strigifier as solr
so the question is - did you report bugs?
[11:41]
pharveythe bugs were already open, IIRC [11:42]
SvenDowideitso andrew might have fixed them? [11:42]
pharveyandrew maintains the cpan lib?
it all seemed very stale... but if it works now, that's great. I'll have to try it out. Solr is certainly lacking an equivalent integration w/SEARCH
[11:43]
SvenDowideitno?
oh, the SEARCH integration is a seperate issue
kino works like solr - in that using itsown topic works
the SEARCH integration is a much more complicated issue
[11:44]
pharveythe sustainable way forward, is to make core search flexible/pluggable enough that SOLRSEARCH/SOLRFORMAT aren't usually needed [11:46]
SvenDowideitcorect
but that requires that i get more time to work on store2.0 and search2.0
so that type="" is extensible, and thaat WebSearch topics are pluggable without the autotemplate hacks
type= and scope= :(
SvenDowideit stops looking at this quagmire for today and goes to look at another
back tomorrow i think
[11:47]
***Alexander__ has quit IRC (Ping timeout: 258 seconds)
leik has joined #foswiki
[11:53]
pharveyg'night [12:02]
***pharvey has quit IRC (Quit: ChatZilla 0.9.87 [Iceweasel 6.0/20110815162918]) [12:02]
.... (idle for 17mn)
foswiki_irc2 has joined #foswiki [12:19]
Alexander__ has joined #foswiki
gac410 has joined #foswiki
[12:24]
sezuan_ is now known as sezuan [12:34]
foswiki_irc2hello [12:36]
***foswiki_irc2 is now known as fomo
Colas has quit IRC (Ping timeout: 258 seconds)
[12:36]
fomoi have a problem with the twisty plugin sind i upgraded foswiki from 1.0.9 to 1.1.3 and i was wondering if anyone could help me here
*since
[12:37]
ArthurClemensfomo: did you upgrade JQueryPlugin and TwistyPlugin? [12:39]
fomoyes, everything is uptodate :)
i ran the google chrome debugger, which says:
Cannot call method 'getPrefList' of undefined
and i get a bad request error for foswikiPref.js, foswikiString.js, foswikiForm.js and pattern.js
[12:41]
***MartinCleaver has joined #foswiki
MartinCleaver has quit IRC (Changing host)
MartinCleaver has joined #foswiki
MartinCleaver has quit IRC (Remote host closed the connection)
MartinCleaver has joined #foswiki
MartinCleaver has quit IRC (Changing host)
MartinCleaver has joined #foswiki
mark_doe has joined #foswiki
Colas has joined #foswiki
[12:44]
andreli has joined #foswiki [13:05]
gac410fomo: Did you clear your browser's cache? [13:05]
andreliHi there
Is there a Perl guru around?
I would like to forward two questions I got from my Linux crack while we were investigating system behaviour of foswiki/fastcgi on Linux.
[13:07]
fomogac410: yes, and twisty wont work on any browser. it just looks like javascript is disabled, which means every table is expanded and the show/hide buttons arent there [13:12]
gac410Did you upgrade "in place" or do a new install of 1.1.3 and copy over your user data?
Hm. Also check the version of jquery selected in bin/configure. It should be at least 1.4.3.
[13:12]
fomoi just downloaded and installed the 1.1.3 debian package
i got jquery v. 1.6.1
[13:20]
***leik has quit IRC (Quit: Leaving)
Alexander__ has quit IRC (Quit: ChatZilla 0.9.87 [Firefox 6.0.1/20110830092941])
[13:25]
gac410fomo, I have no idea if/how debian works in removing obsolete files, compressed javascript, etc.
But by jquery version, I mean the setting in LocalSite.cfg
$Foswiki::cfg{JQueryPlugin}{JQueryVersion} = 'jquery-1.4.3';
[13:34]
***wdenk has quit IRC (Ping timeout: 260 seconds) [13:37]
CDot has joined #foswiki [13:47]
fomoi currently have no access zo the configure site, but im pretty sure its higher than 1.4.3 [13:48]
gac410Did you build a new configuration when you upgraded to 1.1.3? Or use the old 1.0.9 configuration? [13:49]
***wdenk has joined #foswiki [13:50]
gac410fomo, does your site have a public URL that I could visit to see the twisty issue? [13:51]
fomoi dont know if its the old configuration, because i didnt do the upgrade :)
no unfortunatley its not public, but you can just switch off javascript in your browser and u get the same :)
[13:53]
andreliCDot: Have you time for two foswiki/fastcgi issues? [13:53]
CDotandreli: I don't use fastcgi (still stuck in the mod_perl world) but will help if I can. Fire away. [13:54]
gac410fomo, Can you access your bin/configure tool and use FindMoreExtensions to see if Foswiki thinks the latest versions are installed?
btw I know turning off javascript will recreate your issue. The question is why is the javascript not working on your site after upgrade. It is most likely due to a stale file or something out-of-date after the upgrace
[13:55]
andreliOk. First observation: Why do some of the foswiki files have __DATA__ sections?
It seems to us, that this is not necessary and just keeps file handlers open for no reason
[13:56]
fomogac410: no js is working, it just seems that it doest. other plugins which require js are working fine [13:57]
gac410fomo: right - so something is working but the wrong version
andreli: is a file handler opened if the __DATA__ is not referenced? I seem to recall someone saying that was more efficient than parsing the # comments at the head of the file.
[13:57]
fomoi dont have access to the configure atm, but i know that everything is updated to the newest version [14:00]
***terceiro has joined #foswiki [14:02]
gac410fomo: I don't know what to suggest. I'd keep asking if any foswiki debian package experts are around. (aka SvenDowideit )
I only use the Tarballs, and bin/configure for extensions. And Foswiki tarballs do not support upgrade from 1.0 to 1.1 releases. It leaves too much stale stuff behind.
fomo: Is it working for anyone on your site? Was it working for the person who did the upgrade?
[14:02]
fomono its working for noone :) [14:04]
andreligac410: For questions like this I definitely know to little about perl. We just observe, that fastcgi keeps those handlers open. And the longer the process lives the more files end up keep open. So we ask: Is this necessary? I have no idea whether this is relevant anyway! [14:06]
gac410Not sure what else to suggest. I'd run bin/configure?action=FindMoreExtensions and see if it flags anything as out-of-date
gac410 in learning mode: How does one observe the open handlers. Needs to check his own site ;-)
[14:06]
andreligac410: ls -la /proc/<pid>/fd [14:07]
fomoty ill try that when i have access
bye
[14:08]
gac410andreli: So I'm not sure why we use __DATA__ vs __END__ ... the latter of which I doubt would have a file descriptor [14:08]
***fomo has quit IRC (Quit: Page closed) [14:08]
gac410gac410 goes to check [14:08]
CDotgac410: purely historical. A few files have __DATA__, most should have __END__
there are a very small number of files where __DATA__ is appropriate, however.
[14:09]
***foswiki_irc1 has joined #foswiki [14:10]
gac410andreli: interesting - so I see a few on my system as well, where the __DATA__ is nothing but the license/copyright info.
We probably ought to have a task to change __DATA__ to __END__ where appropriate.. ie when __DATA__ is not opened.
[14:10]
andreligac410: yeah, that is why we thought __DATA__ is not necessary [14:11]
gac410gac410 learns something every day [14:11]
andreliOk. But then we observed a more severe code short cut that seem to harm fastcgi processes:
It look like that file handler for uploads are not closed.
We think this should be corrected. What is your opinion?
I mean the file handler to the file in tmp
[14:12]
gac410That would not be good either. Yes I would think that would be important. ... hm... Foswiki 1.1. or 1.0? [14:14]
andreliFoswiki 1.1.3 [14:14]
gac410Do you have any plugins using the beforeAttachmentSaveHandler ??
Check System/InstalledPlugins ... that handler is deprecated. It copies attachment to a tmp file to allow the handler to open it.
The before/afterUploadHandler operates on the stream instead of a /tmp copy.
[14:15]
foswiki_irc1hello - on top of some of my topics above the top bar a double quote and a semicolon are displayed. the topic seems to load very slow. the only macros i use in this topic are %TOC%, %EDITTABLE% and %INCLUDE%. any idea what could be wrong resp. how to fix it? [14:18]
gac410andreli: yup - I recreated it - it seems to be a CGI temp file - /var/tmp/CGItemp54053 [14:20]
andreligac410: My coder tells me, that is common in cgi hacking, as the handler will die anyway when the script dies.
But with fastcgi these files cumulate over time.
So what do I do? File a task?
[14:22]
gac410Both of these seem to be good finds andreli - thanks. Can you open a task - for excessive file descriptors in fastcgi environment?
The __DATA__ will be very easy to fix. Not sure about that upload file.
[14:23]
andreligac410: I will. [14:24]
gac410Hm... Foswiki::UI::Upload does close the file handle that it used for the upload. Is there more to it than just issuing a close? [14:29]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::UI::Upload [ (Foswiki login) PerlDoc ] [14:29]
foswiki_irc1on top of some of my topics above the top bar a double quote and a semicolon are displayed. the topic seems to load very slow. the only macros i use in this topic are %TOC%, %EDITTABLE% and %INCLUDE%. any idea what could be wrong resp. how to fix it? [14:36]
SvenDowideitgac410: 99% of upgrade issues on debian are bacause the upgrader did not re-run and save in configure after upgrading
ie, same issue as for inplace upgrades
which is pretty much due to configure not being runnable from cmdline
if it where, then upgrades would be totally trivial on all shell env's
mmm, big commit pushed to my store2.0 repo
unit tests now run rather than crashing
all fail of course, more todo
s/course/o'curse/
[14:36]
***eberon has joined #foswiki [14:46]
gac410SvenDowideit: I've found that 1.0 -> 1.1 leaves stale templates and javascript behind that breaks things. Re-running configure does not remove the cruft.
Does installing new .deb's remove files that were eliminated in the more recent distribution?
I can't recall the details any more, but a comparison between 1.0 and 1.1 full tarballs shows a bunch of deleted files. Colas's upgrade shell script will find/remove them for the daring.
[14:47]
SvenDowideityes, debian upgrades do remove files
i'm not positive if it does all
but thats kinda one of the points of using a pkging system
and why the result is that it actls like js is off
the files that configure knows are gone
[14:53]
gac410agreed. (about packging). As far as fomo, he claims all js is working fine except twisties. Which would not be the case if a jquery version had been removed, but was still configured. [14:55]
SvenDowideitish. in that its plausibly a cache&cfg issue
upgraded without running configure being a pretty subtle pain point
[14:58]
gac410Changing subject - any idea why Upload of attachment would leave the /tmp/CGIxxxx file descriptor around? Upload.pm does do a close. Is there something more needed? [14:59]
SvenDowideitbut no idea - not eing able to get to configure is the singlyaly most useless stuffup we have
pass :( that code used to be pretty odd tho
[14:59]
gac410Why can't debian users just run bin/configure from browser? [15:00]
SvenDowideitthey can
but they generally don't, because they dont' read the big fat warning that they need to
[15:00]
gac410oh. okay [15:00]
SvenDowideitand really, they should not need to, it should be runnable from theupgrade script
but some ***** wrote configure as a webapp only
and now its so complex, that its more work to fix than working on store
is configure more code than foswiki yet? (me goes to look :))
[15:01]
gac410Yeah - The code that manages the config tree is rather inscrutable. Elegant ... but ..
I was trying to write some code to parse out the documentation from Config.spec - so BuildContrib could merge it into the BlahPlugin topic. Wanted to borrow where configure loads the same code.
[15:02]
SvenDowideitooops :) [15:04]
gac410It wasn't very usable for other purposes.
we should have one module that can read a .spec file - but the one in Configure is deeply embedded into the UI structure.
[15:04]
***denisr has quit IRC (Quit: Parti) [15:05]
SvenDowideitoh shit. the unit tests system creates more than one foswiki obj at a time
there goes any semblance of sanity i was pretending about
[15:05]
gac410Always? Or just certain tests that don't do a finish before opening a new session? [15:06]
SvenDowideitit finally does it in the teardown
but tbh, dunno yet
gone, back in many ours
[15:06]
gac410I know some tests need to finish and re-open sessions to get preferences and authentication re-processed, change userid, etc. But the ones I've come across do a finish [15:07]
***mark_doe has quit IRC (Ping timeout: 245 seconds) [15:10]
foswiki_irc1on top of some of my topics above the top bar a double quote and a semicolon are displayed.
the topic seems to load very slow. the only macros i use in this topic are %TOC%, %EDITTABLE% and %INCLUDE%.
any idea what could be wrong resp. how to fix it?
in the plain html text the following row seems to be the source of the problem:
src='/foswiki/pub/System/JQueryPlugin/i18n/ui.datepicker-de.js'></script>";<!--JQUERYPLUGIN::UI::LANG-->
[15:13]
ModAcOstfoswiki_irc1 its a bug in JQueryPlugin
foswiki_irc1 see Item11005
[15:16]
FoswikiBothttp://foswiki.org/Tasks/Item11005 [ Item11005: Surplus "; in JQueryPlugin/UI.pm ] [15:17]
foswiki_irc1ModAcOst thank you for the hint
can i fix this problem by myself by removing "; from the code in UI.pm?
[15:21]
ModAcOstyes
You'll need to reload apache if you have a consisten config
[15:29]
foswiki_irc1sorry for this question but - what is ment with "consistent config" [15:32]
ModAcOstmod_perl or mod_fcgid
wrong word
I meant persistent
[15:33]
***foswiki_irc0 has joined #foswiki [15:37]
CDotandreli: good find; sounds like a bug in CGI rather than Foswiki. [15:37]
gac410andreli: Found a fix [15:37]
CDotgac410: I think andreli is looking at the file handle in CGI, isn't he? [15:38]
foswiki_irc1ModAcOst thank you again - it works [15:38]
***foswiki_irc0 has quit IRC (Client Quit) [15:38]
CDotSvenDowideit: which tests create multiple Foswiki objects?
CDot is not surprised, but is curious
CDot has noted several tests that don't shut down the existing session before creating a new one
[15:38]
gac410Yes --- Simple fix in Upload.pm ...
close($stream) if $stream;
$query->close_upload_files(1);
[15:39]
CDotoh, CGI publishes that method? I thought it was private
CDot has learned something new today (besides the fact that Polish people think we are godless traitors)
[15:39]
gac410Add the close_upload_files call after foswiki closes the stream. Had to look at source - don't know if it is officially published. But it does close the file. :-) [15:40]
CDothmmm. If it's not published, make sure you test that it exists before calling it
otherwise it'll dissappear (on not appear on old CGI versions)
or, better, find where it's called in CGI, and deduce what *should* be called to cloase it down cleanly
[15:41]
gac410The docs in CGI.pm says it leaves files open so you can just rename after upload to avoid the file copy. There is a global to cause it to close all files. or explicitly call it. [15:42]
***ModAcOst has quit IRC (Remote host closed the connection) [15:42]
CDotOIC. Yeuch. [15:42]
***huggybear has joined #foswiki
foswiki_irc1 has left
[15:42]
huggybearwas wondering if there is a forum plugin for foswiki [15:43]
gac410CDot: Seems to imply that it only applies to mulitpart uploads - maybe we enable that by default?
gac410 doesn't know all that much about CGI.
[15:44]
CDotgac410: the "multipartness" comes from the request, not from our usage of CGI
so it's quite possible that a multipart request might come in. Though I can't think of a case where the core would use it.
but your proposal - closing when the request terminates - ticks all the right buttons for me.
[15:45]
gac410when andreli opens a task I'll commit the change for trunk. What's the best way to test if the method exists ... just in case. Eval it? or something easier [15:47]
BabarCDot: as you commented, better godless traitors than pussies :) [15:48]
gac410I might as well handle Item11114 as well - I have the open tmp files left dangling on my fastcgi installs. [15:48]
FoswikiBothttp://foswiki.org/Tasks/Item11114 [ Item11114: __DATA__ sections keep file handler open ] [15:48]
gac410(er ... and open __DATA__ sections) [15:48]
huggybearIs There a way that when you enter something into a table link that the table link will autmoatically become a link [15:49]
gac410There were way too many links in that question :-) sorry - I didn't follow it. [15:49]
huggybearlike when you have the edit button have one field of the table set for links [15:50]
Babargac410: find -nofollow? :) [15:50]
gac410Hm Reading through CGI.pm --- there is also another global. # 2) CGI::private_tempfiles(1); or $PRIVATE_TEMPFILES = 0; [15:52]
CDotgac410: defined &CGI::close_upload_files should work
reading the CGI code is bad for the heart. There's some really crufty code in there.
[15:53]
***Colas has quit IRC (Ping timeout: 258 seconds)
MartinRowe has quit IRC (Quit: Leaving.)
[15:55]
gac410Foswiki::Request defines itself as @ISA = ('CGI'); I suspect that the close_upload_files and private_tempfiles should be set when the CGI aka Request is created? [15:55]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Request [ (Foswiki login) PerlDoc ] [15:55]
gac410As these are methods to override globals, and not needed per file. [15:56]
***TJCRI has joined #foswiki [15:58]
gac410nope - setting them at the top of the request break things badly. :-(
I guess the call in Upload.pm is the right location.
[16:02]
***Lynnwood has joined #foswiki
Lynnwood has quit IRC (Changing host)
Lynnwood has joined #foswiki
[16:02]
ArthurClemens has quit IRC (Quit: Leaving...)
mark_doe has joined #foswiki
[16:09]
gac410No good. With that change after repeated uploads, I'm still finding tmp CGI files left open ... but not consistently. :-( [16:17]
***GithubBot has joined #foswiki [16:17]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/-2gKSw
[foswiki/master] Item10247: SASSified buttons - ArthurClemens
[16:17]
***GithubBot has left [16:17]
FoswikiBothttp://foswiki.org/Tasks/Item10247 [ Item10247: Implementing Base Skin ] [16:17]
huggybearCan I do textarea, 1x15 but have this area on an editable table as a link [16:22]
CDotgac410: the code does some acrobatics with the stream provided by CGI. Could that be holding it open? [16:25]
gac410No idea.
Have to look at the afterUploadHandler calls.
[16:25]
***milkman has quit IRC (Read error: Operation timed out) [16:28]
CDotI guess all the FW data is finish()ed, so any handles ought to have been cleared by that
CDot converted all glob handles to refs a long time ago
[16:30]
gac410Yeah - though we provide a local finish for Request::Upload that unlinks a tempfile but it appears no the temp name used by CGI [16:30]
CDotwhy would we unlink something provided by CGI? sounds like a foot-shooting enterprise to me [16:31]
gac410I think it might be the tempfile that we create for the deprecated afterAttachHandler - but I have not traced it well enough yet.
Though that all confuses me. Supposedly the purpose of changing from afterAttach to afterUpload was to avoid a temp file and use the stream. But CGI seems to create the temp file anyway.
gac410 confused
[16:33]
CDotCGI always creates a tempfile. It then hands Foswiki a handle to that tempfile.
The afterAttach handler, IIRC, requires data rather than a file handle, however. So we have to suck the handle dry, then re-generate it
in an ideal world, we'd *remove* that handler.
[16:36]
gac410ah - okay - can't seek the stream back to the beginning? Well as long as no plugin provides one the handler isn't used. [16:38]
CDotcan't seek the stream because the handler may modify the content
presumably the intent was to do stuff like, for example, scaling uploaded images
and if there is only *one* afterAttachHandler you could get away with it. However, as many plugins as want to can register that handler.
[16:39]
gac410Ah - understood. BTW found minor bug. we test $tmpFilePath = $query->tmpFileName($fh) and if tmpFilePath is not provided, we report a zero size. So changing global to close uploaded files may actually be breaking that function, and not actually causing a zero size upload. [16:40]
CDotCDot didn't really understand that, but nods and smiles as if he did. [16:41]
BabarBabar does the same [16:41]
gac410gac410 does too.
gac410 realizes that the more he reads CGI.pm the further over his head he is.
gac410 has no clue where to look to fix this one.
[16:42]
CDotgac410: I agonised over that for some time. The inelegant solution that is there is the best I could do. :-(
identifying *which* handle is left open is, however, the #1 requirement, because that might not be the problem
[16:50]
gac410Well it's definitely the file that is uploaded. And I don't have any plugins installed with after*Handlers installed. I didn't see that there would be more than one fh opened for an uploaded attachment.
pita to test in fastcgi - have to kill handlers after each change.
doesn't help I'm using my prod server. :-(
Hm. The finish() routine for Foswiki::Request::Upload is never entered. Added a print STDERR "finish entered" on my local test server - not running fastcgi.
[16:51]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Request::Upload [ (Foswiki login) PerlDoc ] [16:55]
gac410Does finish() have to be explicitly called, or does the magic of perl call it? [17:02]
Babarnah, perl's magic might call DESTROY
not finish
but finish should be called from the main call... the Foswiki::Engine
[17:03]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Engine [ (Foswiki login) PerlDoc ] [17:03]
gac410Maybe part of the problem - I don't see where engine ever calls finish() for Request::Upload. It creates it. But finish() has the tempfile cleanup
nope - Engine.pm and Engine/CGI.pm don't have any reference to finish
[17:03]
Babarworth trying
they should, iirc
[17:04]
gac410grep -ri finish Engine ... no hits
and for Engine/* either.
[17:05]
Babarah? [17:07]
gac410finialize deletes the uploads, but doesn't call finish - but I don't really understand the code :-( [17:08]
Babarbut Foswiki's finish should call it
but I'm guessing it's never called
[17:10]
gac410I put a print at the top of sub finish in Foswiki::Request::Upload ... no hits. [17:10]
***andreli has quit IRC (Quit: ChatZilla 0.9.87 [Firefox 7.0/20110824172139]) [17:11]
Babarbut that's a good point... as the Foswiki object is never finished, what's getting killed? [17:12]
gac410Engine/CGI.pm issues a call for prepareUploads and then finalizeUploads for the attach. finalizeUploads deletes the request object ... but doesn't call finish.
well - it deletes the name provided by the object, I don't understand the code yet.
[17:14]
Babarlemme check [17:15]
gac410gac410 wonders if he can change $req->delete($_) foreach keys %{ $req->uploads }; to also call finish.
not that this has anything to do with the issue - but it does seem to be some dead code.
[17:15]
Babarit does that already
the delete in Foswiki::Request is supposed to do just that
[17:17]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Request [ (Foswiki login) PerlDoc ] [17:17]
Babarand the code is stupid :) [17:18]
gac410Hm. print in finish() routine would dispute that. ... for some reason it's not called.
gac410 admits he is totally lost
[17:18]
Babarok, I'll check it after my bike ride then :) [17:22]
gac410in any event the close_upload_files(1) seems to be totally unrelated.
Thx Babar.
[17:23]
***MartinCleaver has quit IRC (Ping timeout: 276 seconds) [17:30]
...... (idle for 25mn)
Babargac410: the code in Foswiki::Request::delete makes no sense at all [17:55]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Request [ (Foswiki login) PerlDoc ] [17:55]
gac410I wonder if that is just an unrelated find, or if it is causing the stale file handles. [17:57]
Babarif I code some patch, can you try it out for me? [17:58]
gac410sure. [17:59]
BabarfinalizeUploads is called, right? [18:00]
gac410yes
(had to check log)
[18:00]
Babarthere is something wrong here...
great... I nuked my foswiki apache config on my laptop
time for lighttpd :)
which doesn't work... just my luck
[18:05]
ok, I understood the problem, but I don't really see the fix yet
as it's too obvious
[18:17]
gac410okay - I've got to step away for a bit - I'll check back every so often. Have to get some errands done.
At least I don't feel too bad if the code is difficult for you to understand. I wouldn't have a chance.
[18:18]
BabarI mean, the code is OK
I just don't understand how it can work
I mean...
it seems to be able to support multiple file upload at once
but: 1. I have no idea how to test it
[18:19]
gac410why would it have to - each upload would be a new request - unless we get some javascript based uploader. [18:20]
Babarthis part I don't understand
but basically, it's calling delete with the wrong name
hence, it's not deleting anything
hence, it's not finalising anything
[18:22]
gac410hopefully that is the root cause of leaving the file handles open [18:23]
BabarI have a quick and dirty patch to test that :)
replace $req->delete( keys %{ $req->uploads } );
[18:23]
gac410Though there is an explicit close in UI/Upload.pm [18:23]
Babarin Foswiki::Engine::CGI.pm [18:23]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Engine::CGI [ (Foswiki login) PerlDoc ] [18:23]
gac410with... [18:23]
Babarwith $req->delete( 'filepath' );
ah sorry, it's not like that in your version
there is a foreach
[18:23]
gac410yes.
so replace the whole foreach line?
[18:24]
Babaryes [18:24]
gac410hang on [18:25]
Babarin my tests, it ought to work [18:25]
***ArthurClemens has joined #foswiki [18:25]
Babarbut... it means half the rest of the code is awfully complex for nothing
ah ArthurClemens, maybe you know...
is there a possibility for Foswiki to upload more than one file at a time?
[18:25]
ArthurClemensprobably, but not implemented in core
MichaelDaum has an upload plugin using pupload
[18:27]
Babarok... because Foswiki::Engine::CGI seems to support it somehow. But I can't see how the code could work [18:27]
ArthurClemensand twiki has something in the attach form [18:27]
gac410Hm.. partial fix - it deletes the filepath but leaves the fd open.
lrwx------ 1 apache apache 64 Sep 6 14:27 9 -> /var/tmp/CGItemp53753 (deleted)
[18:27]
Babargac410: but does it call finish? [18:27]
***CDot has quit IRC (Quit: Leaving.) [18:27]
MichaelDaumBabar, have a look at the TopicInteractionPlugin's backend impl [18:28]
gac410If I ls -la the proc/<pid>/fd
hm... Those debug statements are on a different system :-(
[18:28]
Babargac410: ok, doesn't matter
MichaelDaum: hum... does it use the core engine?
[18:28]
MichaelDaumFoswiki::Plugins::TopicInteractionPlugin::Action::UploadAttachment that is [18:29]
FoswikiBothttp://trunk.foswiki.org/System/PerlDoc?module=Foswiki::Plugins::TopicInteractionPlugin::Action::UploadAttachment [ (Foswiki login) PerlDoc ] [18:29]
MichaelDaumBabar, nope can't. it needs to cope with different upload styles.
as well as chunked uploads and the like
[18:29]
***MartinCleaver has joined #foswiki
MartinCleaver has quit IRC (Changing host)
MartinCleaver has joined #foswiki
[18:30]
Babarso why on Earth did Gilmar code the CGI engine that way??? [18:30]
gac410Babar, yes finish is called. [18:30]
MichaelDaummultipart vs streamed n stuff [18:30]
Babarmaybe he dreamed of a time when the layer underneath would support uploads flawlessly? [18:30]
MichaelDaumGilmar's work is in danger to be totally obsoleted by plack [18:31]
Babargac410: sure. cya
Gilmar's work works on plack, according to ...
http://foswiki.org/Support/BestPracticeTip24
[18:31]
gac410thx for the help looking at this. If you have any ideas on fixes I'll check logs. [18:31]
BabarMichaelDaum: with use CGI::Emulate::PSGI; [18:32]
MichaelDaumBabar, ... was about to say :/ [18:32]
Babarright, if you want to go pure plack, then it needs rewriting
but still, his work is almost abstracted enough
[18:32]
MichaelDaumit needs removal I am afraid [18:32]
Babarhaven't played enough with plack to know
so, trusting you on that
[18:33]
MichaelDaumthe idea of psgi is to "outsourcing" stuff like that
Babar, did you try BPT24? ... is it as fast as it says?
[18:33]
Babarnope. I have it open in one tab to try it out
wanted to even try it out on f.o :)
and I'm guessing it's just as lightning fast as any other persistent environment
[18:35]
MichaelDaum... that does not use apache
am pretty happy with lighty
[18:37]
Babaryes, but you really think apache adds such an overhead?
ok, it adds some, which is the all point of nginx
[18:39]
gac410: ok, as I have no idea what else you're working on, I will upload my "fix" to github (so githubbot should tell you were), and you'll tell me more when you come bac [18:52]
***GithubBot has joined #foswiki [18:57]
GithubBot[foswiki] Babar created gac410Upload (+1 new commit): http://git.io/0oOKXg
[foswiki/gac410Upload] For George to review - OlivierRaginel
[18:57]
***GithubBot has left [18:57]
MartinCleaver has quit IRC (Quit: MartinCleaver) [19:02]
MichaelDaumBabar, apache _is_ fat. and seems to choke occasionally, presumably when maintaining backends.
it is like a fat swiss army knife ... that doesnt fit into anybody's pocket.
[19:08]
***MartinCleaver has joined #foswiki
MartinCleaver has quit IRC (Changing host)
MartinCleaver has joined #foswiki
[19:11]
....... (idle for 31mn)
Plazma-Rooolz has joined #foswiki [19:42]
MichaelDaum is now known as MichaelDaum_
MichaelDaum_ has quit IRC (Remote host closed the connection)
[19:47]
Plazma-Rooolzjust got foswiki installed and trying to go to the configure script the first time it's telling me to run configure :/
centos and apache .. nothing fancy
i feel like it's probably something silly on my part.. it's configued to use port 8080
url is http://localhost:8080/foswiki/bin/configure
or that i'm using (obviously won't work for you guys :P )
[19:49]
ArthurClemensPlazma-Rooolz: there is a tool to help with apache settings; http://foswiki.org/Support/ApacheConfigGenerator [19:58]
Plazma-Rooolzi acutally did use that
and put it in /etc/httpd/conf.d/foswiki.conf
restarted apache
apache:apache owns everything too
well in /var/www/foswiki
also when i try http://localhost:8080/bin/configure it gives me a 403 error thrown from apache
[19:59]
***mark_doe has quit IRC (Read error: Connection reset by peer) [20:13]
ArthurClemensPlazma-Rooolz: did you protect configure before adding yourself as user? [20:14]
***mark_doe has joined #foswiki [20:14]
Plazma-RooolzArthurClemens: i setup an .htaccess file in /var/www/foswiki/data as it instructede
AdminConfigure i think is what i used..i don't get that far yet though
[20:18]
ArthurClemensif you use apache config, you probably don't use .htaccess
where is the instruction to set up .htaccess in /data ?
[20:20]
Plazma-Rooolzhmm
i know it's dated and mentions that but http://foswiki.org/Support/CentOS
[20:32]
ArthurClemensok, that's htpasswd not htaccess [20:38]
Plazma-Rooolzhmm
that's what i meant then, sorry
[20:38]
ArthurClemensI am not a apache expert. but do you need to use a port number? [20:39]
Plazma-Rooolzi tried both without and with 8080 and it fails [20:41]
ArthurClemenswhat is in the apache error log? [20:43]
Plazma-Rooolzhttp://ideone.com/qUotW [20:48]
***GithubBot has joined #foswiki [20:48]
GithubBot[foswiki] foswiki pushed 1 new commit to master: http://git.io/KosJoQ
[foswiki/master] Item11113: remove reference to date.js - ArthurClemens
[20:48]
***GithubBot has left [20:48]
FoswikiBothttp://foswiki.org/Tasks/Item11113 [ Item11113: date.js missing ] [20:48]
Plazma-Rooolzwoa [20:48]
ArthurClemenssomething to do with LocalSite. This file is created the first time you save your settings in bin/configure
that it does not exist is not the error
[20:51]
***MartinCleaver has quit IRC (Read error: Connection reset by peer) [20:59]
Plazma-RooolzArthurClemens: well i think it's ok since i'm TRYING to acutally create said file
but i saw somethin gelse
in the virtualhost it has allow from all too for /var/www/foswiki
[21:08]
***Zenopus_ has joined #foswiki
Zenopus has quit IRC (Read error: Connection reset by peer)
Zenopus_ is now known as Zenopus
[21:14]
Plazma-Rooolzwoa, it works now [21:21]
***eberon has quit IRC (Quit: eberon) [21:23]
ArthurClemensgreat [21:34]
***TJCRI has quit IRC (Remote host closed the connection) [21:38]
gac410Babar - thx - I'm back - sort of - I'll take a look at your fix and test it on my fastcgi server soon [21:46]
Babarmy fix is the same as the one I told you to try, more or less
just done the other way around :)
[21:46]
gac410Hm. okay - So if the results are the same, it deletes the file, but leaves the file descriptor "open" per /proc/<pid>/fd/* [21:47]
Plazma-Rooolzman.. this thing is kinda overwhelming [21:47]
gac410Babar - your fix works - /tmp file is deleted, but the file handle remains open. So something is not being properly closed out in CGI::upload. Wish there was an easy way to find out which open opened the file descriptor [21:54]
Babarhum, there is probably some Devel:: module which can do that [21:54]
gac410I set my server to only have a single FCGI handler. Each subsequent attach closes the prior fd, and leaves a new one in the open (deleted) state. [22:00]
***ArthurClemens has quit IRC (Quit: Leaving...) [22:03]
...... (idle for 26mn)
mark_doe has quit IRC (Read error: Operation timed out) [22:29]
........ (idle for 35mn)
huggybear has quit IRC (Quit: They see me trollin' They hatin') [23:04]
..... (idle for 24mn)
gac410Well, I'm out of ideas. I've identified where the stream on the file handle is opened, verified that it really is closing it. Confirmed the name. and yet after an attach, linux shows the file still having an open file descriptor [23:28]
***pharvey has joined #foswiki [23:37]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)