#foswiki 2017-09-24,Sun

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
FoswikiOnSlack<itaki> Hello everybody. The above support question belongs to me. Anybody here, to help me? [10:39]
uebera||FTR, itaki is referring to https://foswiki.org/Support/Question1898 (this has not been propagated over the IRC 'bridge')
itaki: Please try to minimise/compact the example as much as possible--it's enough to demonstrate the problem with a single entry. Also, this drastically reduces the risk that someone might think of it as SPAM (no offense) if s/he does not speak the language and the machine translation is funny.
[10:48]
FoswikiOnSlack<itaki> Maybe, my problem results from such a big data as I mentioned in the comment. [10:52]
<itaki> In order to minimise the size of the topic I have deleted some of the rows. [10:58]
uebera||itaki: You can answer that part of your question yourself if you create a second topic with (much) less entries and try to edit it. If it works with the smaller table, it might indeed be size related. Also, it is important to know whether the modifications were huge (e.g., you added 50 entries at once) when the problem arose. In that case--does adding few entries a time solve the problem?
The above answers to questions would speed up potential analysis by those addressing the questions.
The more precise the problem description, the easier it is/might be to pin it down.
[10:58]
FoswikiOnSlack<itaki> As it can be seen from the table, top part of the Turkish alphabets are correct, the bottom part of the table is not correct. [11:00]
uebera||itaki: Last remark before I need to step away: It might be helpful to provide the Windows/Excel version you're using in case there are problems to replicate the scenario. Did you copy from "native Excel" or was this a browser based view? (In the latter case, the version of the browser in question might be helpful as well) [11:06]
FoswikiOnSlack<itaki> I created all of the excel cells something like these:
<itaki> ||İşyerine ait yıllık eğitim programları planlanmış mı?|Yıllık eğitim planı istenir.||||||Yıllık eğitim programında, eğitimlerin konusu, hangi tarihlerde düzenleneceği, eğitim süresi, kimlerin katılacağı belirlenmelidir.|||||
<itaki> which enables Foswiki to detect the table correctly.
<itaki> Please have a look at the original file in my foswiki web which WikiGuests can view:
<itaki> https://istenovaisguvenligi.com/bin/view/Herkese/RiskDegerlendirmeRaporuCopy
<itaki> I corrected the topic for my above last link which you can view.
<itaki> I copied and pasted the correct data, everything seems OK now. But when I revise the table the alphabets are collapsed for some part of the table.
[11:15]
.......... (idle for 45mn)
***ChanServ sets mode: +o Lynnwood [12:10]
.................. (idle for 1h25mn)
ChanServ sets mode: +o gac410 [13:35]
gac410itaki: Are you editing with the Wysiwyg (TinyMCE) editor, or with the "Edit Wiki Text" plain text editor. Or are you using EditTable or EditRow plugin? [13:40]
FoswikiOnSlack<itaki> I use both of them, Wysiwyg and "Edit Wiki Text". Both of them causes same error. [13:42]
gac410Itaki: If you _are_ using Wysywyg / TinyMCE, could you try recreating it on https://trunk.foswiki.org/Support/Question1898? trunk.foswiki.org is running a new editor, it might be better (or worse) .
Another question, does the browser matter? chrome vs. firefox vs. ie / edge?
Did you actually recreate the corruption on foswiki.org, or did you paste in the bad data to show the issue.
Issues with character sets and utf-8 / unicode can be very dependent upon the perl version too.
[13:42]
FoswikiOnSlack<itaki> I copied from the correct data, but when I paste it transform to another language's alphabets.
<itaki> As I said here is the correct version, just before any revision:
<itaki> https://istenovaisguvenligi.com/bin/view/Herkese/RiskDegerlendirmeRaporuCopy
<itaki> By the way I use VPS1 package of "https://www.tmdhosting.com/vps-hosting.html"
<itaki> I had also been informed by their support team that, I have almost consumed SSD space of my storage. Do you think it can also be a reason?
[13:49]
gac410I don't think so. Can you get the raw - un-corrupted data onto foswiki.org somehow? Maybe upload your Herkese/RiskDegerlendirmeRaporuCopy.txt file as an attachment to Question1898? [13:57]
FoswikiOnSlack<itaki> Ok. I will. [13:57]
gac410We'll need some way to actually recreate the corruption event to figure it out.
Not that I'm very good at all with unicode/utf-8 issues.
Is it always the size of the table? Starting at row 39? Or have you seen it happen in some of the first rows?
[13:57]
FoswikiOnSlack<itaki> I think it can be anywhere, not always same position. [14:03]
gac410ugh. For the exact same data, if you repeat it multiple times, does the start of the corruption "move around" Sometimes row 39, sometimes other places? [14:04]
FoswikiOnSlack<itaki> I attached the txt file:
<itaki> https://foswiki.org/pub/Support/Question1898/EditTable.txt
[14:06]
gac410Thanks [14:06]
Hm Could you look at your topic System.FoswikiServerInformation ... What version of perl is being used. [14:12]
FoswikiOnSlack<itaki> Foswiki Version v2.1.2 Foswiki Release Foswiki-2.1.2 Foswiki Plugin API 2.4 [14:14]
gac410How about Perl version [14:14]
FoswikiOnSlack<itaki> Perl version 5.10.1 (linux) [14:15]
gac410Ah... That's really really old. Dates from August 2009. Very early in unicode development.
I do have 5.10.1 installed here, so I can try to recreate your issue on that version. So far I'm not seeing any issues, but I'm running perl 5.22.
I need to step away. Time to cook breakfast.
[14:17]
FoswikiOnSlack<itaki> What do you suggest? Should I request from my hosting provider to upgrade my perl version?
<itaki> Thank you vey much for your support George.
[14:19]
gac410I'll try to play with this a bit today and try to recreate it. I don't know yet if it's perl version or not. So don't upgrade yet. [14:20]
FoswikiOnSlack<itaki> Thanks. [14:20]
gac410I can say that performance on new perl is much better. But no idea if this is the cause of the problem.
Did you actually re-create the corruption on foswiki.org ... or did you just paste in the bad data?
[14:20]
FoswikiOnSlack<itaki> I re-copied the correct data again and pasted it "https://foswiki.org/Support/Question1898" as r2 of Topic.
<itaki> The table seems OK.
[14:27]
gac410I've got to go take care of breakfast. See if you can cause the corruption on foswiki.org And trunk.foswiki.org (They share the same data directory) Foswiki.org is running Perl 5.22.2 [14:31]
FoswikiOnSlack<itaki> Thank you very much for your time. [14:32]
gac410you are welcome.
Interesting. In rev=1 of the topic, at line 5, it seems to switch from Turkish to a completely different character set. And then at Row 39, it becomes illegal utf-8
One more piece of information that might be helpful. In FoswikiServerInformation, in the list of modules, what version of Encode is in use?
[14:33]
FoswikiOnSlack<itaki> When I try to revise "https://istenovaisguvenligi.com/bin/view/Herkese/RiskDegerlendirmeRaporuCopy" which was a correct table, it is corrupting to another alphabet set. [14:39]
gac410But if you try the same thing on Foswiki.org, does it work okay?
Down in the core of Foswiki, for every topic read/write, Foswiki has to convert utf-8 (on disk) to Unicode in memory, and on save, it converts back. So bugs in an old version of Perl or Encode could cause this type of problem.
[14:40]
FoswikiOnSlack<itaki> Trying to revise the topic at https://foswiki.org/Support/Question1898 working properly.
<itaki> May I learn your final suggestion after your breakfast?
[14:45]
gac410I'll try. [14:49]
..... (idle for 22mn)
itaki - What version of Encode are you running (See your System.FoswikiServerInformation. I am unable to recreate the problem on Perl 5.10.1 ... But it might be Encode [15:11]
FoswikiOnSlack<itaki> Encode 2.35 /usr/lib64/perl5/Encode.pm Encode::Alias 2.12 /usr/lib64/perl5/Encode/Alias.pm Encode::Config 2.05 /usr/lib64/perl5/Encode/Config.pm Encode::Encoding 2.05 /usr/lib64/perl5/Encode/Encoding.pm [15:13]
gac410Damn. I'm running the same version. I really don't understand why this is being corrupted. [15:14]
FoswikiOnSlack<itaki> Are you sure I don't need to upgrade my VPS package? [15:16]
gac410No I'm not sure. It's my best guess on what is going wrong. But as I cannot recreate the issue, I cannot say for sure.
I do know that Perl 5.20 5.22, etc. are MUCH much better with Unicode. Especially performance.
[15:17]
FoswikiOnSlack<itaki> When I had two times an error message like this :
<itaki> "Please wait... retrieving page from server."
[15:19]
gac410I *roughly* timed loading your page on my test server. Perl 5.10.1 about 4-5 seconds, Perl 5.22 around 2 seconds. [15:21]
FoswikiOnSlack<itaki> My hosting provider informed me that I have consumed lots of my storage capacity, and they suggested to upgrade my VPS package or, create some empty space. [15:21]
gac410The log files can get quite big. You could look at working/logs and remove old logs. [15:22]
FoswikiOnSlack<itaki> I see, I will forward your suggestion to my hosting provider. [15:24]
gac410working/tmp can also get large if you are not removing stale session files.
I'm not sure what else can grow lots of space in foswiki
But really if your provider has it, getting onto a modern perl would help you a lot and *might* fix the issue you are seeing.
[15:24]
FoswikiOnSlack<itaki> When I faced with this message, I couldn't reach to the related topic. [15:26]
gac410Perl 5.26 is the latest. Perl 5.22 is very reasonable. Anything older than Perl 5.18 is not recommended.
Y, I have no idea what would cause the Please Wait. It could be that the request for the page crashed for some reason.
[15:26]
FoswikiOnSlack<itaki> So, I should also upgrade my Perl, too. [15:27]
gac410The Wysiwyg editor has to "fetch" the page contents using a javascript request. If that fetch fails then you get stuck on the please wait.
I would recommend newer perl ... yes.
[15:28]
You are also running Foswiki 2.1.2 ... Version 2.1.4 is the latest. There are a few fixes related to non-English characters. But more related to sort order. Nothing that would cause this issue that I can think of. [15:34]
FoswikiOnSlack<itaki> Thank you very much George. [15:36]
gac410I just switched my server to Foswiki 2.1.2 / Perl 5.10.1 ... Everything still works with your test data.
So I am very much at a loss. I really have no idea why you are seeing corruption.
My "best guess" is Perl version, but there is no evidence :(
[15:38]
FoswikiOnSlack<itaki> It's so strange. Poor me. [15:38]
.... (idle for 17mn)
gac410You might ask back if you see cdot online. He did a lot of the utf-8 work. But my "money" is bet on it being related to the Perl or cpan versions. Getting onto 5.22 with latest cpan modules, and Foswiki 2.1.4 *might* make the problem go away.
No guarantees though :(
itaki, your support question says "Edit Table Plugin" but I assume that is actually unrelated? You have created the problem using both Wiki Text editor, and the Wysiwyg editor. ???
Just making sure.
Anyway, good luck with it. I've got to get some stuff done today.
[15:55]
FoswikiOnSlack<itaki> I think so. It seems that it's not a plugin problem. It's editing problem. [16:02]
.... (idle for 16mn)
***ChanServ sets mode: +o Lynnwood__ [16:18]
.... (idle for 16mn)
gac410itaki ... I think it just occurred on Foswiki.org. I edited the topic to remove the EditTablePlugin reference. When I saved it, corruption started at Row 108
No... It was your Edit - Rev 4, which is corrupted.
What browser are you using?
[16:34]
FoswikiOnSlack<itaki> Chrome and Internet Explorer. [16:37]
gac410Do you know which one you used when editing Rev 4?
Which version of IE?
[16:37]
FoswikiOnSlack<itaki> Mainly Chrome. [16:38]
gac410Okay ... which version of Chrome? [16:38]
FoswikiOnSlack<itaki> How can I see? [16:38]
gac410Probably the Help -> about
(Pulldown menu on right -> Help -> Help -> about
[16:39]
FoswikiOnSlack<itaki> 61.0.3163.100 (Resmi Derleme) (64 bit) Sürümü [16:39]
gac410Okay. I restored rev=3. Let me try to edit it with my own chrome browser. I also have a version of 61 installed. [16:41]
This is really strange. I've edited it multiple times with Chrome both Wysiwyg and wiki text. Does not seem to be corrupting. [16:47]
FoswikiOnSlack<itaki> By the way another surprise here:
<itaki> You remember the txt file.
<itaki> I copied the data from this txt file, and pasted it here:
<itaki> https://istenovaisguvenligi.com/bin/view/M%C3%BC%C5%9Fteriler/Onn/RiskDegerlendirmeRaporu
<itaki> It works very well, there is no corrupting.
[16:49]
***ChanServ sets mode: +o Lynnwood [16:51]
gac410I am completely baffled. It is most likely NOT the Perl version. I wonder if there is some unusual data in the initial version which is causing the utf-8 conversion to go bad. [16:52]
FoswikiOnSlack<itaki> But this time I can not succeed for the topic you have right to view:
<itaki> https://istenovaisguvenligi.com/bin/view/Herkese/RiskDegerlendirmeRaporuCopy
[16:55]
gac410Y, I get a login prompt.
I'm sorry but I really cannot spend more time on this right now. Things to do.
And I just don't have any ideas.
We do know that MS Excel does to some special things with encodings. At least here, Windows typically uses cp-1252 with special microsoft characters. I have no idea what recent versions in non-English areas will be using.
It may be that a copy/paste from excel is inserting some characters that cause problems in the utf-8 conversion later.
[16:55]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)