Full Board URL: http://forum.solbu.net/
XMB Version: 1.9.11

I run a norwegian forum. Whenever someone posts anything containing the norwegian special characters «æ», «ø» and «å», the entire post is totally BLANK both on preview and when posting.

The apache logs give no errors.

Anyone have any idea as to what is going on?
(The forum is running on Debian 7)

Which character set are you using on the database - see http://forums.xmbforum2.com/viewthread.php?tid=776873

Has this just started to occur?
Was there an upgrade to mysql or php on your server?

I think if the tables were set to UTF-8 it might resolve that issue for you.

Quote: Originally posted by lottos

Which character set are you using on the database - see http://forums.xmbforum2.com/viewthread.php?tid=776873

It seems to be latin1_swedish_ci.

Quote:

Has this just started to occur? Was there an upgrade to mysql or php on your server?

It happened sometimes this summer. I don't know exactly when, as the forum is not a wery active one, and I don't use it each week myself.
But we did upgrade from Debian 6 to Debian 7 this summer, so there most definetly was an upgrade of both MySQL and php.

Quote: Originally posted by eman86

I think if the tables were set to UTF-8 it might resolve that issue for you.

Interesting. And how do we do that?

you have go and change that manually on each xmb table. give some some time and I can write up a nifty script for you

Quote: Originally posted by eman86

you have go and change that manually on each xmb table. give some some time and I can write up a nifty script for you

Hmm. I found something that did that, and the special characters still blanks the posts. And changing it somehow broke every configured setting throughout the entire forum. Like Sub forum names, the rules, etc.. So we had to revert all changes by importing from backup.

Quote: Originally posted by solbu

It happened sometimes this summer. I don't know exactly when, as the forum is not a wery active one, and I don't use it each week myself. But we did upgrade from Debian 6 to Debian 7 this summer, so there most definetly was an upgrade of both MySQL and php.

Perhaps you can find out what versions you were running and what versions you upgraded to.

Always good to make a back-up if you do changes like that.

However, changing the charset should not break all configured settings.

Could you please show us a screenshot of what you changed?

First, sorry for late response.

Quote: Originally posted by Mouser

Always good to make a back-up if you do changes like that.

No problems there. We have an automated (crontab) daily database backup routine that keep backups for 5 months back.

Quote:

Could you please show us a screenshot of what you changed?

We reverted to the old nonworking backup, so all is the same as it where when the operating system was upgraded, so I don't have a screenshot. I also remember the php script I used (and ran in the terminal) had problems converting some of the tables and exited, resulting in me having to run all of the steps manually anyway, and stil had some tables that would not convert, so I can't use it again.

Any other ideas?

Quote: Originally posted by eman86

you have go and change that manually on each xmb table. give some some time and I can write up a nifty script for you

I think I'll have take you up on the offer.

I'll try to whip something up this weekend and will post back when I'm done

here it is, this will convert every xmb table to UTF-8

Attachment: xmb-sql-utf8.php (3kB)
This file has been downloaded 576 times

Quote: Originally posted by eman86

here it is, this will convert every xmb table to UTF-8

Thanks

On my forum, the charset is "latin1_swedish_ci" and have not had any issues so far. *crosses fingers*


The provided PHP file, works if the table_prefix is "xmb_".
Q; Why not include the config.php and pull the database info and table_prefix from there?




7 Years ago I made some handy scripts for versions like XMB1.9.6.
I've taken one of those files and modified the queries to be run.
This will work without the need to modify it, just upload it to the forum-root and run it... Then click on the link to start the procedure to update your tables to charset "utf8_unicode_ci".


Please, create a backup of your database before you start this procedure !!

Attachment: TablesEncoding.php (5kB)
This file has been downloaded 421 times

I was going to do that mouser but since XMB's codebase is a bit new to me I wasn't sure if the config had anything sort of "anti-theif" check to it. 99.99% of people don't change the prefix imo at from my experience most don't do it unless theres a real reason to.

Hi, problem in my forum is when one member write message with this chars:
à è ì ù à ò

If I use php 5.3 or lower work correctly, but I see blank window with php 5.4 or higher. Only solution is change charset with utf8_unicode_ci

I've now tested both scripts, and nothing changes. As in, the error of blank posts still remains.

Seems to be a known issue with php 5.4:

http://stackoverflow.com/questions/15028339/odd-issue-with-f...


This implies reloading the data into the database fixed it:

http://stackoverflow.com/questions/14049079/php-5-4-9-mysql-...


vbulletin also had issues:

http://www.vbulletin.com/forum/forum/vbulletin-4/vbulletin-4...

with this the possible fix, if implemented into xmb:

http://www.vbulletin.com/forum/forum/vbulletin-4/vbulletin-4...

http://www.vbulletin.com/forum/forum/vbulletin-4/vbulletin-4...


Along with SMF, with this their possible fix:

http://www.simplemachines.org/community/index.php?topic=4971...


and Wordpress:

http://core.trac.wordpress.org/ticket/24121


Another possible workaround is step 4 from

http://xaviesteve.com/1223/issues-with-accents-and-strange-c...



I suspect the culprits are the post.php file:

$rawsubject = htmlspecialchars_decode($threadname, ENT_QUOTES);
$rawusername = htmlspecialchars_decode($username, ENT_QUOTES);
$rawemail = htmlspecialchars_decode($subs['email'], ENT_QUOTES);
$rawbbname = htmlspecialchars_decode($bbname, ENT_NOQUOTES);

Example:
$rawusername = htmlspecialchars_decode($username, ENT_QUOTES);
probably needs change to:
$rawusername = htmlspecialchars_decode($username, ENT_QUOTES, 'utf8');

In functions.inc.php,
function htmlspecialchars_decode($string, $type=ENT_QUOTES)
probably also needs changing.

If that works, there are another six instances in u2u.inc.php, possibly others in other script files.


Although... probably best to have the settings or config.php have the character set type rather than hard code.

Quote: Originally posted by lottos

probably best to have the settings or config.php have the character set type rather than hard code.

Then the question becomes, how do one do that.

I'm a bit far from the xmb but I'll try to help.

The problem is the php 5.4 like lottos said.

Maybe this can fix this problem but is not a permanent solution

Just replace the file validate.inc.php

Attachment: validate.inc.php (14kB)
This file has been downloaded 548 times