Mr. Tact wrote on Feb 19, 2021, 14:29:
Frans to the rescue, again!

Nah, the Bluester returned and finished it himself.

Shock wrote on Feb 19, 2021, 13:15:
Logo hasn't changed for me. Still looks like Command & Conquer

Not just you, the Bluester forgot to rotate it and then apparently left the Blue Tower.

eRe4s3r wrote on Feb 10, 2021, 22:28:
Did you declare @charset "utf-8"; in the primary CSS and *SAVE* that .css file encoded in UTF-8 (with notepad++ or some other file editor that allows specifying..) ?

Since the style sheet contains no non-ASCII characters, that doesn't have any effect, but it doesn't hurt either.

eRe4s3r wrote on Feb 10, 2021, 22:28:
I dunno which CSS is loaded for NewBlue,

Hit Ctrl-U and look at line 12: <link href="/css/styles.css"...> Click it to view its contents.

fds wrote on Feb 10, 2021, 08:41:
Which version of MySQL is it? Older versions of MySQL, and all versions of MariaDB, required the use of their specially named "utf8mb4" character set instead for true utf-8, one that also supports 4-byte characters such as all emojis.

5.7, and yes I'm aware of the mb4 charset situation, but was hoping to avoid that. And the eacute is 2-bytes so I'd think the mb4 angle wouldn't explain the problem.

But the OS is due for a major LTS upgrade anyway, so I have some small hope that that might sort out the problem as yet, if it isn't a coding bug in our scripts.

fds wrote on Feb 10, 2021, 08:41:
Currently for me, your #2 link consistently serves the title of the story as Cyberpunk 2077 Expos0xE9, which is invalid utf-8, and as such, consistently shows broken. Both in the h2 posting title, and the top-level title tag which then ends up as the tab's title.

How did you determine the received byte(s), packet sniffing?

fds wrote on Feb 10, 2021, 08:41:
However, the issue is only on this /s/ "Share" link, which I gather was statically generated one time.

No, the only truly static page is the old HTML archive index (and 3 more that aren't relevant here). And as noted, the frontpage is a static .html file served via the switch script.

Share links have been shortened for SEO purposes, but are executed (via Apache rewrite) by the board.pl script, action viewstory. This shows replacement characters for me as well.

fds wrote on Feb 10, 2021, 08:41:
If I click over to the comment viewer board.pl version, it is always correct. It's served as a proper Cyberpunk 2077 Expos0xC3A9…

For title element and story header yes, but most comment subjects show ?'s for me. Although those come out of a separate table/column, and were copied from the news story header when that was still an incorrect 1-byte Latin1 version of eacute, thus the ?'s are to be expected.

fds wrote on Feb 10, 2021, 08:41:
If I had to guess, not all statically generated pages got refreshed after you have fixed whatever was borked in the perl settings earlier, and some corrupted static pages remain.

No, that's not it then.

But thanks for helping to investigate.

Frans wrote on Feb 6, 2021, 05:29:
I was going to add one more weird observation about the replacement characters, but since last night I'm not getting them anymore during dozens of refreshes in multiple browsers. So perhaps the issue resolved itself just as mysteriously as it popped up...

Wishful thinking... the replacement mark returned for me too (some days ago but I didn't have time for this until now).

The earlier observation is that in a browser's dev tools (F12) in the Network panel, one can view the response headers and body of the GET request for the URL with the problem. In that response body, the accented e is indeed shown as such in <title> tag and news post header, even when the browser's main window renders it as the replacement character in those spots. But the dev tools renderer may behave differently from the main window in this respect, so I'm not drawing conclusions, it merely contributes to the weirdness of the problem.

It's been pointed out multiple times before, but you folks shouldn't bring up and discuss site issues under a news post. A General Discussion topic already existed for it at the time of this story, and yesterday I added a comprehensive update there. One fix for the front-page should address the malformed characters there, another issue doesn't seem to occur anymore.

But if you still notice malformed characters, please report in the GD topic.

Frans wrote on Feb 5, 2021, 14:50:
But I don't know the reason nor remedy for this yet; it's really weird that the browser only renders UTF-8 characters correctly and consistently if they're 3-byte ones (and then the 2-byte ones too), but not if a page includes only 2-byte characters, given that HTML and HTTPS instruct it to use that character set.

Yes, a workaround remedy may be to insure that the page always includes a 3-byte character somewhere, but I'd still like to understand the underlying problem if possible.

I was going to add one more weird observation about the replacement characters, but since last night I'm not getting them anymore during dozens of refreshes in multiple browsers. So perhaps the issue resolved itself just as mysteriously as it popped up...

This isn't about "Word formatted" characters but about all UTF-8 (non-ASCII) ones. The system has to process them as UTF-8 every step in a long chain:

• Blue writes a story in Frontpage, which is configured to UTF-8
• copies it into the Blammo admin script, which runs with Perl settings for UTF-8 processing
• it gets stored in MySQL, which along with its tables and string columns are set to UTF-8 (utf8_unicode_ci)
• the front-end script (with the same settings*) retrieves two days of news and writes the static .shtml files with UTF-8 characters
• the switch script serves the .shtml page with or w/o ads**
• alternatively, the board script (with the same settings*) does the same with news stories dynamically, for Share and Comment pages; ditto for the articles, logos, and other scripts
• all generated HTML5 starts with meta charset="utf-8"
• Apache sends all HTML over HTTPS to the browser with Content-Type: text/html; charset=utf-8
• the browser should render the received bytes as UTF-8 characters, but doesn't always

* Those I fixed early October, which I thought covered all bases, until the problem re-emerged three weeks ago.

** In an oversight, this script did not yet fully use Perl UTF-8 settings until this morning. So I believe the frontpage now correctly shows UTF-8 characters again. At least, in dozens of refreshes I haven't seen malformed "Word" quotes, ellipses, mdashes, etc. anymore. Please let me know if you catch them as yet.


Separately, we still have a problem with UTF-8 characters in single stories. In fact, the two Cyberpunk Exposé stories showing up in the popular threads box mid-January is how I first noticed it. And one of them is still erratic on the Share/Comments links:

• 1: Cyberpunk 2077 Development Debacle Exposé
• 2: Cyberpunk 2077 Exposé Follow-up

For Blue and me, on story 1 the headline and browser title bar always show the accented e (eacute): é.
But on #2, they erratically flip between the é and the question mark on a diamond background, i.e. the replacement character �.

What I noticed as a possibly relevant difference between the two stories is that é is a 2-byte character (C3 A9), and while in story 1 multiple 3-byte characters occur (e.g. mdash —, E2 80 94), story 2 has none. Adding an mdash character to story 2 too (in testing) causes it to no longer show the replacement character in dozens of refreshes.

But I don't know the reason nor remedy for this yet; it's really weird that the browser only renders UTF-8 characters correctly and consistently if they're 3-byte ones (and then the 2-byte ones too), but not if a page includes only 2-byte characters, given that HTML and HTTPS instruct it to use that character set.

Yes, a workaround remedy may be to insure that the page always includes a 3-byte character somewhere, but I'd still like to understand the underlying problem if possible.

Btw, I hardly spent any time on this until yesterday, because I initially hit a trying figure out what happened when, and because I was immersed in another programming project (non-BN, in fact I haven't done any real BN development since the smilies modal early November). But at least today there's some progress.

eRe4s3r wrote on Jan 31, 2021, 18:48:
Why is the quote window having this issue though? [...] And then sometimes it just works fine... it seems completely random even.

If I knew, I could (probably) resolve it.

eRe4s3r wrote on Jan 30, 2021, 12:35:
Well at least I can pinpoint it to many particular symbols

’ <-- causes issues
´ ` ' " <--- no issues issues too

I'm not sure it's related to quoting characters, rather than non-UTF8 ones. But I was pretty sure I covered all the bases there when I last worked on this a few months ago.

And either way, the erratic nature of this little problem is frustrating.

eRe4s3r wrote on Jan 30, 2021, 12:35:
Testing more it seems it's Firefox doing something stupid on first load.

Blue is seeing it in his primary browser Chrome too.

Not anymore.

Blue and I are seeing it occasionally, and I did a lot of investigation last week. But because the effect doesn't appear consistently -- it appears/disappears when refreshing the same page, and occurs on some pages of the same .pl script but not others -- I couldn't find the cause.

Blue wrote on Jan 10, 2021, 03:05:

The Flying Penguin wrote on Jan 9, 2021, 16:49:
Happy BDay Frans! Thanks for all the hard work on the BBS. The mobile browsing experience is so much better now.

Frans worked his tail off on the responsive/mobile conversion, but the project was the brainchild of Kxmode, who spearheaded its implementation.

Seconded, without Kxmode this project probably would still not have happened, and I am very grateful for everything I learned from him. The final responsive conversion (the smilies modal) didn't involve him anymore, but all this only happened because I could stand on his shoulders. I hope my grip sole boots didn't cause any bruises.

Thanks Blue and everyone for their kind words.

gpk_og wrote on Jan 8, 2021, 19:11:
Thanks for helping keep the awesomeness going, I can't believe it's been 25 years already. Holy crap I'm getting old (am?) .
Best way to start the day is with a good coffee and Blue's News, here's to another 25!

Just to clarify: the site is in its 25th year (since July 1996) but my active involvement started in October 1999. So a few more years to go until my 25th year.

Rhialto wrote on Dec 2, 2020, 13:12:
Do comments and forum posts get a permalink or something for when you want to link to a specific one?

Yes, the post number left of the subject has been linked to the individual post in threads mode for a decade+.

Acleacius wrote on Nov 4, 2020, 20:19:
Omg, hate the new Smiles, sorry. Just lost a full post because I tried to move the dang window it LOCKS too low.

It's not a window but a modal, which couldn't be moved anyway. But now you can.

Frans wrote on Nov 5, 2020, 04:21:
Don't know what to make of that, a modal is not a window and cannot be moved by Bootstrap. If this is a "request" to make it draggable, then I'd have to look into how to accomplish that.

Well, Blue liked the idea and I found some short jQuery code not requiring an additional library (like jQueryUI, which includes draggable support). So now you can grab the smilies modal by its header to move it around.

eRe4s3r wrote on Nov 7, 2020, 16:10:
Hi there, [ url = ] [ / url ] feature is kinda broken when the url is longer.. not even really THAT long...

Link that works
https://politics.theonion.com/jubilant-reaction-to-trump-defeat-quickly-soured-by-new-1845551327

Link that DOES NOT work within URL (remove spaces, you get "the word is too long"
Test

A counting discrepancy - fixed. Thanks.

Short quoting "q [ ] q" was processed before long "[ quote... /quote ]". Swapping that fixed it. Thanks.

Don't know what to make of that, a modal is not a window and cannot be moved by Bootstrap. If this is a "request" to make it draggable, then I'd have to look into how to accomplish that.