LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 02-22-2022, 01:25 AM   #1
ZlatkO
Member
 
Registered: Oct 2020
Location: Bad Vöslau, Austria
Distribution: Slackware. What else?
Posts: 38

Rep: Reputation: 26
Question Unicode/ISO-Latin/non-US-ASCII logging broken?


Look what I found in /var/log/debug recently (this is what it looks like when viewed with less, and $LESSCHARSET="utf-8"):
Code:
Feb 21 21:54:15 next gcr-prompter[2368]: GLib-GIO: _g_io_module_get_default: Found default implementation gvfs (GDaemonVfs) for <E2>M-^@M-^Xgio-vfs<E2>M-^@M-^Y
Now, is it just me, or wouldn't that line actually be supposed to look rather like this?
Code:
Feb 21 21:54:15 next gcr-prompter[2368]: GLib-GIO: _g_io_module_get_default: Found default implementation gvfs (GDaemonVfs) for ‘gio-vfs’
Before I go and hunt this down any further, I'd like to know if this is a general Slack-15.0 problem, or a problem with my specific system. So, if you could you please verify the following on your own Slack-14.2 and/or Slack-15.0 systems and share your findings here, I'd be very grateful! Now on to the facts ... on a newly installed Slack64-15.0 system, syslogd seems to have a problem logging non-US-ASCII characters. I've been using Slackware since 1997 now, converted all my systems (terminals, fonts, X, filenames, file contents, ...) from ISO/Latin to Unicode on the fly back in the early 2000s, and this has never been a problem so far:
Code:
[root@next:~]# logger -p debug -s -t test "iso/latin - öäüÖÄÜ߀¢§"; logger -p debug -s -t test "unicode - –…‘’•"; tail -2 /var/log/debug
<15>Feb 22 06:22:19 test: iso/latin - öäüÖÄÜ߀¢§
<15>Feb 22 06:22:19 test: unicode - –…‘’•
Feb 22 06:22:19 next test: iso/latin - öäü�M-^V�M-^D�M-^\�M-^_�M-^B�¢§
Feb 22 06:22:19 next test: unicode - �M-^@M-^S�M-^@��M-^@M-^X�M-^@M-^Y�M-^@�
When viewed with less, it looks like this:
Code:
Feb 22 06:22:19 next test: iso/latin - öäü<C3>M-^V<C3>M-^D<C3>M-^\<C3>M-^_<E2>M-^B<AC>¢§
Feb 22 06:22:19 next test: unicode - <E2>M-^@M-^S<E2>M-^@<A6><E2>M-^@M-^X<E2>M-^@M-^Y<E2>M-^@<A2>
On a Slack-14.2 system, it Just Works™ as expected:
Code:
[root@disclosure:~]# logger -p debug -s -t test "iso/latin - öäüÖÄÜ߀¢§"; logger -p debug -s -t test "unicode - –…‘’•"; tail -2 /var/log/debug
<15>Feb 22 06:22:07 test: iso/latin - öäüÖÄÜ߀¢§
<15>Feb 22 06:22:07 test: unicode - –…‘’•
Feb 22 06:22:07 disclosure test: iso/latin - öäüÖÄÜ߀¢§
Feb 22 06:22:07 disclosure test: unicode - –…‘’•
A few random observations that might or might not make a difference: all my 14.2 systems are 32bit, the newly installed 15.0 one is 64bit. The characters are displayed fine on the terminal, and I used the -s flag on logger to see if it receives & handles the characters properly, which does seem to be the case. Logging lower-case umlauts ä/ö/ü work fine, upper-case umlauts Ä/Ö/Ü are broken. Logging the Euro sign € is broken, cent ¢ works. Unicode filenames also work as expected. When viewing/editing other files with Unicode content with less/vim/mcview, everything's fine as well. My locales are set properly, although syslogd does not seem to make use of any of it (this is the same on both 14.2 and 15.0):
Code:
[root@next:~]# locale
LANG=en_US.UTF-8
LC_CTYPE=de_AT.UTF-8
LC_NUMERIC=de_AT.UTF-8
LC_TIME=de_AT.UTF-8
LC_COLLATE=C
LC_MONETARY=de_AT.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_PAPER=de_AT.UTF-8
LC_NAME=de_AT.UTF-8
LC_ADDRESS=de_AT.UTF-8
LC_TELEPHONE=de_AT.UTF-8
LC_MEASUREMENT=de_AT.UTF-8
LC_IDENTIFICATION=de_AT.UTF-8
LC_ALL=
[root@next:~]# xargs -0 -n1 < /proc/$(pgrep syslogd)/environ | sort
BOOT_IMAGE=/vmlinuz-generic-5.15.19
CONSOLE=/dev/console
HOME=/
INIT_VERSION=sysvinit-3.01
PATH=/sbin:/usr/sbin:/bin:/usr/bin
PREVLEVEL=N
PWD=/
RUNLEVEL=4
SHELL=/bin/sh
SHLVL=3
TERM=linux
_=/usr/sbin/syslogd
Edit: just tested: stopping & starting /etc/rc.d/rc.syslog from a terminal with all the proper locales set doesn't make a difference. The locale settings do show up in syslogd's environment, but the logging is still broken.

I have a 14.2 test system which I intend to upgrade to 15.0 Really Soon Now™, so I can check whether non-ASCII-logging breaks on upgrading as well. But until then, the whole thing begs the Grand Universal Question: WTF?!

Last edited by ZlatkO; 02-22-2022 at 01:32 AM. Reason: additional test
 
Old 02-22-2022, 03:26 AM   #2
ZlatkO
Member
 
Registered: Oct 2020
Location: Bad Vöslau, Austria
Distribution: Slackware. What else?
Posts: 38

Original Poster
Rep: Reputation: 26
Update: I used the original Slack 15.0 sysklogd.SlackBuild on my 14.2 test system and built and upgraded to sysklogd-2.3.0. Result: logging is now broken on 14.2/i686 as well.

Code:
[root@d2:/usr/src/slackware/a/sysklogd]# logger -p debug -s -t test "iso/latin - öäüÖÄÜ߀¢§"; logger -p debug -s -t test "unicode - –…‘’•"; tail -2 /var/log/debug
<15>Feb 22 09:46:33 test: iso/latin - öäüÖÄÜ߀¢§
<15>Feb 22 09:46:33 test: unicode - –…‘’•
Feb 22 09:46:33 d2 test: iso/latin - öäüÖÄÜ߀¢§
Feb 22 09:46:33 d2 test: unicode - –…‘’•
[root@d2:/usr/src/slackware/a/sysklogd]# bash sysklogd.SlackBuild | tee sysklogd.SlackBuild.log
sysklogd-2.3.0/
[...]
Slackware package /tmp/sysklogd-2.3.0-i586-1.txz created.

[root@d2:/usr/src/slackware/a/sysklogd]# upgradepkg /tmp/sysklogd-2.3.0-i586-1.txz
+==============================================================================
| Upgrading sysklogd-1.5.1-i586-2 package using ./sysklogd-2.3.0-i586-1.txz
+==============================================================================
Pre-installing package sysklogd-2.3.0-i586-1...
Removing package: sysklogd-1.5.1-i586-2-upgraded-2022-02-22,09:51:09
[...]
Package sysklogd-1.5.1-i586-2 upgraded with new package ./sysklogd-2.3.0-i586-1.txz.
[root@d2:/usr/src/slackware/a/sysklogd]# /etc/rc.d/rc.syslog restart
[root@d2:/usr/src/slackware/a/sysklogd]# logger -p debug -s -t test "iso/latin - öäüÖÄÜ߀¢§"; logger -p debug -s -t test "unicode - –…‘’•"; tail -2 /var/log/debug
<15>Feb 22 09:56:46 test: iso/latin - öäüÖÄÜ߀¢§
<15>Feb 22 09:56:46 test: unicode - –…‘’•
Feb 22 09:56:46 d2 test: iso/latin - öäü�M-^V�M-^D�M-^\�M-^_�M-^B�¢§
Feb 22 09:56:46 d2 test: unicode - �M-^@M-^S�M-^@��M-^@M-^X�M-^@M-^Y�M-^@�
Reverting to sysklogd-1.5.1-i586-2 fixes things up again, as expected. I haven't tested the other way round yet (ie. building the 14.2 sources on 15.0), but I have a very strong idea about the result.
 
1 members found this post helpful.
Old 02-22-2022, 08:53 AM   #3
ZlatkO
Member
 
Registered: Oct 2020
Location: Bad Vöslau, Austria
Distribution: Slackware. What else?
Posts: 38

Original Poster
Rep: Reputation: 26
Okay, so it looks like the move from infodrom.org's sysklogd to troglobit's sysklogd is where things broke. troglobit's 1.6 release (the first after the fork from infodrome) is the last one where non-ASCII logging works properly (for me, at least), and 2.0 (which was almost a complete rewrite) and upwards munches & garbles non-ASCII characters on-disk. In case you're interested and/or this affects you as well, I've opened an upstream bug report - let's see how it works out.
 
3 members found this post helpful.
Old 03-07-2022, 11:25 AM   #4
ZlatkO
Member
 
Registered: Oct 2020
Location: Bad Vöslau, Austria
Distribution: Slackware. What else?
Posts: 38

Original Poster
Rep: Reputation: 26
Update: yup, looks like this is really a bug. Joachim has suggested to add a workaround from the FreeBSD version of sysklogd (which his 2.0 rewrite is based on) to his next release, 2.4.0, as an intermediate "fix". So if you have any opinion on this matter (I can't believe that I'm the only Slack 15.0 user who has an issue with garbled non-ASCII characters in his logs, am I? FWIW, this breaks at least logrotate and logcheck, which I think are quite essential tools), feel free to comment on the buf report over at GitHub (see link in my previous post).
 
2 members found this post helpful.
Old 06-21-2022, 01:38 PM   #5
ZlatkO
Member
 
Registered: Oct 2020
Location: Bad Vöslau, Austria
Distribution: Slackware. What else?
Posts: 38

Original Poster
Rep: Reputation: 26
Lightbulb I can haz 2.4.0?

I just found out that sysklogd-2.4.0 has been released, including the fix/workaround to allow proper logging of 8-bit data. Can we have this version in Slackware-15.0, pretty pretty please?
 
1 members found this post helpful.
Old 06-21-2022, 04:23 PM   #6
henca
Member
 
Registered: Aug 2007
Location: Linköping, Sweden
Distribution: Slackware
Posts: 982

Rep: Reputation: 668Reputation: 668Reputation: 668Reputation: 668Reputation: 668Reputation: 668
Unless it is a security update I would not hold my breath waiting for a new version of some package to come into a stable version of Slackware.

The easiest way for you to get version 2.4.0 into your Slackware 15 installations is probably to download the source of 2.4.0, compile and install it yourself. Best is probably to first remove the existing package of sysklogd. To use the slackbuild script for sysklogd included in the source directory is probably an easy way to get a Slackware package of the newer sysklogd.

More likely this version will find its way into Slackware current. Once it does, you might be able to take the slackbuild script and sources from current to compile a package for Slackware 15.

regards Henrik
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
non-ascii characters in bash script and unicode igor.R Linux - Newbie 31 12-29-2012 03:45 AM
How do I add support to type in Unicode or non-QWERTY ASCII letters. robofighter Linux From Scratch 2 05-12-2011 05:59 AM
FOSS translation tools? English to Spanish, Spanish to Latin, Latin to Icelandic, etc brynjarh Linux - Software 0 06-19-2006 12:34 AM
How to detect non ascii filenames from an application which doesn't support UNICODE pankajtakawale Solaris / OpenSolaris 0 02-05-2004 06:31 AM
How to detect non ascii filenames from an application which doesn't support UNICODE ( pankajtakawale Solaris / OpenSolaris 0 02-05-2004 06:28 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 06:23 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration