LinuxQuestions.org - [SOLVED] slackware-current: hp-setup from hplip-3.23.12-x86_64-4 throws UnicodeDecodeError after selecting printer

- Slackware (https://www.linuxquestions.org/questions/slackware-14/)

- - slackware-current: hp-setup from hplip-3.23.12-x86_64-4 throws UnicodeDecodeError after selecting printer (https://www.linuxquestions.org/questions/slackware-14/slackware-current-hp-setup-from-hplip-3-23-12-x86_64-4-throws-unicodedecodeerror-after-selecting-printer-4175736494/)

slackware-current: hp-setup from hplip-3.23.12-x86_64-4 throws UnicodeDecodeError after selecting printer

When I try to add a printer using hp-setup to add a network printer I've successfully added in the past I get the error below.

Steps:

Run hp-setup from cli
Select "Network/Ethernet/Wireless network (direct connection or JetDirect)
Select my printer
Click "Next" followed by the crash

I see that there were patches applied to -4 to fix unicode errors, but they don't seem to work for whatever this is. I assume this is a system misconfiguration problem on my end, but I don't know where to start checking things.

Things I've tried:

downgrading hplip to 3.23.12-x86_64-1 through 3 (same error)
downgrading to hplip-3.20.6-x86_64-7 (ModuleNotFoundError: No module named 'cupsext')
reinstalling hplip

The trace:

Code:

HP Linux Imaging and Printing System (ver. 3.23.12)

Printer/Fax Setup Utility ver. 9.0



Copyright (c) 2001-18 HP Development Company, LP

This software comes with ABSOLUTELY NO WARRANTY.

This is free software, and you are welcome to distribute it

under certain conditions. See COPYING file for more details.



Searching... (bus=net, timeout=5, ttl=4, search=(None) desc=0, method=mdns)

...clipped some stuff here...

hp-setup[17284]: debug: Found 1 devices

hp-setup[17284]: debug: Cache miss: officejet_pro_9010_series

hp-setup[17284]: debug: Reading file: /usr/share/hplip/data/models/models.dat

hp-setup[17284]: debug: Searching for section [officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat

hp-setup[17284]: debug: Found section [officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat

hp-setup[17284]: debug: Cache miss: hp_officejet_pro_9010_series

hp-setup[17284]: debug: Reading file: /usr/share/hplip/data/models/models.dat

hp-setup[17284]: debug: Searching for section [hp_officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat

hp-setup[17284]: debug: Found section [hp_officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat

hp-setup[17284]: debug: Unable to open file /var/lib/hp/hplip.state for reading.

hp-setup[17284]: debug: {}

hp-setup[17284]: debug: HP_OfficeJet_Pro_9010

hp-setup[17284]: debug: (CUPS 1.2.x) Getting list of PPDs using CUPS_GET_PPDS...

hp-setup[17284]: debug: CUPS PPD base path = /usr/share/cups/model

hp-setup[17284]: debug: Foomatic PPD base path = /usr/share/ppd

Traceback (most recent call last):

  File "/usr/share/hplip/ui5/setupdialog.py", line 1395, in NextButton_clicked

    self.showAddPrinterPage()

  File "/usr/share/hplip/ui5/setupdialog.py", line 709, in showAddPrinterPage

    self.findPrinterPPD()

  File "/usr/share/hplip/ui5/setupdialog.py", line 798, in findPrinterPPD

    self.print_ppd = cups.getPPDFile2(

                    ^^^^^^^^^^^^^^^^^

  File "/usr/share/hplip/prnt/cups.py", line 540, in getPPDFile2

    family_class=getFamilyClassName(model)

                ^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/share/hplip/prnt/cups.py", line 178, in getFamilyClassName

    dict=m.read_all_files(False)

        ^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/share/hplip/base/models.py", line 456, in read_all_files

    self.read_section(self.released_dat)

  File "/usr/share/hplip/base/models.py", line 487, in read_section

    line = fd.readline()

          ^^^^^^^^^^^^^

  File "/usr/lib64/python3.11/encodings/ascii.py", line 26, in decode

    return codecs.ascii_decode(input, self.errors)[0]

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 913: ordinal not in range(128)

Aborted

For posterity, I get the same error running

Code:

$ hp-setup -ldebug -i

but patching /usr/share/hplip/base/models.py:481 to force UTF-8 lets me at least add the printer. I'm assuming something in my environment is telling python my default encoding is ASCII? Interested to know what the correct fix would be...

Code:

$ diff -u models.py.orig models.py

--- models.py.orig      2024-04-27 19:58:49.276940079 -0400

+++ models.py  2024-04-27 19:59:41.413799589 -0400

@@ -478,7 +478,7 @@

            cache = self.__cache

 

        try:

-            fd = open(filename)

+            fd = open(filename, 'r', encoding="utf-8")

        except IOError as e:

            log.error("I/O Error: %s (%s)" % (filename, e.strerror))

            return False

I have also encountered the same problem and look forward to solving it.

I look forward to Pat could see this issue and providing a solution.

Quote:

Originally Posted by zalois (Post 6498632)

I look forward to Pat could see this issue and providing a solution.

I do not own a working HP printer and will never buy another one. So I'm probably not the one to fix it (I did try to see if I could reproduce the issue here, but could not).

Quote:

Originally Posted by volkerdi (Post 6498636)

I do not own a working HP printer and will never buy another one. So I'm probably not the one to fix it (I did try to see if I could reproduce the issue here, but could not).

:) thank you!

Quote:

Originally Posted by zalois (Post 6498639)

:) thank you!

this is the output of mine:

Show the output of command 'locale'.

Quote:

Originally Posted by Petri Kaukasoina (Post 6498657)

Show the output of command 'locale'.

this is the output

Quote:

Originally Posted by zalois (Post 6498682)

this is the output

looks fine.

hp-setup

HP Linux Imaging and Printing System (ver. 3.23.12)
Printer/Fax Setup Utility ver. 9.0

Copyright (c) 2001-18 HP Development Company, LP
This software comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to distribute it
under certain conditions. See COPYING file for more details.

Searching... (bus=net, timeout=5, ttl=4, search=(None) desc=0, method=mdns)
Traceback (most recent call last):
File "/usr/share/hplip/ui5/setupdialog.py", line 1395, in NextButton_clicked
self.showAddPrinterPage()
File "/usr/share/hplip/ui5/setupdialog.py", line 709, in showAddPrinterPage
self.findPrinterPPD()
File "/usr/share/hplip/ui5/setupdialog.py", line 790, in findPrinterPPD
ppdName = cups.getPpdName(self.model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/hplip/prnt/cups.py", line 196, in getPpdName
dict=m.read_all_files(False)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/hplip/base/models.py", line 456, in read_all_files
self.read_section(self.released_dat)
File "/usr/share/hplip/base/models.py", line 487, in read_section
line = fd.readline()
^^^^^^^^^^^^^
File "/usr/lib64/python3.11/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 913: ordinal not in range(128)
fish: Job 1, 'hp-setup' terminated by signal SIGABRT (Abort)

Similar locale here.

Code:

$ locale                                                                                                                                                            in bash at 09:01:41

LANG=en_US.UTF-8

LC_CTYPE="en_US.UTF-8"

LC_NUMERIC="en_US.UTF-8"

LC_TIME="en_US.UTF-8"

LC_COLLATE=C

LC_MONETARY="en_US.UTF-8"

LC_MESSAGES="en_US.UTF-8"

LC_PAPER="en_US.UTF-8"

LC_NAME="en_US.UTF-8"

LC_ADDRESS="en_US.UTF-8"

LC_TELEPHONE="en_US.UTF-8"

LC_MEASUREMENT="en_US.UTF-8"

LC_IDENTIFICATION="en_US.UTF-8"

LC_ALL=

I just tried exporting LC_ALL=C.UTF-8, which didn't work, but switching to LC_ALL=C did work!!

Alternatively, exporting PYTHONUTF8=1 also lets it work.

Relevant PEPs:

So what do most people set LC_* to?

Quote:

Originally Posted by pdags (Post 6498685)

So what do most people set LC_* to?

I have not yet moved to utf8. I only set LC_CTYPE to an old fashioned 8-bit (non-unicode) locale, the others are unset (default):

Code:

$ locale

LANG=

LC_CTYPE=fi_FI

LC_NUMERIC="POSIX"

LC_TIME="POSIX"

LC_COLLATE="POSIX"

LC_MONETARY="POSIX"

LC_MESSAGES="POSIX"

LC_PAPER="POSIX"

LC_NAME="POSIX"

LC_ADDRESS="POSIX"

LC_TELEPHONE="POSIX"

LC_MEASUREMENT="POSIX"

LC_IDENTIFICATION="POSIX"

LC_ALL=

Looks like there is a single non-ascii character in /usr/share/hplip/data/models/models.dat (looks like a unicode space https://unicodeplus.com/U+2002)

Code:

diff -u /usr/share/hplip/data/models/models.dat.orig /usr/share/hplip/data/models/models.dat                                                                      in bash at 10:26:11

--- /usr/share/hplip/data/models/models.dat.orig        2024-04-28 10:23:49.885064906 -0400

+++ /usr/share/hplip/data/models/models.dat    2024-04-28 10:25:10.436847862 -0400

@@ -78088,7 +78088,7 @@

 io-support=14

 job-storage=0

 linefeed-cal-type=0

-model1=HP Color LaserJet Enterprise MFP X58045dn 

+model1=HP Color LaserJet Enterprise MFP X58045dn

 model2=HP Color LaserJet Enterprise MFP X58045

 monitor-type=0

 panel-check-type=0

If I remove it, everything also seems to work fine, but I have no idea if that will break other things for the long suffering HP users :(
I'd think any impact would be limited to people with a HP Color LaserJet Enterprise MFP X58045dn, but who knows.

Looking at the output of locale in post#9, there are a couple of oddities:
LC_TIME=en_GB.UTF-8 when I would expect LC_TIME="en_US.UTF-8" and
LC_TELEPHONE=zh_CN.UTF-8 when I would expect LC_TELEPHONE="en_US.UTF-8".

The screenshot shows the fish shell running in a konsole window. Is the output from locale run in konsole the same? Apparently the fish shell expects to be passed those variables.