slackware-current: hp-setup from hplip-3.23.12-x86_64-4 throws UnicodeDecodeError after selecting printer

pdags · 04-27-2024, 06:26 PM

When I try to add a printer using hp-setup to add a network printer I've successfully added in the past I get the error below.

Steps:

Run hp-setup from cli
Select "Network/Ethernet/Wireless network (direct connection or JetDirect)
Select my printer
Click "Next" followed by the crash

I see that there were patches applied to -4 to fix unicode errors, but they don't seem to work for whatever this is. I assume this is a system misconfiguration problem on my end, but I don't know where to start checking things.

Things I've tried:

downgrading hplip to 3.23.12-x86_64-1 through 3 (same error)
downgrading to hplip-3.20.6-x86_64-7 (ModuleNotFoundError: No module named 'cupsext')
reinstalling hplip

The trace:

Code:

HP Linux Imaging and Printing System (ver. 3.23.12)
Printer/Fax Setup Utility ver. 9.0

Copyright (c) 2001-18 HP Development Company, LP
This software comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to distribute it
under certain conditions. See COPYING file for more details.

Searching... (bus=net, timeout=5, ttl=4, search=(None) desc=0, method=mdns)
...clipped some stuff here...
hp-setup[17284]: debug: Found 1 devices
hp-setup[17284]: debug: Cache miss: officejet_pro_9010_series
hp-setup[17284]: debug: Reading file: /usr/share/hplip/data/models/models.dat
hp-setup[17284]: debug: Searching for section [officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat
hp-setup[17284]: debug: Found section [officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat
hp-setup[17284]: debug: Cache miss: hp_officejet_pro_9010_series
hp-setup[17284]: debug: Reading file: /usr/share/hplip/data/models/models.dat
hp-setup[17284]: debug: Searching for section [hp_officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat
hp-setup[17284]: debug: Found section [hp_officejet_pro_9010_series] in file /usr/share/hplip/data/models/models.dat
hp-setup[17284]: debug: Unable to open file /var/lib/hp/hplip.state for reading.
hp-setup[17284]: debug: {}
hp-setup[17284]: debug: HP_OfficeJet_Pro_9010
hp-setup[17284]: debug: (CUPS 1.2.x) Getting list of PPDs using CUPS_GET_PPDS...
hp-setup[17284]: debug: CUPS PPD base path = /usr/share/cups/model
hp-setup[17284]: debug: Foomatic PPD base path = /usr/share/ppd
Traceback (most recent call last):
  File "/usr/share/hplip/ui5/setupdialog.py", line 1395, in NextButton_clicked
    self.showAddPrinterPage()
  File "/usr/share/hplip/ui5/setupdialog.py", line 709, in showAddPrinterPage
    self.findPrinterPPD()
  File "/usr/share/hplip/ui5/setupdialog.py", line 798, in findPrinterPPD
    self.print_ppd = cups.getPPDFile2(
                     ^^^^^^^^^^^^^^^^^
  File "/usr/share/hplip/prnt/cups.py", line 540, in getPPDFile2
    family_class=getFamilyClassName(model)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/hplip/prnt/cups.py", line 178, in getFamilyClassName
    dict=m.read_all_files(False)
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/hplip/base/models.py", line 456, in read_all_files
    self.read_section(self.released_dat)
  File "/usr/share/hplip/base/models.py", line 487, in read_section
    line = fd.readline()
           ^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 913: ordinal not in range(128)
Aborted

pdags · 04-27-2024, 07:07 PM

For posterity, I get the same error running

Code:

$ hp-setup -ldebug -i

but patching /usr/share/hplip/base/models.py:481 to force UTF-8 lets me at least add the printer. I'm assuming something in my environment is telling python my default encoding is ASCII? Interested to know what the correct fix would be...

Code:

$ diff -u models.py.orig models.py
--- models.py.orig      2024-04-27 19:58:49.276940079 -0400
+++ models.py   2024-04-27 19:59:41.413799589 -0400
@@ -478,7 +478,7 @@
             cache = self.__cache
 
         try:
-            fd = open(filename)
+            fd = open(filename, 'r', encoding="utf-8")
         except IOError as e:
             log.error("I/O Error: %s (%s)" % (filename, e.strerror))
             return False

zalois · 04-27-2024, 08:19 PM

I have also encountered the same problem and look forward to solving it.

zalois · 04-27-2024, 09:04 PM

I look forward to Pat could see this issue and providing a solution.

volkerdi · 04-27-2024, 09:39 PM

Quote:

Originally Posted by zalois

I look forward to Pat could see this issue and providing a solution.

I do not own a working HP printer and will never buy another one. So I'm probably not the one to fix it (I did try to see if I could reproduce the issue here, but could not).

zalois · 04-27-2024, 09:44 PM

Quote:

Originally Posted by volkerdi

I do not own a working HP printer and will never buy another one. So I'm probably not the one to fix it (I did try to see if I could reproduce the issue here, but could not).

thank you!

zalois · 04-28-2024, 02:45 AM

Quote:

Originally Posted by zalois

thank you!

this is the output of mine:

Petri Kaukasoina · 04-28-2024, 03:06 AM

Show the output of command 'locale'.

zalois · 04-28-2024, 07:58 AM

Quote:

Originally Posted by Petri Kaukasoina

Show the output of command 'locale'.

this is the output

Petri Kaukasoina · 04-28-2024, 08:09 AM

Quote:

Originally Posted by zalois

this is the output

looks fine.

zalois · 04-28-2024, 08:16 AM

hp-setup

HP Linux Imaging and Printing System (ver. 3.23.12)
Printer/Fax Setup Utility ver. 9.0

Copyright (c) 2001-18 HP Development Company, LP
This software comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to distribute it
under certain conditions. See COPYING file for more details.

Searching... (bus=net, timeout=5, ttl=4, search=(None) desc=0, method=mdns)
Traceback (most recent call last):
File "/usr/share/hplip/ui5/setupdialog.py", line 1395, in NextButton_clicked
self.showAddPrinterPage()
File "/usr/share/hplip/ui5/setupdialog.py", line 709, in showAddPrinterPage
self.findPrinterPPD()
File "/usr/share/hplip/ui5/setupdialog.py", line 790, in findPrinterPPD
ppdName = cups.getPpdName(self.model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/hplip/prnt/cups.py", line 196, in getPpdName
dict=m.read_all_files(False)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/hplip/base/models.py", line 456, in read_all_files
self.read_section(self.released_dat)
File "/usr/share/hplip/base/models.py", line 487, in read_section
line = fd.readline()
^^^^^^^^^^^^^
File "/usr/lib64/python3.11/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 913: ordinal not in range(128)
fish: Job 1, 'hp-setup' terminated by signal SIGABRT (Abort)

pdags · 04-28-2024, 08:22 AM

Similar locale here.

Code:

$ locale                                                                                                                                                            in bash at 09:01:41
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=C
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

I just tried exporting LC_ALL=C.UTF-8, which didn't work, but switching to LC_ALL=C did work!!

Alternatively, exporting PYTHONUTF8=1 also lets it work.

Relevant PEPs:

So what do most people set LC_* to?

Petri Kaukasoina · 04-28-2024, 08:58 AM

Quote:

Originally Posted by pdags

So what do most people set LC_* to?

I have not yet moved to utf8. I only set LC_CTYPE to an old fashioned 8-bit (non-unicode) locale, the others are unset (default):

Code:

$ locale
LANG=
LC_CTYPE=fi_FI
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

pdags · 04-28-2024, 09:36 AM

Looks like there is a single non-ascii character in /usr/share/hplip/data/models/models.dat (looks like a unicode space https://unicodeplus.com/U+2002)

Code:

diff -u /usr/share/hplip/data/models/models.dat.orig /usr/share/hplip/data/models/models.dat                                                                      in bash at 10:26:11
--- /usr/share/hplip/data/models/models.dat.orig        2024-04-28 10:23:49.885064906 -0400
+++ /usr/share/hplip/data/models/models.dat     2024-04-28 10:25:10.436847862 -0400
@@ -78088,7 +78088,7 @@
 io-support=14
 job-storage=0
 linefeed-cal-type=0
-model1=HP Color LaserJet Enterprise MFP X58045dn 
+model1=HP Color LaserJet Enterprise MFP X58045dn
 model2=HP Color LaserJet Enterprise MFP X58045
 monitor-type=0
 panel-check-type=0

If I remove it, everything also seems to work fine, but I have no idea if that will break other things for the long suffering HP users

I'd think any impact would be limited to people with a HP Color LaserJet Enterprise MFP X58045dn, but who knows.

allend · 04-28-2024, 10:31 AM

Looking at the output of locale in post#9, there are a couple of oddities:
LC_TIME=en_GB.UTF-8 when I would expect LC_TIME="en_US.UTF-8" and
LC_TELEPHONE=zh_CN.UTF-8 when I would expect LC_TELEPHONE="en_US.UTF-8".

The screenshot shows the fish shell running in a konsole window. Is the output from locale run in konsole the same? Apparently the fish shell expects to be passed those variables.