Added something to lyx, clicked reconfigure and you get the news that “reconfiguration failed….”.

Been there done that, got the errors and much sadness. So This post is a quick a)reminder to me what I did, b)some notes to help others fix/debug similar problems.

I’d just added these book links from editorium from the link for book layouts at https://wiki.lyx.org/Layouts/Layouts

Which provides new layout files ( LyXBook–Default.layout and LyXBook–Archaic.layout ) which have to be placed in /usr/share/lyx/layouts folder. Once done you need to tell lyx to reconfigure (Tools-Reconfigure). But at that point I got an error message like this:-

The system reconfiguration has failed.Default textclass is used but LyX maynot be able to work properly.Please reconfigure again if needed.

Although fairly obvious what had broken the setup was the new layout files, Ithough it would be useful to document how you can debug this type of problem.

Launch Lyx from command line so you can see error messages

This gave some clues.

Traceback (most recent call last):  
File "/usr/share/lyx/configure.py", line 1901, in <module>   
 ret = checkLatexConfig(lyx_check_config and LATEX, bool_docbook) 
File "/usr/share/lyx/configure.py", line 1422, in checkLatexConfig    
 for line in open(file, 'r', encoding='utf8').readlines():  
File "/usr/lib/python3.8/codecs.py", line 322, in decode    
 (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 845: invalid start byte
support/Systemcall.cpp (276): Systemcall: 'python3 -tt "/usr/share/lyx/configure.py" --binary-dir="/usr/bin/"' finished with exit code 1

The immediate clue is in one of the files there is something that was expected to be an utf-8 encoded character, but wasnt. Now need to find it…..We have only added two layout files so they are where the problem lies, but which line of which file?

Now just in case you have added lots of layout and want to identify the specific file you can modify the python script as explained at https://www.lyx.org/trac/ticket/11736

modify /usr/share/lyx/configure.py around line 1420 which looks like this
catline = ""        
for line in open(file, 'r', encoding='utf8').readlines()

add an extra line so it looks liek this instead. Its Python so be careful with those indents spaces vs tabs

catline = ""        
logger.info("class: " + classname)       
for line in open(file, 'r', encoding='utf8').readlines()

Then tell Lyx to reconfigure again, the difference being we now get told which file had the problem.

class: LyXBook--Default
Traceback (most recent call last):  
File "/usr/share/lyx/configure.py", line 1901, in <module>   
 ret = checkLatexConfig(lyx_check_config and LATEX, bool_docbook) 
File "/usr/share/lyx/configure.py", line 1422, in checkLatexConfig    
 for line in open(file, 'r', encoding='utf8').readlines():  
File "/usr/lib/python3.8/codecs.py", line 322, in decode    
 (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 845: invalid start byte
support/Systemcall.cpp (276): Systemcall: 'python3 -tt "/usr/share/lyx/configure.py" --binary-dir="/usr/bin/"' finished with exit code 1

Well that confimed a few things the file is LyXBook–Default, and the error related to unicode/utf-8.

So lets find where the non utf-8 characters are in the layout files.

For this need a little grep incantation and explained here

(as shown by Pablo Bianchi at https://stackoverflow.com/questions/29465612/how-to-detect-invalid-utf8-unicode-binary-in-a-text-file)

Which says

grep -axv '.*' LyXBook--Default.layout     

the arguments here are 
-a, --text: treats file as text, essentialy prevents grep to abort once finding an invalid byte sequence (not being utf8)    
-v, --invert-match: inverts the output showing lines not matched    
-x '.*' (--line-regexp): means to match a complete line consisting of any utf8 character.

So lets run the command

grep -axv '.*' LyXBook--Default.layout 
# LyXBook layouts, modules, and documentation copyright � 2013 by the Editorium. All rights reserved.
# LyXBook� is a trademark of the Editorium:

You can spot the non utf-8

And you can see which lines have something non utf-8 in them.

So you can now go and remove those characters, reconfigure Lyx and and all will be happiness….ish