Lyx Configuration Fails When new Layouts Added

Added something to lyx, clicked reconfigure and you get the news that “reconfiguration failed….”.

Been there done that, got the errors and much sadness. So This post is a quick a)reminder to me what I did, b)some notes to help others fix/debug similar problems.

I’d just added these book links from editorium from the link for book layouts at https://wiki.lyx.org/Layouts/Layouts

Which provides new layout files ( LyXBook–Default.layout and LyXBook–Archaic.layout ) which have to be placed in /usr/share/lyx/layouts folder. Once done you need to tell lyx to reconfigure (Tools-Reconfigure). But at that point I got an error message like this:-


The system reconfiguration has failed.Default textclass is used but LyX maynot be able to work properly.Please reconfigure again if needed.

Although fairly obvious what had broken the setup was the new layout files, Ithough it would be useful to document how you can debug this type of problem.

Launch Lyx from command line so you can see error messages

This gave some clues.

Traceback (most recent call last):  
File "/usr/share/lyx/configure.py", line 1901, in <module>   
 ret = checkLatexConfig(lyx_check_config and LATEX, bool_docbook) 
File "/usr/share/lyx/configure.py", line 1422, in checkLatexConfig    
 for line in open(file, 'r', encoding='utf8').readlines():  
File "/usr/lib/python3.8/codecs.py", line 322, in decode    
 (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 845: invalid start byte
support/Systemcall.cpp (276): Systemcall: 'python3 -tt "/usr/share/lyx/configure.py" --binary-dir="/usr/bin/"' finished with exit code 1

The immediate clue is in one of the files there is something that was expected to be an utf-8 encoded character, but wasnt. Now need to find it…..We have only added two layout files so they are where the problem lies, but which line of which file?

Now just in case you have added lots of layout and want to identify the specific file you can modify the python script as explained at https://www.lyx.org/trac/ticket/11736

modify /usr/share/lyx/configure.py around line 1420 which looks like this
catline = ""        
for line in open(file, 'r', encoding='utf8').readlines()

add an extra line so it looks liek this instead. Its Python so be careful with those indents spaces vs tabs

catline = ""        
logger.info("class: " + classname)       
for line in open(file, 'r', encoding='utf8').readlines()

Then tell Lyx to reconfigure again, the difference being we now get told which file had the problem.

class: LyXBook--Default
Traceback (most recent call last):  
File "/usr/share/lyx/configure.py", line 1901, in <module>   
 ret = checkLatexConfig(lyx_check_config and LATEX, bool_docbook) 
File "/usr/share/lyx/configure.py", line 1422, in checkLatexConfig    
 for line in open(file, 'r', encoding='utf8').readlines():  
File "/usr/lib/python3.8/codecs.py", line 322, in decode    
 (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa9 in position 845: invalid start byte
support/Systemcall.cpp (276): Systemcall: 'python3 -tt "/usr/share/lyx/configure.py" --binary-dir="/usr/bin/"' finished with exit code 1

Well that confimed a few things the file is LyXBook–Default, and the error related to unicode/utf-8.

So lets find where the non utf-8 characters are in the layout files.

For this need a little grep incantation and explained here

(as shown by Pablo Bianchi at https://stackoverflow.com/questions/29465612/how-to-detect-invalid-utf8-unicode-binary-in-a-text-file)

Which says

grep -axv '.*' LyXBook--Default.layout     

the arguments here are 
-a, --text: treats file as text, essentialy prevents grep to abort once finding an invalid byte sequence (not being utf8)    
-v, --invert-match: inverts the output showing lines not matched    
-x '.*' (--line-regexp): means to match a complete line consisting of any utf8 character.

So lets run the command

grep -axv '.*' LyXBook--Default.layout 
# LyXBook layouts, modules, and documentation copyright � 2013 by the Editorium. All rights reserved.
# LyXBook� is a trademark of the Editorium:

You can spot the non utf-8

And you can see which lines have something non utf-8 in them.

So you can now go and remove those characters, reconfigure Lyx and and all will be happiness….ish

Posted in Uncategorized | Tagged , | Leave a comment

OpenCV Function exploring and Image Stabilisation for DICOM

The open CV library www.opencv.org provides many useful functions for image processing. But when it can be difficult and slow to learn the best parameter values and the optimum order to apply them. My first application of opencv was to count the flock sizes of birds and I wanted to optimise the operations for enhancing the image prior to applying the counting mechanism.

For the next project I wanted to be able to experiment “interactively” with the order of the processes and also to convert grayscale images to a 3D representation. (the project was to find an slightly embossed area on a rusty sign on a canal aqueduct).

So I wrote apython application to facilitate this and a version of it is shown below.

( For anyone wanting to learn about opencv an excellent resource is https://www.pyimagesearch.com/ )

The ‘pipeline’ of processes is shown on the left and their order can be changed and they can be enabled or disabled via the checkbox. The process panel (in blue above) show the image being input to a processing stage and the output from that stage. The large image shows the result after the image has been processed by all the enabled stages of the pipeline. The 3d image shows the grayscale values converted to heights.

I also wanted a testing environment for future android projects and so decided to try a java implementation. The current version is shown below and its operation is the same as the python version shown above.

The UI of the JAVA version showing effect of canny edge detection

The next addition was to use template matching to provide image stabilization of a series of jpeg images in which an object was not always in the same location in the photo but for comparison needed to be overlaid.

Image Stabilisation

The image processing pipeline is use to improve contrast of the image. A section of the enhanced image is then used as a template which is is searched for in each image of the image set. The co-ordinates of the located item are then used to extract the selection from the ORIGINAL image set and saved as a new file.The enhanced images and template are only used to locate the region of interest.

Medical Application DICOM Image stabilisation

Many medical imaging modalities save their resultant images as a sequence of images in the DICOM format.

In some image sequence the item of interest moves due to movement of the patient or their organs for example due to breathing. The work-around can be for each image to be examined and processed manual to select and crop the item of interest. This is time consuming as such images may be a set of hundred of images.

To assist some medical colleagues I modified the code to process dicom series. The specific scenario here is wanting to identify and extract the kidney in the sequence shown. This was to enable co-registration of the images for use by another diagnostic tool.

The constrained matcher shows the selected reference template (in a green rectangle). The output of the filter shows a white rectangle around the matched area in the test image, the large image on the right shows a zoom view.

The filter can either show the matched region (show in first section of the video) or as the extracted rectangle on its its own (as shown later in the video).

The matching method can also be changed via the dropdown in the filter.

To “help” the matcher a “contraint area” has been selected (the blue rectangle) and this limits where in the input image the matcher is limited to searching. There are other subtleties involved in this process such as selecting which “slice” of an image has the best shape to use as a template and means of filtering the relevant dicom positions.

The next version really needs some deep learning enhancement 🙂

Example of stabilising image

A download of the application will be available in the next few days.

Posted in android, coding, computing, dicom, imaging, JAVA, matplotlib, opencv, pandas, python, Uncategorized | Tagged , | Leave a comment

Replacing Switches on Boss Multi-effects

As another weekend project during lockdown I resurrected an old BOSS ME multi-effects unit. An ME30, described in one forum post as “the worst sounding pedal ever”

It was one of the early digital effects units but with a bit of care and tweaking is OK, especially for portable practice purposes.

Once back in use it lasted a day before one of the footswitches failed. So time to fix it, there are plenty of resources online for the Boss service manual(link to manual). The manual also lists various self tests for the pedal and setup scope traces etc.

Finding the exact replacement switches was less successfull, but I found some that are of comparable size and easily available. These would likely fit other generations of Boss effects.For anyone wanting to do refurb and ME30 the process is relatively painless.

  • Order new switches
  • Dismantle
  • Desolder old switches
  • Fit new switches
  • Reassemble
  • Rrrrrrrrrrrrrock, or whatever

The switches I used were from RS Components and made by Omron.

Yellow Plunger Tactile Switch, SPST-NO 50 mA @ 24 V dc 3mm RS Stock No.718-2411 Cost 12p Each!!!!!! The originals had a soft rubber actuator which had a bit of “give” but in the underside of the pedal there is also a squashy rubber buffer. If this worries you then there are switches with smaller actuators to which you could attach a soft top.

The only problem part of the project was one of the switches is obscured by the screening plate that covers the some of the electronics. You could unsolder it, but as the switch is so close to one edge you can/could bend it out of the way slighting (aka a bit of a bodge or a pragmatic solution depending on your view point). I opted bending it out the way. The pictures below show the old/new switches and the screening plate.Have fun.

The Screen Plate bent to give access to the switch terminals
The New Omron Switches next to one of the original switches

Posted in electronics, guitar | Tagged , | Leave a comment

Sending UDP Packets with Phonegap and the In App Browser

To send some instructions to a server process I needed to send UDP packets from an Android device. Normally I would “go native” and java/kotlin but for this one I tried phonegap. This is a reminder of a couple of tweaks needed.

You need this package:-

cordova-plugin-chrome-apps-sockets-udp

ie cordova plugin add cordova-plugin-chrome-apps-sockets-udp

I also wanted to check a web page so used the cordova “in app browser

That worked on older Android but on on newer versions gave the error net::ERR_CLEARTEXT_NOT_PERMITTED .

To fix that needs tweaks in the project config.xml needed the xml android namespace adding as below

<widget id="com.whatever.app" version="1.0.0" 
xmlns=”http://www.w3.org/ns/widgets” 
xmlns:android=”http://schemas.android.com/apk/res/android"
xmlns:gap="http://phonegap.com/ns/1.0">

and in the platform stanza

<platform name="android">
<edit-config file="app/src/main/AndroidManifest.xml" mode="merge"   target="/manifest/application">
  <application android:usesCleartextTraffic="true" />
</edit-config>
........rest of the platform entry
</platform>

For more info on this at stackoverflow see here

Posted in android, coding, computing, Uncategorized | Tagged , | Leave a comment

Effect of Resistor in Parallel with Pot

Following on from the post on the capacitor bleed circuit, a comparison in the behaviour of the pot with and without the resistor bypass.

The diagram below shows the voltage out from a potentiometer as it is turned..and the effect of a resistor in parallel with one end of the pot and the wiper.

For example at half rotation, rather than the output being at 50% its around 64%.

And a reminder, these are showing comparisons against a linear taper pot.

If you want to know more about modelling pots and changing the taper see http://qucs.sourceforge.net/docs/report/potentiometer.pdf and an example of qucs with modelling a potentiometer here. For more general “how to use sweep parameters see https://steemit.com/utopian-io/@thinkingmind/qucs-circuit-simulator-or-how-to-use-parameter-sweep-in-simulation

Posted in coding, computing, education, guitar, music, qucs, simulation | Tagged , | Leave a comment

Guitar Bleed Capacitor Vs RC Network

Many guitars have a “bleed” capacitor across the volume potentiometer so that as the volume is reduced the high frequencies are not as attentuated as much.Companies such as Fender also use an RC network to give more bass as the volume is reduced.

Typical combination are explained in this article https://www.fender.com/articles/tech-talk/how-a-treble-bleed-circuit-can-affect-your-tone So just out of interest (and as I wanted a demo circuit to use with qucs The Quite Universal Circuit Simulator ) I did some circuit simulation.

I did some frequency plots comparing the single capacitor configuration vs the new Fender network. I’ve shown plots for 3 settings of volume. These are settings of the pot from the point of view of resistance from the wiper to ground. So 5 is the pot at halfway FROM an electrical point of view, 9 at 90%, 1 at 10%

It should be remembered that volume/tone control pots are likely to be audio or logarithmic taper rather than linear. So halfway through the rotation wont be 50% signal level.

If you need an explantion of log/linear pots have a look at http://www.resistorguide.com/potentiometer-taper/

As you would (probably) expect the difference are most obvious when the volume pot is at the lower setting showing a significant amount of high frequency signal getting through.

The overal signal level is higher for a given pot position (other than at max) on the Fender network due to the effect of the top of the pot and the wiper being in parallel with the resistors in the bleed circuit (a total of 170K). So when the pot is at 10% between wiper and ground ie 25K the remaining 225K is in parallel with that 170K yielding about 96K so that effective pot position is nearer 20% (ish)…and yes I could have accounted for that in the circuit but didnt, but if you want to there is a link to the qucs file below.

Anyway long and short of all this, is yes the new Fender combination will seem less toppy….But we knew that at the start 🙂

The qucs file can be downloaded here

The circuits in Qucs

Posted in coding, computing, guitar, music, Uncategorized | Tagged , | Leave a comment

Bike Lidar and Social Distancing Alarm.

Time is strange at the moment, but weekends are still a good time for project building. So over a few weekends this project evolved and turned into two projects.

The main project came about as I wanted to a) try a VL53l1X time of flight/Lidar device b)write some  Bluetooth LE code.

The result was this board (breadboard prototype shown in pictures) and an mobile application.

As a standalone board the “Ninelocks Bike Lidar” (all projects need a name ) can sound an alarm when an object is within a certain distance. But to make it more adjustable I gave it an Bluetooth LE GATT profile so it can be configured from a phone or other Bluetooth LE device. That meant writing an phone application as well.

The phone app shown here the phone gets updates from the Lidar board and logs the current distance along with timestamp and GPS location data and for eye candy shows the most recent reading on a graph.

The phone can also sound an alarm if anything gets closer than the distance set as the trigger threshold. There is a ‘video’ below showing the device and application at work with some suggested uses.

In its bike guise it could potentially warn you of vehicles behind you (ok only up to 4 meters away and its still being tested but….) as a social distance alarm it can live on your desk to warn of approaching colleagues/ potential infection vectors.

You could maybe use it to check if you wheel have wobble or any other lidar/distance/graph  type things you can think of. Code will eventually make it to github.

The Application

There is a video (aka vid-ay-oh as they say in stoneybridge)

Video of Lidar Board

Posted in arduino, coding, computing, cycling, music | Tagged , , , | Leave a comment

Midi System Exclusive Control of Arduino Application

Most recent project has been an midi foot controller based around an Arduino/PJRC Teensy and a companion windows application to configure it.

Details and source over at github

The configuration tool in Windows(Works in wine on Linux)

Sysex Format

The sysex format is in this format.
====================================================
   System Exclusive Configuration Messages
=====================================================

Configuration of the project is by Midi System Exclusive MEssages, the format of which are now described

The midi header counting from byte 0

0 0xF0 Start of sysex
1 0x7D indicating research/school/not  a manufacturer
2 0xNN 4 Character key , the same in the arduino application
3 0xNN
4 0xNN
5 0xNN
6 0xMM A device ID, you choose identify your arduino project
7 0xMM A Command 
.
. Variable number of bytes depending on message
.
. 0xF7 End of SysEx marker

The 4 character code is set in the arduino source and can be configured in the windows pedal manager (see its documentaion )

=====================================================
  Request arduino send out its current configuration
=====================================================

0 0xF0 Start of sysex
1 0x7D indicating research/school/not  a manufacturer
2 0xNN 4 Character key , the same in the arduino application
3 0xNN
4 0xNN
5 0xNN
6 0xMM A device ID, you choose identify your arduino project
7 0x02 Command 0x01 is send config
8 0xf0 End of SysEx marker


On receipt of correctly formatted message the Arduino should reply

 0 0xF0 Start of sysex
 1 0x7D indicating research/school/not  a manufacturer
 2 0xNN 4 Character key , the same as in the arduino application
 3 0xNN
 4 0xNN
 5 0xNN
 6 0xMM A device ID, you choose to identify your arduino project
 7 0x02 indicates this is response to a "give me your configuration" request 
	
Posted in arduino, coding, computing, midi, teensy, Uncategorized | Leave a comment