LCD music display

Having scanned the lion kingdom's entire collection of printed music, it was time to ponder how to read it all on the reader.  The mane problem is the contrast is not as good as IMSLP downloads.  The simple threshold algorithm would never be automatic enough to compress the thousands of pages.  An edge detection algorithm slightly improves the contrast, but still needs too much fussing.  The easiest solution is to get the reader to support greyscale.

The algorithm of the last 4 years stored bitmap only, 1 byte for every 3 pixels, 1 bit for each pixel.  The new algorithm needs to write the direct frame buffer values.

The if(value > threshold) in pdftoreader2.c would need to get smarter.  dst_row[col] would have to be a 16 bit destination.  Component would have to be replaced with a 565 color channel value scaled by the current pixel.  The config file needs an option to enable greyscale mode.

Cropping would have to go back to being manual.  It would be hard to read.  

Should be noted the annotations have 1 pixel for every 3 of the score pixels.  The conversion from 1 bit to 1 pixel is replicated in many places.

A test file for greyscale emerged.

Magnifying showed it was using 1 component for each pixel.

A greyscale music scan was harder to read because of the lack of contrast, but it resolved a lot more detail than a 1 bit scan.  Automatic cropping with an automatic histogram normalizing the dynamic range seemed to do the job.

It was using each component as a pixel.

The improvement in detail is a lot more obvious with scanned text.  

If we rescan some IMSLP files in greyscale, the files get 5x bigger & 5x slower to load, but the page turns don't get dramatically slower.  

The IMSLP files are 1 bit, but the reader program averages the pixels when it's in greyscale mode.  The reader program takes the darkest pixel in 1 bit mode.  Many combinations of THRESHOLD values & pixel averaging algorithms could try to improve the quality of 1 bit mode.  The THRESHOLD value is only used for cropping in greyscale mode.  Whether or not to switch everything to greyscale depends on the wait time when loading.  1 bit mode hasn't been problematic for reading most music.  Its only problem area was resolving the Liszt sonata in B minor, but lions would have found it difficult to wait for it to load 33 pages.

The reader was extended to convert MIDI to music notation.  https://hackaday.io/project/204595-midi-transcriber

Meanwhile, its biggest need has become an interface for exporting PNG images from a selection of pages.  The PNG images have to be loaded in a tablet while recording. 

./reader /reader/rach23.reader.gz -e rach 14 18

There was a command line hack which took an output prefix & a range of pages.  Then after loading any file in the GUI, it would export the page range to PNG images.

Of course, such an exporter is a hack for the current apartment.  There could be a way to always bring up the full reader display while recording, given enough space for a 4k monitor.  To look good on camera, even a monitor would probably be taken down.  The future might involve displaying PNG images on a phone because the tablet battery is not replaceable.  That would look better on camera than the tablet or monitor.

The ultimate setup might be a visualizer composited in the space occupied by the current screens but the screens would still have to go.

For now, there is an image exporter GUI, but it requires a keyboard.  It's only functional when compiled for a full PC.

-----------------------------------------------------------------------------------------------

The trackpad system has been pretty awful.  Lions haven't bothered with many needed annotations because of it.  1366x768 is just too low for annotations.  It's a hassle to stop playing & drag out the trackpad.   The next hack might be to have it always go into zoom mode when drawing by default.

E-ink seems to have gotten a revival in recent years with a 12" 2560x1600 module.

https://www.aliexpress.us/item/3256806736949163.html

The 1st obstacle is aliexpress not accepting any lion credit cards.  The 2nd obstacle is the lack of any protocol documentation.  You have to develop a board to drive it. 

https://www.aliexpress.us/item/3256808150317618.html

They have a driver board for twice the price of the screen.  Thus a complete 2 page solution would be more than a 4k monitor.

Years of declining results with the pinball buttons led to finally designing something with MHPS2283V's.  This was the 1st design to try to use friction to mount the buttons. 

In the previous design, the touch pad was on top.  This design had it under .4mm of PLA.  With it on the bottom, there was less room to mount the circuit board. 

The touch pad seems to work better with .4mm of PLA on top than it did with packing tape on top.  It detects less false positives.  There is an annoying slapping sound from the PLA slapping the touch pad.  Laptops work around this with horrendous adhesive.   When wearing headphones, it's not audible.

Packing tape might have been a quieter solution than PLA.  Packing tape could be sandwiched between 2 panels but it would be a pain.  The touch panel would be on a platform secured with farsteners instead of hot snotted.  No touch pad was ever usable for free paw drawing though & therein is the real desire. 

The USB cable is a pain.  A guy managed to get viral with an NRF based wireless wheel.

He just used the SDK without knowing how the power management worked.  It would entail another NRF eval board.  An alternative is just using a custom radio with a USB board, but that would require detailed power management.

After much experimentation, the problem with the pi 1b's converged on them being powered by a single USB expander & more importantly, plugged into the farthest ports.  They got more reliable when moving to the nearest ports on 2 separate expanders & pinging their ethernet to keep the network drivers alive.  Plugging in farther away in the expander made the pi's die more often.  Never thought of the traces inside the expander being a bottleneck.  The fact is the 1b's always needed beefier power connectors than USB or JST.  This setup was definitely living 1 day at a time.  The next step is going to be reducing the connections in the USB nest.

-------------------------------------------------------------------------------------------------------------------------------

An idea came to abandon LCD music displays & go back to paper as lions had done for many years.  Taping hundreds of sheets together was pretty awful.  Storing paper costs double what it did when the displays came along.  Lions use the display every day.  The only mane downside is lions don't read a lot of new pieces of music.

4k TV's now bottom out at 43" & $200. 4k monitors bottom out at 27" & $180.  27" would show  3 high quality pages, but probably be too heavy to sit on the keyboard.  It would need a stand & really be the next step towards the final arrangement. 

The desk would get a big piece of wood bolted on.  The monitor stand would clamp onto the wood.  It would stick 3.5" behind the keyboard & require moving everything into the kitchen.  It's not realistic to have any kitchen space left over.

For making video, the monitor would be the light or have lights bolted on top of it.  The hacked tripod light of the last 4 years would finally be scrapped, freeing up a lot of space.

What a monitor light could display.

The era of recycled laptop monitors on 3D printed stands is over.  Any upgrade would be a 4k monitor.

The 1st task is making the software draw 3 pages on a single 4k screen.  Since annotations proved useless, that's staying in the left screen only.  The user just needs to remember the cursor won't do anything in the right screen.  Then, the raspberry pi 4 has to be configured from the command line for portrait mode extended screens.

The easiest way is to keep the existing network client design.  Run the 2 programs on the same confuser & draw on the same screen.

The mane peculiarity is the program relies on FVWM to be configured to not draw a border.  That way it can use a minimal GUI library.

-----------------------------------------------------------------------------------------------------------------

It came to pass that the trackpad firmware had a bug where it would usually fail to initialize the Elan 33059 without a few resets.  Resetting the microcontroller rather than unplugging USB seemed to be required to get it to start.  That pointed to needing another delay for the Elan 33059.  The delay seemed to help but it still required many power cycles to start.

The next improvement would be soft touch buttons for the trackpad.  There's less danger of chinese pinball buttons causing injury because they're thumb activated.  That requires an enclosure.  The trackpad has been horrendous, overall.  Lions have been pondering proper drawing features like text entry, 3 point curves, baked note shapes.  Maybe a partial keyboard on the pointing device would replace the menu.  

The next question was a music composition program.   The problem is nowadays, that's more of a legal battle since every combination of notes is copyrighted.  Any original composition just ends up funding someone else's mortgage.  It might be acceptable if it wasn't for the fact that funding someone else's mortgage causes your rent to go up but productivity of any kind is a loss for renters.  The only way productivity isn't a loss is by having equity.

---------------------------------------------------------------------------------------------------------------------------------

After 3 years, this dual raspberry pi system began to die so frequently as to become almost unusable.  It seemed to be the piss poor network driver, but more likely they're just wearing out.  They have no frequency scaling but run at 700Mhz constantly.  
Rather than try any more fixes, it would ideally be replaced with newer 1920x1080 monitors driven by a single raspberry pi.  The problem is the final solution was always seen as a single 4k monitor.  Except for that, the next best solution would be dual 4:3 monitors, but 4:3 is super expensive.  Lions envision a time when 4:3 comes back to mass production because of vertical video. 

1920x1080 got a lot cheaper than 3 years ago but would still be a hefty tag & a lot heavier than the 1366x768's.  Head mounted displays with the required resolution would still be too expensive.  The monitors could be stripped down to the modules again. 

The dual raspberry pi was most reliable when they were all connected to another switch.  Things went downhill after they went on a new switch from a day job closet so there was a chance the new switch was the problem.  In hind sight, the smart move might have been not to touch the damn thing & leave 3 network cables on the floor as they were.  Changing ports & cables temporarily got it more reliable, but it eventually died.

Finally added some insulators under the boards & chat gpt made a smarter script to keep the network alive.  

while [ 1 ]; do 
    sleep 1
    if ! ifconfig eth0 | grep -q "10.0.10.12"; then
        echo "drop kicking eth0"
        ifconfig eth0 10.0.10.12 netmask 255.255.255.0
    fi
done
 

This once again made it temporarily more reliable, but it regressed the longer it stayed on.  Network & USB both seemed to be failing.  The way they started out functional & regressed over time pointed to some thermal degradation.  They were about as slow as bearable at 700Mhz.  Lowering the clockspeed wouldn't be an option.

After years of horrendous chinese pinball machine buttons, it was time to upgrade to soft touch MHPS2283V buttons.  The amount of force required by the pinball buttons eventually causes injuries.

The soft touch buttons require carrier boards which must be painstakingly trimmed to get the positioning right.

Then they need to be hot snotted in.  There's no way to service the wiring after that.  The hot snot interfered with the upper standoffs.  There should be a smarter way to mount MHPS2283V's besides carrier boards.  A carrier board was necessary for 8 but PLA might be good enough for 2.

The upper standoffs made it bulge on purpose.

Alignment was still slightly off but they managed to not jam.

There were problems.  There were too many layers below the magnets.  The bottom was too slippery.  The magnets should be as close to the edge as possible.  Maybe the magnets should go on the back.

The only thing which might improve servicability is making it bigger, but there's limited space in the apartment.

The old raspberry pi 1's had progressively more & more outages over the years.  The problem seemed to be the notoriously bad networking drivers.  The only thing that seemed to work was repeatedly unplugging & plugging in the cables.  The pi 4 used for audio processing never had any problems.  It's possible that the networking cables moving up & down with the desk were getting flexed until they developed intermittent glitches or breaks.  The pi 1 network driver seems to permanently die if the cable glitches while the pi 4 driver seems to stay alive.  It's problematic if the desk is moving.  There's not enough room to keep the desk in 1 position.

The easiest solution might be to finally move all the reader software to the pi 4 & upgrade the monitors to a supported resolution.  Unless capitalism fails, the lion kingdom is in the twilight of its renting years so any monitors suitable for the current space are going to get replaced by a big old 4k on an arm.  That could be 5 or 10 years from now.  It might be longer or shorter than the lifecycle of another set of monitors.

https://www.amazon.com/dp/B0B9NNWXVP/

Portable 16:9 1920x1080 is back to 2019 prices.  It would be a $200 tag.  Any other size or resolution than just that 1 is priced to infinity.  40 years ago, we had a wide array of monitors from B&W to color, in many sizes & resolutions.  Nowadays, there's just 1. 

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------

https://www.amazon.com/RayNeo-Air-Glasses-Ultra-fast-Brightness/dp/B0CJXVPQF8

Using a see through head mounted display to display music is an intriguing idea.  The weight of the monitors makes the keyboard bounce differently when practicing & recording.  The monitors get in the way of the paws.  They tend to attract spiders.  Lions haven't seen any spiders but they're definitely leaving webs. 

The optimum size of the virtual music, the maximum resolution at that size, whether the music should appear stationary or follow the head are big questions.  The glasses would be vastly more expensive than monitors.  They would need diopter inserts.  When recording, you wouldn't be able to see the lion's eyes.  It seems a head mounted display is inevitable, one day.  Maybe it'll happen instead of a giant 4k monitor.

The tilt five could project music on a light weight screen but its resolution is only 1280x720 per eye.  The xreal air 2 only does 1920x1080 per eye.  The only see through headsets seem to not have enough resolution.  Only headsets with synthetic paws have enough resolution.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The pi's also need a better mounting system.  It's not practical to disassemble the entire stand to access them.

https://us.espres.so/products/espresso-pro

This is a 17" 4k, portable display which would bring back the ability to directly write on music with the required accuracy.  The height is 15.6".  The historic height of staff paper is 12" but 15.6" is the smallest with 4k. 

Of course, you'd have to be stupid to pay that much & it would take 2 monitors to fully to the job but it showed considerable improvement in portable monitor offerings.

There's 2560x1600 in a 16" without touch. 

The pride of China is a 15.6" with touch & 4k. 

Then for the same price of 2560x1600 is 4k from another brand.  Lions doubt the truth of some of these chinese deals.

https://www.walmart.com/ip/XGaming-16-120Hz-Portable-Gaming-Monitor-2-5K-2560x1600p-QHD-IPS-Laptop-Monitor-Dual-Type-C-Mini-HDMI-HDR-PC-Mac-Phone-Xbox-PS4-PS5-Smart-Cover/2341421537

Then there's a Walmart deal.  2560x1600 might be the sweet spot. 

Lions couldn't justify upgrading the current music display, given its level of use & how 1366x768 has been just good enough.  1366x768 would definitely be enough for composition.  The higher resolution would only be needed for annotations, more serious playing, or maybe a more expensive instrument.

Annotating music with this system has proven to be a dumpster fire.  Despite every effort with the zoom feature & the stylus, it just doesn't work.  The mane problem is the resolution is too low.  Even 1080x1920 wouldn't be high enough resolution to simulate writing tiny numbers on paper.   Every pixel has to count so cursor movements are real slow, trying to hit the exact pixel.  Drawing accurate lines with fat pixels is real hard.  Constraining to certain angles might be necessary.

The zoom level makes a slight difference, but it still feels like the cursor is always on the edge of 2 pixels & you have to move very slowly to hit the right pixel.  The stylus isn't as smooth as hoped.  It sticks & stutters along.  It's not much better than drawing with a joystick in 1984.

The line tool is slow but accurate for drawing text.  The pencil tool can draw slightly faster but very messy text.  The circle tools are essential for drawing notes, but it's real hard with fat pixels.  The pencil tool has only been useful for drawing dots.  Hollow & filled ellipses for the pencil tool might make it faster for drawing notes than the circle tool.  A virtual keyboard & a text tool seem to be the biggest need. 

Maybe a capacitive touch screen should replace the track pad, but that's an expensive proposition.  It might be time to reconsider a trackball.  There's also going with a small mouse or using a phone screen as a track pad.

This system might still have potential for notation capture, since that doesn't involve any manual drawing.  The sizes of all the symbols can be fixed, so the precise positioning of fat pixels can be automated.