Silly software wishlist

Lions originally wanted everything to have a GUI.  In old age, lions just want as much computing power devoted to the task as possible instead of the interface.  After writing a video editor, lions have accumulated many usages of ffmpeg & mplayer  from the command line.  The mane use of mplayer is extracting decompressed audio.

mplayer -ao pcm:file=audio.wav -vc null -vo null 

But specifying the output file doesn't work.  ffmpeg has a usage which extracts decompressed audio, but this works for fewer sources.

ffmpeg -i <input file> -vn <output file.wav>

Then there's extracting the audio from a movie without any transcoding.

ffmpeg -i <input file> -vn -acodec copy <output file>.mp4

Extract the video from a movie without any transcoding:

ffmpeg -i <input file> -an -vcodec copy <output file>.mp4

The other thing lions use ffmpeg for is transcoding video from a variable framerate to a fixed framerate file.  This degrades the quality, but if the specified bitrate is higher than the input bitrate, it'll encode as high as the input bitrate.

ffmpeg -i <input file> -c:v mpeg4 -vb 5000k -an <output file>

Lions use ffmpeg to encode from an uncompressed source, for debugging machine vision programs.  This isn't as fast as linking ffmpeg & calling the functions with a memory resident frame buffer but it's simple.

ffmpeg -y -f rawvideo -y -pix_fmt bgr24 -r 15 -s:v <width>x<height> -i - -c:v mpeg4 -vb 5000k -an <output file>

It would be a lot simpler if these most common ffmpeg usages were condensed into yet another wrapper program.

https://cdn.hackaday.io/files/1380505906331200/countfreq.c


If you're into knowing the relative frequency of something to very high precision, oscilloscopes of lion kingdom vintage are pretty bad, but you can transfer a waveform to a computer & run this program on it.  Newer oscilloscopes can definitely do better.  It can't get absolute frequencies to very high precision because that depends on the oscilloscope crystal.

Some sample waveforms show 2 surface mount 32.768 crystals running at 32766 & a 3rd through hole crystal at 32768.  They can also be affected by the probing method.  As lions discovered from creating Heroineclock II, crystals can vary by a lot.  .01% precision for a resistor is high precision, but for a crystal is way out.

It gives the standard deviation of the wavelength in samples, useful for knowing how stable the signal is.  In the sample data, the through hole appears much more stable than the surface mount, but the standard deviation also depends on the samplerate.  Frequency is not as affected by samplerate as the standard deviation.  The surface mount standard deviation drops to 20 when sampled at 100 megsample.

Best results have come from counting zero crossings with a debounce delay, rather than counting peaks & troughs with hysteresis.  The best debounce depends on how noisy the signal is.  The zero crossings are where the waveform is steepest, so the time of the zero crossing can be known to the most precision.  Slight errors in where each cycle begins can create huge errors in the frequency measurement & the standard deviation.

Of course, to just see if 2 crystals are way off, load them in an audio editor & play them back.

Dreamed about software being used to determine the most efficient way to pack items into boxes. It was based on either analyzing video of the products or 3D scanning everything that entered the warehouse. Amazon.com was using it in fulfillment centers. All the humans did was move items from the shelf to a box the computer selected & where in the box the computer said to put them.

Lions don't believe any such system exists. Workers currently pick the box by eye & pack it by eye. Random variations in packaging might make it impractical to prescan items when they enter the warehouse, so workers would need to scan the dimensions of every item in an order, then the computer would pick the box.

Making the scanning process fast & transparent would be a good machine vision problem.

Then, there was this craptastic program. The lion kingdom long dreamed of a way to manage a phone's filesystem in a browser instead of ADB, to download & upload files without ADB.

After many years, the desire finally hit critical mass, there was enough time without commuting, & the deed was done. It's 25 year old HTML, no javascript, no AJAX, no jquery, no node.js, no react.js, no angular.js, & would never get a job, but it does the job. There's no security. It's fast when it works & easy to manetain, like all web applications 25 years ago.

A key difference with past lion programs is this one doesn't run in the background.  Once it's cleared from the app manager, the server is gone.

There are a few limitations.

It can't create symbolic links, since lions have never needed a symbolic link on a phone.

The phone has to be rebooted occasionally, as Android has memory leaks which cause the network performance to get slower & slower.

It can't write to external storage.  Android has dropped any support for writing to an SD card from a user program.  It can only read.

There was originally a desire to sort by size & date, but the kind of file operations lions have done so far have been too simple to justify it.

https://github.com/heroineworshiper/webphone

------------------------------------------------------------

11/15/24

It grew quite a bit since 2020, gaining an edit function, sorting. The next frontier is a dual copy function. 1 copy function would copy the selected files to the same directory with different names. Another copy function would copy the selected files to another directory.

At least on the  lion kingdom's obsolete 4Ghz 8 core Ryzen, it's

% ./random |catbps

main 33: 1154458213 bits/sec
main 33: 1155342065 bits/sec
main 33: 1158795636 bits/sec
main 33: 1161013450 bits/sec
main 33: 1153026045 bits/sec
main 33: 960231704 bits/sec
main 33: 1077391216 bits/sec

From a phone through ADB, it's

% adb shell /data/random | catbps 
main 33: 9080670 bits/sec
main 33: 5169716 bits/sec
main 33: 4291419 bits/sec
main 33: 4482882 bits/sec
main 33: 8428983 bits/sec
main 33: 8613113 bits/sec
main 33: 9701913 bits/sec

Native linux on a laptop to ethernet:

root@heroine:/root% netcat -l -p 1234 | catbps
main 41: 1309 bits/sec total_bytes=1024
main 41: 939087920 bits/sec total_bytes=117504400
main 41: 940861506 bits/sec total_bytes=235229696
main 41: 941440127 bits/sec total_bytes=353027392
main 41: 941604955 bits/sec total_bytes=470845712
main 41: 941564611 bits/sec total_bytes=588658984
main 41: 941590569 bits/sec total_bytes=706475504
main 41: 941602141 bits/sec total_bytes=824293472
main 41: 941494025 bits/sec total_bytes=942097912
main 41: 941467236 bits/sec total_bytes=1059899000

Linux on a virtual machine to ethernet:

root@heroine:/root% netcat -l -p 1234 | catbps
main 41: 5333 bits/sec total_bytes=1024
main 41: 14750500 bits/sec total_bytes=1850368
main 41: 16163477 bits/sec total_bytes=3876864
main 41: 10647144 bits/sec total_bytes=5209088
main 41: 9818954 bits/sec total_bytes=6438912
main 41: 10589858 bits/sec total_bytes=7763968
main 41: 22450331 bits/sec total_bytes=10575872
main 41: 12099960 bits/sec total_bytes=12091392
main 41: 15158875 bits/sec total_bytes=13991936
main 41: 15942073 bits/sec total_bytes=15986688
main 41: 14217453 bits/sec total_bytes=17767424

The random program just prints random ASCII characters as fast as possible.  Recognizing that the rand function has a finite speed, it precalculates a fixed buffer & resend it.


Obviously, adb doesn't do any compression.  The catbps program just measures how fast data is coming from stdin but doesn't write it to stdout.   These are basic needs lions have had for a while.  Next comes hacking ADB to improve this bandwidth.  You can't hack the ADB server on a normal phone, but this is a special phone.

https://cdn.hackaday.io/files/1380505906331200/catbps.c

https://cdn.hackaday.io/files/1380505906331200/random.c


We all want to know how fast our networks are, so these programs can be run with netcat.

netcat -l -p 1234 | catbps

random | netcat 10.0.0.14 1234

Management of the audio interface is notoriously bad in Goog Chrome, so we've all adopted the same ritual in the 15 years since the Goog's IPO.

killall -9 google-chrome-stable chrome chrome-sandbox

Then, you have to run ipcs & ipcrm to delete all the IPC objects it leaves behind, one by one.  Goog employees don't know how to properly delete shared memory when their programs crash.  For those of us not completely focused on passing interview questions with leetcoding, the secret is to call 

shmctl(shmid, IPC_RMID, 0);

right after calling 

shmat(shmid, NULL, 0);

But anyways, it was finally time to automate this process.

https://cdn.hackaday.io/files/1380505906331200/ipcclean.c

This program just deletes every IPC object.  It has to be run as root.

.  

It's a problem we've all come to know in the last 10 years.  We want to download a video from a site other than the goo tubes, so we start up the network panel in Chrome & load the page.  Then, we look for the largest files being downloaded & files ending in .ts, .mp4, or .webm.  The problem is, it downloads a bunch of small files containing a number like 00002. 

It's a pain in the ass segmented video.  The trick is finding the 1st & last segment numbers.  Seek to the end of the video to get the last segment number.  Seek to the start of the video to get the 1st segment number.  After being stymied by these files for years, the lion kingdom finally decided to automate the process with a dead simple program.

https://cdn.hackaday.io/files/1380505906331200/segments.c

It just searches the URL for the starting number & replaces that block of text with every number until the ending number. If the result is a stream format like .ts, the files can all be concatenated.  If it's a .mp4, the files have to be loaded into an editor.  Currently, it downloads base 10 numbers.  It could be hacked to download hex numbers.

While it works for some PBS videos & some local TV stations, it doesn't work for discovery channel.  They use some kind of encryption.

https://cdn.hackaday.io/files/1380505906331200/udptest2.c

This program began life many years ago when the lion kingdom was trying to speed up ethernet by using UDP instead of TCP.  Lions quickly realized this was useless, so it became manely a way of testing UDP connections through large file transfers.  It also works with IPV6, but lions only tested it with numeric IPV6 addresses.

The general idea is it sends a certain number of packets at a time & waits for an ACK.  There's no windowing or asychronous communication.  It's useful for half duplex like radios.  It uses the same port for receiving on the source & destination.  This was necessary to get around a certain firewall.  The packet size is hard coded.

The journey begins by starting the receiver on the destination.

udptest -6 -r 1234

This waits for connections on IPV6, all interfaces, port 1234.  Then the transmitter starts on the source.

udptest -6 2017:0000:0000:0000:7690:5000:0000:0001 1234 file.txt

This sends a file over IPV6.  By default, it sends 1 packet & waits for an ACK.  Leave out the -6 to use IPV4.

udptest -r 1234

udptest 10.0.0.14 1234 file.txt

To increase the number of packets per ACK, provide the -n option to the receiver.

udptest -r -n 8 1234

udptest 10.0.0.14 1234 file.txt

This only works with the very latest cameras, manely the EOS RP.  The journey begins by compiling the v4l2 loopback device.

https://github.com/umlaeute/v4l2loopback

insmod v4l2loopback.ko

This creates a new /dev/video device which can be simultaneously written & read.  The same device is used for writing & reading.

Then, with the camera turned on, run

gphoto2 --stdout --capture-movie | ffmpeg -i - -vcodec rawvideo -pix_fmt rgb24 -threads 0 -f v4l2 /dev/video1

Take note of the frame size.  The same frame size must be used for capturing.

Then, in the case of Cinelerra, recording must be configured to use uncompressed RGB, Video4Linux2, /dev/video1, 960x640, 15fps, frames to buffer in device: 2.  If the framerate is too high, it'll studder.   If it's too low, it'll studder less but be bearable.  The kernel driver has a bug in its double buffering, as most double buffering newbies tend to have.

Who knows what settings will get invoked with browser based video conferencing.  An ideal video conferencing system would have gphoto2 send frames to a program that performed some realtime effects, defishing, sharpening, color correction.  

ffmpeg can perform defishing with some convoluted command line parameters.

-vf "lenscorrection=cx=0.5:cy=0.5:k1=-0.1:k2=-0.1"

It uses nearest neighbor scaling & looks absolutely horrid.  1 of the lion kingdom's 2 computers only captures video with the camera in still photo mode.  Another captures video with the camera in video mode.  It's either a gphoto2 version problem or a USB problem.  In still photo mode, the autofocus is disabled.

After a while, the lion kingdom decided the EOS RP wasn't worth the trouble & went back to an old fashioned webcam for video conferencing.

IPV6 has been everywhere & nowhere for 20 years now, but for the lion kingdom's day jobs, it's already everywhere.  The problem is the shortpaw notation for the 16 byte addresses is a disaster.  After using it for a long time, lions have only started to intuitively recognize the :: to mean stuff with 0's until the left & right side of the :: are justified.  /128 represents the number of most significant bits comprising the subnet mask.

Despite some intuition, lions never type even the shortpaw notation but copy & paste all the addresses, so they might as well all be 8 hex numbers.  If IPV6 ever becomes manestream, most animals are going to write the addresses as 8 16 bit hex numbers.  The decision was made to provide a utility for converting the shortpaw notation to a full address.  

This is most useful for writing IPV6 software.  Linux has the getaddrinfo function which accepts shortpaw notated IPV6 addresses, but lions most often have to program embedded systems which require a full 16 byte address.

https://cdn.hackaday.io/files/1380505906331200/parseipv6.c

Let's parse some addresses.

% ./parseipv6 2017::7690:5000:0:2/64

2017:0000:0000:0000:7690:5000:0000:0002 prefix=64

% ./parseipv6 2017::

2017:0000:0000:0000:0000:0000:0000:0000 prefix=128

% ./parseipv6 ::ffff:0:0:0/96
0000:0000:0000:0000:ffff:0000:0000:0000 prefix=96

./parseipv6 2600:1:9a28:46f3:0:35:f8b4:f601/64
2600:0001:9a28:46f3:0000:0035:f8b4:f601 prefix=64

Despite so many decades of being just around the corner & supposedly allowing every device to have a fixed address, IPV6 addresses are still being assigned dynamically like the good old days.