Modern HP5N

After my log entry, star date 12345 yesterday, about my idea to simply inject PJL code at the beginning and the end of the stream, Ken Yap, again, gave some valuable pointers on what to look for.

He told me to look at the filters in cups, which I did and I found a massive amount of potential there. Examples of using PJL with PCL looked like what I wanted to try was perfectly possible. Here is a link to a site where a lot of examples are given : Using your own filters to print with cups

Looking into it further, I realized that a lot of examples would only work on older CUPS versions. The way filters work on the latest version is often somewhat different and the various degrees of up-to-date documents make it a bit confusing at times. Luckily, while trying to figure out how data is routed through CUPS (I have use cups for decades, but never dove in so deep), I figured out that the PPD file for the 5N held the target filter that outputs the data that the printer can read. In this case it was rastertogutenprint5.3 which is an binary executable.

Also in this folder /usr/lib/cups/filters/ were a number of wrapper scripts for various filters, most of them for legacy purposes, so I copied the code ( from rastertopclm ) of a wrapper file, put it in a script and after some trial and error, I got the wrapper file, which I set as the filter in the HP5's PPD file to pass though the data and get a good print.

#!/bin/sh

# Wrapper filter for PJL injection

echo "DEBUG: rastertopclm argv[$#] = $@" >&2
echo "DEBUG: PPD: $PPD" >&2

if [ -n "$6" ]; then
    exec <"$6"
fi

OUTFORMAT=PCLM $CUPS_SERVERBIN/filter/rastertogutenprint.5.3 "$1" "$2" "$3" "$4" "$5"

Bare in mind that I have no clue what I'm doing here, and the subsequent experiment will attest to that fact. After this script neatly passed the data from the wrapper to the filter, I thought to go for it and try and combine some of the stuff from the earlier link, for what I suspect were examples for cups < 2.0. So I took this command :

echo -en "\033%-12345X@PJL RDYMSG = Printing\n\033E"

And I just pasted it in there. It did not work! But it did do something. The print job was a success and the result was 1 page with "-en" on it and the second with the actually print on it. If it put this line after the OUTFORMAT line, the last page would be the "-en" page and unsurprisingly, two lines before and after resulted in 3 pages! "-en" - "Hello World" - "-en".

Without the "-en" flags, there is no effect at all. Perhaps the PJL arrived correctly but did not set the message, so I put this in :

echo "\033%-12345X@PJL INFO STATUS\n\033E"

Hoping to get a first page with the output of the INFO STATUS command.. No such page.

 
I guess it is a start, but I am still a bit in the dark by ECHO is behaving that like and frankly how the whole thing works. I have a basic understanding, but many key details and mechanisms elude me.

On another note, this wrapper could be a better way of turning on power to the printer, so even if I am completely mistaken, something rather good has come from this.

Thanks to Ken Yap for pointing me in this direction. Often a small nudge is what gets balls rolling, or printer drums in this case.

EDIT : Just tried to power up the printer from the wrapper with this code :

/home/pi/automation/./hs100.sh -i 192.168.1.4 on  > /dev/null
sleep 30s

The sleep command makes the spooler wait for the printer. Sometimes the job was lost without it. Of course this needs a bit more work with a check if the TP-Link device is already on. In which case, nothing has to be done.

While this little group effort has given some interesting insights into 90's printer protocols, the practical side of things has yet to be completely satisfied! While we have prodded our ancient laser-printer tanks in all sorts of interesting ways, I at least am still not able to determine by a shell script if the printer is actually done.

Last thing we tried was to determine is power-save mode was on via a PJL command that gets the current status which includes the display text. For me this text was always the "processing job" text, so that was completely useless.

I did notice one odd behavior which was when I set the "ready message" by the PJL command: @PJL RDYMSG DISPLAY = "Some Text" , the status would not return the job message, but the new message. In itself, this is also not very helpful, but then I started to thing how CUPS talks to the printer.

Effectively, CUPS does not do a whole lot different than we did via Telnet or Netcat. It opens up a connection to the printer via port 9100 and essentially just dumps the job data there. After that is done, CUPS really could not care less as the printer will handle the rest.

Here is my latest idea to abuse PJL to get this working. What if we could attach a PJL command before the data stream and afterwards as well? The stream is just another bunch on commands that are executed mostly in a serial manner.

Suppose we'd start the data with setting a RDYMSG that reads "printing". Then the data is send and at the end of it, after the last command that has anything to do with the document itself, we set RDYMSG to something like "Bazinga".

Assuming all commands are executed in serial and the last command will always signal the end of the job, using the Netcat method to poll status would only resolve "Bazinga" if the job is actually done. Any fault would cease operation so jams and paper out scenarios won't cause premature power-off.

Anybody out there with experience in messing with CUPS printer drivers?

Over the last couple of days, there has been a lot of tinkering going on. Trying several things and exchanging ideas with user Instruct, tutor, train... and with some help from Ken Yap we dove into the wonderful world that is PJL. A HP developed printer language that governs just about every aspect of printing.

The premise was that if we could read the display status while printing, we'd know when the job was done. This would mean no complex hacky scripts counting jobs and pages going through CUPS.

Alas for me it seems this approach is a dead end. I have tried various methods to use PJL to get status information, but it just does not seem to work for me. Regardless of the state of the printer and regardless of the method I use to communicate be it Telnet or Netcat, I only get this result :

I some cases I can change the display text, which is fun , but not helpful in this case. What I think is going on is that every PJL session is regarded as a job on the 5N. There isn't much there about the current job. No ID's no setting, not a single printer state that would indicate something useful.

I guess the correct information would exist on the session that CUPS opened to deliver the job, but CUPS just breaks the connection after sending is done and considers it done!

One interesting piece of information did come out of this and it is a testament to the way things were build back when my head was freakishly large but my belly wasn't

That is right, My 5N is slowly closing in on a cool million. One megaprint! Surely that is more than one tree! Despite of this, the quality of the print is still fantastic. In fact, you can FEEL the toner on the paper. It does not smudge and it could possibly produce readable Braille!

Well, until something new crops up, I think I'll start looking at catching the spooler information again and start scripting around that.

As other printer may behave differently, I'll put the bash and PJL files online. The later can be fed into Netcat any which way you like, they all seem to work the same for me!

I've been poking and prodding around the bits and pieces that make up my print environment with some community help to boot. I was especially interested where the messages about number of pages and percentages came from. Ken Yap chipped in and offered a couple of suggestion how to go about figuring this out. Using Wireshark, I logged some network traffic between the client, printer server and printer to see who was talking to whom about what!

Turns out, the "Printing page X, xx%" is generated by the cups driver and set as the last "message" of the spooler, regardless of what the printer is doing (or not doing).

There is some communications going on between the printer and the print server because it knows it's out of paper. I also know this from the syslog server.

I still need to do some analysis on the Wireshark data to figure out precisely which cups driver is doing this messaging, but it seems reasonably the same across all devices, although on the tablet, it behaved differently than on the laptop.

The laptops report the same page and percentage data as found in "lpstat -p" and continue until "rendering" is completed, while the tablet generated the same messages, but stopped at a certain percentage when "out of paper" was reported, hinting at the local cups driver generating these messages. This should not be much of a problem as all devices here use cups as we only run Linux boxes and Apple machines. But mileage may vary on windows OR with other drivers. Not sure yet!

Essentially, I think we have plenty of data to manage and control the automated on/off function relatively accurately. We have two questions we need answered.

1. Do we have a reasonable idea how much pages are put in the queue?
2. Do we have a reasonable indication if the printer is running or not?

I think we can answer these question with the data we have so far and I think I have found a way to script in such a way that adding to the queue at any time will make the printer stay on for longer.

The method will rely on two scripts. The first is the timer and the second the monitor.

The timer will be a very simple while loop that, once a minute, subtracts 1 from the timer variable. When it reaches 0 AND if a state variable is set true, the printer powers down. When the state variable is false, the timer also does not subtract.

The monitor script will continuously try to intercept the number of pages and add minutes to the timer variable calculated from the intercepted highest page number. This script will also check syslog and lpstat for any indication that something is wrong. If paper is out, a tray is open or something does not communicate, the state variable will be set false.

This way, the printer will not shut down if there is doubt about the state of the printer and jobs will not be lost or cut short.. hopefully.

Playing around with various methods to determine a workable more advanced method to control the HP 5N printer, I have come to realize how naive we were back in the 90's. So I figured I might be able to use the syslog server functionality of the printer to actually get the data I need to figure out the state of the printer and this is partially true, but seemingly nobody ever thought it a good idea to actually log jobs! I can find no way to "set" the log level.

So now I have a nice centralized log on my Raspberry Pi on whether or not the covers are open or not! This DOES report paper jams and is there for useful to stop the script from turning off the printer when something goes awry, but it tells us nothing about starting or finishing a job. Why?

Anyway, I stated in the description of this project that CUPS does not count pages.. This is apparently not true. I can't find any evidence in any log of numbers of pages, but when running "lpstat -p" in a loop, it got me this :

I definitely see some pages and percentages going on there. If I could catch that data and distil the highest number, adding some margins, I could come pretty close to the correct amount of time needed to print the batch and power off the printer after cool down.

I did play around with printing multiple blank pages, both as one document and as one page duplicated a number of times, and as I suspected, the spooler queue is useless to determine anything other than "A job has been added".

At this point in time, I think the most effective way forward is to do something like this.

• Check for job
• Job detected, power on printer and start scanning lpstat -p for page count
• determine power on time from page count
• continue checking syslog for errors
• if error occurs, stop script from turning off printer.

In this scenario, a print would always be ready with the printer neatly turned off unless it ran out of paper or had a jam.

So we found out where the "hack" lies. Good times.

The core of this project is the ability to reliably turn on and off the printer when there are job in the printer spooler. Seems simple enough. Alas, it is never so! There are many edge cases to factor in and it all depends on how reliable we can get certain data.

In the most simple model, we'd do something like this.

if job {
    waitTime = startupTime + (pages * perPageTime) + cooldownTime
    wait waitTime
    powerDownPrinter
}

But lives never that easy is it? The first problem is that by default, CUPS does not count pages. We have a document size, but that tells precious little about the length of the physical document. It is also uncertain if "lpstat" can be accurately used to determine the state of the job. For instance, what if you print 10 copies of 1 page and CUPS sends off the data with the command, ten times please. The spooler would show no job after the first page came out. And then there are the cases where we have paper jams or ran out of paper. This all has to be tested and considered.

There is a CUPS wrapper called TEA4CUPS which gives much more tools and option which I'm looking at, but for the moment I'd like to keep it simple.

Another option would have been to buy the TP link hs110 version of the "smart" plug. Essentially it is the same as the HS100, but with the added functionality that it can monitor and report on power usage. Depending on how this power data worked, It would be rather simple to monitor the power usage after a job started and only shut the printer off after it returned to "stand-by". Only problem here would be when the printer jams or runs out of paper as it would stop using power and probably hit that low power state.

But for now, we'll just do it the Q&D way. Detect job turn on for 10 minutes and then turn off unless there is still a job in the spooler.

echo "Printer power socket controller 0.1b"
while true; do
        PRINTERSTAT=`lpstat -u`
        if [ -n "$PRINTERSTAT"  ]; then
                echo "Turn On Printer!"
                ./hs100.sh -i 192.168.1.4 on
                sleep 10m
                PRINTERSTAT=`lpstat -u`
                if [ ! -n "$PRINTERSTAT"  ]; then
                        echo "Turn Off Printer"
                        ./hs100.sh -i 192.168.1.4 off
                fi
        fi
        sleep 2

done

Put this in rc.local and we're golden! For now.