Since I have 2 eyes, I need 2 cameras. To get over the £136 import TAX, and because the store just so conviniently had a bulk listing for them, I'm also planning to get 20 VGA cameras that are £2 each.

Most of these cameras are going into the #SecSavr Suspense [gd0105] (as a kind of doll-house CCTV since the machine is fully sealed) but 6 of them are going to be used for 360 degree Simultaneous Localization and Mapping (SLaM) and a 180+ degree "rear view mirror" if I ever get this project to a point where it can run without being connected to a PC. 

For what I need to accomplish for this project, I think SLaM is entirely optional but it could be useful for the dynamic spatial audio feature or being able to have virtual monitor points in space. I will admit that there's a tinge of FOMO because basically any AR HMD has some level of SLaM. 

Hopefully, 30FPS is good enough to get something. I couldn't find a video of the GC0307 but I did find one of the very similar GC0308:

I also couldn't find any M5 lenses to use with these, so it sounds like I'd have to manually remove the IR filters in the stock lenses or keep the SLaM to visible light only and revisit the feature if it shows potential.

Speaker system

I was able to find the speaker module I remembered seeing >1 year ago:

There's at least 2 similar speakers that are 85mm and 90mm long, but it seems that this one should fit in the space provided. This 80mm speaker doesn't say, but the other speakers all have a thickness of 12mm so I'm going to assume the same for this speaker. The reviews I was able to scan across listings say that this speaker is "as good as the original", and since these are for Samsung TV's, I'd assume that they're actually pretty good.

If I recall correctly, reproduction of bass frequencies require a good amount of air movement, which can be obtained by a higher cone surface area or higher excursion. It looks like this speaker has both, looking at these video:

For reference, the below speaker looks like the 53mm speaker I found when I first did audio research, and the excursion is drastically less:

It also looks nicer and higher quality. I know the aesthetic of the speaker is of little importance, but it's something I wanted to mention. It just feels like using higher quality ingredients.

I also would like to believe a full-length speaker cone would emulate spacial sounds more accurately than a smaller one.

This is the speaker (or one of its similar derivatives) in their enclosures and it sounds like it rip 'n' chews (it sounds good). The test songs are nice because I can hear the response across the enitre range, from bass beats to vocals to violin strums:

Yeah... ok... I know how I said I don't care that much about audio quality in the previous audio log but that was just me trying (and failing) to reduce my expectations. I do need some quality, just perhaps not enthusiast grade. 

Eye tracking system

On the eyetracking side of this project, I think I've found a very good lens for this applicaiton:

As long as the lens MP rating exceeds that of the sensor, I shouldn't run into problems. I believe something like a 16mm lens is ideal for this application because I can just have 2 mirrors inside a tube (see below) and there's less of a chance of the camera picking up the tube walls or the two images having unusably different focus levels (because the light path is longer on one).

I've also found an IR led that looks a lot like the EyeTrackVR recommended one.

This one is rated for 20mA whereas the recommended one is 30mA, so I should be in safe radiation levels.

Exciting stuff!

[09:30] Okay. I've been wondering about this for a while but I think it's finally time to start looking into what options or components I need for eye tracking. I feel that, even if I don't obtain a full TyMist solution, eye tracking would speed up / streamline my PC work, just like #Tetent [gd0090]. 

The best solution would be one that I can use to detect the assumed focal point of the user. The closer something is, the smaller the angle is (if 90 degrees is the eye looking straight ahead). I doubt I'd get that accuracy, but I also don't think I need it as long as the screen focus is user adjustable.

Lets see if I can not invent here. For starters, I know of the Tobi 5 eye tracker which is a Wii-looking bar that's too large for this application (as well as seemingly incompatible with my portable monitor #Teti [gd0022]  setup). I also know prices for these kinds of things are usually >£200 and have mentally set pricing expectations accordingly.

So the first option was looking promising when I was looking in the docs, until I found the price of the device:

"It's… 3.4 grand? Grand? Three thou four? Not 344? 344 zero?"
Then I looked into the DIY version and the cameras are 120Hz. Considering even basic mice have a 125Hz polling rate and eyes move so much faster, I'm not impressed. The DIY edition also has a non-commercial licence, which sounds iffy. 

There weren't any hardware solutions after the first one, so I'll have to invent here. I just watched a video of what happened when Target tried to outsource a software solution (and now they do 70% inhouse, according to a comment) so inventing here might not be so bad.

It seems that the main thing I need is an infrared camera. Well most cameras can detect infrared (and thus actually require a filter for them), so I have the 330fps camera I planned to use for the #SecSavr Sublime [gd0036] in the back of my mind, but I first wanted to see if I could use a mouse sensor camera. I hopefully don't need a great deal of resolution for tracking the eye, and those sensors are much faster and cheaper. They're also small so, along with the low price, I could have more than 1 sensor per eye for increased accuracy / reliability. 

So I the watched the above video. It seems that the sensors are 40x40px and those high mouse DPIs are a result of interpolation. Is that enough to detect an eye? Could I even get the raw data off fast enough?

Well there's this paper https://www.hindawi.com/journals/js/2019/3931713/ about using optical mouse sensors for eye tracking so there's probably some promise in there.

[11:00]

Very interesting how all these cool things are only showing up very recently. This is the kind of camera setup I hoped for when I was looking for SecSavr Slime cameras. The only kind at the time was the sensor mounted onto some large 38x38mm module when ribbon-cabled versions of this sensor also existed on Aliexpress (with no board to use them with). 

WOAH! It does 1080p 120fps?! That's a new one! The version I had originally found prior only did 60fps at that resolution. Anyway, for this application I'm only interested in the 330fps options. 

I also got an idea to create a custom lens that cuts the image sensing area in half so that something like dual 320x320px images of the eye can be captured with a single sensor. That sounds like a solid plan; better than wasting free data by capturing the surrounding skin or something. Wait... couldn't I use dual images for triangulation? 😏

The original plan was to have the sensor and illumination take the same path as the displays, but I don't think there would be space in the focal plane for that (oh and I'd lose about 50% of the light just to illuminate and another 50% on the capture, due to the losses from the lenses and beamsplitter) so I thought I'd made the optics holder more useful by integrating the eye tracker gear in there.

I can worry about converting the data into eye position at a later time. Right now, I just need to avoid Garbage In Garbage Out.

[12:50] Here are more up-to-date renders, including the lenses and the 20mm diameter beams that hold them and house the eye trackers:

Potentially useful research links

• Near eye detection using a mouse sensor: https://www.hindawi.com/journals/js/2019/3931713/
• Open source project for VR headset eye tracking: https://docs.eyetrackvr.dev/getting_started/intro
• Detecting eye focus from reflections: https://www.cs.unc.edu/~cpk/data/papers/purkinje-tracker.pdf

So that the helmet is easier on the eyes, I've swapped out sharp corners for smooth fillets. I'm also trying different textures again.

I think it actually looks quite relaxing and comfortable with this walnut skin, thus the aesthetic geometry of the helmet is good enough for me. (If the helmet didn't look good no matter what texture I slapped on, I'd have to go back to the drawing board.) This helmet does make me think of a hood dryer, which is probably a decent initial thought for someone to have.

Oh right. Whilst I'm writing this log I should mention this helmet I found:

Due to space constraints, I'm thinking that the air pathway of TyMist is going to be similar to this. It's already got a similar front visor. The VOC filter input is expected to be around about here:

I've also imported the headphone cup concept and the speakers I'm planning on using might be a tight fit:

So I bought the 37mm lenses and 5pcs 60mm F180 lenses (whereby I need 4 per eye) and thought of what else needs my attention whilst waiting for that long AliExpress delivery time.

For the sound system, I first looked to see if I could just take a headphone I liked out of the thousands probably out there by now. There's so many out there that I asked myself the question "What kind of headphone would I like to try?". 

So I watched a few videos talking about things like trebble and bass response and soundstage and imaging and I came to the conclusion that I wasn't actually interested in better sound quality but an improved headphone experience. I wanted to try:

• Open ear (not sealed) headphones or sealed headphones that sent a slow stream of air over the ear.
• Multiple speaker drivers per ear. I've seen it a lot in the enthusiast earphone space (I assume to get a wide frequency response from multiple small drivers) but I'm more interested in directional audio (which is what multiple drivers in headphones do).
• "Dynamic spatial audio" is what I think the term is for being able to hear sounds omnidirectionally and not just a 360 degree plane such as 7.1 surround sound. 
  • This is important for sound TyMist virtual monitors. If there's a video coming from the top virtual monitor and I'm looking at the centre virtual monitor, I want the video to sound like it's still originating from above my head (where the virtual monitor should be) instead of straight forward. 
  • Even the description for dynamic spacial audio doesn't sound 100% like what I want, because I also want it to be able to take translational movement into account. It doesn't need to rival VR tracking, but being able to move my head 30cm in any direction without head rotation and things sound like they're locked to a physical point in space.

Well I'm assuming my headphone choices dropped to 0 like a pallet of bricks. That's fine because most HUD / AR / VR devices come bundled with their own audio solution anyway. It would just have been nice not to have to invent everything here.

Holding up my portable monitor and moving the speaker grill around my ear, I came to the conclusion that I certainly needed 3D audio for TyMist and have decided to opt for 4 speaker directions instead of 3 for a bit more pinpoint accuracy and because I can arrange them in a diamond-square. I'm currently printing this initial concept of an earcup:

The top 45mm thing is where I could put a 4010 blower fan to try out the air stream if I take this concept further.

The speakers will be facing inwards, towards the ear. Since most sounds would probably be coming from infront of me, I think having 2 speakers working together would be an improvement. I'm also hoping that the diamond gets over the ear with a smaller inner square dimension and the air pathways are more ideal.

I do worry about the 2 edges circled in red though. I'm looking at the print and it might be a bit on the large side. For the large TyMist helmet, it's probably less of an issue.

For the speakers themselves, I'm planning on using a pre-sealed one like the ones below:

This aids manufacturing and eliminates the need to be able to print a quality cavity. Right now, since I just need any speaker (that I'd probably be able to EQ later), I'm thinking that this 64x23x18mm speaker is a good choice, at least visually:

The hope is that, for underutilized speaker directions, those speakers will be more bass focused. For example, if the virtual sound point is centered infront, then the front 2 speakers will try to reproduce the full frequency range and the rear 2 speakers will try to reproduce the bass range.

Again, I'm not that concerned about actual audio quality right now at these initial stages. Honestly, the main thing I want to do is reproduce the feeling of listening to overhead speakers in a restaurant / coffee shop / supermarket. I've actually listened to Local Forecast - Elevator WAYYYYYY more than Local Forecast, for example.

Trying the print

Oo nice the print finished just as I was about to publish and it's... actually comfortable? But... this is plastic... how could this be?

The airspace around the ear is really nice, and I think there's actually enough space for me to actually be able to make a magnetic seal attachment (copper part) for an optional seal:

The print looks really cool and modern too. I can't believe diamond-squares was the missing link to my brand aesthetic. Thanks #Tetent TestCut [gd0139].

I'm now thinking of branching this sub-project out so that I can develop without waiting for other TyMist things and then do a pull request.

I've gotten around to modelling the concept with the mouth cutout and PDLC covered camera bar.

Instead of the Chrome-tab like cutout I first thought of, I thought I should stay on-brand and make the cutout easier to cut by going with a half-square-diamond that starts halfway down. It seems large enough to drink from a cup.

First up, I saw the below image in this video and it sounds like I'm onto a winner here. Since the view is locked to the head, I don't want the image to be so large that I move my head instead of my eyes.

The 30 degree FOV I'm planning for this is the same as the vertical FOV of my 16cm tall Realme phone from 30cm (1 A4 sheet) away. PPD of that 1080x2400px display is 78PPD. That's another one for "I'm probably looking for 80 - 90PPD" bingo.

Anyway I found a pan I've seemingly never used and it's actually 30cm in diameter. Thus I don't have to print a tube to try it out. Yeah it looks kinda wide in the mirror, but from my perspective (the one that actually matters for more than 10 seconds) and honestly I won't consider anything less. It actually feels like a good amount of space infront of the face. 

With kitchen paper towels simulating the mist mode, it felt like I was in a micro fort/tent and was suprised on how fast I felt more focused. Is seeing the rest of the world at 16K 480fps RTX ON XDR actually distracting? Since light could still diffuse though the paper towels, it was actually quite nice. I can't really imagine saying the same if it was a fully opaque material.

I'll have to increase the visible tube angle to something like 170 degrees as peripheral vision is suprisingly wide. It's currently 150 in the CAD model

I'm cleaning up my AliExpress basket and I get an idea:

Double down and get even more squares into this T^2 tech. 

30cm * 3.14 / 2 = 47cm, which is 2cm longer than 3 * 15cm. I can have the 3 white ones on the front of the cylinder tube and the 3 black ones on the rear of the tube where there's a mouth cutout. (I haven't modelled it in yet, but I'm going to keep most of the acrylic tube intact and just cut an internet browser tab shaped cutout so that I can eat.)

The reason for putting the black PDLC on the cutout side is simple: air quality is less of a priority when I'm likely to use it. I'm less likely to be doing intellectual work and more likely trying to get an IMAX cinema vibe going if I desire a black PDLC behind the virtual monitor. If I am doing intellectual work and I still desire a black PDLC , it's likely that it's because I'm working in the fresh outdoors (away from pollution sources) and want to improve contrast since the outside world is much brighter.

With 1 square infront and 2 on each side, I should gain some quality of life improvements. 

The main one is an animation that turns on the centre PLDC and then turns on the two side PDLCS 0.33 seconds later. (Remember, turning the PDLC on makes it transparent.) This animation would also be used in reverse and makes it much more comfortable to switch from full transparency to full mist and back, compared to suddenly switching one large PDLC. My instant thought was "Wow that's certainly some Iron Man Armoured Adventures aesthetic!" but all I can remember is a void of darkness whenever the camera was looking at the suit wearer. Honestly, the only instance I can think of where I've seen this UI/UX is in Total Fantasy Knockout where the control room metasurface of spacelike sparkles opens up to a view outside:

I also would have the opportunity to only mistify the front PDLC for contrast and keep my peripheral vision transparent, or have things the other way around to feel like I'm in some frosted glass office of the future (and hide distractions). I could also have a slow "mexican wave" animation to get the Thunderbird 5 rotation vibe I've mentioned in a log a while ago without physically moving the acrylic tube. Like the video below but rotated 90 degrees, way slower and much thicker strips:

Oh and I just noticed that the acronym is PDLC not PLDC. Now I have to be on the lookout for that typo.

To conclude, I think this white and black PDLC film idea is a good solution to the problem of high light sources behind the screen. Using a polarising light gate (1 pixel LCD w/ no backlight) would only work for flat surfaces, won't cover all the vision, blocks 50% of all light in the most transparent state and only comes in specific sizes. Dimming electrochromic film takes a while to switch (which isn't really a problem in this application) but I can't seem to find any on AliExpress (which is a serious problem in this application).

PDLC seems to have a number of benefits:

Seems that full transparency requires 4W * 0.45 * 0.15 = 0.27W of power. I also like the sound of the safety as I don't want acrylic flying if I trip+fall. Putting the film on the inside might also be a consideration, since 29 * 3.14 / 2 = 45.53cm and even closer to the length I need, so the film could be completely gap free all around and reduce exposure of the film to the elements. Yeah lets go with that plan instead. 

Man, this is turning out to be a very cool helmet. Fairly expensive tho... but very cool.

[09:05] Considering that most VR and AR systems today... actually basically every one I know of (even the Meta prototypes) have PPDs well under 80, I'm thinking that I should be more cautious with my optical path. This was brought to my attention when I found out about this AR lens module:

It's also notable that it's up to 25% transparent in this industry. Perhaps that means that I could use a R60:T40 beamsplitter instead of a 50% splitter.

Anyway, I'm still unconvinced that the fresnel lens is going to give me the optical quality I desire, so I went on AliExpress.

Hm? A lens in the 50 - 60mm diameter range that has a usable focal length? ASPHERIC TOO!? Wow that didn't take long to find!

Then I found a bunch of achromatic lenses that all had rather high focal lengths for this application. I also found a parabolic mirror, but the simulation didn't reflect light like I hoped it would (90 degree reflection + focusing).

This lens looks suprisingly good in the AliExpress listing video for being 20p per each. I've assumed the actual optical diameter is 34mm because it looks like there's a bit of a flat section for mounting, though there's also a youtube vid that looks to be an edge-to-edge lens:

A solution seems to exist for 1x 60mm + 3x 34mm lenses, though I still have the unsimulatable problem of chromatic abberation to worry about. Hopefully, the longer the focal length the less of an issue this is. It's also going to be a very tight fit with that 60mm diameter lens, and I'm using the outer reflective surface of the 20mm prism instead of TIR like in the previous optical path. (The 200unit ruler on the mirror also implies where the glass face is.)

[15:50] It's interesting that AliTools only has historical data for both lenses that only go back 1 week. Is the reason why I never found them before related to the part where they never existed until very recently? I also noticed that the lens is F45mm not F47mm as I thought in the simulation. I doubt it'll change much. (Can confirm. I was able to move the screen forward 4mm though.)

[Feb 2 at 02:33] Me: Wrapping things up and about to buy

Aw the exchange rate went up? Wait. F100mm? 

checks bottom of basket

clenched teeth facepalm

Second time I've had a solution and then got hit with Out Of Stock. I'm guessing in this case it was a listing error.

A solution... will have to be recalculated.

 I mean... this lens I found from the same seller still exists. It was in my basket and I was about to remove it.

nvm now I remember why I was supposed to remove it from my basket.

A solution... still has to be recalculated.

The distance from the top of the fresnel to the top of the helmet (above the fresnel) is just over 10cm:

That means that the optical path I had probably was going to intersect with the user head area.

I've got this new path that has a couple of benefits:

• There is no lens near a focal point
• Only 2 glass lenses are needed, so that (starting from the fresnel lens), it's F40 -> F30 -> F40
• The screen is less than 10mm behind the front of the eye.

The steepest angle emmited from the screen is under 15 degrees, and according to this blog post (see below), the OLED brightness at a given angle is cos(angle) * 100%. The result for 15 degrees is 96.6% brightness so I'd probably never notice. I wouldn't be suprised if I see steeper angles from normal sized OLED panels.

Overlaying the path, it seems that I've still got space to go up even further, perhaps to allow more head padding space.