[HN Gopher] Show HN: DIY virtual HDMI monitor using "AR" glasses
___________________________________________________________________
Show HN: DIY virtual HDMI monitor using "AR" glasses
I am making a virtual HDMI monitor using Viture Pro XR glasses and
an SBC ( currently OrangePi 5 Plus because it has HDMI-in ). What
it does is map the frames from the HDMI input onto a virtual
display that is controlled by the IMU data from the glasses ( 3DOF
only ). I've put AR in quotes in the title because many won't view
those display glasses as true AR but by tracking the head movement
it comes close. I am trying to build kind of a "low cost" version
of a virtual screen that acts like a monitor and can be connected
to anything that has an HDMI output I started off using the
official Viture SDK to interact with the glasses but have since
switched to a reverse engineered implementation of the protocol
because their SDK is not available for ARM Here is a video showing
the first version: https://www.youtube.com/watch?v=D6w5kAA22Ts Big
caveat: Performance still needs to improve a lot because the whole
frame reading/converting is completely unoptimized for now. What
other solutions do exist out there? * Streaming the computer screen
to a headset like Meta Quest/Vision Pro * Connecting a HDMI capture
dongle to the Meta Quest directly * XReal Beam ( basically the same
as this project but official and for XReal glasses ) And for the
obvious question, why I am not use something like a Quest or Vision
Pro 1. Comfort 2. Price 3. Comfort After using those display
glasses over HMDs it's hard to convince myself to use a headset for
productivity again
Author : blensor
Score : 99 points
Date : 2025-06-11 09:20 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| stavros wrote:
| I'm actually wearing the Xreal Air 2 right now, and yes, this is
| basically exactly what their app does. Personally, I only need
| one screen, so I'm using the glasses without the app, but this is
| great if you want a virtual second display somewhere in the room!
| Very well done.
| blensor wrote:
| I'm actually a bit jealous about the XReals. Back when I bought
| the Viture I wanted to buy XReal first but they had such
| strange shipping policies ( they shipped to every county around
| me but not to Austria ) so I went for the Viture.
| stavros wrote:
| The Xreals are the only ones I've tried, so I don't know how
| they compare, but I have trouble with fit. I have trouble in
| general, because of my nose bridge being too wide, but even
| when I put the glasses right against my forehead, I still
| can't get the entire screen in focus, the edges are blurry.
|
| I really wish the screen were smaller and more to the center,
| then they'd be perfect. I guess it depends on whether you
| want them for games (the important stuff is usually in the
| centrer) or for work (the important stuff is usually at the
| edges).
|
| I really like the focal distance, though, it's around 5m away
| and pretty comfortable for the eyes.
| blensor wrote:
| With the viture I have the problem that the bottom of the
| screen is almost always not visible. I initially used the
| accessibility feature that let's me zoom into the desktop
| with mouse follow moving the viewport but that gets old
| quickly, which is one reason why I started that project.
|
| I'm also planning to add other "display" modes like a
| curved monitor and in theory we could connect multiple
| monitors using several hdmi capture devices but that's
| probably even less useful
| gavinray wrote:
| I also have this problem with the Viture glasses.
|
| Do you know if this accessibility feature is available
| when using it with phones?
| blensor wrote:
| I haven't found a way to do it on the phone unfortunately
| which is really annoying.
|
| You could in theory use the approach I posted here and
| use a USB to HDMI adapter to connect the phone to device
| and it would do the same thing it does with a regular PC
| but that loses all the benefits of being small and
| portable if you have to carry around all that additional
| stuff
| mentos wrote:
| I'm staying at a friends and using my laptop and a second
| monitor of his to work. I've wondered if the Xreal Air 2 would
| be better than a large monitor. My hunch is that having a
| dedicated smaller monitor not on my face is more comfortable
| and productive than a large virtual one.
|
| From your experience would you agree?
| stavros wrote:
| Hm, no, I much prefer the Xreal to the laptop's screen
| (except for the blurry edges). I don't care about the monitor
| size, only the resolution, but I'd rather have the monitor
| always be straight ahead in my vision (no matter how my head
| is tilted), and I really like the Xreal for that.
|
| As I said, though, I always only use a single monitor anyway,
| so YMMV.
| 0xffff2 wrote:
| I haven't previously heard of Xreal. Looking at their
| glasses, it seems like they're all 1080p. I find it very
| difficult to deal with text on a regular 1080p monitor
| after having uses 4k monitors for a while. Is there
| something about the way the glasses work that makes 1080p
| glasses equivalent/better than at least a 1440p monitor in
| terms of clarity?
| stavros wrote:
| Well, 1080p is 1080p, what counts is density. The display
| looks huge on the glasses, so density isn't great, but it
| doesn't bother me personally. YMMV, though.
| k__ wrote:
| The Xreal Air 2 Ultra look pretty interesting.
|
| After using both 3DoF and 6DoF for years, I just can't imagine
| using a 3DoF headset for more than watching videos or simple
| writing tasks.
| sitkack wrote:
| Just so it is clear, the "Xreal Air 2 Ultra" appear to have
| 6DoF tracking
| k__ wrote:
| Yes, but the previous versions didn't.
| KeplerBoy wrote:
| Is the xreal usable outdoors in bright conditions?
| stavros wrote:
| Yes, I live in Greece and it's fine. The dark cover is pretty
| dark.
| deivid wrote:
| Cool project, but I'm missing what does this "remapping" do that
| the glasses can't do by themselves?
|
| I see the glasses are connected (via usb-c?) to the OrangePi.
|
| Is this project converting HDMI signal to the Viture protocol? if
| so, can't you run a screen grabber directly on your machine and
| send it to the glasses?
| blensor wrote:
| If you connect the glasses to your computer you get a fixed
| screen in front of your eyes (like those display glasses back
| in the day that promised to give you a huge screen), which is
| fine for video viewing but if you want to use it for
| productivity you want to be able to look at certain parts of
| your screen vs. straining your eyes to glimpse the edges.
|
| That's where this comes in, it allows you to look around the
| virtual screen with the added bonus that you can connect it to
| almost anything and behave the same not just your PC
| xeonmc wrote:
| If you have 3dof tracking only, consider making the displayed
| image to have no stereodisparity, this way it is optically
| correct to being a screen projected onto the celestial sphere.
| This is also how reflex sights works on fighter jets and guns.
| blensor wrote:
| I have to look into that, thanks.
|
| I've been thinking about combining it with our work on the
| reverse engineered HTC Vive Ultimate Tracker stuff here
| https://youtube.com/shorts/U4vhEPQw-Uo to get 6DOF
|
| But realistically I ( and nobody else ) won't strap the tracker
| to my head all the time
| LonelyWolfe wrote:
| Just venting and ranting about AR glasses here:
|
| One thing about AR glasses that I don't get so far: Why do _ALL_
| of the AR glasses use only 1080p displays per eye whereas all the
| other VR solutions that also have a display right in front of
| your eye have even bigger resolutions?
|
| I really do want AR glasses that can act as an alternative to a
| physical display. But they're always 1080p for some reason. The
| companies I've seen making them appear to me to keep fixing
| important issues and adding really useful features - but seem to
| avoid touching anything that would improve the resolution.
|
| Edit : There's a part of me that believes the situation here is a
| lot like how Linus Tech Tips described the Dashcam situation.
| k__ wrote:
| Because the glasses are much smaller than headsets, I'd assume.
| riedel wrote:
| One of the reason is IMHO that the optical resolution with
| those lense designs does not get better than 1080p. It seems
| even that resolution is blurry at the edges. So anything beyond
| probably would be a waste without totally different optics.
| LonelyWolfe wrote:
| This feels like the most probable reason. Likely a big
| redesign would be needed for larger resolutions.
| blensor wrote:
| Not disagreeing with the resolution but I assume for most last-
| gen use cases it didn't make much sense because you also need
| to be able to drive that somehow and you can be happy that you
| get a 2 Lane display port interface up until the more recent
| phones.
|
| So if you have high res displays that no one can realistically
| use for the intended mobile cinema use case then you are
| wasting money and energy
| spookie wrote:
| AR solutions value mobility over fidelity. This has numerous
| repercussions, leading to those displays.
| andrewmcwatters wrote:
| The screens are two entirely different products. The current
| Sony micro-OLED screens being used by product manufacturers for
| AR glasses require out-of-plane reprojection versus VR headset
| screens which require focal lenses.
|
| The top-of-the-line solutions by Sony used in current products
| only go up to 1080p120.
| LonelyWolfe wrote:
| I heard something like this in the Linus video. That video
| also mentioned Sony as well.
| andrewmcwatters wrote:
| Yep https://www.sony-
| semicon.com/en/products/microdisplay/oled.h...
| stronglikedan wrote:
| Apples to oranges, since VR glasses are a much bigger form
| factor.
| kevin_thibedeau wrote:
| It's a packaging problem. They don't have the luxury of placing
| relatively large screens in front of the eyes.
| charcircuit wrote:
| AR glasses have smaller FOV which means that a lower resolution
| can still achieve a higher peak pixels per degree compared to a
| VR headset.
| numpad0 wrote:
| The display panels used for AR(and some pre-Oculus VR) displays
| are completely different thing from normal LCDs. Lots of them
| used to be a reflective panel made of silicon wafers with RGB
| front lights flashing in sequence. Driving circuits and
| interfacing protocols are all non-standard, under NDA, and
| quite complicated. The panels also needs to be coupled to a
| complicated optical lens - they're just magnifying glasses,
| only fancier. The complete thing that has a digital input and
| an optical output is often referred to as an Optical Engine.
|
| What that means is, these glasses are made by someone paying
| $$$ once to Asian engineering companies to have it figured out,
| and everyone reuse that exact setup for years on.
|
| And companies like Sony or Canon, they develop stuffs by
| scraping engineers off a wall instead of first throwing them at
| the wall, so every Sony or Canon cameras and projectors come
| with novel viewfinders and crazy patented lenses. Or panel
| suppliers like Kopin, Epson, or Sharp can arrange contacts to
| engineering consultants. I guess. Same likely goes for all the
| Chinese companies too, though I'm not familiar enough on that
| front to be able to offer googlable keywords.
|
| Startups and even mid-sized consumer electronics companies
| don't have that kind of time or financial backings or margins
| to do the same. They barely manage to pay for assembly and ship
| it. And so the spec of the final product is whatever spec of
| parts that they could buy off the shelf.
|
| What about Apple? Well, they pay to have 2 extra display
| factories built for redundancy by policy - I'm sure that most
| military organization don't do that. And even then they use a
| rather simplistic, completely concentric and rotationally
| symmetric optical design.
|
| (I kind of have a crazy idea to bypass some of that, but
| unfortunately I'm crazy and so is the idea)
| blensor wrote:
| now I'm intrigued, any hints?
| numpad0 wrote:
| Straight up homebrewing display devices. How hard can it
| be.
| sitzkrieg wrote:
| some are very hard :-) fun problem space tho
| underlipton wrote:
| It's depressing but not surprising when you remember that
| that only reason any of this exists is because Lucky
| Palmer pointed out that you can kickstart modern VR R&D
| with off-the-shelf smartphone displays (IIRC).
| z3t4 wrote:
| In my experience you need at least 8k per eye to get sharp
| text. But if you just want to watch a movie 4k will look very
| crisp and 1080p@30hz also works for movies.
|
| So what you get is basically a larger mobile screen for
| watching videos, something that competes with video projectors,
| but you can use it on the bus. It's an easier sell compared to
| a device that will give you a good reading experience and
| competes with paper books.
|
| Those of us that want to use them as terminal emulators are
| still too few.
| msgodel wrote:
| I don't know what I would do with higher resolution personally.
| This is already enough that xterm's "tiny" font is extremely
| hard to read. Personally I'd prefer lower power draw to more
| resolution.
| amelius wrote:
| I looked at those Viture glasses, but the pitch of the glasses
| (how far the eyes are apart) looks strangely low. Perhaps this
| works in the Asian market, but it just seems inadequate for the
| Western world.
| gavinray wrote:
| I own them, bought just recently. The eyes have these circular
| wheels near your eyebrow above the frame that lets you
| independently adjust the distance for each eye, if that's what
| you're talking about.
| amelius wrote:
| Good to know, but does that move the arms too? Some people
| have bigger heads than others.
| gavinray wrote:
| It does not, but for what it's worth, I have been told I
| have quite a large head and they fit comfortably.
| amelius wrote:
| Ok, one more question: does the wheel also make the black
| glasses move? Because I'd imagine that if only the
| internal optics move to the sides, you get to have less
| protection from external light on the sides of your view.
| blensor wrote:
| I think they suit bigger heads as well is because the
| hinge on the stems can go slightly beyond 90deg
| unsupp0rted wrote:
| I, for one, would like to see AR glasses that use e-ink. Black
| and white with a slow refresh rate. Would be perfect for my use-
| cases.
| blensor wrote:
| you still would need a backlight but it's an interesting idea
| unsupp0rted wrote:
| Would be cool to wear a "light panel" on your sleeve.
|
| Outdoors you see the e-ink fine. Indoors you look down at
| your illuminated sleeve to see the e-ink in your glasses.
| sleepybrett wrote:
| You can't backlight an eink display. They are opaque, by
| design. Thats why they make such great ebook displays, so you
| can take them outside and still read them. Eink displays
| require .. i guess 'forelight'. Light that reflects off the
| display front.
| blensor wrote:
| I didn't know that, I just wonder how the kindle with light
| works though. Is that light coming from the front too?
| sleepybrett wrote:
| frontlit
| ashdksnndck wrote:
| If you look closely at a backlit kindle screen, you can
| see that the e-ink pixels are a decent distance behind
| the surface. It's more noticeable if there are scratches
| or dust on the screen and you rotate the display. The
| surface reflects the light from the sides back to the
| e-ink.
| numpad0 wrote:
| IIRC, side illuminated microscopic dots on a protective
| cover. Light only escapes through the dots, illuminating
| the panel, without shining into the user.
| sitkack wrote:
| I think what is not clear from your project description is that
| your AR glasses don't provide an _environment locked virtual
| screen_ , meaning a screen that stays fixed in space while your
| head moves. The native screen of your glasses provide a screen
| that is fixed relative to the glasses, like having a display
| locked to your face.
|
| Some AR glasses _do_ provide this feature, so it overcomes a
| product flaw in AR glasses and will at some point not be
| necessary.
|
| Your project could also allow for other features that the AR
| glass manufacturer didn't think of, or has gated behind upgrades
| and product segmentation.
|
| Is my assessment correct?
| andrewmcwatters wrote:
| It's also a Viture-specific project, but I suspect if the
| author wanted, this could expand to other video display glasses
| in the future and he could simply rename the project.
| blensor wrote:
| Spot on. And yes expanding it to other glasses ( which are
| popping up quite a lot recently ) would for sure be possible
| blensor wrote:
| Yes that's correct, and it could be expanded to other features
| yes, one thing I've been mulling over is mult monitor input
| which you don't even get on devices that support what I did
| here natively
| msgodel wrote:
| I use AR glasses heavily for work and other things. I'm
| actually typing this on a pair right now. I've never understood
| what the application for environment locked screens is other
| than novelty/marketing. My glasses provide enough sensor data
| to implement this but I just can't be bothered.
| veilrap wrote:
| I'm curious how/what you use the screens via AR for if you're
| not using environment locked screens. Particularly in a
| productivity/work environment.
|
| Unless I'm misunderstanding the feature, it seems like
| enironment locked screens allows for more natural usage and
| interactions with the screens in the virtual space?
|
| My experience is mostly with VR/AR products like Oculus has
| been mostly with environment locked AR information.
| msgodel wrote:
| I suppose they could? I prefer having my posture decoupled
| from what I'm looking at though.
|
| It's like having a very nice monitor that uses ~1 watt of
| power and happens to be positioned exactly wherever is most
| comfortable without even having to think about it. It's way
| better than a normal monitor if you don't have to do eg
| pair programming.
| stavros wrote:
| How are you finding the focus? I use the Xreal Air 2, but
| the edges are blurry, and I can't get the glasses close
| enough to my face to see the entire screen in focus, even
| if the top of the glasses is touching my forehead.
| msgodel wrote:
| That's interesting. I think mine are the original Xreal
| glasses? I had to slightly adjust the nose part but
| otherwise the focus has been great for me. That would be
| a shame if their regressing.
| stavros wrote:
| I think it's just the shape of my face.
| derefr wrote:
| I think the use-case for these is more VR focused, with the
| AR just being a "being able to notice when something needs
| your attention" feature (where you would respond to such an
| interrupt by taking the glasses off, not by trying to look
| at the interrupting thing through the glasses.)
|
| I've heard people propose that these "screen in glasses"
| devices (like the Xreal Air) are useful for situations
| where you want a lot of visual real-estate but don't have
| the physical room for it -- like in a dorm room, or on a
| plane. (Or at a library/coffee shop if you're not afraid of
| looking weird.)
|
| ---
|
| Tangent: this use-case could likely just as well be solved
| today with zero-passthrough pure-VR glasses, with a small,
| low-quality outward-facing camera+microphone on the front,
| connected only to an internal background AI model running
| on its own core, that monitors your surroundings in order
| to nudge you within the VR view if something "interesting"
| happens in the real world. That'd be both a fair bit
| simpler/cheaper to implement than camera-based synced-
| reality AR, and higher-fidelity for the screen than
| passthrough-based AR.
|
| + Which wouldn't even need to be a novel model -- you could
| use the same one that cloud-recording security cameras use
| in the cloud to decide which footage is interesting enough
| to clip/preserve/remote-notify you about as an "event".
| underlipton wrote:
| The most obvious advantage is being able to "zoom in" on the
| screen by moving closer to it (as with a real monitor), which
| is impossible with 3DOF or view-locked XR.
| bodge5000 wrote:
| I guess it makes a lot more sense if you're emulating more
| than one monitor, without environment locking you'd only ever
| be able to see the sides of the monitor(s) beside the one
| you're looking at.
|
| But yeh for a single monitor, I guess it takes a bit of
| getting used to but non-locked seems far more reliable
| gsf_emergency wrote:
| It is indeed hard to assess from the video at 19s? (Relatively
| impressive thru-the-glass perspective ntheless!)
|
| https://www.youtube.com/watch?v=D6w5kAA22TsP&t=19s
|
| Is it me or is there some dynamic here that could/should be
| further exploited to implement what you suggest
| skeptrune wrote:
| Wow, $409 getting you a AR display that's viable to hack on is
| crazy. I can't believe how approachable hardware is for tinkerers
| some time.
| verdverm wrote:
| https://immersed.com is a great piece of software that gives you
| multiple virtual screens. It supports all major OS and VR
| devices, with their own https://visor.com headset coming soon
| (tm)
| wkat4242 wrote:
| Nice! I wish this worked on my XReal Air :( It does have IMU
| sensors because their own app (nebula) uses them, and even does
| 3D effects.
|
| By the way I find my Quest 3 way more comfortable than the Xreal.
| The problem with the xreal is, the prescription inlays are too
| small so I can see only a part of the display sharp unless I put
| in lenses every time. And it is kinda heavy on my nose and ears.
| the Quest is super nice, has a wider FoV. But it's heavier and
| bulkier to bring along of course.
|
| PS I had no idea you could connect an HDMI dongle to the quest.
| I'll check that out.
| blensor wrote:
| If I had XReal I could look into that too, it's probably pretty
| similar to how it works for the Viture.
|
| Regarding the HDMI on Quest 3, I believe it was sometime last
| year when they made that possible
| https://www.meta.com/help/quest/1350679463006443
| wkat4242 wrote:
| Yeah I bet it's real similar. But I understand.
|
| Thanks for the heads up about that Quest thing, I actually
| have some of those adapters so I will give it a try.
| benhoff wrote:
| Did you fix the op5plus HDMI in driver?
|
| Thing was unstable like 8-12 months ago
| blensor wrote:
| So far I haven't had any issue but I haven't done long tests
| yet. I'm using BredOS not their official OrangePI images though
___________________________________________________________________
(page generated 2025-06-12 23:02 UTC)