Accessibility

[kommander]

Currently screen readers have a hard time with TUIs. My experience making TUIs accessible is limited. I know that some terminal emulators handle it better than others.

But OpenTUI should be able to help out here, by leveraging accessibility APIs on the platform it is running and let Renderables/Elements take aria-labels and other hints that are communicated to the accessibility APIs.

[lp20065220]

I would think it should be able to use windows UIA for windows screen readers. I'm not well versed in the API su I'm not sure how I can help besides testing with NVDA.

[kommander]

Okay I read up a tiny bit on that for Windows, MacOS would be similar and I guess linux variants as well. This is roughly how I would do this:

RFC: Ship a Windows UIA “provider backend” in the native library (Win32 + COM only)

Implement a UI Automation provider that exposes the OpenTUI renderable tree as a UIA accessibility tree.
Microsoft UI Automation

Rough Architecture

• Create a tiny Win32 window (HWND) owned by the OpenTUI process when accessibility is enabled
• Handle WM_GETOBJECT and return a UIA provider root for that HWND
• Map OpenTUI renderable tree nodes -> UIA elements:
  • role/control type (button, list, textbox, menu, etc.)
  • Name/HelpText/Value
  • bounding rectangles (already available from layout)
  • keyboard focus state (already a thing in Renderables and the renderer)
• Implement the key provider interfaces for a tree:
  • IRawElementProviderSimple (baseline)
  • IRawElementProviderFragment + ...FragmentRoot (for navigation)
• Implement control patterns per Renderable node type:
  • Buttons -> IInvokeProvider
  • Inputs -> IValueProvider / ITextProvider
  • Lists -> ISelectionProvider / ISelectionItemProvider
• Raise the right UIA events:
  • focus changed
  • property changed (name/value/selection)
  • structure changed (children added/removed)
  • notification events for toasts/status updates

This would map real semantics (not “terminal text reading”) and would work regardless of alternate screen / ANSI. Exposing options on all Renderables via the base Renderable to set semantics and some lower level methods on the renderer to to for example trigger UIA notifications would allow something like OpenCode to make its complex interface accessible easily, as most would come out-of-the-box.

[lp20065220]

Sounds good to me. I'm happy to test once this is implomented

[lp20065220]

Any progress on this so far?