Screen Reader Technology

Web Based User Agents

Everyone that is browsing the web has a user agent. This is the software that acts as the bridge between the user and the internet. When your browser loads a web page, it identifies itself as an agent when it retrieves the content you have requested, and the browser sends a host of information about the device and network that you are on. This is a set of data that allows web developers to customize the web experience. That is, the browser must understand the Application Protocol Interface (API) building blocks and command structure of the web page, so as to display meaningful information on the user device and allow the user to interact with the page elements. There are many different browsers, each using a specific API structure and thus delivering a different user experience on different devices. However, for the most part, the browser API is designed for visual users and Mouse interaction, but some browsers have accommodated blind users by providing an accessibility interface structure for screen readers.
TPG: A Tale of Two Rooms – Understanding screen reader navigation

In order to enjoy access to websites and other digital media, many people with visual impairments rely on assistive technologies such as braille displays and screen readers. A screen reader is a software application that converts the text displayed on a computer screen into synthesized speech. Text-to-speech capabilities are a vital component of AI Assistants, such as Apple’s Siri and Amazon’s Alexa, which talk with users by converting their replies into digital speech. Screen readers are a specific use case of text-to-speech technology that improve accessibility for people with visual disabilities. These days the market offers many options for screen readers. Some are intended for use with different operating systems and applications; some are free and open source; others are commercial software. Screen readers are able to look for and process any kind of text that is displayed on the screen of a computer or mobile device, including website content, icon labels, documents, spreadsheets, file menus and more. The software also reads the title of web pages and the alternative text captions for any images on the page. Once it identifies this text, the screen reader speaks it out loud at the user’s request, pausing at punctuation such as periods, commas, and exclamation marks. The user can adjust settings such as the speed and volume of the speech. To learn more see
Wikipedia: List Of Screen Readers
However, Augmented Reality (AR) applications like the Seeing-AI go beyond the capabilities of screen readers.
Microsoft Seeing AI, Artificial Intelligence for Blind and Visually Impaired Users, By Steven Kelley

Using The JAWS Screen Reader

The Freedom Scientific JAWS screen reader can be a powerful tool in helping to evaluate the usability of a website. JAWS, a Microsoft Windows based screen reader assistive technology, presents Web pages using the JAWS Virtual Cursor. This allows users to read and navigate a Web page as though it were a text document. Users press the ARROW keys to read line by line, word by word, character by character, and so on. JAWS also provides Navigation Quick Keys, which are alpha-numeric keys that move the Virtual Cursor to features of the page such as links, headings, and form controls. In addition, users can press the TAB key to move between focusable elements on the page. Using the ARROW keys or Navigation Quick Keys to change the position of the Virtual Cursor does not change the actual focus point in the application. This means that even if JAWS reads the text of a given link on a Web page for example, that link doesn’t necessarily have the keyboard focus. Conversely, pressing the TAB or SHIFT+TAB key to navigate moves the focus point and the Virtual Cursor follows the focus. The JAWS Navigation quick keys make it faster and easier to move around on a Web page and anywhere else the Virtual Cursor is active. These commands are all assigned to keys on the main part of the keyboard. A web page well structured with the HTML tags, make it possible for JAWS user to quickly navigate the page and gain an understanding of the page content.

    JAWS Cursor Modes:
  1. The PC Cursor is linked to the keyboard functions of Windows and applications. This is the cursor that is used when typing information, moving through options in dialog boxes, and selecting options or icons. As you type information, the PC Cursor follows along with each key you press. pressing NUM PAD PLUS activates the PC Cursor.
  2. The JAWS Cursor is linked to mouse pointer functions in Windows and other applications. It is used to read information the PC Cursor cannot read, such as toolbar information. The mouse follows along with the JAWS Cursor when it is moved. press NUM PAD MINUS to activate the JAWS Cursor.
  3. The Touch Cursor enables you to navigate the Windows environment using a touch screen, found on many tablet computers running Windows 8 or later. To control JAWS from a touch screen, you will use one or more fingers to perform various gestures directly on the screen, such as tapping, flicking, and swiping. press SHIFT+NUM PAD PLUS to activate the Touch Cursor.
  4. The virtual PC Cursor mimics the functions of the PC Cursor, but is activated by default when entering an HTML document, such as a Web page on the Internet. The virtual PC Cursor speaks the number of elements on the page when it first opens. You can use the ARROW keys to navigate and read the document, or use Navigation Quick Keys to move to specific elements, such as paragraphs, tables, or headings. Press Insert+z keys to toggle the Virtual PC Cursor on and off, or Insert+z twice quickly for all opened Window applications.

Using The NVDA Screen Reader

NVDA (NonVisual Desktop Access) is a free, open source software, Microsoft Windows based assistive technology, screen reader which enables blind and vision impaired people to use computers. It reads the text on the screen in a computerized voice. You can control what is read to you by moving the cursor to the relevant area of text with a mouse or the arrows on your keyboard. NVDA can be installed on your computer, or a portable version can be use on an USB thumb drive. NVDA uses a virtual buffer concept common to all Windows screen readers on the market. NVDA takes the HTML of the page and converts it into a flat document with semantic information in the order the HTML appears in the browser source. As you navigate, NVDA will speak semantic information such as link, heading level 1 through heading level 6, button, images, and much more.

Using The Voiceover Screen Reader

VoiceOver is a screen reader program that comes on Mac computers, iPhones, iPads, and iPod touches. VoiceOver is not a standalone screen reader assistive technology. It is deeply integrated into the iOS operating system and all the built-in apps on iOS devices, and as developers update their apps to take advantage of the accessibility interfaces provided by Apple, their apps can start working with VoiceOver right away. It can also access most OS X native applications and functions. VoiceOver gives auditory descriptions of each onscreen element, and provides helpful hints along the way (whether you prefer using gestures, a keyboard, or a braille display), and it supports more than 30 languages, including multiple voice options.

VoiceOver is a sophisticated technology that provides many powerful features to users with disabilities. Although you do not need to become an expert VoiceOver user to test your app with it, you do need to know a handful of basic gestures. The Accessibility Inspector, available in the iOS Simulator, lets you simulate VoiceOver interactions, and examine the accessibility information that is available for the controls in your app. However, Accessibility Inspector does not have speech output, so it is a debugging tool rather than a testing tool. When it comes to testing, there is no substitute for using your app on an iOS device with VoiceOver turned on, or better still asking VoiceOver users to test your app. You can toggle VoiceOver on and off quickly by setting it to the triple-click setting in the Accessibility Settings. When VoiceOver is turned on you will need to use a different set of gestures. After you have turned VoiceOver on, you will notice that many familiar gestures have different effects. For example, a single tap causes VoiceOver to speak the selected item and a double tap activates the selected item. When an element is selected, VoiceOver draws a black rounded rectangle around it, which is called the VoiceOver cursor.

To simulate the experience a visually impaired user might have with your app, you can run it with the VoiceOver screen curtain in place. When you activate the screen curtain, VoiceOver turns off the device display so that no one can read. Testing with the display turned off obliges you to rely on the information VoiceOver speaks and removes the temptation to use your app as a sighted user would. To turn off the display while you use VoiceOver, triple-tap the screen with three fingers. To turn the display back on, perform the same gesture again.
YouTube: A video explaining how to use the VoiceOver screen reader

Using The Windows Narrator Screen Reader

The Microsoft Windows Narrator screen reader is a screen-reading app built into the Windows operating system. Microsoft Narrator was written by Professor Paul Blenkhorn in 2000 and has been included with every version of Windows since that time. While Microsoft recommends that the visually impaired purchase a full-function screen reader for general computer use, Narrator is a useful piece of built-in accessibility software. Narrator is included with every copy of Microsoft Windows, providing a measure of access to Windows without the need to install additional software as long as the computer in use includes a sound card and speakers or headphones.

Narrator can assist a blind person in installing a full-function screen reader, assisting the user until their screen reader of choice is up and running. Because Narrator is a lightweight screen reader that requires minimal “hooks” into the operating system, Narrator can provide speech when a full-function screen reader might be unable to do so, such as during the process of updating hardware drivers. The Ease of Access settings in Windows are easy to discover, learn and use. Settings are grouped by ability (vision, hearing, and interaction), with the most frequently used settings listed first. To get directly to Narrator settings, press Windows logo key + Ctrl + N.

Some Additional Resources