Testing Scenarios for User Assistance

Last updated 5/18/2009

About this Test Suite

The test suite is intended to test areas which the JUnit tests do not cover. Some of the tests have been converted to cheat sheets, these are all included in the composite cheat sheet titled "User Assistance Test Suite" which is in the file cheatsheets/usTests.xml. The remaining tests are described in detail.

Welcome

Setup

Please ensure that you run this test plan with the org.eclipse.ui.intro/debug=true debug option set. This is the only intro debug option that you need turned on, and if there are any errors or warnings in the .log then this is a bug. Please open a defect and include the log.

Scenario A: Basic behavior

Launch a fresh workbench. Make sure that Intro is automatically opened and is in full mode.

Test that root page come up/looks ok. Icons should be there. Verify that the root page icons all have labels.

Test clicking on each of the links in the root page. Verify that on each page visited the size of the text can be increased or decreased by clicking on the two buttons on the toolbar that looks like a letter 'A' with a small arrow next to it.

In Overview page, test that each of the links brings you to a corresponding topic from the Help System documentation.

In the Tutorials page, make sure that clicking on each of the links brings you to the appropriate cheat sheet.

In the Samples page, click on any of the links, you should get a pop-up asking you to download the samples. Click Yes. Check that all samples can then be imported automatically into the workbench and that they can be run.

In the What's New page, the New and Noteworthy links and the Migration link should launch the Help System and take you to the appropriate page. The New Updates link should launch an Install/Update dialog to update features. The latest news from eclipse.org should also be available as links.

Click on the Eclipse Community link, it should open the eclipse.org web site.

Click on the curved arrow icon to “Go to the workbench” and make sure that you get the launch bar at the bottom right trim of the workbench window. It should have a button for each welcome page. All should have tooltip, and clicking on each should open Intro in full mode and take you to the appropriate page.

Hover over the Return to Workbench and the Close icons in the launch bar to get the "move" cursor. Test drag and drop of Launch bar.

Scenario B: Remembering state and Standby Behavior

Close the workbench with the Intro view opened in full mode. Test that re-launching a workbench opens Intro.

Close the workbench with the Intro view opened in standby mode on a cheatSheet (ie: go to Tutorials and click on any of the links). Test that re-launching a workbench opens Intro with the same cheatSheet opened in standby mode.

Close the workbench with the Intro view opened in standby mode on the SDK Welcome standby page (ie: double click on the Intro view to bring it to the Standby page). Test that re-launching a workbench opens Intro with the SDK Welcome standby page opened.

From a cheatSheet in standby mode, click on Return to Welcome and make sure it takes you back to the last page you were at.

Scenario C: Icons and Toolbar Icon behavior

Go to Help -> Welcome, note the icon for the menu item

Make sure the same icon is used for the Welcome view.

In the welcome view toolbar, make sure that the Home icon is enabled and that it has a tooltip when the view is in full mode.

Make sure that the left and right icons are initially disabled, and are subsequently enabled when you navigate to some of the pages. These icons should be disabled if there is no history to navigate to.

When the welcome view is in standby mode (e.g. when you open a cheat sheet, or double click on Welcome view) the home, back, and next icons are disabled.

Scenario D: Universal Intro

Click the customize button at the top of the Welcome view.

Change the theme, verify that it works. Make sure that you view every page of every theme.

Add and remove some pages, make sure it updates accordingly (some pages may be empty)

In the Overview, Tutorials, etc tabs, move some items to the Available Extensions and confirm that they no longer show up

Move some items to different columns and confirm that the welcome pages react accordingly

Change the priority on some items by clicking the icons. Verify that different priorities appear differently in the welcome.

Scenario E: Platforms

Ensure that Intro uses the HTML presentation when a browser is properly configured on the platform. If the links for all the pages (Overview, Tutorials, etc.) are at the top left corner of the page, you have HTML presentation. If they are centered at the bottom of the page, you have SWT.

Help

Scenario A: Integration with the workbench

These tests exercise interactions with help system from workbench.

 

NOTE this scenario is part of the UA test cheatsheet. The steps used to create the cheat sheet can be found here.

Scenario B: Navigation and browsing

NOTE this scenario is part of the UA test cheatsheet. The steps used to create the cheat sheet can be found here.

The main things to look for here are:

moving around the navigation views (Contents, Search Results, Links, Bookmarks)

opening a book, expanding/collapsing/selecting topic nodes

navigating links inside the help pages

using toolbar actions for displaying the main table of contents, for hiding or maximizing the navigation frame, for topic/navigation synchronization, printing

performing simple searching.

 

Note: this test is to be done on browser adapters that are based on Internet Explorer or Mozilla. For other browser, refer to the "basic" test.

Scenario C: Search

NOTE this scenario is part of the UA test cheatsheet. The steps used to create the cheat sheet can be found here.

Scenario D: Customization

D1: Help preferences and Webapp extensibility

  1. Download the eclipse automated test framework and install into your Eclipse installation.

  2. Download the eclipse automated tests and copy the plugin org.eclipse.ua.tests into your Eclipse dropins directory.

  3. Launch Eclipse, with the parameter -product org.eclipse.ua.tests.testproduct, e.g. eclipse -product org.eclipse.ua.tests.testproduct

  4. From the workbench menu: Help->Help Contents. This should open the help browser. Verify existence of banner frame above search toolbar and a footer at the bottom of the screen.

  5. The page displayed in the content frame should be eclipse.org page.

  6. There should be only two Eclipse books displayed Plugin Developer Guide and JDT Development Guide, in that order. Other Eclipse books should not appear.

  7. Search for "workbench". Verify there are no hits from Workbench User Guide that is not present.

  8. Verify that the Contents, Index and Search tabs show at the bottom of navigation frame, the bookmarks view is not present.

  9. Verify the browser title shows product name.

  10. Verify help UI background is yellow, and green in the toolbars.

  11. Verify fonts are bold sans-serif in the toolbars, and italic sans-serif in the rest of the UI.

  12. Type "active help" (no quotes) in the search field in the browser, and search.

  13. Choose a document titled "Active help" from the search results.

  14. Click once "Open a cheatsheet" link at the bottom of the document.

  15. Verify a message informing that active help is not enabled is displayed, and "Hello World" workbench dialog does not appear.

  16. If you don't have org.eclipse.ua.tests installed stop here

  17. Open Platform Plug-in Developer's Guide/Programmer's Guide in the browser and verify that the breadcrumbs have a gray background

  18. Open Platform Plug-in Developer's Guide/What's New in the browser and verify that the breadcrumbs are in  a black box

  19. In the help view open Open Platform Plug-in Developer's Guide/What's New. Verify that the breadcrumbs and every header is in a black box

  20. Verify that there is an additional tab with a welcome icon, a button over the content frame which is a welcome icon and a narrow frame beneath the content frame titled 'Extra frame inserted by extension point'

  21. Click on the welcome button in the content toolbar, verify that the new view shows. Verify that it allows topic titles to be searched and test it by entering the word 'platform' and hitting the Go button/

 

D2: BIDI

Launch eclipse passing "-dir rtl" option on the command line. Open Help.

Verify Help UI is rendered right to left.

Verify that each page of Intro is usable.

Scenario E: Accessibility

Accessibility support should test for how the help view

uses system colors and fonts,

uses browser's accessibility support,

can be navigated using only the keyboard.

Note: Platform considerations for the accessibility tests:

on Windows things should work as described;

on Linux using the Mozilla browser adapter keyboard navigation works slightly different: Ctrl-Tab and Ctrl-Shift-Tab are to be replaced by Ctrl-F6 and Ctrl-Shift-F6 (this is for frame navigation, see below).

on Solaris/AIX/HP, unless you use Mozilla, the default browser adapter for Netscape 4.x has very little accessibility support, both for fonts/colors and for keyboard navigation.

System colors and fonts:

Observe if the colors and fonts match your system settings. Window backgrounds for navigation views (Contents/Search Results/Links/Bookmarks) should match the system background color (usually white on Windows and Linux). The toolbars should match the button face colour (usually gray). The font should match the system icon font. Highlight colour should match the system highlight colour.

Change your system colors/font settings and see if the changes take effect.

On Windows, try the high contrast settings: Control Panel -> Accessibility Options -> Display - > Use high contrast.

Browser accessibility support:

Windows: in IE - > Tools-> Internet Options -> General ->Accessibility play with the settings for Format. Help should behave accordingly.

DO NOT turn off style sheet support, help will not work well.

Navigation with keyboard only:

 

Here are the keyboard navigation rules for help (as supported by browsers):

 

- To move to the next topic in the left frame, press TAB or DOWN arrow.
- To move to the previous topic , press SHIFT-TAB or UP arrow.
- To expand/collapse a tree node press RIGHT/LEFT arrow.
- Press Enter to select a topic and have its content displayed
- To scroll all the way up/down press HOME/END.
- To go to the next frame (there are quite a number of frames in the help system) press CTRL-TAB (On Mozilla 1.2 press Ctrl-F6).
- To move to previous frame, press SHIFT-CTRL-TAB (On Mozilla >1.2 press Ctrl-Shift-F6).
- To print the current page or active frame, press CTRL-P.

 

F1 infopop keyboard accessibility:

Press F1 (on GTK - Ctrl-F1, on carbon - Help key) in the workbench Package Explorer view . This should open the Related Topics section of the Help view with some text and related links. Tabbing once should focus on the first link ("Views"). Use tab and ctrl-Tab to move up and down the links. Click on a link and the help view should open to the selected topic.

On Windows, use Windows Narrator (or JAWS if installed ), repeat the same tests and see if description an links read out correctly.

Scenario F: Other / Miscellaneous

In addition to the eclipse workbench help scenario, there are two other help uses scenarios: standalone help and info-center (server based) help.

F1: Infocenter

Assuming eclipse is installed as d:\eclipse, change directory to d:\eclipse and run the following command: java -cp plugins/org.eclipse.help.base_3*.jar org.eclipse.help.standalone.Infocenter -command start -data infocenter -port 8888

Open a browser and navigate to http://localhost:8888/help. You should see the help system, but without the links and bookmarks tabs.

Define a working set, search help. Close the browser and reopen. Verify working sets are remembered.

Perform ad-hoc testing or follow the steps for the regular workbench scenarios, with the exception of bookmarks, infopop and active help.

Shutdown the infocenter by running: java -cp plugins/org.eclipse.help.base_3.1.0.jar org.eclipse.help.standalone.Infocenter -command shutdown -data infocenter

F2: Stand-alone help

Assuming eclipse is installed as d:\eclipse, change directory to d:\eclipse and run the following command, replacing the '*' with the actual plugin version information if testing on Windows: java -cp plugins/org.eclipse.help.base_3*.jar org.eclipse.help.standalone.Help -command displayHelp -data standalone

The above command should automatically open the browser so you can browse the help. Perform ad-hoc testing or follow the steps in the regular workbench scenario, with the exception of infopop and active help.

Shutdown the standalone help by running: java -cp plugins/org.eclipse.help.base_3*.jar org.eclipse.help.standalone.Help -command shutdown -data standalone

F3: Basic help ui (any browser with Javascript disabled or browsers other than IE, Mozilla or Netscape6+)

Disable javascript in the browser that is used by the default browser adapter (normally, that's IE on Windows, Mozilla on Linux and Netscape6+ on others)

Launch help and you should see a simple 3-frame layout with links instead of tabs and just the synchronize navigation button available. Books are always expanded and there is no state preservation across views (i.e. when switching between contents/links/search/bookmarks) the page always reloads)

Try ad-hoc testing or follow the regular workbench scenario steps above.

Instead of step 1, you can launch Netscape 4.7 and navigate to the help system URL. To get this URL, right click in the help browser and select "Properties". Check what the URL is.

Cheat Sheet

Scenario A: Search for Cheat sheet

Help/Search

Search for "Creating Rich Client"

Expected result: The cheat sheet "Creating a rich client application" should show in the matches at or near the top of the list of search results..

Expected Result: The icon should be overlapping blue and white rectangles.

Double click on this cheat sheet in the search results.

If this is not a clean workbench and you have run this test before restart all tasks using the view menu.

Expected result: The cheatsheet opens in the cheat sheet view and displays two panes, a tree and a section containing a description of the cheat sheet and a hyperlink.

Resize the cheat sheet view so that it is taller than it is wide.

Expected result: The tree appears above the description.

Resize the cheat sheet view so that it is wider than it is tall.

Expected result: The tree appears to the left of description.

Click on the link "Go to Create a plug-in"

Expected result:  "Create a plug-in is selected in the tree and its description is displayed, with a hyperlink to "Start working on this task".

Start the cheat sheet by clicking on the hyperlink and complete the Introduction and the following step.

Scenario B: Self testing cheat sheet

NOTE this scenario is part of the UA test cheatsheet and is not described here.

Scenario C: Open simple cheat sheet

Help -> CheatSheets

Expected Result: The Cheat Sheet Selection dialog is opened.

Select the radio button "Select a cheat sheet from the list"

Select the cheat sheet  "Java Development/Hello World Application" and hit OK.

Expected result: The cheat sheet opens.

Complete the introduction and the first steps.

Expected result: Next step is "Create a Java project"

Click the "Click to perform" hyperlink.

Expected result: A dialog titled "New Java Project" opens with the cheatsheet docked to the right of the dialog.

Cancel out of this dialog.

Scenario D: History

Click on the cheat sheet view menu.

Expected result: 5 cheat sheets are shown in the dropdown menu as well as "Launch Other ..." and "Restart all tasks". One of the cheat sheets in the list is "Building a Rich Client Application", select that menu option.

Expected result: Building a rich client Application is opened, the selected task is "Create a plug-in" and the introduction and first step have been completed.