The Seek Item feature finds any screen item.

Overview
Recording a valSeekItem command
Parameter List
Additional Notes



Overview  ^

The 'Seek Item' feature allows the user to identify and record a validation checkpoint on any screen item, including those items not represented as individual objects in the Windows scheme of things, for example, bitmap buttons and menu items. The playback engine will know to search for any item recorded this way if the item is not found at its original location. If the item is found somewhere else, subsequent mouse events on that item (and optionally, all mouse events on the same window) will be adjusted to be relative to the new coordinates.

The Problem

Certain Windows objects such as Explorer-style (browser-style) toolbars (e.g. the main IE toolbar or an add-on such as the Yahoo! Companion toolbar) contain a number of 'buttons', to perform different actions, but these 'buttons' are not buttons in the Windows object sense; they cannot be identified by an object id, title or classname. The other main feature of these 'buttons' is that they can be rearranged by the user - or even by the system in response to some event (some add-on toolbars, for instance, will show buttons to suit the context of the current page). The combination of no Windows objects and frequently moving buttons makes it extremely difficult to record any action (namely a mouse-click) on one of these toolbar buttons in such a way that it will always play back successfully, regardless of where the button is on the toolbar. Using the Windows object scheme only the toolbar itself is recognized as an object, and the click on that toolbar is mapped to coordinates on the whole toolbar. Once the buttons have been rearranged, the next playback will send a click event to the wrong button.

Items on a menu may likewise be changed or moved; consider the File Menu 'recent files' list provided by most applications. An item on a changeable menu may need to be searched for within the menu in order to send the next menu message to the correct place.

The 'Seek Item' feature can be used in many other contexts also. For example some Java Application do not use Windows objects at all. The only Windows object is the main application frame itself. This makes recording events on such a GUI difficult, especially if some of the objects are known to be positioned differently from one execution to the next.

The Solution

TestSmith overcomes this problem by identifying such objects using a small bitmap of the central pixels, which is recorded as an MD5 hash (see the vpixelhash parameter). This requires a simple action to be taken when recording (described below), and guarantees that on playback, if the item is not found at the expected x,y coordinates, it will be searched for within its main application window (or if it is a menu item, even outside the bounds of its main application window).

Once an item has been located, subsequent clicks on that item can be adjusted to the item's new x,y coordinates. The adjustment is set up automatically during the recording for left mouse-clicks, but other mouse actions can have the adjust parameter manually set if required.


Recording a valSeekItem command   ^
  1. Activate the window containing the object of interest.

  2. Position the mouse pointer at the center of the key feature on the item of interest. For a toolbar button the 'key feature' would usually be the bitmap image, but the button text should be considered as the 'key feature', if the same bitmap is used on more than one button. For a menu item the 'key feature' is the menu item text itself. Try to position the mouse pointer as close to the center of the key feature as possible (but don't worry, absolute accuracy is not essential).

  3. Hold down the Control (Ctrl) key and press the appropriate hot key. By default the seeking is optimized for speed, at the expense of accuracy. This means that on rare occasions the playback engine will - incorrectly - match an almost-similar item to the recorded vpixelhash. To optimize seeking for accuracy at the expense of speed (it will be ~3 times slower) hold down the Shift key along with the Control key before hitting the hot key.

  4. Continue recording as normal. If the adjust parameter is not recorded on subsequent mouse commands it can be manually added, as required. Be careful when adding it to mouse commands that are specified for a different Windows object, i.e. not the Windows object that contains the item. You should be absolutely sure that adjustment is required on this other object.

Parameter List  ^

The full parameter list is shown with the valSeekItem command in the TestSmith Commands page of the User Guide.

Additional Notes  ^
  1. Clicks, and other mouse actions, on drop-down menus generated by Explorer-style toolbar buttons will cause a warning (WRN) to appear in the TestSmith Report, stating "Mouse event is occurring outside the Windows object." This is because the toolbar window size is specified by the visible toolbar itself (excluding the drop downs). To avoid this message appearing set the suppress parameter to suppress=W for any command that produces this inaccurate warning message.

  2. If any item being sought is on a drop-down (a.k.a. pop-up) menu that overlaps the main frame (i.e. is outside of the main frame's confines) the voutside parameter should be set at voutside=1 to ensure the whole menu is included in the search.

  3. When searching a menu vertically (on the y-axis) for an item it is important that the mouse cursor does not go over the items on the main menu bar otherwise a new menu may pop up for a different (main menu) item. TestSmith handles this problem in a default way; it assumes all main menu bars are of the newer, movable style of menu and are at the default position (i.e. directly below the window's title bar and suppresses mouse-over actions at that position. This works in the majority of cases, however please note the following:
    • If the main menu bar of the application is of the older 'fixed' type of menu bar the vmenufixed parameter should be set at vmenufixed=1 to ensure successful playback.
    • If the moveable menu has been moved to a new position, e.g. below the main button bar the mouse-over suppression will not work and the command may fail.

  4. A toolbar button that moves from the main toolbar to the drop-down menu part of the toolbar, or visa-versa, will not be found. To seek for an item in both places some additional manual tweaking of the script is required. This dual-seeking is not officially supported at this time; it may or may not work.