# UIDeviceBridge module

This module implements the **UiDeviceBridge**, a driver component which provides communication with the UI automation service in Kepler device for UI interaction, and its factory. The module also implements the **ScreenContext** and **UIElement** which represents the UI tree and element of the Kepler app.

## Usage

### UiDeviceBridge life cycle
#### Connect with UI automation service in Kepler device

To start communicate with UI automation service, developers can import **UiDeviceFactory** from `ui_automation_bridge` module to call **set_device_bridge** member function.

```python
from database import Database
from ui_automation_bridge import UiDeviceFactory
from user_profiles.options import Options

options = Options()
output_dir = ...
test_case = ...

db = Database(options, test_case, output_dir)

UiDeviceFactory.set_device_bridge(db=db)

```

**set_device_bridge** takes **Datebase** object as argument, which stores the test session context. 

Within [`perf_core.py`](https://code.amazon.com/packages/VegaUIReact-PerfTests/blobs/mainline/--/src/kpi_collector/perf_core.py#L521), the **UiDeviceBridge** is instaniated thru **UiDeviceFactory** for *KPI Visualizer* and *KPI Collector* tooling when `--use-appium` options is passed to `perf` command so that developers do not have to instantiate **UiDeviceBridge** instance for the tooling. In the future, if the Kepler Performance team find more use case, **UiDeviceBridge** will be supported in other toolings as well.

#### Get UIDeviceBridge instance

Once the **UiDeviceBridge** instance is instantiated, developers can call **get_device** to get the instance.

```python
from ui_automation_bridge import UiDeviceFactory

ui_device = UiDeviceFactory.get_device()
```

developers can use **UiDeviceBridge** to communicate with UI automation service in Kepler device to perform UI interaction. For details of the **UiDeviceBridge** APIs, please refer to [Perform UI interaction thru UiDeviceBridge](#perform-ui-interaction-thru-uidevicebridge).

#### Release UIDeviceBridge instance

When the test completes, developers can call **release_device_bridge** to release the resource for communication with UI automation service and disconnect with the service.

```python
from ui_automation_bridge import UiDeviceFactory

ui_device = UiDeviceFactory.release_device_bridge()
```

Within [`perf_core.py`](https://code.amazon.com/packages/VegaUIReact-PerfTests/blobs/mainline/--/src/kpi_collector/perf_core.py#L574), the **release_device_bridge** if the **UiDeviceBridge** instance is instantiated. Hence, the developer do not need to release the **UiDeviceBridge** on their end.

### Perform UI interaction thru UiDeviceBridge

**UiDeviceBridge** expose following public members for developers to perform UI interaction with the Kepler app thru UI automation service.

* **reconnect**: Function to reconnect to UI automation service in Kepler device. Use if
  * If developers are rebooting the device between tests
  * If developers see exceptions raised from **UiDeviceBridge** due disconnection.
* **get_screen_context**: Function to get UI element tree of the Kepler app's page at the time point of the function execution. Return `None` object if runtime exception is catched. Developers may use **reconnect** if the runtime exceptions are due to disconnection with UI automation service.
  * returns:
    * **ScreenContext**: A parsed screen context object containing UI hierarchy. `None` object is returned If any step fails during the process.
* **find_element_by_xpath**: Function to get **UIElement** from the Kepler app's page at the time point of the function execution using Xpath. XPath is language to query XML document which is used in Appium to search for element. For details of the XPath syntax, please refer to [`Using XPath Selectors with the Appium Kepler Driver`](https://tiny.amazon.com/cuzui7h4/wikilabcnetconfdispPROJUsin) page.
  * arguments:
    * **xpath**: XPath to search for UI element in string.
  * returns:
    * **UIElement**: UIElement found. If XPath is invalid, or element is not found, `None` object is returned instead.
        
* **click**: Function to click on the Kepler app pages coordinate.
  * arguments:
    * **x**: X coordinate to click
    * **y**: Y coordinate to click
  * returns:
    * **bool**: *True* if the click succeeded else *False*
* **swipe**: Function to click on the Kepler app pages from start to end coordinate.
  * arguments:
    * **x1**: X coordinate for the swipe start point
    * **y1**: Y coordinate for the swipe start point
    * **x2**: X coordinate for the swipe end point
    * **y2**: Y coordinate for the swipe end point
    * **interval**: The time interval between each movement along the path, in unit microsecond
  * returns:
    * **bool**: *True* if the click swipe else *False*
* **inject_input_key_event**: Function to inject key event. Analogous to pressing keys in remote control.
  * arguments:
    * **input_key_event_code**: Key event code. For supported key event code, please refer to [`Appium Commands`](https://developer.integ.amazon.com/docs/kepler-tv/appium-commands.html#:~:text=button.id)%7D%5D)-,D%2Dpad%20navigation,-The%20execute_script%20function) page
    * **hold_duration**: Time to keep the button pressed, in milliseconds
  * returns:
    * **bool*: *True* if the key press succeeded else *False*

Example:

```python

from ui_automation_bridge import UiDeviceFactory

...

# Assuming UiDeviceFactory.set_device_bridge was called from
# perf_core.py and connection with UI automation service in
# Kepler device is established.
ui_device = UiDeviceFactory.get_device()

# Get UI element tree of the current app page
screen_ui_tree = ui_device.get_screen_context()

# Find element in current app page using XPath
element = ui_device.find_element_by_xpath(
    "//*[@test_id='fadeContainer-ro-RO']"
)

# Click on coordinate (100, 200) of the current app page.
ui_device.click(100, 200)

# Swipe from coordinate (100, 200) --> (300, 400) of
# the current app page with interval of 500 ms.
ui_device.swipe(100, 200, 300, 400, 500)

# Send key event to press HOME button of your remote.
# control
ui_device.inject_input_key_event(170)

# Reconnect with UI automation service.
ui_device.reconnect()
```

### Analyze app UI

Once developers get the UI tree of the app page thru **get_screen_context** or find element with XPath by calling **find_element_by_xpath**, developers can analyze the UI elements of developersr app page.

#### Search for UI element shown in the app page

Once developers get **ScreenContext** object representing developersr app page, developers may can explore the UI element tree to find elements according to the element attribute. Following are the list of public members exposed for developers to search for UI element in shown on the app page.

* **get_focused_element**: Returns the focused UI element on the app page. Prioritizes elements in windows with zOrder = 0, which represents the topmost window in the UI hierarchy.
  * returns:
    * **UIElement**: **UIElement** object representing the focused UI element on the app page. Returns `None` if focused elements are not found.
* **get_all_focusable_elements**: Return list of UI element which are focusable.
  * returns:
    * List[**UIElement**]: List of **UIElement** object representing the focusable UI element on the app page. Returns `None` if focused no elements in the app page are focusable..
* **get_element_by_id**: Find an element by its ID.
  * arguments:
    * **element_id**: ID of the **UIElement** to search for.
  * returns:
    * **UIElement**: **UIElement** object whose `id` match to the argument **element_id**. Return `None` if no element whose `id` value match to the argument **element_id**.
* **get_element_by_test_id**: Find an element by its test_id.
  * arguments:
    * **test_id**: The test_id to search for.
  * returns:
    * **UIElement**: **UIElement** object whose `test_id` matches the argument. Return `None` if no element with matching `test_id` is found.
* **get_elements_by_role**: Find all elements with a specific role.
  * arguments:
    * **role**: The role to search for.
  * returns:
    * List[**UIElement**]: List of **UIElement** objects with matching role.
* **get_elements_by_description**: Find all elements with descriptions containing the given pattern.
  * arguments:
    * **description_pattern**: The pattern to search for in descriptions.
  * returns:
    * List[**UIElement**]: List of **UIElement** objects with matching descriptions.

Example:
```python

from ui_automation_bridge import ScreenContext, UiDeviceFactory, UIElement

...
# Assuming UiDeviceFactory.set_device_bridge was called from
# perf_core.py and connection with UI automation service in
# Kepler device is established.
ui_device = UiDeviceFactory.get_device()

# Get UI element tree of the current app page.
screen_ui_tree = ui_device.get_screen_context()

# Get focused element in current app page.
focused_element = screen_ui_tree.get_focused_element()

# Get list of focusable elements.
focusable_elements = screen_ui_tree.get_all_focusable_elements()

# Search for UI element whose ID match to "123456".
element_matching_id = screen_ui_tree.get_element_by_id("123456")

# Search for UI element whose test_id is "button-submit".
element_by_test_id = screen_ui_tree.get_element_by_test_id("button-submit")

# Find all button elements.
button_elements = screen_ui_tree.get_elements_by_role("button")

# Find all elements with descriptions containing "settings".
settings_elements = screen_ui_tree.get_elements_by_description("settings")
```

#### Check UI element attributes

Once the **UIElement** representing the UI element of interest is found using either **find_element_by_xpath** of **UiDeviceBridge** or using the members of **Screen** in [`Search for UI element shown in the app page`](#search-for-ui-element-shown-in-the-app-page), developers can check the attribute of the UI element using the following member functions.

* **get_parent**: Returns the parent element if it exists, `None` otherwise.
  * returns:
    * **UIElement**: The parent element or `None` if this is a root element.
* **is_focusable**: Check whether the UI element is focuable.
  * returns:
    * **bool**: Indicator of UI element is focuable.
* **is_focused**: Check whether the UI element is focused.
  * returns:
    * **bool**: Indicator of UI element is focused.
* **get_focus_value**: Check whether the UI element is focused.
  * returns:
    * **int**: The focus value or `None` if this is element is not focusable or does not have focus value.
* **get_bounds**: Get bound property of UI element.
  * returns:
    * List[List[int]]: List of x, y coordinate pairs of UI element vertices.
* **get_path_to_root**: Returns the path from this element to the root.
  * returns:
    * List[**UIElement**]: List of UI elements between this element to the root in that order.
* **get_siblings**: Returns all siblings of this element (excluding self).
  * returns:
    * List[**UIElement**]: List of sibiling UI element sharing same parent.
* **get_test_id**: Get the test_id of this element
  * returns:
    * **str**: Get test ID of this element if exists, `None` otherwise.
* **get_description**: Get the description of this element.
  * returns:
    * **str**: Get description of this element if exists, `None` otherwise.
* **get_trait**: Get a trait value from the element's traits.
  * arguments:
    * **trait_name**: The name of the trait to retrieve.
    * **sub_trait**: Optional sub-trait name within the trait.
  * returns:
    * **Any**: The trait value or `None` if not found.
* **click**: Click the UI element
* **send_keys**: Send string to the UI element.
  * arguments:
    * **keys**: A string for typing, or setting form fields.


Example:
```python

from ui_automation_bridge import ScreenContext, UiDeviceFactory, UIElement

...
# Assuming UiDeviceFactory.set_device_bridge was called from
# perf_core.py and connection with UI automation service in
# Kepler device is established.
ui_device = UiDeviceFactory.get_device()

# Get UI element to check the attributes
element = ui_device.find_element_by_xpath("//*[@test_id=\"title\"]")

# Get parent of the UI element.
parent = element.get_parent()

# Get siblings of the UI element.
siblings = element.get_siblings()

# Get list of elements between the UI element
# and root ordered from the UI element upto the root.
parent = element.get_path_to_root()

# Get test ID attribute of the UI element.
test_id = element.get_test_id()

# Get description of the UI element.
description = element.get_description()

# Check if the UI element is focused.
is_focused = element.is_focused()

# Check if the UI element is focusable.
is_focusable = element.is_focusable()

# Get focus value of the UI element.
focus_value = element.get_focus_value()

# Get boundary of the UI element.
boundary = element.get_bounds()

# Get a trait value from the element's traits.
visibility_trait = element.get_trait("visibility")

# Get a specific sub-trait value.
z_order = element.get_trait("visibility", "zOrder")

# Click element
element.click()

# Send keys to element
# For example, send ID to the login text box
element.send_keys("id_for_login")
```

### Support for dump.json Format

The `ScreenContext` class now supports parsing the new response format, which represents the UI hierarchy in a Kepler device application. This format includes multiple applications and windows, with additional metadata like zOrder and traits.

When processing a UI dump, the `process_ui_tree` method automatically detects whether the input is in the original format or the new response format and handles it accordingly.
