# Telemetry module

This module implements APIs to emit telemetry of the [`VegaUIReact-PerfTests`](https://code.amazon.com/packages/VegaUIReact-PerfTests/trees/mainline) activity. The module implements the **KeplerPerformanceMetric** class representing the telemetry metric schema, factory for telemetry client and instrumentation functions.

For details of telemetry service for `VegaUIReact-PerfTests`, please refer to following quips:

* [`Design Document`](https://quip-amazon.com/TSY9AKAYb3Ga/Design-Kepler-Perf-Tooling-Behavioral-Metrics-Collection)
* [`Metric Schema`](https://quip-amazon.com/iiaqA5CpXbXL/Kepler-Performance-Tooling-Kindle-Fast-Metrics-Schema)

## High level design

<img src="https://d32acggsvimfuc.cloudfront.net/KSPE_Metrics_Emittion-High-level-design-Perf-CLI-Perspective-3.jpg?Expires=4128624000&Signature=ImSJg~pda5e0BYBZkF1zSosJyDwADXqfsTMAk-J~77PuCdsicRumtTuvDdcd3kBUX9lxlGAQTlxHKauxcEFOfVp0KJft9YyHHqa0fuXdaUF--ShvcXyORcJaY2wgPbdQfFJjmoVROhtu8E9Ij0G~tnDqY62pIW8lyiaiNHpthbl62W3ILVj3zUCepMQG0BxQVtFSftJGRLG4a4hLiVNWr5Aio-q6zFyOt9~DxkEpANho5fGkeJy0kRwYEmDX8vRbq0ChJMssGJ8MWyY9qGaqyw-7XmeTxZt3CUK-CYttO2-HIKuurla6PkIXcbW-9NNntsApFZbmQnVB-OxBGdEsuw__&Key-Pair-Id=KG9NA58BI3UFQ" width="700"/>

**Kepler Studio Performance CLI** (tooling implemented in the `VegaUIReact-PerfTests` package) serves as both CLI and backend of VSCode extension [`KeplerStudioPerformanceExtension`](https://code.amazon.com/packages/KeplerStudioPerformanceExtension/logs) to provide tooling for measuring and monitoring the app performance for Kepler developers.

To align metric emit among KDTs (Kepler Developer Tool), [`Kepler Insight & Analytics`](https://wiki.labcollab.net/confluence/display/PROJVEGA/KIA+Customer+Documentation) (KIA) has distributed the telemetry server and client API for each KDT owing team to onboard on their tooling. 2Ps has not fully onboarded to KIA telemetry. Kepler Studio Performance team have implemented telemetry server to support 2P clients who have not onboarded to KIA telemetry.

Within **Kepler Studio Performance CLI**, the telemetry factory automatically detect whether the customer platform has onboarded to KIA telemetry to instantiate client which communicate with the backend telemetry server accordingly.

## Usage

### Start telemetry client to communicate with backend server

To start a telemetry client instance which communicate with backend telemetry server to emit metric, you can import the `TelemetryClientFactory` from `telemetry` module and call **setTelemetryClient** member function.

```python
from telemetry import TelemetryClientFactory

TelemetryClientFactory.setTelemetryClient(options=options)
```

The function takes [`Options`](https://code.amazon.com/packages/VegaUIReact-PerfTests/blobs/mainline/--/src/kpi_collector/user_profiles/options.py#L150) which records the context of command call.

Within `VegaUIReact-PerfTests`, the client is instantiated inside [`perf_core.py`](https://code.amazon.com/packages/VegaUIReact-PerfTests/blobs/mainline/--/src/kpi_collector/perf_core.py#L117) at the beginning of the test. Hence, the customer do not need to start telemetry client on their end.

### Get telemetry client

To get telemetry client, call **getTelemetryClient** member function of `TelemetryClientFactory` class.

```python
from telemetry import TelemetryClientFactory

client = TelemetryClientFactory.getTelemetryClient()
```

### Emit metric

#### Create metric instance

To create metric instance to record metric, you can import [`KeplerPerformanceMetric`](https://code.amazon.com/packages/VegaUIReact-PerfTests/blobs/mainline/--/src/kpi_collector/telemetry/_telemetry_metric/_kia_telemetry/_kia_telemetry_metric.py#L29).

```python
from telemetry import KeplerPerformanceMetric

metric = KeplerPerformanceMetric(
    operationId=<OPERATION NAME>,
    subOperation=<SUBOPERATION NAME>
)
```

`KeplerPerformanceMetric` class take 2 arguments as following:
* **operationId**: The name of the operation of the metric being published.
* **subOperation**: Optional argument for another level of granularity for the metric being published.

`KeplerPerformanceMetric` expose 2 members which is called to record metric value to the metric instance.

* **with_metric**: Function to populate the metric type and value to metric object.
  * arguments:
     * **metricType** ([`MetricTypeEnum`](https://code.amazon.com/packages/VegaUIReact-PerfTests/blobs/mainline/--/src/kpi_collector/telemetry/_telemetry_metric/_telemetry_metric.py#L19)): Type of metric. Can be one of *COUNT*, *ERROR*, *FAULT*, *LATENCY*, *VALUE* or *EVENT*.

       For details of metric type, please refer to [`Integration Mechanism For KDT and Telemetry Service`](https://quip-amazon.com/XS2HAPFRfiV9/Integration-Mechanism-For-KDT-and-Telemetry-Service)

     * **metricValue**: Metric value. Either string of integer.

* **with_error_stack**: Function to populate the error stack to metric object.
  * arguments:
    * **err**: Python Exception object to record on metric.

Example:

```python
from telemetry import KeplerPerformanceMetric, MetricTypeEnum

metric = KeplerPerformanceMetric(
    operationId=<OPERATION NAME>,
    subOperation=<SUBOPERATION NAME>
)
metric.with_metric(
    metricType=MetricTypeEnum.VALUE,
    metricValue=100
)
```

You can also use `using_metric_of` with context manager which will automatically release the data stored in metric object for better memory management.

> Note: `using_metric_of` takes same argument as `KeplerPerformanceMetric`.

Example:
```python
from telemetry import MetricTypeEnum, using_metric_of

with using_metric_of(operationId=<OPERATION NAME>, subOperation=<SUBOPERATION NAME>) \
    as metric:
    metric.with_metric(
        metricType=MetricTypeEnum.VALUE,
        metricValue=100
    )
    ...
```

#### Emit to Telemetry Server

To emit metric to server, you can get telemetry client instance from `TelemetryClientFactory` and call **emit** member with metric as argument:

> Note that to emit metric, the metric should either have metric value or error stack logged by calling either `with_metric` or `with_error_stack`.

```python
from telemetry import KeplerPerformanceMetric, MetricTypeEnum, TelemetryClientFactory

metric = KeplerPerformanceMetric(
    operationId=<OPERATION NAME>,
    subOperation=<SUBOPERATION NAME>
)
metric.with_metric(
    metricType=MetricTypeEnum.VALUE,
    metricValue=100
)

client = TelemetryClientFactory.getTelemetryClient()
client.emit(metric)
```

#### Decorate function to emit metric upon completion

You can also decorate the function with `emit_metric` to emit metric with return value of the function as metric value. If function crash, the metric with the error stack will be emitted.

* **emit_metric**: Function to populate the metric type and value to metric object.
  * arguments:
     * **operation_id**: The name of the operation of the metric being published.
     * **sub_op**: Optional argument for another level of granularity for the metric.
     * **metricType** ([`MetricTypeEnum`](https://code.amazon.com/packages/VegaUIReact-PerfTests/blobs/mainline/--/src/kpi_collector/telemetry/_telemetry_metric/_telemetry_metric.py#L19)): Type of metric. Can be one of *COUNT*, *ERROR*, *FAULT*, *LATENCY*, *VALUE* or *EVENT*.

       For details of metric type, please refer to [`Integration Mechanism For KDT and Telemetry Service`](https://quip-amazon.com/XS2HAPFRfiV9/Integration-Mechanism-For-KDT-and-Telemetry-Service

Example:

```python
from telemetry import emit_metric, MetricTypeEnum

@emit_metric(
    operation_id=<SOME OPERATION>,
    sub_op=<SUBOPERATION NAME>,
    metricType=MetricTypeEnum.VALUE)
def some_function():
    ...
    return return_value
```
