Query Builder

Graphing

Graphs are the way you can visualize the data coming from your monitoring systems. Most of the times that you visit OddEye, you look at dashboards made up of graphs. Graphs are at the heart of monitoring and observability, so it is essential to understand how to define great graphs.

We have two modes of interacting with Graphing subsystem:

  • Graphical editor (Easy but powerful method to create charts using UI tools)
  • Json Mode Editor (The most complete and powerful way for creating charts using Json editing interface)

When you start creating charts, graphical editor with empty dashboard will appear.
Here you can:

  • Set name for for this dashboard (default name will be something like "Dashboard #N")
  • Create Row (At least one row is mandatory) This is for grouping Charts inside Dash.
  • View or edit entire Dashboard in Json mode (Useful if you want to export entire dashboard)
  • Delete dashboard
  • Save dashboard after making changes

MKDash

After adding new row we can create charts inside that raw. Number of rows is not limited so you can as have much as you want rows. Here you can: HeatmapqHeatmap Set name for the raw Create Charts View/Edit row in Json mode Save the row.

MKRaw

At the moment of writing this document we had 3 main types of charts:

  • Single stat (single value of one metric)
  • Heatmap (Heatmap charts)
  • Graph (Line, Bar, Pie/Donut, Funnel, Treemap, Gauge)

Signgle Stat and Heatmap charts are special ones so these charts cannot be converted to other types. Graphs (Line, Bar, Pie/Donut, Funnel, Treemap, Gauge) are more common so you can change types of these types whenever you need it.

Single Stat

To create Single Stat chart click on Single Stat button at Row's menu. App will redirect you to Query Builder for Single Stat chart.

Single Stat

At first (General) tab you can set name and size for chart

Metrics tab is where you should create you query for getting needed data. Here you should select tags like Cluster, Hostname, etc ... fo getting exact value and actual name of metric. Tags are for making queries which exactly matches you need. Selecting metric is the only mandatory part for getting values. Just click on + button and start filling your values. Autofill field will appear to help you easily get tags and metrics that you need. When you complete writing filter, vlivk on v button to execute the filter or x to cancel it.

SS Query Metrics

Metrics builder comew with two types of Aggregators: for aggregating values of different metrics and for downsampling by time interval. Foe example you can query for cou_user of group of servers, aggregate two values into single one by average of values and then downsample it to 99percentile for 10 minutes interval. After all just set unit of measurement of metrics, and you will see fill picture in nice format.

SS Query Metrics

At Display tab you can set Background/Font colors and font sizes. Time Range tab for overriding default time ranges and refresh intervals.

SS Display Time

The last tab will open visual Json Editor where you can alter all parameters of charts. Be ware that Json Editor is only for Advanced users.

Json Editor

Heatmap

Heatmap

Heat maps show the distribution of values for a metric which evolves over selected time interval. Each column shows a distribution of values during a particular time slice. Where each shading shows the amount of entities reporting that particular value during that particular time
Illustration above displays the amount of Servers with Old Gen GC pauses of ElasticSearch cluster during last 3 hours downsampled with 2 minutes intervals with max value aggregator. So we can easily determine the issues which servers and how many servers suffers from long time GC pauses.

The procedure of creating queries for Heatmap is similar to Single Stat with one main exception.

  • You need to set appropriate downsampling with aggregator in order to see valuable information .

This is because we need fixed time interval for grouping similar values and showing it for same time interval. So in Heatmaps are showing grouped values aggregated in selected time interval, and its essentially useful when you have so much values to monitor that viewing it in Line Charts will make it hard to understand full picture. Do demonstrate the different refer to next illustration which shows same information but in Line Charts

LineHeatmap

These two charts shows exactly same information with difference of several minutes. So you can see that it's very hard to determine exact source of exact values, especially when tooltip of Line Charts at each point of time will show reports of all servers matching queries parameters.
Usually Heatmaps are called 3 dimensional charts, which quite correctly describes the nature of Heatmap charts.

Heatmapq query builder consists of following tabs:

  • General (Contains titles and dimensions of chart)
  • Metrics (Main query builder interface)
  • Axes (Settings to X and Y axes)
  • Data Zoom (Settings for Zooming chart)
  • Legend (Colors and backgrounds of Legend of chart )
  • Display (Various parameters for displaying)
  • Time Range (Selected time range and auto refresh interval)
  • Json (Visual Json editor fir advanced users and use cases)

Metrics tab is pretty similar for all types of charts, the biggest difference for Heatmaps is the need of adding downsampling with appropriate aggregator

HM Metrics Editor

Axes is another important tab for making Heatmaps Here you can set various configuration option for X and Y axes.
All parameters of Axes tab contains predefined defaults so making changes here is not mandatory. Parameters for Y access are:

  • Text for axes
  • Position of Axes in chart (Right, Left)
  • Split Number: How many splitting levels should have Y axes
  • Colors and Font sizes
  • Units for measurements for particular metric

Parameters for X are are mostly decorative and self explaining:

HM Axes

Data Zoom let's you can Enable/Disable zooming of chart as well as set types of zoom for each axes. There are two types of Zooms :

  • Inner (Will not show any zoom label and will do zooming when you scroll your mouse holding alt key of keyboard)
  • Scroll (will diplay seprate zoom label which you can use to select and zoom needed time range)

Legend tab allow you to decorate fonts and borders or entirely enable or disable showing of legend .

Display Setup display parameters Heatmap Here you can:

  • Enable/Disable , Set size oand position of Heatmaps color map
  • Select color scheme for you heatmap (over 300 schemes are available),
  • Enable/Disable showing labels for each bucket of heatmaps
  • Setup positions and labels for

HM Display

Lines

Lines

Line graphs are the simplest and in most cases the default way to visuslize the data. To create new Line chart click on Add Graph widget button at Edit Row menu

Add Lines

This will create dummy line chart and open UI to edit the chart:

Add Lines

As for all other chart types General tab allows to set names and dimensions for chart.

Metrics tab is the first and the only mandatory one for creating charts. Click on Add button to start editing the chart:

Line1

Tags are a way to add dimensions to your metrics , so you filter your data to match you needs. With tags you can select only that part of data which you wan to see chart. For example if you have servers in two locations (us-east-1 and eu-west-1) and want to create chart to see network activity of servers located ate us-east-1 region. You will select Tag Location and chose us-east-1. OddEye have limit of 8 tags per metric, so you should build you query with keeping that in mind.

Metrics Here you can select desired metric(s) for this chart.

Aggregator is for aggregating multiple metrics or tags into one. For example selecting avg from drop-down menu of Aggregator you will see ones line showing average traffic from all servers in location us-east-1.

Down Sample allows to aggregate metrics for particular period of time. In forst field you should write time interval in number value format without spaces. For example 10m (10 minute) or 20s (20 seconds) or 2h (2 hours) etc ... We have number og aggregation algorithms like Average, Max, P99, Count etc ... Select desired one and you will see downsampled chart. Downsampling time frames and Aggregators are muttable, so you can change it anytime. You can also entirely disable Downsample by Clicking on Disable downsampling button.

Rate button converts increasing counters into rated values. For example if agent is reporting increasing counters of accessing your web service since its start, but you want to see hits per second.

Inverse Simple multiplies your values with -1 so you have same value but with minus value. This is useful if you want to compare values of similar metrics without having to create complicated queries

Image below illustrates how we can see sum of sent and received traffic for group of servers, for device eth0, for last 3 hours, Down sampled with average value for metrics in 1 minute and inverse value of transferred traffic for better comparability.

Line2

Next Tab is Axes. This allows to add/remove and configure X and Y axes as well set it's positions
With Axes control you can:

  • Decorate fonts and colors of X and Y exes
  • Set positions and number of splitting points
  • Add or Remove Axes
  • Set measurement unites for Y
  • Set Scale type of X
  • Set minimum and maximum values for Y

Line Axes1

Data Zoom Tab allows to control zooming of X and X Axes. OddEye have two types of Zoom

  • Inner: this will allow zoom in and zoom out axes from position of your mouse by pressing alt key of keyboard and scrolling the mouse wheel.
  • Scroll: Will display seprate zoom line where you zoom in/out and more zoomed positin .

You can have separate Zooms with different types assigned to Axes

Line Zoom

Legend Tab is just for showing/Hiding and decorating the legend og chart. In our case its butes_rx and bytes_tx at middle of upper side of chart.

Display* Is one of the most important tabs in Query builder. The most important think is that Display Tab allows to change types o charts. Yuo can chose between Lines, Bats, Pie, Gauge, Funnel, Treemap. When you select Type, Parameters of display tab will be changed in accordance to needs of that perticulat type of chart. For line charts you will see following option:

  • Set background colors and transparency
  • Switch on/off the animation
  • Switch on/off Labels
  • Set styles and positions of Labels
  • Staircase the chart
  • Control the animamation of chart
  • Stack/Unstack Line graphs
  • Draw points ofr Line Graphs

Line Display

Time Range tab is to override default time range and refresh interval of chart.

Json opens visual Json Editor

Bars

Bars

Bar graphs are essential for viewing metrics over time as well as for using series as scale for X access. Image above displays current values for cpu_user, cpu_sysem and cpu_idle for certain group of 3 servers, where Axes Y is value and X is tag host . Query for above chart looks like this:

QBars

Here I have removed cpu_ text from metric name and took only first part of FQDN of server. Just for making texts shorter and more informative.

Axes Tab looks exactly as for Lines, except that I have changed Scale for X from Time to Series . By default Alias 1 will act as Y and Alias 2 a X

Dta Zoom, Legend, Time Range and Json tabs have the same functionality for all widget charts, so let see what is different in Display tab.

DBars

Essential parts here are Stacked button and Labels. Labels can be placed inside and around of Bar and can have following labels or combination of labels :

  • {a1} : Alias 1
  • {a2} : Alias 2
  • {value} : Value of metric

Stacked button can be used to stack values by Alias 1 and for our case see CPU usage of machine split by cpu_user, cpu_sysem and cpu_idle

DBars

Pie , Donut

Pie

The process of creation of Pie charts is almost identical to Bars, these are very similar charts and the only difference is with decorations and overall look and feel. So from previous example with Bar charts, we can just take Bar chart and change type to Pie. We can change height and width of Pie and get Donut chart. In example below I have also replaced Aliases 1 and 2 to get more informative chart .

Donut

Funnel

The Funnel chart is used to visualize the progressive reduction of data passing from one phase to another. Data in each of these phases is represented as different portions of 100% (the whole). Like the Pie chart, the Funnel chart does not use axes. Like Pie chart Funnel can be sized decorated using Display* tab of Query builder. All other parameters for Funnel ar the same as with Pie charts.

Funnel

Gauge

Gauge is used to visualize position of value of metric of 100%. Thus it's meaningless to use Gauge charts for values of metrics which are not showing percentage. Also its more useful to create Separate Gauges per single metric. Example below shows percentage uf used memory per server .

Gauge

While there is no hard limit for showing values from multiple metrics in single Gauge.

Treemap

Treemaps shows tree-structured data as a set of nested rectangles. Data, organized as branches and sub-branches, is represented using rectangles. The treemap chart shown below showcases values of inbound and outbound network traffic for set of servers of specified cluster. The size of each rectangle represents the amount of value, the color represents the host which reports that value, the size of rectagle is proportional to reported value.

Treemap

Creation and decoration of Treemap chart is pretty much similar to Pie, and Funnel, there are no special differences in Query builder for threemap.

Json Mode

The reason for having two composers is to provide Easy and Advanced chart building mechanisms. While Visual composer is quite feature rich it may appear limited for those who wants to have full flexibility and create sophisticated charts like this:

Smile

This chart shows the power of Json Editor interface where we have create Emoji chart with combined Donut, Pie , Gauge and Funnel chart-types. Flexibility here comes with complexity so knowledge of Json E-Charts is required to play with Json Editoe

Dsplay