In Sysdig Monitor today we have 3 key concepts as explained in the image below:
1 ) Grouping - A grouping is a hierarchical organization of labels. It is used to organize your infrastructure view in the Explore page. For example, namespace.name > deployment.name or host.hostname > container.name are groupings.
2) Scope - Scope is collection label-value pairs used to filter out data. Scope can be found while defining dashboards, alerts, teams and more. For example, namespace = prod and deployment contains java are scopes.
3) Segment - Segments are used to split aggregated data by labels , thereby generating multi-series comparisons, multiple alerts and so forth. For example, segment by container.id will split a time series into multiple lines.
1. Grouping : First, lets talk about the changes for grouping. In Sysdig Monitor, earlier , we allowed arbitrary hierarchies. While this provided flexibility , it often led to inaccurate data. For example you could create a grouping configured as pod.name > host.hostname. Now a host can have many pods from many different deployments running on it. If you try to view host metrics like host.count at the pod level, it would be erroneous.
With the new release, we are going to filter out labels that don't make sense to be a part of the hierarchy based on the previous selection. Ensure you follow a logical hierarchy.
An example of logical groupings for Kubernetes is shown below.
2. Explore Scope : Second, we have done a bunch of work to provide applicable scope and segmentation for metrics. In Explore page, if a scope is selected by row selection, only the applicable dashboards and metrics will show up in the drill-down page. Appropriate message will be displayed for invalid selections. Note: custom metrics such as StatsD will be filtered based on scope in a subsequent release.
3. Dashboard and Alert Scope and Segments: Third, when you are viewing a dashboard or alert, if you had applied scope and segments that does not follow the new logical hierarchies, then you will get a visual indicator asking you to change the selection. For example, host level metrics like host.count are not applicable to a container.
The dropdowns for each field has the right list of values. Pick from the new dropdown list to create accurate charts and tables. Follow the logical hierarchies to ensure you can scope and segment by the right values.
4. Team Scope: Fourth, for teams, if you want to scope the team by orchestrator metadata like kubernetes.namespace.name = x or swarm.service.name = y you must choose Scope by 'Container'. For example if you created a Team scoped by 'Host' but attempted to use container metadata such as kubernetes.namespace.name, you will need to modify the team to Scope by Container.
Also note, Kubernetes State metrics will be available only for pure Kubernetes groupings i.e., groupings in which the hierarchy is formed with kubernetes.* labels only
5. Using in-built kubernetes cluster ID/name instead
If you are using groupings and scopes that have agent tags to identify clusters consider switching to the newly introduced kubernetes.cluster.name or kubernetes.cluster.id out-of-the-box labels. For example lets say you have a grouping or scope: agent.tag.cluster -> kubernetes.namespace.name -> kubernetes.deployment.name , please change it to kubernetes.cluster.name -> kubernetes.namespace.name -> kubernetes.deployment.name.
Sysdig populates kubernetes.cluster.name with the value from the agent yaml config k8s_cluster_name. You can choose to set k8s_cluster_name = agent.tag.cluster in order to continue seeing the exact same values as you have now.
Use the guidance above and the visual guidance in the application to create the right groupings, scope and segmentation to avoid data inaccuracy. If you have any questions, please feel free to reach out to customer success via Live Chat or to support.