CTO's Guide to Prioritizing Business Metrics with Dynatrace

European IT company Cegeka has bolstered its capabilities in Digital Experience Management through the acquisition of Belgian firm Key-Performance. Specialising in ‘observability’ and with expertise in APM solution (incl. Dynatrace), the company is a strategic complement to Cegeka’s proficiency in providing customers with visibility and transparency in their digital experiences. Key-Performance has been assisting companies in monitoring and improving app performance with over 20 years of experience. 

Effective communication between IT and management is key for success. Using the same observability tool and metrics can improve communication. This white paper will show managers how to understand key metrics and make dashboards to monitor their application performance. 

Service Level Metrics (SLM) by importance 

By ordering SLM’s according to importance, you can see your application status in a clear and fast way. The five crucial pillars or groups of metrics, by order of significance, are: 

1. Availability or Uptime: If the application is not accessible, other metrics become irrelevant. 
2. Performance: Slow performance hinders user experience and renders other observations futile. 
3. User Behaviour of the application: Understanding user actions and potential issues. 
4. Critical Paths (The Funnel): Sequences of actions leading to desired objectives, e.g., completing a purchase. 
5. Business Metrics: Tracking metrics related to orders, revenue, etc. 

To keep track of these metrics accurately you need to do 4 things: 

1. Refer to a daily chart for a clear overview.
   
2. Set up an alerting system to proactively notify you of anomalies or issues.
3. Use a single source of data that everyone understands, facilitating better discussions and alignment. 
4. Integrate with other tools (ITSM, Jira, Ansible or Jenkins) to foster a unified workflow, enabling teams to leverage Dynatrace's insights throughout development, deployment, and operations, rather than relying on it solely during alerts.

Pillar 1 : Availability or Uptime 

Uptime metrics tell you if the application can be accessed from various user locations. It is vital to get early alerts to avoid losing revenue and keep a good user experience. Synthetic monitoring (bots interacting with your application) is recommended to identify problems when real visitors (usage) are not present. 

Recommended metrics for availability and uptime include: 

• Error rate
• Errors on critical script executions or pages from proactive bot calls
• Errors encountered by real visitors and the number of impacted users 

Pillar 2: Performance  

Performance metrics are important for user satisfaction and revenue generation. Research shows that an application or website that responds quickly results in longer user sessions, lower bounce rates, and more returning visitors. By tracking performance, you can establish objectives and prevent possible crashes.

 Recommended metrics for performance include: 

• Visually complete time for each page, indicating how long it takes to display the visible part
• Visually complete average for the critical path
• Time to interactive, measuring how long it takes for the page to become interactive
• Apdex (Application Performance Index), indicating user satisfaction with performance
• Third-party performance, assessing their impact on your performance 

Pillar 3: User Behaviour  

Keeping track of user requests and traffic can help you measure the demand on your infrastructure and detect problems. Examining errors can reveal information about the conditions that cause them, such as certain browser versions or operating systems that trigger JavaScript errors. 

Recommended metrics on user behaviour include: 

• User session length (time or number of pages visited)
• Bounce rate, indicating the percentage of visitors who leave the site or application without exploring further
• Error occurrences, identifying the frequency and context of user errors

Pillar 4: Critical Path   

A business manager needs to define and optimize the critical paths for users. These paths show what users want to do, like buying something. Monitoring these paths helps find any problems that stop users from doing their goals.

Recommended metrics for the critical path include: 

• Funnel metrics, tracking the number and percentage of visitors who successfully complete the defined path.
• Performance and availability of critical paths.
• Conversion rate, indicating the number of users who started and completed the process. 

Pillar 5: Business Metrics   

By measuring business-oriented metrics, you can understand how well your application or website is performing and how it affects your business. These metrics let you monitor orders, income, and the results of A/B testing. 

Recommended business metrics include: 

• Order intake and daily revenue.
• Payment processing metrics.
• Average order value. 

Conclusion   

By integrating the key metrics in one overall dashboard, you get a complete view of the situation. You can also drill down for more depth when needed. 

Your context determines what metrics you should choose and how they matter. With the right Service Level Metrics (SLMs) or Key Performance Indicators (KPIs), you can measure how anomalies affect your business and customer experience. Hiring professional help can speed up the process of choosing and provide a holistic view of key metrics. After you make a choice, setting Service Level Objectives (SLOs) allows you and your team to track and accomplish objectives effectively.