In part one of this series I introduced the Microsoft Call Quality Methodology and have a high level overview of what CQM was and how you could start your journey to improving user experience for your Lync or Skype for Business Server deployment. In Part 2 we dive into running Key Health Indicators or (KHI) for short on your servers. Microsoft has published KHI guidance for your Lync and Skype for Business Servers. The notion is that if your servers are not running optimally then you might expect a less than desirable user experience. In part 3 of this series we dive into running Call Quality Metrics (CQM) Reports to gain insight into the users’ call quality experience.
CQM is a scorecard built in an Excel spreadsheet which loads all the CQM query results and creates a number of views that will help you identify, analyze and fix call quality issues in your Skype for Business deployment. The data used is from the existing Quality of Experience (QoE) and Call Detail Record (Cdr) databases that have been deployed so these are a prerequisite before attempting to run CQM reports for your organization.
Getting Started
1. Ensure you have deployed Skype for Business Monitoring which will enable the capturing of vital call quality metrics and detailed call records for the users of your Skype deployment. You will need SQL Server with SQL Server Reporting Services to be able to do this. Ensure data is properly being captured by running the Microsoft built in reports that are included with Monitoring.
2. Download CQM and copy the contents to any Skype server with the administrator tools deployed such as a front end server. The extracted folder structure will look as follows:
Figure 1: CQM folder structure
3. Open a PowerShell window and change directory the folder where you copied the CQM extracted files and run the following to gather data:
.\CQM.ps1 -StartTime "10/01/2016" -EndTime "11/11/2016" -QoECdrInstance "SQLServer01.mydomain.ca" -QoECdrVersion "SfB2015" -DataRoot "D:\Microsoft Call Quality Methodology\Results"
Where,
StartTime is the first date in the range of data you would like to analyze.
EndTime is the last data in the range of data you would like to analyzeQoECdrInstance is the FQDN of the SQL server where you have deployed the QoE and Cdr databases.
QoECdrVersion is your current deployment version "SfB2015" or “Lync2013” are the only valid choices
DataRoot is where you would like to store the results
If the cmdlet is run correctly you will see the following:
It may take a few minutes depending on the volume of data you are collecting. The script will create a new folder within the folder you specified as the dataroot with the current data. If you run another report on the same day it will overwrite that data.
Generating the CQM Excel Output
Once the script has finished running you want to copy the folder created in the folder you specified in as the DataRoot folder to your local workstation which has Office installed as well as a copy of the downloaded CQM files.
1. Find the Microsoft Call Quality Methodology Scorecard for Skype for Business Server v15.xlsm file and open it.
2. You will see a multi tabbed Excel file and you should be on the “Start” tab which shows the following:
3. We will follow steps 1-11 next.
4. Try using the default settings for steps 1-4 as they are generally correct for most deployments. The Streams Cutoff number specifies how many streams a given row needs to have to be taken into consideration for Top 10 Issues. In this way, you can filter out rows with only a few streams included in the results, many of which are poor few streams, since these will skew Top 10. The larger the environment the larger you should make this setting.
5. If you want to remove the results from the weekends click Remove Off-Work Day results. This will remove you days you set as off work days above from the reports and graphs.
6. Click on each of the buttons from #7-#11. After clicking on each one you will be taken to the corresponding tab in the spreadsheet. Return to the Start Tab to click on the next button until you finish clicking on them all.
Looking at the data
The very first screen shown will be your Scorecard. This represents the overall score of the period sampled:
Figure 2: CQM Scorecard
At a basic level, you are looking to ensure all your scores are in the “Green”. The idea behind the CQM is to continually be making improvements in the areas required until you achieve a level Green across the board.
The provided documentation will give explanations to all the various metrics that are measured and possible root causes. In general, you will want to take a top down approach using the score card. Starting with the “Red Road” at the top ensure your Server Health is good. If you previously ran KHI’s and rectified any issues, then you should expect to see Green here.
Next move on to the Last Mile or the Blue Road which measures the quality of the audio stream itself for both wired and wireless networks.
Next, the Green Road looks at Devices and System performance.
Lastly unmanaged streams are looked at as well as reliability dial tone, conferences and P2P sessions.
While the default targets are a good starting point, you can update the targets to a score of your choice and then update the scorecard to which will show you where you rank based on the new targets. I find this useful to understand where mean scores are at within organizations.
Useful Metrics
There is a plethora of data provided from the CQM report. Here are some examples of a few:
Next Steps
Hopefully this has shown you how you can use CQM to improve the call quality across your Skype for Business deployment.
In summary, you should:
· Run the CQM Scorecard to:
· Get an idea which areas are problematic and prioritize where you will focus first. Look at the StreamDistribution and TrendCharts tabs to prioritize elements by poor streams
· Establish an initial baseline by going through the three “road” as show in the Scorecard tab
· Analyze the results and create a list of the elements you need to focus on. For example,
· Server Health
· Devices
· Wired Subnets
· For each of the items on your list, use the scorecard to analyze which servers, subnets or devices show the poorest streams. Remember, you can modify the Scorecard targets if different values are more appropriate for your users and environment.
· Use the TopIssues tab to focus your analysis on the current hot spots. Create an action plan for remediation and implement it.
· Regenerate the scorecard after each potential fix, and see how your results improve. When you have met the target for an element, move on to the next one on the prioritized list.
· When all elements are within the targets (GREEN on the scorecard), continue monitoring trends and start remediation should quality levels drop below your targets.