New updates for Dictate, including voice commands in Word in the web and mobile

by Scott Muniz | Sep 1, 2020 | Uncategorized

This article is contributed. See the original author and article here.

Just recently, it was announced that voice commands are available for Dictate in Word. In this post we will cover some tips and tricks on how to use these commands and other recent updates the next time you are brainstorming, creating an outline, authoring content, or giving feedback!

Why dictate?

Dictating can be 3x faster than typing with a traditional keyboard and mouse, plus gives both educators and students flexibility in how they work. This feature lets you draft content hands-free while on-the-move, multi-tasking, or repetitive stress injury. The option to use speech-to-text is a powerful tool for early learners or students with cognitive differences too – whether just learning to type to having dyslexia or dysgraphia. It’s also much easier to do your best thinking when you can speak at the speed of thought.

Getting Started

1. Sign into your Microsoft account on a supported browser – Edge, Chrome, or Firefox
2. Go to the Home tab > Dictate button
   • Alternatively, toggle the feature on and off using the keyboard shortcut ALT + ` (this is ALT + backquote – the same as the tilde (~) key left of the number 1 on the keyboard)
3. Enable microphone permissions if prompted for the first time
4. Speak clearly and see the speech-to-text at work
5. Try phrases to add punctuation and symbols or even edit and add structure to the document
   • Try saying punctuation such as “new line”, “question mark”, or even “dot dot dot”
   • Try saying symbols such as “percent sign”, “multiplication sign”, “degree symbol”
   • Try using voice commands such as “backspace”, “delete”, “bold that, “start list”
6. Explore the various other items on the dictation toolbar including
   • In settings, the language dropdown  
   • In settings, the auto-punctuation toggle
   • In help, the list of “What you can say” per language
   • If you see a gray squiggle under any dictated text, click on it to see alternate correction options
7. For more help check out Dictate your documents in Word

Here are some quick notes about availability

1. Dictate is free on Word for the Web, and Office mobile (iPhone and Android Phone)
   • Currently has the new experience described below
   • Although dictation is available in various languages, voice commands are English-only at this time
2. Dictate requires a Microsoft 365 subscription for the desktop versions
   • New toolbar and voice commands coming soon

The new dictation toolbar 

This new dictation toolbar now exposes new settings such as auto-punctuation and help when you need it. It’s also draggable so that you can move it around and is more flexible in case you want to leave the Home tab, such as to insert a picture.

New Dictation toolbar

New settings and Auto punctuation

Let Word insert periods, exclamation marks, and question marks for you based on your inflection and pauses if that works best for your style of dictation. Explicitly saying your punctuation will still work when this is on.

New punctuation, symbols, and voice commands

All the supported things you can say are being added in the help section of each application or at Dictate your documents in Word. We are regularly adding more support. Some scenarios to try include:

• Editing
• Formatting
• Making Lists
• Commenting

Mobile availability

These features are also available on the latest version of the Office app on iPhone and Android phones.

We hope you try out these updates and send us your feedback.

At any time, check out all our dictation offerings at the Dictate in Microsoft 365 page.

Dealing with Imbalanced Data in AutoML

by Scott Muniz | Sep 1, 2020 | Uncategorized

This article is contributed. See the original author and article here.

What is Imbalanced Data? 

For a given classification problem, if the classes/targets within the dataset are not represented equally, then the dataset is said to be imbalanced. The classes with a higher representation are called majority classes, while the ones with lower representation are called minority classes. Sometimes imbalance exists due to limited availability of data for certain classes, while in other cases it could be attributed to the nature of the dataset. For instance, if there is a spam classification dataset, we’d expect more non-spam emails than spam emails. Similarly, for fraud detection, we would expect the fraudulent transactions to occur only rarely. 

It is harder for learning models to generate predictions for the minority classes because the model has access to limited training examples representing those classes, making it difficult for it to learn to distinguish these classes. Most ML algorithms for classification perform well when all classes have a roughly equal distribution and hence it becomes important to address the problem of class imbalance. 

How do we detect class imbalance? 

While there is no fixed definition of what constitutes an imbalance, generally it is detected using one of the following: 

1. Ratio of the samples in the least populated class to the overall number of samples in the dataset. Let’s call this Ratio-1.
2. Ratio of the samples in the least populated class to the samples in the most populated class. Let’s call this Ratio-2. 

Let’s evaluate both methods using a couple of sample cases: 

1. For a four–class problem with data distributed among labels like this: {‘a’: 20, ‘b’: 20, ‘c’: 20, ‘d’: 200}, Ratio-1 is 7.7%, while Ratio-2 is 10%. 
2. For a four–class problem with data distributed among labels like this: {‘a’: 20, ‘b’: 200, ‘c’: 200, ‘d’: 200}, Ratio-1 is 3.2%, while Ratio-2 is 10%. 
    

As seen in the above examples, Ratio-2 remains consistent, even if the constitution of the other classes in the overall data changes. 

Hence, we utilize Ratio-2 as the indicator of imbalance in AutoML and imbalance is detected when this ratio is lower than 20%.  

How to address the problem of Imbalance? 

Ideally one would prefer collecting more data for the minority class(es). However, often that is quite expensive or may not be feasible due to the nature of the problem (e.g., fraud or spam detection).  

Therefore, we leverage the following methods for dealing with imbalanced data within AutoML: 

1. Using weights for class balancing: this feature gets automatically applied in AutoML if it improves performance on a subset of the user’s data (more details in later sections)  
2. Using metrics that are sensitive to imbalance: users can pick relevant metrics based on our recommendations. 
3. Leveraging the sample weights provided directly by the user via the AutoML config 

Additionally, users can apply over/under sampling to rebalance the data before feeding it to AutoML. Methods #1 and #3 will impact the ML algorithm’s cost function in the same manner (covered in the next section), but the difference is that in method #1 AutoML will automatically do it for you. This feature of Class Balancing using weights is the focus of this blog and we shall elucidate that next. 

Weights for Class Balancing: 

Without actually over–sampling the minority classes or under-sampling the majority classes, we can simply apply weights to the samples belonging to a class, in the inverse proportion of the number of samples representing that class (Fig 1 elaborates on this calculation). The intent is that the underrepresented classes would have a higher weight than the overrepresented ones.  
 

We leverage this method because it allows us to apply balancing without increasing the size of the dataset, and hence without modifying memory requirements. Additionally, even if the classes are well balanced, applying this method would simply apply uniform weights, and in theory this method could be applied for all datasets. 

To understand the impact of applying weights, let’s review the cost function J(θ) for a Logistic Regression classifier below. Here m is the number of training samples, x and y are the features and labels respectively, θ refers to the model parameters. 

When we use the Weights for Class Balancing, the above cost function is modified to apply the class weight corresponding to every training sample, as shown below, where c(i) refers to the class weight for the i^th training sample. 

Using metrics sensitive to imbalance: 

For dealing with class imbalance we recommend using AUC–weighted as the optimization metric for AutoML runs. There are several benefits, as follows: 

• Any macro average will independently calculate the average for every class and then take the overall average, thus treating all classes equally; whereas a weighted average will calculate the contribution of every class based on the relative number of samples representing that class, and hence this is more robust to imbalance. Hence, we choose a weighted metric. 
• Additionally, AUC-weighted is threshold invariant since it measures the area under the (ROC) curve over all possible classification thresholds by aggregating them. 
• We also report several other metrics sensitive to class imbalance, such as F1, Precision, Recall and Matthew’s Correlation Coefficient. 

Analysis and Experimentation:  

After selecting the solution and the metric, we performed A/B tests on a variety of datasets with the following setup: 

1. Treatment Group: using weights for treating class imbalance
2. Control Group: with no class balancing
3. Significance Level (Type 1 Error probability) preselected as 5%
4. H₀ (Null Hypothesis): There is no impact on the performance of AutoML after applying weight-balancing
5. Ha (Alternative Hypothesis): Weight balancing improves the performance of AutoML 
6. One tailed t-tests were performed by leveraging the sample statistics like sample standard deviation from multiple runs performed on the same datasets

Extensive A/B testing with this setup demonstrated that we could leverage subsampled data reliably to determine the effectiveness of our class balancing solution. Running A/B tests on subsampled data rather than the full data enabled us to reduce the overall execution time. These tests utilize sample statistics computed using binomial distribution and some heuristics. This helps us decide if balancing the classes using weights would improve the performance and the complete flow is described in the graphic below. 

Additionally, AutoML’s Guardrails feature informs the user if their data exhibits imbalance and if the class balancing solution was applied. 

Figure 1: Flow chart describing the handling of imbalanced data within AutoML

If you’re getting started today with Microsoft Azure’s Automated Machine Learning, here are a couple of helpful links: 

https://azure.microsoft.com/en-us/services/machine-learning/automatedml/ 

https://docs.microsoft.com/en-us/azure/machine-learning/concept-automated-ml#how-automated-ml-works

Contributors:

Arjun Singh – Data & Applied Scientist II

Anup Shirgaonkar – Principal Data & Applied Scientist