Registering and logging in removes this ad.
Registering and logging in removes this ad.
Some test results on V9
Whilst evaluating a new microphone today with a new user file, I took the opportunity to make some accuracy checks after making changes to my user file as described below. Out of interest, I thought I would post the results here.
I read a 166 word passage from Bill Bryson's book "Notes from a small Island". Results:-
Brand new user after reading one "easy reading" enrolment text - 94.6% accuracy
Rechecked on same day as user file created after rebooting the computer - 95.7% accuracy
Repeated on same day as user file created after running a second ?easy reading" enrollment training text - 96.3% accuracy
Repeated on same day as user file created after correction of errors - 97.6% accuracy
Repeated day after user file created after correction of errors - 97.6% accuracy
Repeated straight away after running a third (long) enrollment text (Success is a Journey) - 99.4% accuracy
Repeated straight away after running the Acoustic Optimizer and Language Model - 100% accuracy
Version 9 Professional
IBM T42p ThinkPad laptop
1.6 GHz processor
1 GB RAM
Windows XP Professional
RadioShack headset microphone
Andrea USB adapter
Some observations - it was reassuring to be approaching 98% accuracy with V9 on the second day of the use of the new user file. Accuracy further improved to above 99% of after reading a long enrolment text (I chose " Success is a Journey "). I achieved 100% in the last test after running the Acoustic Optimizer and Language Model. So the Acoustic Optimizer and Language Model didn't degrade accuracy and if anything further improved it. Of course, these results, can't be regarded as statistically valid as I would have needed to have repeated each accuracy reading several times over to obtain a statistically valid average. They are indicative only.
Some purely subjective opinions from me - accuracy and responsiveness seemed more "solid" after running the Acoustic Optimizer and Language Model. An indication that Nuance have made this feature work optimally in version 9? Also, this is again purely subjective but I am getting the sense that accuracy is more robust when dictating at a more "measured" pace than I was using with version 8. If my subjective observation is in fact the case, a possible explanation (which I have seen stated by others already on certain Dragon speech recognition forums) is that Nuance have used quadragrams in version 9 to increase accuracy and therefore, dictation needs to be more "measured" to take advantage of them.
Thanks for that interesting
Thanks for that interesting test. I can't immediately see a flaw in your procedure. Your results seem generally in line with what the manufacturer promises. It will be interesting to see if the level of accuracy holds up over time
Bruce
Thanks Pete. As you may
Thanks Pete. As you may know, I'm a big fan of empirical tests when it comes to understanding how DNS works. Your tests do suggest that despite the impressive initial accuracy of a DNS 9 user file, just as with previous versions of DNS, doing more general training than DNS requires may be beneficial. Also, I've always been skeptical of the acoustic optimizer and have worried that it will corrupt my user files, but your tests have inspired me to give it a try again.
A couple of things to consider when conducting tests though. Your report is not completely clear, but it looks as though you were correcting the errors DNS made when you were reading the test passage. That is a no-no in my book. In actual use, you'll rarely be dictating the same text twice, so in my opinion, that is an artificial accuracy enhancement that has no real practical significance. (Though I suppose if you state that you are doing this there may be no real harm). Also, 166 words is a pretty short passage in my opinion. But of course it takes more time to read longer passages, and one only has so much time to devote to such testing.
Also, thanks for the reminder about quadragrams. My natural dictation mode is short; I guess I need to try to remember to make sure my phrases go at least four words.
Cheers,
Meph