Presented by:

Senior Engineer, TIS – Member of Staff of OPTICOM GmbH

Joachim POMY
Moscow, 27-29 April 2011
“

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

“
” No external funding or debt

“
” Noise-to-Mask Ratio (NMR) 1988
” Spin-Off from Fraunhofer-Institute (Home of mp3)

“
(1996), (1999), (2000), (2004), (2008),
and now (2010)

Moscow, 27-29 April 2011

“ the next-generation mobile voice quality testing standard P.863 ”

“ stands for ‚Perceptual Objective Listening Quality Assessment‚

“ Standardised as Draft , following the history of P.861 ’PSQM’ and P.862

‘PESQ’

“ Specially developed for HD Voice, 3G and 4G/LTE, VoIP

“ Offers a new level of benchmarking accuracy

“  A joint development of the

POLQA consortium in the ITU-T

Moscow, 27-29 April 2011

 “
“
“
“
“
“

“
“
“
“
“
“

Moscow, 27-29 April 2011

 Evolution of ITU-T Recommendations for Voice Quality
Testing (P.86x - Full Reference MOS-LQO)
    )
   B
   W
   S
    (
    d
   n   z
   a   H
    b
  -    k
   e   4
    d
   i    1
   w
  -    e
   r    c
   e
   p
   i POLQA
   u    o
   S    V P.863 (draft)
    )    D ??/2010
   B    H
   W
    ( P.862.3 PESQ- Speech Codecs
    d   z
   n   H PESQ E2E Network
   a    k WB Quality
    b
  -    7
   e Application Guide Variable Delay and
    d
   i 11/2005
Wide-band
Extension
Time Scaling
   W to 7 kHz Level & Linear   e
   d
 .
  m
Filtering Eff ects   o
  c
    )    i
   t
  p
   B   PESQ 
Acoustical   o
 .
   N
    ( PSQM PESQ   Interfaces   w
  w
  w
    d   z MOS-LQO
POTS and HD Voice    –

   n   H    S ITU-T P.861 ITU-T P.862 P.862.1 P.862.2

   0
   1
   a    k    T (NB and WB/SWB)    0
   2
    b
  -    4    O 08/1996 02/2001 11/2003 11/2005 VQE Enhanced
   H
   b
 .   m
   w   3    P (Withdrawn) Speech Codecs MOS Mapping Networks    G
   o
   r Variable Delay for Mobile    M
   O
   r Speech Codecs Network Enhanced Accuracy    C
   I
   a Fixed Delay
E2E Network
Benchmarking of MOS P rediction
   T
   P
   O
   N Quality
   ©

1996 2000 2005 2010

2G VoIP 3G 3.5G NGN UC 4G/LTE
Evolution of N etwork Technologies available at the time of development, i.e. included use cases for e ach Recommendation

Moscow, 27-29 April 2011

 Requirement Specification P.OLQA

May 2008
Call for Proponents

July 2008
Six model candidates announced 

First set of Super- Statistical Evaluation

wideband Database procedure for
for training purposes P.OLQA

Start of model training  February 2009

Submission of mo del candidates to ITU-T 

July 2009

Second set of speech databases for

evaluation purposes

Evaluation of model candidates 

Report to ITU-T SG12 May 2010

Models from OPTICOM, SwissQual and

TNO are selected to form th e new Rec.
P.OLQA with a joint model 
September 2010

Consen t and Appr oval of P.OLQA (P.863) 

Characterization phase 

Moscow, 27-29 April 2011

“

“
”
”
”
”
”
”
”
”
”
”

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

Moscow, 27-29 April 2011
“

“ “

“ “
“ “
“ “

“ “

Moscow, 27-29 April 2011

“

rmse*
rmse*
narrow-band
narrow-band PESQ
PESQ POLQA
POLQA Improvm.
Improvm.
P.862.1
P.862.1
 Averaged
 Averagedrmse*
rmse* 0.1857
0.1857 0.1363
0.1363 27%
27%

wideband
wideband PESQ
PESQ POLQA
POLQA Improvm.
Improvm.
P.862.2
P.862.2
 Averaged
 Averagedrmse*
rmse* 0.3450
0.3450 0.1506
0.1506 56%
56%

  1  
rmse*   
  N   d   N 
 Perror i² 
 
 Where….

 Perror (i)  max( 0,  MOSLQS (i)  MOSLQO(i)  ci95 (i))

Moscow, 27-29 April 2011
 PESQ Performance - NB_8kHz504_SWISSQUAL, rmse* = 0.4204

4.5

4
   )
   2
   6
   8
 .
   P 3.5
   (
 .
   d
  n
  o 3
   C
   O
   Q
   L
  - 2.5
   S
   O
   M
2

1.5

1 1.5 2 2.5 3 3.5 4 4.5 5

MOS-LQS Cond. 27% improvement*

POLQA Performance - NB_8kHz504_SWISSQUAL, rmse* = 0.2311

4.5

4
   )
   3
   6
   8
 .
   P 3.5
   (
 .
   d
  n
  o 3
   C
   O
   Q
   L
  - 2.5
   S
   O
   M
2

1.5
*Narrowband average rmse*
1

1 1.5 2 2.5 3 3.5 4 4.5 5 improvement observed for all ITU tests
MOS-LQS Cond.

Moscow, 27-29 April 2011

 PESQ Performance - WB_16kHz204_FTDT, rmse* = 0.4221

4.5

4
   )
   2
   6
   8
 .
   P 3.5
   (
 .
   d
  n
  o 3
   C
   O
   Q
   L
  - 2.5
   S
   O
   M
2

1.5

1 1.5 2 2.5 3

MOS-LQS Cond.
3.5 4 4.5 5
56% average
POLQA Performance - WB_16kHz204_FTDT, rmse* = 0.2319 Improvement*
5

4.5

4
   )
   3
   6
   8
 .
   P 3.5
   (
 .
   d
  n
  o 3
   C
   O
   Q
   L
  - 2.5
   S
   O
   M
2

1.5
*Wideband Average Improvement
1

1 1.5 2 2.5 3 3.5 4 4.5 5 observed for all ITU tests

MOS-LQS Cond.

Moscow, 27-29 April 2011

 PESQ Performance - WB_PSY_402_POLQA, rmse* = 0.3245

4.5

4
   )
   2
   6
   8
 .
   P 3.5
   (
 .
   d
  n
  o 3
   C
   O
   Q
   L
  - 2.5
   S
   O
   M
2

1.5

1 1.5 2 2.5 3 3.5 4 4.5 5

MOS-LQS Cond. 56% average

POLQA Performance - WB_PSY_402_POLQA, rmse* = 0.1839 Improvement*
5

4.5

4
   )
   3
   6
   8
 .
   P 3.5
   (
 .
   d
  n
  o 3
   C
   O
   Q
   L
  - 2.5
   S
   O
   M
2

1.5
*Wideband average rmse* improvement
1

1 1.5 2 2.5 3 3.5 4 4.5 5 observed for all ITU tests

MOS-LQS Cond.

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

“

5 5 5
Clean speech, 50…14000Hz
“ Clean speech, 300..3400Hz
Clean speech, 50…7000Hz
(WB)
 AMR 12.2kBit/s
4 Clean speech, 300…3400Hz
(NB)

GSM HR

3
“
“

1 1 1
Moscow, 27-29 April 2011
“

Moscow, 27-29 April 2011

   0
  1
  0
  2
 ,
  H
   b
  m
  G
  M
  O
  C
  I
  T
  P
  O
  ©
| f s,Ref   f s,Deg, est |
 1%
f s,Ref 

Moscow, 27-29 April 2011

   0
  1
  0
  2
 ,
  H
   b
  m
  G
  M
  O
  C
  I
  T
  P
  O
  ©

Moscow, 27-29 April 2011

  Very different to PESQ 

© OPTICOM GmbH, 2010

Moscow, 27-29 April 2011

Level too high or too low x x 0

Strong linear filtering x x 0

Noise in the reference signal x x 0

High timbre in the reference signal x x 0

Level variation x x poor

SWB noise on NB/WB signal x x 0

Moscow, 27-29 April 2011

Sample Rate 48kHz 8, 16, 48kHz

Ref. Bandwidth 50..14000Hz 300..3400Hz

Ref. Level -26dBov (73/79dBSPL) -26dBov (79dBSPL)

Deg. Level -21..-46dBov -26dBov  

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

“
for Windows, Linux

“
for Symbian, Android, ...

“
incl. POLQA+PESQ+ECHO “  Voice, Video, or
 Voice+Video
“
“
incl. POLQA+PESQ+ECHO

Moscow, 27-29 April 2011

Europe, Middle East: USA, Canada: Asia-Pac:

China

Taiwan

Korea

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

  Years of profitable Business Experience
 Years of Scientific Expertise
International Standards (= 100% Conformance)
Essential Patents and License Agreements
Excellent Reference Customer Base
:

Moscow, 27-29 April 2011

“

Moscow, 27-29 April 2011

Originally a working title of a new objective ‚instrumental‛ approach for prediction of Listening
Quality, ITU-T SG12 / Question 9

Lead study group on quality of service and quality of experience

Subcommittee of ITU-T Study Group 12, dealing with perception-based objective methods for
 voice, audio and visual quality measurements in telecommunication services

Perceptual experiments where the human listeners and viewers in those experiments are named
‚subjects‛.

Instrumental prediction of quality. Measures made model a certain type of perceptual (subjective)
experiment.

Moscow, 27-29 April 2011

Moscow, 27-29 April 2011
 © OPTICOM GmbH, 2010

Moscow, 27-29 April 2011

 “

Completely Masked
     e
       B
     n
       B Partially Masked
       d
       d
     o
       S
“Smearing”

Bark Bark

       )
     n      g
     o      n
       i
     s       i
     s      n
     e      e
     r      p
     p      r
     p      a
     u       h
S        S
       (
     e Convert to      e
       B
     n      n
d
     o Loudness      o
       S        S

Bark Bark

Moscow, 27-29 April 2011

Moscow, 27-29 April 2011