Echo Canceller Evaluation

Quality of Service is a fundamental issue in the telecommunications environment, particularly given the growing number of players in the market, and ever-stronger competition. Quality issues are a major concern, and can often detract significantly from an operator's competitive edge. Poor quality exacerbates customer churn and can substantially impair long-term revenue generation.

Where operators or manufacturers of echo cancellers prefer to perform the measurements themselves, DSPG can provide the necessary equipment and tools. DSPG has put considerable effort into developing a set of dedicated hardware and software tools to perform echo canceller tests easily and efficiently.

Echo canceller tests can also be carried out on competitive products for a comparison of market analysis.

Tests to assess echo cancellers can be divided into two distinct groups:

Standards Testing - Measurements are made of the echo canceller to verify against relevant echo canceller recommendations. These include G.168 2002, G.168 2000, G.168 1997, G.165 and G.164 Tone Disabler.

VQE Testing - Additional features of echo cancellers such as automatic level control, noise reduction and standard voice quality is assessed. Automatic level control is tested against relevant items from G.169. Voice quality is assessed using the Perceptual Analysis Measurement System, PAMS. Noise reduction testing is accomplished using PAMS.

Standards Testing
These are the most basic tests that apply to all echo cancellers. All the tests are implemented by software running on our DSPGenie ECT1 in a fully digital and automatic procedure. Data is saved on the hard drive and is available for further analysis.

G.168 Testing
DSPG would perform a series of G.168 tests on the Echo Canceller, and would report the results and findings of the tests. Performance is assessed for each of the G.168 tests, and includes simple pass/fail decisions based on the minimum performance requirements specified in G.168. Tests performed are based on ITU-T G.168 2002, G.168 April 2000 and ITU-T G.168 April 1997. Tests for G.168 2002 and G.168 2000 will be performed using the standard G.168 three-branch hybrid model unless the customer specifies a complex hybrid model to use. In general, the tests in G.168 specify performance measures that an echo canceller is expected to meet or exceed over a wide range of values for the test parameters. The ranges of the test parameters are specified as continuous between certain limits. As the test parameter values can affect the echo canceller performance, in theory, an infinite number of tests would be required to check performance with all possible values and combinations of parameter values.

Test list for G.168 and G.168 2000
Test list for G.168 1997

G.165/G.164 Tone Disabler Testing
Echo cancellers covered by Recommendation G.165 should be equipped with a tone disabler that conforms to characteristics defined in those Recommendations. The G.165 tone disabler should activate on receipt of a 2100Hz tone with periodic phase reversals (as defined in ITU-T Recommendation V.25). It should not respond to a 2100Hz tone without phase reversals, or to any other in-band signal. The objective of the G.165 tone disabler is to ensure that the echo canceller is enabled/disabled appropriately during voice-band data transmissions. In general, high-speed data transmissions are preceded by 2100Hz answer tone with phase reversals so that the echo canceller is disabled, whereas facsimile and low-speed data transmissions are preceded by 2100Hz answer tone without phase reversals so that the echo canceller remains enabled.

The ECT-1 performs tests on the disabler using ideal values for disabling frequency, phase reversal period, and phase reversal angle, and also with these parameters set to the limits of their specified range. The ECT-1 also tests that disabling does not occur if the phase reversal angle lies within a specified lesser range. All these tests are performed at both a high signal level and the specified minimum signal level for tone disabling, and with various signal-to-noise ratios.

The ECT-1 performs tests on the G.164 tone disabler using an ideal 2100Hz disabling frequency, and also with the disabling frequency set to the limits of the specified operating frequency range. All tests are performed at both a high signal level and the specified minimum signal level for tone disabling, and with and without noise added to the disabling tone.

VQE Testing
Additional features of echo cancellers such as automatic level control, noise reduction and voice quality are assessed. Automatic level control is tested against relevant items from G.169. Voice quality is also assessed using the Perceptual Analysis Measurement System, PAMS. Noise reduction testing is accomplished using PAMS.

G.169 ALC Testing
Digital signal processing (DSP) techniques and algorithms offer a way to enhance the quality of some voice calls in certain respects. However, although certain aspects of call quality may be improved, without due care such techniques may degrade other aspects of call quality, or cause problems for other call types (e.g. fax and data).

Network echo cancellers offer an ideal signal-processing platform for the implementation of other voice-call enhancements, and several echo canceller manufacturers now offer such features. One of the most popular enhancements to be offered is automatic level control (ALC). While the objectives of ALC are simple - to optimise signal levels for reduced distortion and improved listener comfort - the realization of these objectives without serious side effects is more difficult. ITU-T Recommendation G.169 gives guidance on crucial ALC characteristics. The Recommendation defines certain performance constraints for network ALC devices, and a number of laboratory tests that may be performed to verify these constraints. An ALC device that satisfies all the constraints and passes all the tests in G.169, should ensure (though not guarantee) that the pre-existing standard of overall network performance (e.g. the transmission of speech, voice-band data and other voice-band signals, ISDN etc.) is not degraded when the ALC device is installed in the network.

In particular, compliance with G.169 should ensure satisfactory performance of the ALC device with respect to the following:

Increased deployment of ALC is anticipated to satisfy customers' ongoing expectations of improved quality, and to offset the effects of the more restricted dynamic range typical of low bit rate coding schemes. To manufacturers and network operators, G.169 offers the same type of benefits as G.168. Namely:

PAMS Voice Quality Analysis
The Perceptual Analysis Measurement System (PAMS) is an objective test method developed by BT Laboratories for determining speech quality over both linear and non-linear telephony networks. Traditional objective measurements do not provide a reliable indication of subjective performance, particularly in the case of complex non-linear processes such as low-bit-rate speech codecs. The great value of PAMS is that it predicts the mean subjective opinion score for the network under test, with high accuracy, and without the considerable time, trouble and expense necessary to conduct a subjective test.

In a PAMS measurement, a speech-like test signal is injected into the network under test. A quality prediction is then derived from a mathematical comparison between the transmitted test signal and the received, degraded signal.

To make the comparison between the two signals, PAMS models all the main psychophysical effects of human hearing, and calculates a measure of the audible error. It then interprets the subjectivity of the error signal to predict the mean opinion score of real customers.

In real subjective listening tests, subjects are typically asked to assign scores on two ITU-T five-point listening scales, as shown in the tables below. The output from a PAMS test is a prediction of the mean scores that would be derived from subjective testing. One score relates to the perceived 'quality' of the speech, while the other relates to the 'effort' needed to understand it.

Listening Quality	Score	Listening Effort	Score
Excellent	5	Complete relaxation	5
Good	4	Attention necessary	4
Fair	3	Moderate effort	3
Poor	2	Considerable effort	2
Bad	1	No meaning understood	1

Although real speech signals can be used as the PAMS test signal, BT Laboratories developed a new speech-like test signal to complement the PAMS algorithm, designed to produce quicker, more accurate results. The complete PAMS test signal is 30 seconds in duration for combined male and female voices, but it is possible to use much shorter periods of test signal. The PAMS test signal is made up from a number of separate files, each consisting of a short speech utterance preceded and followed by periods of silence. Any number of these individual files may be concatenated to produce a test signal of the desired duration. However, for best results, it is recommended that a file of at least 5 seconds duration be used.

PAMS is a fundamentally robust approach for predicting quality of any speech transmission network. By comparing PAMS results with a database of subjective test results, the accuracy of PAMS has been proven for many speech carrier technologies, both individually and in combinations.

Noise Reduction Testing

Based on PAMS, this software analyses the ability of the noise reduction voice quality enhancement feature. This operates by inserting noise just before the echo canceller and analysing the effectiveness to return to the original reference file.
In a comparison from a system with noise injection disabled as compared to when it is enabled you are able to tell how much of an improvement has been made.

A DSPG Echo Canceller Evaluation

When commissioned to evaluate an Echo Canceller, DSPG will connect the ECT-1 to the echo canceller under test via an E1 or T1 primary rate PCM interface. The interface details are configured at the tester and the echo canceller to ensure a compatible configuration. The echo canceller control port is also connected to the tester to allow certain echo canceller parameters to be controlled automatically during the tests. It is necessary to configure the control interface at the tester so that proper operation is achieved with the particular model of echo canceller.

A typical sequence of operations would be as follows: