ETL Testing Framework

Introduction

In most ETL technology categories there is a requirement for the manufacturer/supplier to provide testing documentation of the product’s performance when making a new ETL product application. The test document will be reviewed by the assessment team against the criteria for this product, along with other supporting documentation submitted by the manufacturer with the application.

This guidance note sets out the testing requirements that apply in the different technologies and includes details of how representative testing can be used to reduce the number of separate reports required for products within a range.

Some sub-technologies require no substantiation of performance as part of the application process, these are:

• Portable Energy Monitoring Equipment (Energy Monitoring)
• Heat Recovery from Flash Steam and Boiler Blowdown Condensate (Boiler Retrofit Equipment)
• Master Controllers (Compressed Air Equipment)
• HVAC Building Controls (HVAC Equipment)
• Variable Speed Drives
• Evaporative Condensers and Refrigeration System Controls (Refrigeration Equipment)
• Building Energy Management Systems (Energy Monitoring)
• Building Light Controls (Lighting)

“Unlisted” technologies do not require product applications to the ETL for assessment, these are:

• CHP & Absorption and Other Heat Driven Cooling & Heating Equipment

Sub-technologies which cover software shall be approved under the BSC Procedure BSCP601 to read half hourly data from meters and devices[1]. This currently applies to the following product categories in the sub-technology automatic Monitoring & Targeting (aM&T) sub-metering systems:

• Sub-metering software
• Sub-metering systems

For sub-metering systems the software must be verified as specified above, while the sub-meters and sensors must be verified as specified in Table 1. More information about these categories can be found in the Key Information for Manufacturers factsheet.

Product Testing and Verification Methods

There are seven routes to providing valid test evidence for your product application. Table 1 details which routes are available for which technology:

1. In-house testing: self-certified.
2. In-house testing: verified by an independent body. A representative sample of the test data is verified by an independent body. The verifier should also have seen an example of testing take place in the lab and be satisfied that test conditions meet the ETL criteria test standard for that technology category.
3. Witnessed testing. Products are tested in the presence of a witness from an independent body who should sign to verify the accuracy of the data on the test report and confirm that the test was carried out in accordance with the test standard requirements.
4. Independent testing. Products are tested in an independent laboratory that is UKAS accredited or accredited by an accreditation body under ILAC or equivalent. 
5. Acceptance tests or field trials. Only used where laboratory testing is not practical due to product size or lack of suitable facilities.
6. Design calculations: representative testing. Used to reduce the cost of testing products for the same basic design.

Note that if a manufacturer’s own laboratory is certified to BS EN ISO/IEC 17025 the requirements for self-certified products can be followed. There is no need for verified in-house testing, witnessing testing or independent testing[2].

Definitions

• Accredited Test Laboratory: A laboratory that has been accredited by the United Kingdom Accreditation Scheme (UKAS) or other equivalent national accreditation body recognised via the European Co-operation for Accreditation, the International Accreditation Forum, or the International Laboratory Accreditation Co-operation (ILAC) agreements to undertake that particular type of test. A list of laboratories that have been accredited by UKAS for testing and calibration is published at: http://www.ukas.org/. Lists of national accreditation bodies in other countries which may be equivalent can be found at: http://www.european-accreditation.org/ http://www.iaf.nu/ http://www.ilac.org/
• Scope of Accreditation: The document that defines which particular types of test a laboratory is accredited to undertake.
• Independent bodies / Witnesses: e.g. independent laboratories (including accredited test laboratories) and independent consultants (with appropriate qualifications, experience and PI insurance). They are expected to demonstrate their qualifications (e.g. engineer), knowledge (regulations and policy context) and experience (several years in the field for that technology). For example, relevant experience may include:
• Experience of product testing/witness testing and validation of systems in that technology to relevant standards;
• Technical project work providing technical support in the technology;
• Undertaking condition and performance surveys;
• Monitoring compliance to regulatory systems in industry;
• Development and R&D related work in the field.

Acceptable Product Testing & Verification Methods per Technology Type (table 1)

Please note that there are a few technologies where test evidence is not required – see Section 1 (Introduction) of this document. In all the technologies below, test evidence is required. 'P' = Permitted test route for each technology.

Technology & Sub-technology		Testing & Verification Method	Testing					Design Calculations
			In-house testing		Witnessed Testing	Independent Testing	Acceptance Tests or Field Trials
			Self-certified	Verified by an independent body				Representative Testing	Representative Testing (Ecodesign)
Route			1	2	3	4	5	6
Boiler Equipment	Biomass Boilers				P*	P*	P*[3]	P
	Hot Water Boilers		P	P	P	P	P[4]	P	P
	Steam Boilers		P	P	P	P	P	P
	Gas-fired Condensing Water Heaters		P	P	P	P		P	P
Boiler Retrofit Equipment	Burners with Controls		P	P	P	P	P	P
	Condensing Economisers		P	P	P	P		P
	Non-Condensing Economisers		P	P	P	P		P
	Retrofit Burner Control Systems		P	P	P	P
Compressed Air Equipment	Desiccant Air Dryers with Energy Saving Controls			P	P	P		P
	Refrigerated Air Dryers with Energy Saving Controls			P	P	P		P
Energy Monitoring	Automatic monitoring & targeting (aM&T) sub-metering systems		P	P	P	P
Energy Storage	Battery Energy Storage			P	P	P	P	P
	Thermal Energy Storage			P	P	P	P
Heat Pumps	Air to water heat pumps		P	P	P	P		P	P
	Air to Air Heat Pumps, Split, Multi-Split and VRF		P	P	P	P		P	P
	Packaged air to air heat pumps (rooftop)		P	P	P	P		P	P
	Air to domestic hot water heat pumps		P	P	P	P		P	P
	Water or brine to water heat pumps		P	P	P	P		P	P
	Heat Pump Dehumidifiers		P	P	P	P		P
	Heat Pump Driven Air Curtains		P	P	P	P		P	P
	Water to air heat pumps, split, multi-split and VRF		P	P	P	P		P	P
Heat Recovery Ventilation Units			P	P	P	P		P	P
HVAC Equipment	Active Chilled Beams			P	P	P
	Close Control Air Conditioning Equipment		P	P	P	P		P
	Evaporative Air Coolers			P	P	P		P

* Test reports must be prepared by, or verified by, an independent accredited test laboratory

Acceptable Product Testing & Verification Methods per Technology Type…continued (table 1)

Technology & Sub-technology			Testing & Verification Method	Testing					Design Calculations
				In-house testing		Witnessed Testing	Independent Testing	Acceptance Tests or Field Trials
				Self-certified	Verified by an independent body				Representative Testing	Representative Testing (Ecodesign)
Route				1	2	3	4	5	6
Hand dryers				P	P	P	P
Lighting		White LED Lighting Modules for Backlit Illuminated Signs			P	P	P		P
		Efficient White Lighting Units			P	P	P		P
Motors, Drives & Fans		Converter-fed Motors		P	P	P	P		P	P
		Line Operated AC Motors		P	P	P	P		P	P
		Commercial Fans		P	P	P	P		P
Pipework, Tank & Vessel Insulation		Pipework Insulation		P	P	P	P		P[5]
		Tank & Vessel Insulation		P	P	P	P		P
Professional Foodservice Equipment (PFSE)		Combination Steam Ovens			P	P	P		P
		Convection Ovens			P	P	P		P
		Undercounter Dishwashers			P	P	P		P
		Hood-Type Dishwashers			P	P	P		P
Radiant & Warm Air Heaters		Radiant heating equipment		P	P	P	P		P	P
		Warm air heating equipment		P	P	P	P		P	P
Refrigeration Equipment		Air blast coolers		P	P	P	P		P
		Air cooled condensing units		P	P	P	P		P	P
		Automated Permanent Refrigerant Leak Detection System		P	P	P	P		P
		Cellar cooling equipment			P	P	P		P
		Packaged chillers		P	P	P	P		P	P
		Professional refrigerated storage cabinets		P	P	P	P		P	P
		Refrigerated display cabinets		P	P	P	P		P	P
		Refrigeration compressors		P	P	P	P		P
		Retrofit Measures for Open Fronted Refrigerated Display Cabinets			P	P	P		P
Solar Thermal Collectors				P	P	P	P
Uninterruptible Power Supplies					P	P	P		P
Waste Heat to Electricity Conversion Equipment	Organic Rankine Cycle Heat Recovery Equipment					P	P	P[6]	P[7]
	Screw Expanders				P	P	P	P
	Non-condensing or Back Pressure Turbines				P	P	P	P
	Wastewater Heat Recovery Systems				P	P	P

Manufacturers may select any of the product testing options in “table 1”. However, if the manufacturer opts for self-certified or witness testing, there are 5 conditions which need to be met and evidenced by declarations and supporting documents (table 2). Independent laboratories accredited by UKAS, or by another accreditation body under ILAC, operating a BS EN ISO/IEC 17025 system, are considered to be competent. 

General Requirements for Product Testing

Manufacturers/suppliers may select any of the product testing options indicated in Table 1 provided:

1. The test facility complies with the minimum specifications outlined in the test standards and is able to maintain the required test conditions throughout the specified test period or cycle (including any required pre-test stabilisation period).
2. The test measuring equipment meets the requirement for measurement accuracy and uncertainty in the specified test standards/methods, has been calibrated by a laboratory accredited by UKAS/ILAC (or equivalent body) and its calibration is traceable back to national measurements standards.
3. Product testing is undertaken by competent personnel with training and experience in the use of the test facilities, who can accurately and clearly record test data and produce test reports in accordance with the test standard(s) specified in the criteria.
4. A recognised quality management system is in place ensuring and maintaining the quality of the test facilities, procedures and reports; and ensuring document control of the products tested including detailed measurement data, observations and test reports.
5. Appropriate quality assurance procedures are adopted to ensure the accuracy and repeatability of the test results including checking the calibration of measuring equipment and re-testing of a representative sample of products every 2-3 years.

For self-certified and witnessing testing routes, applicants will be required to confirm that the selected product testing option meets these requirements by submitting the declarations and supporting documents set out in Table 2. Independent laboratories that have been accredited by UKAS/ILAC (or equivalent bodies) are considered to be competent to test products using the test standards named in their scope of accreditation under BS EN ISO/IEC 17025:2005.

Table 2: Conditional requirements for self-certified and witness testing routes

Conditions / Requirements	Self-certified	Witness Testing
Test Facilities: complies with the minimum specification in the test standard and throughout the test.	Declaration by the manufacturer against the requirement. Declaration by an independent party where required in the application checklist, that a representative sample of the test data has been cross-checked and verified by the independent body.	Declaration by the witness against the requirement. Declaration by the witness that the test was observed in line with the test standard and verification of the test data.
Measuring Equipment: meets the accuracy requirements from the test standards. Has been calibrated via accredited laboratory or through tracing back to measurement standards.	Declaration by the manufacturer against the calibration requirement, including dates of the last calibration and the method used.	Declaration by the independent body or witness that the calibration requirement was verified.
Technical Competence: competent personnel with training and experience who can accurately record test data and produce test results.	Test person name, signature on the test reports	Details required of the witnesses’ competence to act as a witness, if not recently submitted on another application.
Quality Management System: ensuring and maintaining quality of facilities, procedures and reports.	Details of the QMS used to ensure quality of testing procedure e.g. ISO 9001, ISO 17025 certificate.
Quality Assurance Procedures: ensuring the accuracy and repeatability of test results, including calibration and representative sample testing.	Declaration by the manufacturer against the requirement. Summary details of the procedure used to verify the accuracy of test results.	Declaration by the witness that appropriate QA procedures were applied during testing. Summary details of the procedure used to verify the accuracy of test results.

Notes:
Detailed test reports should contain the name of the accredited laboratory that calibrated the measuring equipment, or an explanation of how calibration is otherwise traceable to national measurements standards as defined in Section 5.6 of BS EN ISO/IEC 17025: 2005.

Representative Testing

Representative models must be selected in accordance with the rules set out below and in Table 3, unless the ETL criteria say otherwise, in which case the criteria rules apply:

1. Product Range: the manufacturer shall define a range of products, which are all variants of the same design. For example for packaged chillers they would all use the same refrigerant, have the same compressors and fit within the same product category.
2. Product Group: within the range, the manufacturer shall define the product groups, with each group having similar design characteristics and efficiencies.

Once the range and then the products groupings within it have been defined a representative sample of models are tested:

• For each range, a minimum of at least two models shall be tested
• For each group, within the range,
  • a model from the lowest quartile of predicted performance shall be tested
  • the performance of each model in the group shall be predicted using a validated mathematical model

With the following exception:

The following 18 sub-technology groups have different representative testing requirements as they are covered by the Ecodesign Directive:

• Heat recovery ventilation units
• Air to air heat pumps, split, multi-split and VRF
• Air to domestic hot water heat pumps
• Air to water heat pumps
• Heat pump driven air curtains
• Packaged air to air heat pumps (rooftop)
• Water to air heat pumps, split, multi-split and VRF
• Water or brine to water heat pumps
• Gas fired condensing water heaters
• Hot water boilers
• Radiant heating equipment
• Warm-air heating equipment
• Converter fed motors
• Line operated AC motors
• Air cooled condensing units
• Packaged chillers
• Professional refrigerated storage cabinets
• Refrigerated display cabinets

The requirements for these sub-technology groups is:

1. A full test report is required for at least one representative model per range. The Ecodesign technical data is required for the other products in the range (where no further test reports are provided for these products).
2. A summary table of all the key data which is included in the technical data sheet or data fiche required by Ecodesign for each of the products within the application.
3. Details of the extrapolation / interpolation method or other calculation model that has been used to determine calculated results.
4. A description of how any calculation method or model has been validated (e.g. through comparison of product test reports with the outputs of the calculation model).

Table 3: Standard rules for selecting representative models for product testing
 

Circumstances	Variation between Models	Selection Rule
Variations or cosmetic changes	For example, in the product exterior or internal layout, that do not materially change its constructional design, functionality or affect its performance specification.	Any model may be the representative model.
Different constructional design	For example, in terms of materials specification like the type of subcomponents and subassemblies used.	These products may be in the same range, but should be placed in different groups.
Different functionality, performance specifications or operation modes	For example, at the test points set out in the criteria.
Different product rating	For example, expressed in terms of kW or electrical power input/output, thermal input/output or maximum heating or cooling capacity.
Modifications and enhancements	Should not affect product performance or alter its constructional design, basic functionality or mode of operation at the test points in the criteria.	Differences need to be explained and products with modifications and enhancements clearly identified. These can be in the same range / groups as other products.

Application

Manufacturers are advised to separate out their product applications, with each application covering a specific range. Each application shall include the:

1. Design data that demonstrates the products are of similar design
2. Details of how the representative sample of products was chosen
3. Test reports

If the representative models are already on the ETL then the test data for the representative model needs to be resubmitted with each new application.

Removal

• If a manufacturer voluntarily removes a representative model from the ETL, the other products linked with that representative model will normally be removed unless alternative test data can be submitted.
• If any product submitted under the representative testing rules is later found to not meet the performance criteria when independently tested, then the entire product group will normally be removed from the ETL.

Determining the Type of Acceptable Test Report

A test report shall be prepared that conforms to the essential requirements of section 5.10 of 17025.

For some product testing options summary test reports can be accepted but for others detailed test reports must be submitted. The requirements are set forth in Table 4 below.

Table 4: Type of report that must be submitted by product testing option

Product Testing Option	Volume of Applications	Verification Method
		Self-certified	Independently Verified
In-house Testing	First 2 product applications within a product range	Detailed	Summary or Detailed
	Subsequent applications within the same product range	Summary
Witnessed Testing	N/A		Summary or Detailed
Independent Testing	N/A		Summary or Detailed
Acceptance Tests or Field Trials	N/A	Detailed
Representative Testing	First 2 representative models for a product range	Detailed	Summary or Detailed
	Subsequent representative models for a product range	Summary

What Constitutes a Detailed and Summary Test Report?

Table 5: Typical contents of a detailed and summary test report
 

Detailed Test Report

Summary Test Report

 

Scope: details of product tested (name & model), full reference to test standards used and confirmation of applicable QA procedures / management system

Same as Detailed Test Report

 

Introduction: location and date of testing and confirmation of whether the test facilities used are permanently under control of the test laboratory.	Same as Detailed Test Report
Products being tested: product identification data, product installation, configuration and commissioning information and details of control settings used during the test. Details of the product’s condition at the start/end of testing and any modifications made to the product during testing.	Same as Detailed Test Report
Test procedures: 1) a description of how each test was conducted, test standard used and any deviations; 2) test rig schematic; and 3) a list of measuring equipment.	Point 1)
Test results: 1) a table showing the average value of all measured parameters including maximum and minimum values recorded; 2) details of any calculations and corrections made; 3) a discussion of the test results, including any observations made during the tests.	Points 1) & 2)
Appendices: 1) ISO9001 / ISO 17025; and 2) calibration certificates[8]; 3) details of procedures used to cross check accuracy of results; 4) measurement data collected during testing, in Excel format.	Points 1) to 3)
Representative testing: an explanation of the methodology used to group and select products for testing.	Same as Detailed Test Report

[1] BSC Procedure BSCP601 (‘Metering Protocol Approval and Compliance Testing’) specifies the process by which meter manufacturers demonstrate that their Metering Equipment is of an appropriate type and remains fully functional and accurate.

[2]Some products might still require an independent verification or witnessed assessment in order to be eligible to be placed on the market (e.g. under the scope of the PED, Pressure Equipment Directive, or some boilers). 

[3] Category 2 & 3 only

[4]Category 1 & 2 above 400kW and Category 3 above 900kW

[5]Representative testing for categories 4, 5 and 6 only (see clause 1.5.1 of the Pipework Insulation criteria)

[6] See Method A – Section 1.4.1.1 in the Organic Rankine Cycle Heat Recovery Equipment criteria, Direct Measurements.

[7] See Method B – Section 1.4.1.2 in the Organic Rankine Cycle Heat Recovery Equipment criteria Validated Design Calculations.

[8] Detailed test reports should contain the name of the accredited laboratory that calibrated the measuring equipment or an explanation of how calibration is otherwise traceable to national measurement standards as defined in section 5.6 of 17025