PACT Conformance Testing

1. Context

The Partnership for Carbon Transparency (PACT) initiative is continuously evolving and bringing together organizations and industry initiatives to exchange accurate and primary data-based Product (Carbon) Footprints irrespective of geography and industry. The technical foundation for this exchange is currently the Pathfinder Framework and Technical Specifications for achieving interoperability.

If a company wants to exchange Product Footprint information in a truly interoperable way, it then either needs to implement these foundations or look for an existing solution. In both cases, companies need to be able to assess the conformance of their software and solutions.

This document defines the overall process of testing the conformance of software implementations, and the requirements for having a solution listed publicly online on the PACT Online Catalog.

2. PACT Conformance Testing Overview

Conformance testing is a process of verifying if a product or service meets specific standards or specifications. In the context of PACT this is the process of verifying that a solution or system implements the Pathfinder Framework and technical specifications.

As the technical and solution landscape in the PACT context is rapidly evolving, in a first step a very “lean” and minimal process is defined. This process will evolve further in scope and complexity, and include software-based automation in the future.

What is a PACT Conformant Solution?

A PACT conformant solution “A” is defined as a solution which:: Which implements the PACT specified mandatory (and / or optional) endpoints for PCF Data exchange; Was tested for conformity with a specific version of the technical specifications with 2 or more other solutions; Where the other solutions independently confirm the conformance of solution “A”

2.1. What is the focus and scope of PACT conformance testing?

The PACT Conformance testing process focuses on testing the technical interoperability of a specific implementation (called “host system under test”):

A. Technical interoperability testing is scoped by data exchange protocol version:
an implementation is tested for conformance against a specific version of the data exchange protocol (such as version 1.0.1 or a later version such as version 2) by a 3rd party.

B. Technical interoperability testing focuses on the HTTP REST API:
for this, the 3rd party makes conforming calls to the HTTP REST API of the host system under test. The 3rd party then compares the witnessed behavior with respective specification.

C. Optionally, technical interoperability testing covers interoperability testing of a data model extension:
for this, the parties test technical interoperability against a specific data model extension version. If the data model extension provides instructions for doing so, the testing process shall cover them.

2.2. When does a host system under test satisfy a PACT conformance test?

A host system under test has satisfied a PACT conformance test if the 3rd party declares it has performed a PACT conformance test with the host system under test, and that the behaviour of the host system under test was conforming to the specification (see e.g. § 4.2 PACT Conformance Test criteria (Checklist) of this document for a checklist).

2.3. What is not yet in scope of the PACT conformance testing?

For the first iteration of the Conformance Testing - testing of conformance with the Pathfinder Framework, GHG Protocol (likewise ISO 14044 etc.), sector-specific or product-specific rules is not in scope.

The first iteration of Conformance Testing will validate solutions conform to the PACT technical specifications only. Solutions will not be assessed for conformance to Product Carbon Footprint methodologies, such as Pathfinder Framework, GHG Protocol (likewise ISO 14044 etc.), sector-specific or product-specific methodologies.

If a host system under test implements the HTTP REST API but contains a (critical) bug in its GHG protocol implementation, the solution would still meet the PACT conformance testing requirements.

Note: Future definitions of the conformance testing process will extend their scope.

3. Connectathons

A Connectathon is an event organized by WBCSD PACT. It is geared towards simplifying interoperability and conformance testing between implementers and their respective solutions.

A Connectathon is open for invited parties and logically consists of

Key Date when the Connectathon takes place and collaboration starts

List of companies participating

Dedicated MS Teams channel to collaborate and share feedback, test progress, and outcome

3.1. Connectathon Process

Organizations can Onboard their Solution to PACT Online Catalog before or after the Connectathon.

Please refer the process below -

Note: Sign up for our next connectathon here

4. Conducting a PACT Conformance Testing

Conformance Testing commences once Organizations identify their Testing partner(s) before or during Connectathons.

4.1. Process

The key stages of a PACT Conformance Test performed by the parties are depicted below:

Organizations will Initiate the Conformance Testing process by aligning on Timeline, Process, and Communication mechanism (dedicated Private MS Teams Channel).

Conformance Test begins based on the Test Data / Environment setup initially. It is assumed, the parties perform the testing in accordance with the [PACT Criteria Checklist] for the testing. The parties are free to use Teams or different channels during the Connectathon or after. Additionally, the parties are free to use the PACT-sponsored SharePoint to share data among each other.

Organizations will continuously / iteratively run test scenarios and evaluate the PCF Data exchange as per the PACT Conformance testing criteria Checklist. Issues / queries can be posted and resolved via the MS Teams Channel. After successful conclusion of a conformance test, Organizations will Sign Off the Solutions (Pass/Fail)

Organizations are encouraged to retest their solutions during the next scheduled Connectathon, especially in case when issues have been resolved or support for new versions (related to the data exchange protocol or 1 or more data model extensions) was added to a previously-tested implementation

Organizations are encouraged to onboard their solution to the PACT Catalog and to upload conformance results there as well (see here for details)

The Conformance Testing process is periodically reviewed, updated, and scope expanded (see § 2.3 What is not yet in scope of the PACT conformance testing?) to reflect changes in the organization’s standards and requirements, as well as feedback received from members of the network.

Process when the testing Partner doesn’t have a Solution yet or has a "Data Recipient Only" solution There are situations where an Organization chooses to implement "Data Recipient only" capability in the immediate term due to business model or product constraints. PACT still encourages these Organizations to participate in the PACT Conformance Testing process and to be onboarded to the PACT Catalog.

For both these scenarios, the concerned Organization can still participate in the Connectathon and Conformance Testing and test another Organization’s solution by participating in the data exchange as recipient and validate the conformance of the other solution based on the PACT standards / specification. The other Organization can add the Conformance Test Result from this testing to the Catalog by following the Scenario 2 as depicted in these instructions

Note:For both the above scenarios – The concerned Organization will still need to submit a Pull request in the PACT Catalog repo to add their Organization as a ”User”. Please follow these instructions to do so.

The concerned Organization as per the above scenarios will appear in the Catalog under the “Collaborators” page, even though their Solution may not be in the PACT Catalog

4.2. PACT Conformance Test criteria (Checklist)

Recommendation from WBCSD to be used as a reference for the Bilateral Conformance Testing criteria for Technical Interoperability. A Solution is deemed PACT Conformant if it meets all the criteria’s as defined in this checklist.

4.3. PACT Conformance Test Cases Template (TestCases)

Access the sample PACT Conformance Test Cases Template here. Please Note - This is a template created with the intention to guide you on what possible test cases you need to consider, but we also encourage you to add any other test cases you feel are necessary for your test coverage.

4.4. PACT Conformance Test - Sample Payloads {#payloads}

Access the sample payloads for the Pathfinder API demo endpoints here.

5. Onboarding Solutions to the PACT Online Catalog

Companies can onboard their Solutions to PACT Online Catalog at any time – regardless if their Solutions were tested for Conformance or not tested yet. Depending on whether the Solution was Conformance tested or not, the Status of the Solution in the PACT Online Catalog will reflect accordingly once they are onboarded to it. Please Refer below -

5.1. Process to Submit a Solution in Online Catalog

Navigate to the “Contribute” section
Click the first “Go to GitHub” button under the “Developing Solutions?” option
- You will be taken directly to the WBCSD PACT Online Catalog repo.
- Follow the process as described in the CONTRIB_SOLUTION.md to Submit a Pull Request (PR) for your Solution
Once the Pull Request is submitted (above step), the Pull Request will be reviewed for completeness. If all details are filled out correctly, the PR will be merged and the Conforming Solution will appear under the “Solutions" page on the Online Catalog
After clicking a particular “Conformant Solution”, you will be taken to its Details page
You can review the details of all your Conformance Tests under the “Conformance” tab

6. FAQs

Do participants need to identify a partner BEFORE the Connectathon kickoff?

PACT Team will conduct a Kickoff before every Connectathon which will cover – Overview of the Conformance Testing Process, Introduction of the participants, Logistics (MS Teams channel, Communication / collaboration mechanisms between testing partners), Demo of the Online Catalog and How to add Conformance Test Results and Solution to the Catalog, and any Q&A. Participants can decide during this kickoff or post Kickoff who they want to collaborate with for the Conformance Testing

What are the pre-requisites for participating in the Connectathons?

The main pre-requisites for participating in the Connectathons is that you have a Solution (fully ready or partially ready) that follows Technical Specification V1 or V2.

In the cases when your organization doesn’t have a Solution yet Or has a Solution but it offers only Data Recipient capabilities (i.e. your Solution doesn’t provide PCF data as Data Owner per PACT Technical Specifications) then your Organization can still participate in the Connectathon and Conformance Testing process, but in these cases your Solution will not be onboarded to the Catalog, but your Organization will be onboarded as "Collaborator" in the PACT Catalog. Please refer to "this documentation" for more detailed overview of the process in such scenarios.

Is there a limit on the Testing Partners an Organizations can conduct Conformance Testing with during the Connectathons?

Organizations are welcomed and encouraged to pair-up with multiple Partners who are available for Testing. Note: Minimum 2 independent Organizations need to conform the Solution for the Solution to be deemed PACT Conformant

How many times can we offer up our solution to be tested in Connectathons?/How many Connectathons can we participate in?

Organizations are welcomed and encouraged to participate in multiple Connectathons and conduct multiple Conformance tests within a Connectathon.

Is there a component of "learning" as part of Connectathons?

Once the conformance Testing is concluded, Organizations are recommended to share the Test Results and the observations. (In the format they choose to). These observations can be leveraged as learning opportunities. PACT Team will also plan on organizing Retrospective sessions after a Connectathon is concluded, so that Organizations who participate in the Connectathons and conformance testing can share their experience and learning.

Our Solution passed the Conformance Tests during a Connectathon. Can we still participate in any upcoming Connectathon?

Yes you can participate in any / every Connectathon based on your availability and readiness.

Can Organizations participating in the Connectathon collaborate over their preferred Communication Platform or communication via WBCSD MS Teams is a mandatory requirement?

Organizations are welcome to use their preferred Communication platform based on the alignment with their Testing Partner(s).

When should we upload the Conformance Results to the online Catalog?

Organizations are encouraged to upload the Conformance Test results to Online Catalog if the Solution passes the Conformance test. If the Solution fails the Conformance Test, it is up to the Organizations to upload that Result to the Online Catalog and it is not mandatory.