Author: Tom Cornelius| Senior Partner at ComplianceForge | Founder & Contributor at Secure Controls Framework (SCF)
Originally published on LinkedIn on August 13, 2020
This episode of Coffee Thoughts With Tom addresses CMMC as a conformity assessment, since conformity assessments are intended to use a risk-based approach to determine a confidence point (e.g., materiality threshold) instead of a binary approach that requires 100% compliance without compensating controls or Plan of Action & Milestones (POA&M) items. This article does provide a quantifiable solution that is based on industry practices, so read on!
From other cybersecurity practitioners, I’ve heard that in respect to CMMC there are so many “elephants in the room” that it feels a bit like a circus. Arguably, the biggest elephant in that circus is the DoD’s binary approach to pass/fail. While it may disappoint certain readers that this article is not about circus animals, others might enjoy reading about a proposed solution to define a materiality threshold for a risk-managed approach to CMMC assessments. Instead of merely complaining, I want to see a rational discussion on the subject, since the DoD PMO’s approach to a “100% pass or 100% fail” in evaluating CMMC practices and processes is both misguided and goes against industry-recognized practices. CMMC will fail without a POA&M component, so the DoD PMO and CMMC-AB needs to have a way to incorporate that need into CMMC assessment criteria and this article addresses that need.
For transparency, the approach described in this article is based on industry-recognized practices and is the basis of what is under development for the Information Assurance Program (IAP) that is part of the Secure Controls Framework (SCF). Both the IAP and CMMC are conformity assessments.
Conformity Assessments
To appreciate the points of this article, you really need to understand the basics of conformity assessments. NIST published an excellent primer on the subject, if you want to read more about conformity assessments: NIST Special Publication 2000-01 – ABC’s of Conformity Assessment. In summary, conformity assessments examine “an object of conformity (such as a product, process, system, person, or body) and determines whether the object meets specified requirements.”
For the conformity assessment hierarchy of CMMC assessments:
- The CMMC-AB is the Accrediting Body (AB).
- The CMMC-AB accredits Certifying Bodies (CB) which are the organizations that actually perform CMMC Assessments.
- Certified 3rd Party Assessment Organizations (C3PAOs) are CBs.
- A DoD contractor is an Organization Seeking Certification (OSC).
If you read through NIST SP 2000-01, you will not find any mention of the words “fail” or “pass.” Conformity assessments are designed to assure that a particular product, service, or system meets a given level of quality or safety. Instead of a 100% pass criteria, conformity assessments rely on a “confidence point” that determines a risk-based threshold to establish if the intent of the objective(s) has been achieved.
NIST Special Publication 2000-02 – Conformity Assessment Considerations for Federal Agencies is a great follow-on read on the subject, since it goes into good detail on the topic of confidence points. Essentially, a confidence point establishes pre-defined criteria that an objective has been achieved, weighing the risk of non-conformity against its associated consequences. What the DoD PMO fails to appreciate is the confidence point is not meant to be a 100% perfection expectation. This is where the concept of materiality comes into play.
“Perfection is the enemy of progress.” — Winston Churchill
Confidence Points & Materiality
To help determine what constitutes the confidence point for a CMMC assessment, it is important to leverage industry practices. In particular, the concept of “materiality” is a relevant approach to address this need. In legal terms, “material” is defined as something that is relevant and significant:
- In a lawsuit, “material evidence” is distinguished from totally irrelevant or of such minor importance that the court will either ignore it, rule it immaterial if objected to, or not allow lengthy testimony upon such a matter.
- A “material breach” of a contract is a valid excuse by the other party not to perform. However, an insignificant divergence from the terms of the contract is not a material breach.
For those in the Governance, Risk Management & Compliance (GRC) space, materiality is often relegated to SOX compliance. However, the concept of materiality is much broader than SOX and can be used in any form of conformity assessment.
“Specific to the CMMC, a material weakness is a deficiency, or a combination of deficiencies, in an organization’s security controls where it is probable that reasonable threats to regulated data (FCI/CUI) will not be prevented or detected in a timely manner.”
In practical terms, a material weakness a deficiency, or a combination of deficiencies, in the internal security practices of an OSC where it is probable that reasonable threats to CUI will not be prevented or detected in a timely manner. Following this logic, the CMMC assessment methodology could identify non-compliant practices and processes as either a (1) minor deficiency or (2) material weakness. A material weakness would be an immediate failure for the CMMC assessment, but a minor deficiency would be something capable of being addressed through compensating controls and temporarily placed on a POA&M. POA&Ms have a long history going back to DITSCAP and DIACAP, but it is still a valid way to remediate risk in NIST SP 800-171, RMF, FedRAMP, FISMA and even outside of the government with compliance obligations such as PCI DSS.
For those OSC already in scope for DFARS, the OSC would be expected to follow established requirements to submit POA&M items for approval by the DoD CIO. For those not in currently in scope for DFARS, POA&M items could be self-regulated until a contract is awarded and the OSC falls under DFARS. The C3PAO would be required to validate if any POA&M items are legitimate per DFARS applicability. That process would allow for the use of a “temporary deficiency” to be used in the CMMC assessment model that would be tracked per CA.2.159 (POA&M requirement within CMMC).
Similar to DITSCAP, DIACAP, RMF, there could be three straightforward ways to define the confidence point for a CMMC assessment:
- Authority To Contract (ATC)
- Interim Authority To Contract (IATC)*
- Denied Authority To Contract (DATC)
*If an OSC had an IATC designation, they could get a 180 or 365 day grace period to remediate the deficiencies that would require re-evaluation of those deficient controls by a C3PAO. That approach allows for flexibility in applying compensating controls without disrupting the DoD’s supply chain.
Material Threshold
At the end of the day, the CMMC requires a defendable and repeatable approach to calculate materiality. Based on an understanding that any non-compliance in a high-risk practices or process would be considered a material weakness of the OSC’s security program where no POA&Ms should be accepted, this could open the door for working with low and moderate risk practices. NIST SP 800-171 DoD Assessment Methodology already tags NIST SP 800-171 controls with a score of 1, 3 or 5, so that work could be leveraged to assign corresponding “low, moderate and high” risk rankings to CMMC practices and processes. For the non-NIST SP 800-171 controls, the CMMC-AB could assign a low, moderate or high risk ranking to each practice.
What is being described is a straightforward approach for the DoD to establish a confidence point. For example, any combination of deficiencies that exceed a 10% threshold become a material weakness of the OSC’s cybersecurity program since it indicates a failure to protect regulated data (FCI/CUI). That threshold could also be 20% (or any other reasonable value) – it would be up to the DoD to determine the confidence point that reasonably manages cybersecurity risk to the DoD against the financial impact to the DIB.
Specific to this CMMC example:
- A deficiency in a process would be a material weakness to the security of regulated data (FCI/CUI), since CMMC processes involve policies, standards and resourcing that are crucial to the OSC’s overall cybersecurity program.
- A deficiency in a high-risk practice would be a material weakness of the OSC’s cybersecurity program.
- A deficiency in a low or moderate-risk practice would not by itself materially impact the security of regulated data (FCI/CUI).
- Within the same domain, more than one deficiency in a moderate-risk practice becomes a material weakness.
Below is a quantitative solution that uses a 90% confidence point as the threshold to determining materiality:
- Within the CMMC there are:
o Level 1 – 17 practices
o Level 2 – 72 practices
o Level 3 – 130 practices
- Each practice/process can be assigned a point value:
o Low-risk practices: 3 points
o Moderate-risk practices: 5 points
o High-risk practices: 20 points
o Processes: 20 points
- The next step is to identify the total points available per level. Since high-risk practices would be material weaknesses, the focus of calculating a point system would focus on low & moderate risk practices. Using the previously-discussed DoD point values comes up with a numerical score associated with low and moderate-risk practices for each level:
o Level 1 = 14
o Level 2 = 134
o Level 3 = 153
- Using a materiality threshold of 90% compliance (10% control deficiency) comes up with the number of points an OSC could accumulate before the combination of deficiencies becomes a material weakness. This is calculated by multiplying the previous numbers by 10% and rounding to whole numbers:
o Level 1 – 1 point (this means no low or moderate practices can be deficient)
o Level 2 – 13 points
o Level 3 – 15 points
The charts below show an example of how various combinations of low and moderate-risk practices could be deficient to still “pass” the CMMC assessment on an interim (e.g., IATC) basis.
The end result is this would allow for a small number and combination of low and moderate-risk practices to be deficient (tracked and approved via POA&M through the use of compensating controls) and still enable the OSC to participate in the DoD supply chain, since the risk would be sufficiently mitigated. The cells highlighted in green would meet the conformity point. The cells highlighted in red would mean the conformity point was not met and a material weakness would exist to protect regulated data (FCI/CUI).
Summary
As it currently exists, CMMC is not structured as a “real” conformity assessment – it is merely designed as a checklist-based assessment performed by an independent third-party that does not take into account the OSC’s technology or business processes to legitimately manage risk. This would be an easy fix by the DoD PMO, since both POA&Ms and the concept of materiality are well-established and would allow for a risk-managed approach to CMMC assessments.
In the end, compensating controls are a good security practice, when done properly. It sufficiently manages the risk of the client (DoD) and takes into account real-world limitations affecting the DIB. The objective solution that is described in this article would also remove subjectivity from the C3PAO by using a point system that is centrally-managed by the CMMC-AB.
As the DoD’s “Pathfinder” proof of concept is executed later this year, it will be interesting to see how the topic of deficiencies plays out and if “critical” prime and sub-contractors are allowed exceptions to this draconian pass/fail concept. My bet is that one elephant will ruin the rest of the circus.
About The Author
If you have any questions about this, please feel free to reach out. Tom Cornelius is the Senior Partner at ComplianceForge, an industry leader in cybersecurity and privacy documentation. He is also the founder of the Secure Controls Framework (SCF), a not-for-profit initiative to help companies identify and manage their cybersecurity and privacy requirements.