Author: Amira Armond, the president of Kieri Solutions – an authorized CMMC Third Party Assessment Organization (C3PAO) providing CMMC assessments, CMMC consulting, and Compliance Documentation packages designed for small/medium business.
This graphic depicts my personal experience talking with defense contractors about their 800-171 and CMMC compliance (and what score they entered in SPRS) over the last three years. This is non-scientific and is based on anecdotal evidence only.
The blue line is the “actual score” – what the company SHOULD be reporting for their compliance score in the Supplier Performance Risk System (SPRS).
The orange line is the “reported score” – what the company actually reports in SPRS.
What are the trends in 800-171 reporting?
At least half of defense contractors shouldn’t submit a SPRS score (technically “not applicable”).
According to the Department of Defense’s NIST SP 800-171 Assessment Methodology, a defense contractor should not report a score at all if they do not have a fully written System Security Plan.
The sad truth is that we have a huge cybersecurity skills shortage, especially for the topic of Governance, Risk, and Compliance (GRC). GRC experience is the baseline expected in order to write a System Security Plan. (want to become that GRC person? check our article and video on how to write a quality SSP)
How many defense contractors have someone on-staff with GRC experience? The answer is: almost none if they are small businesses. Not many if they are medium businesses. Only the large defense contractors have been consistently throwing resources at this problem.
What % of defense contractors are small? 67% according to Defense.gov.
These statistics match up with my own personal experience talking with hundreds of defense contractors over the last three years.
The reported SPRS scores become more accurate as real scores improve
There are two reasons why reported scores start matching real scores as cybersecurity improves: use of GRC specialists and incentives to falsely report.
Governance, Risk, and Compliance (GRC) Specialists
The first reason why scores become more accurate is that companies doing 800-171 / CMMC correctly will have that GRC expertise that I just mentioned above.
Companies need someone who understands GRC in order to determine if you are correctly performing the cybersecurity requirements. Many of the requirements have deep expectations that aren’t obvious to an untrained eye. Having a non-cybersecurity business owner or salesperson perform their own assessment and scoring typically results in MUCH higher scores than should be assigned.
If a company uses a GRC specialist to assess their compliance and calculate a score, it is much more likely to be accurate.
If a company doesn’t have a GRC specialist involved, their real score will go down (because they don’t know how to be compliant) and their reported score will go up (because they incorrectly think that they are performing the requirements).
Incentives to report a false SPRS score
Let’s look at two imaginary contractors:
WidgetsUSA reports a score of “110” – a score that indicates they are doing 100% of the requirements.
FreedomCorp reports a score of “3” – a score that indicates they are doing about 60% of the requirements.
Which company is actually more secure?
Which company doesn’t know what they are doing, or is falsely reporting?
Hopefully you are thinking like I think – that the company with a lower score is being more accurate, and likely is more secure in reality.
However, large primes are treating this in the opposite way. If you (a subcontractor) report a low score to your large prime, they will penalize you by investigating your company and potentially removing you from contracts. The contractors that report a perfect score get an easy pass.
The system incentivizes ignoring cybersecurity responsibilities
Contracting companies know that the odds they will be audited by the DoD are extremely low (not even 1 %). When they calculate the cost of being compliant (sometimes more than the yearly revenue of the entire company), a significant number decide to take their chances.
And know what? Those companies that are falsely attesting without paying for cybersecurity are the ones that win the contracts.
Allison Giddens from WinTech (a small defense contractor), explains it best in this YouTube clip:
Highly compliant companies report lower SPRS scores than they deserve
800-171 and CMMC requirements have a “lower bar” than similar requirements used within DoD and federal networks. Controlled Unclassified Information is not SECRET information.
Contractors are allowed to define their risk tolerance for many requirements. For example, in Federal networks, there are specific requirements to lock accounts after 3 bad password attempts, and force interaction with the helpdesk before unlocking the account. In contrast, defense contractors are allowed to lock accounts at any threshold they think is appropriate (5? 10? 15?) and automatic unlocking over time is allowed.
Contractors who hire strong Governance, Risk, and Compliance (GRC) talent, are normally hiring people who used to work on Federal networks. These specialists often set the bar too high for their own organizations because they are used to Federal requirements. This often results in the GRC specialist marking the requirement NOT MET when it would be considered MET by a typical CMMC assessor.
Having compliance documentation dramatically improves SPRS scores
About 70% of the CMMC assessment objectives are met with policy, procedure, and other documentation-centric activities.
Contractors that focus on “set it and forget it” technical security systems and ignore their documentation and manual procedures will fail almost all requirements. This is because each 800-171 / CMMC requirement typically includes at least one assessment objective that requires documentation. (Each CMMC requirement has one-to-many assessment objectives)
There is a steep drop in real security scores between the companies that have their documentation in order, and companies that are ignoring documentation. This line rapidly drops to a “not applicable” score for the companies that don’t even have a system security plan.
This is just my personal experience. How does it match with what you’ve seen?
Do you think Defense Contractors are acting rationally when they report perfect 110 SPRS scores, even if their real score should be in the negative numbers?
How can we fix this issue, other than requiring third party assessments?
Do you think that companies reporting a perfect 110 should be specifically targeted for audits?
Do you think that companies reporting a non-perfect score should be targeted instead?
Do you think companies choose to report close-to-perfect scores (like 105) in order to reduce their risk of audit?
Author: Amira Armond, President of Kieri Solutions, an authorized CMMC Third Party Assessment Organization (C3PAO). Kieri Solutions provides assessment services, high-quality CMMC consulting, and an easy to use compliance documentation package geared toward small and medium businesses.