June 1, 2017 | SambaSafety

CSA Reform Status – Vigillo Comments


PDF Version Here


Panel on the Review of the Compliance, Safety, and Accountability (CSA) Program of the Federal Motor Carrier Safety Administration

Initial Comments by Steve Bryan, President & Founder of Vigillo, A SambaSafety Company

June 28, 2017

Yesterday, June 27, 2017, 18 months after the FAST Act removed CSA Scores from public view, the National Academies of Science, Engineering and Medicine (NAS) published their preliminary findings outlining their recommendations to reform the Compliance, Safety and Accountability program we all know and love as CSA.

Their 132 page report starts with a nod to the FMCSA for the work they have done on CSA and for the funding for this study (amount undisclosed).  NAS starts by thanking Joseph DeLorenzo, director, FMCSA Office of Enforcement and Compliance, for his presentations, which were not just useful, but “enormously useful”, way to go Joe!

That was followed by acknowledgements to the other twenty-five presenters the first of which was yours truly (it was alphabetical). We are all thanked but our contributions are not characterized on a usefulness scale.  Item Response Theory (IRT) applied to the relative informational value of each of us would have been nice, but hey, more on that later.

NAS categorizes their report into the following four sections:

  1. Assessment of the Current Safety Measurement System
  2. A More Natural Statistical Model
  3. Data Improvements
  4. Transparency, Reproducibility, and Public Disclosure of Safety Rankings

In one paragraph, I’ll summarize. My take on this entire 132 page report is stated as follows.

“Good job FMCSA on what you have done with CSA so far, we (NAS) give you a solid 5 out of 10 for your efforts so far.  But there needs to be a sound statistical model applied to replace the ad hoc nature of how CSA is formulated today. More, and more reliable data needs to be gathered, and the data utilized needs to be more transparent and software developed to make such data more useful by all stakeholders.  Only then should CSA Scores be released to the public.”


National Academies CSA Reform Report to be Released Tuesday

NAS Building


Happy Friday everybody,

It seems to be pretty rock solid now that the CSA Reform Report, 18 months in the making, will be released to the press on Monday, and the Public on Tuesday, June 27.

I’ll be reading the report as soon as I can get a hold of it, I have a team at Vigillo ready to dive into it, and we’ll be issuing comments by Wednesday, perhaps sooner.  Watch this blog for those comments.  Where possible, we’ll be running the numbers in addition to issuing opinions.

I posted a status report last December HERE and outlined the 8 items that the FAST Act cited as areas that need review.  Those 8 areas of reform are:

Reform Requirements

1. The accuracy with which the BASIC’s identify high risk carriers and predict or are correlated with future crash risk, crash severity, or other safety indicators for motor carriers. (Safety Event Groups)

2. The accuracy of safety data, including the use of crash data from crashes in which a motor carrier was free from fault. (Crash Accountability)

3. Whether BASIC percentiles for carriers of passengers should be calculated separately from motor carriers of freight. (Freight vs Passenger systems)

4. The differences in the rates at which violations are reported to the FMCSA by various enforcement authorities, including States, territories, and Federal inspectors (State Difference)

5. How members of the public use the SMS and what effect making the SMS information public has had on reducing crashes

6. Whether the SMS provides comparable precision and confidence, through SMS alerts and percentiles, for the relative crash risk of individual large and small motor carriers;  (Small Carriers largely unscored)

7. Whether alternatives to the SMS would identify high risk carriers more accurately

8. FMCSA has fully implemented or satisfactorily addressed the issues raised in the report titled ‘‘Modifying the Compliance, Safety, Accountability Program Would Improve the Ability to Identify High Risk Carriers’’ of the GAO dated February 2014.

As I conduct my review, these are the areas I’ll be looking for action.

Have a good weekend, I’ll talk to you all early next week