FMCSA | SambaSafety
Considerations for FMCSA Crash Preventability Program

Considerations for FMCSA Crash Preventability Program

On Wednesday, July 31, 2019 the FMCSA Administrator gave notice¸ that the Pilot Crash Preventability Program begun in July 2017 will be made permanent with modifications. Here is what is being modified:

  1. Process will be streamlined
  2. Non-Preventable Crashes Removed from SMS (CSA Scores)
  3. The eight Categories of acceptable crash submissions will be extended to 15 Categories
  4. A 60-day comment period will be opened upon publication of this proposal in the Federal Register

I encourage all stakeholders to read the 19 page proposal and share your thoughts through the public comment process. Motor carriers, insurers, lawyers, trade associations, and others should all take time to understand this program and its proposed changes and speak up. This is important, folks.

To get you thinking, I will try and summarize the Notice. But please, be thorough, do your homework, read the proposal and form your own opinions before the comment period ends.

What follows is my reading, and my own analysis.

Introduction

FMCSA reports that between Aug. 1, 2017 and May 31, 2019 (22 months), they received 12,249 submissions through the DataQ’s system, which they call Request For Data Review (RDR’s). Of those requests, 5,619 met the criteria of at least one of the eight acceptable categories. That’s 45.9% of RDR’s that were actually scrutinized, rejecting 54.1% that were not reviewed because they did not fall into one of the pre-defined categories. FMCSA does nothing to summarize the rejected RDR’s. That would have been interesting to know what was there.

I have always believed that the pilot program should have been open to all crashes with the burden on the motor carrier to submit compelling evidence of non-preventability. It would have been a better pilot if we had used all crashes to organize categories, rather than pre-selecting based on opinions. So, we should be clear, a MINORITY of submitted RDR’s were reviewed. (NOTE: FMCSA states that 56% met the criteria. This looks like a small math error as 5619/12,249 does not equal 56%.)

Of the RDR’s reviewed, 69.9% were in the category: Commercial Motor vehicle (CMV) was struck from the rear. A distant second place, at 7.9% was: CMV struck while legally parked. You can see the breakdown of all categories on page 6/7 of the Notice.

Streamlining the Process

During the pilot period, all qualified RDR’s were held for 30 days following determination so that any interested party could submit information in opposition to the determination. FMCSA reports that not a single public comment was submitted during the pilot period. They have, therefore, proposed to streamline the permanent program by removing this public comment component.

SMS will reflect Determinations

The Notice states that the Safety Measurement System (SMS) website will be updated when non-preventable crashes are determined. To translate, this means official CSA Scores will now reflect the removal of non-preventable crashes for both Motor Carrier and Driver.

Eight plus eight become 15

The category shifts are the meat of the Notice. There are some interesting proposals in here. Some are positive, some less so, some need further clarity, and some are just downright mysteries. I still wonder why we need to pre-categorize at all.

On page five of the Notice, FMCSA lists the eight acceptable categories from the pilot program. My table below aligns pilot to permanent for ease of comparison. The order does not represent any priority or number of RDR’s in that category. There is nothing alphabetical about the categories and they don’t seem to have astrological significance.

Modifications to Pilot Categories

Thoughts, Concerns, and Questions

  • They determined a partial crossover “going wrong direction” is acceptable, but nothing bad ever happens on partial crossovers, right?
  • “No rear sideswipes” (see New category 8/9) technically makes sense as one cannot rear end and sideswipe at the same time.
  • Why is there a new category for “stopped in traffic”?
  • Suicide intent? How does a MC document suicidal intent?
  • Animal swerves no longer qualify. Still, why is Disabling dropped?
  • Infrastructure/Cargo drops reference to falling trees/rocks. Is this all considered debris?

Then there are the new categories to consider:

More Thoughts, Concerns and Questions

  • 5:00 – 7:00? This seems like artificial precision that rarely exists or is determinable at the scene.#8 and #9 taken together:
    • #8 CMV at 1MPH in traffic can only submit if side impact is between 5:00 and 7:00.
    • #9 The instant the CMV reaches 0MPH, all side impacts count.
  • If OV is making a Legal U-turn and CMV operates illegally – say runs a light – why would OV making a legal U-turn automatically qualify?
  • Medical issue: Submitter not only has the burden of knowing the intent of a suicidal person but must also be able to determine the health issues of OV Driver. As soon as we’re done talking to PETA, we better get the HIPPA folks on the line, we’re going to need to see those medical records.
  • The Driver admits to fatigue or distraction is a presumptuous expectation. “Yes officer, I’ve been on the phone and texting the last hour and it made me sleepy. I’m so sorry,” said no one…ever.
  • Under the influence? Presumably this is an intoxicated person – perhaps in a vehicle, perhaps just stumbling down the road – but manages to cause the crash or steps into traffic? OV and CMV crash; all kinds of chaos ensue; law enforcement is summoned; drivers exchange information, give assistance, and perhaps things get heated. When the cops arrive, what are the chances that intoxicated person is still around?
  • OV wrong direction: So, OV2 is traveling in front of CMV. OV1 crosses head-on into the wrong lane, but not completely. OV2 reacts and swerves to save his young family causing CMV to run off the road. Not qualified because OV1 did not completely enter the wrong way lane. Sir, you should not have swerved!

My overall feedback comes down to three areas of concern:

Why do we need to pre-define categories?

We don’t need pre-invented categories or to show the data on all the rejected crashes and explain why they could not be reviewed, and why the data supports these category definitions. Use the trained reviewers burdened with the submitting motor carrier to show enough evidence using the PAR’s and other data, technology, video, telematics, AV Tech, witnesses, etc. There are as many fact patterns to crashes as there are crashes. Let’s determine which are preventable and which are not.

Assume we go forward with Categories.

A significant amount of clarity is needed in many of these categories. For instance:

  • Are falling trees and rocks still qualified? They’re no longer specifically called out. Can we assume all details of pilot categories carry forward unless specifically removed?
  • 5:00 to 7:00 sideswipes, but only if moving. How did they support this?
  • Proof of suicidal intent? Almost impossible to see how the motor carrier gets this information.
  • Medical issue caused crash? No way Motor Carrier gets access to the health and medical records of other drivers. Better go run this one by HHS and get HIPPA exemptions.
Category conflicts abound

I’ll make one up as a hypothetical to illustrate potential conflict.
OV Driver under the influence partially crosses the center line, which causes a deer to be spooked and leap onto the roadway. This scares OV2 driver who has a medical issue and faints, resulting in OV crash into a bridge abutment causing a big chunk of concrete to fall off into the path of a CMV. The CMV swerves and successfully avoids the concrete chunk by partially crossing the center line…but ends up striking the very same deer whose lifeless body flies into a tree, that which then falls onto the roadway and into the path of a second CMV, who ultimately has the recordable crash.
So, again, why do we need to pre-define categories?

I encourage all interested parties to do their due diligence and read the Notice. Then, make your opinions heard in the now active 60-day window. You can get started giving them your thoughts here.

The State of the CSA Score: IRT and Facing the Future – Fully Informed

The State of the CSA Score: IRT and Facing the Future – Fully Informed

In the dozen years that I have been involved in the Trucking Industry, I have met many amazing people and seen advances in safety technology, data, and analytics that are mind-boggling.  I have met hundreds of business owners and executives who operate their businesses with a vision of literally delivering America, but a mission built on Safety.  If I have heard it once, I’ve heard it 100 times – competition stops at safety’s door.

In 2007, I founded Vigillo and now proudly serve as an executive with SambaSafety. While SambaSafety is a supplier to the industry and exists as a for-profit business, we live and breathe the mission of safety.

Change has been constant in my industry tenure and perhaps none as momentous as the pending changes to the foundation of the CSA scoring methodology – used by industry and the FMCSA to gauge a carrier’s commitment to safety.

The specific topic is the work SambaSafety, and previously Vigillo, has done to work through the transition from CSA to the new IRT Model as defined by the National Academies of Sciences, Engineering and Medicine (NAS) in their report of June 2017:

https://www.fmcsa.dot.gov/sites/fmcsa.dot.gov/files/docs/mission/policy/82226/nas-report-final-june-2017.pdf

Change as big as the shift from CSA Classic to an IRT model has led to a robust debate among industry participants. Below are excerpts from some concerns we’ve heard and our response.

 

Question: While it is possible to guess what the new IRT model will look like, how can any early representation produce results that will align with the final model?

We didn’t guess at the design of the model. Just the opposite. NAS provided a 130-page roadmap that describes in detail the IRT model.  With counsel from IRT experts, we followed that roadmap to produce our preview.

Moreover, we have a strong track record helping the industry to anticipate and interpret the CSA scoring methodology. Vigillo released the industry’s first CSA management platform in 2008, two full years before the official launch by FMCSA.

We have helped support changes to the CSA program, culminating in the FAST ACT and the IRT recommendation. In its early years, I provided data to illustrate CSA deficiencies to Congress as CSA criticism grew after its launch. I collaborated with the American Trucking Associations and others in the development of the language that was inserted into the FAST Act which ultimately launched CSA Reform.  At the request of the ATA, I gave the industry comments at the first meeting of the NAS and accompanied the ATA Chairman in the delivery of those comments. Over the next 18 months, I attended every public meeting held by NAS. Before the draft report was published publicly, I was one of the primary reviewers. Ultimately, I led the efforts at SambaSafety to retain the services of IRT experts including one of the NAS Panelists.

Back in 2008, Vigillo gave thousands of motor carriers a preview of what CSA was, how it worked, and how it would impact their safety programs. Advance knowledge of these kinds of significant changes at FMCSA is essential. That’s why we’re doing it again with IRT.

 

Question: Why worry about IRT now when it’s not near implementation?

The NAS report provided for a two-year timeline following the initial 18-month study.  That would call for implementation in June of 2019. FMCSA has stated publicly that its target is September 2019, not significantly behind.

Our best information says that we’ll see careful rollout and solicitation of industry feedback over the following six months.  Full implementation is expected in Q2 2020. The reforms are moving along pretty much as outlined in the FAST Act.

IRT produces very different results for many carriers.  IRT uses a methodology that is easy to interpret but nearly impossible for a non-statistician to calculate. Many of our customers want a preview of their IRT score to be included in the discussion, right from the start. It’s imperative for them to know how their safety culture will be assessed and it allows them to provide informed feedback to the FMCSA during the solicitation period.

At SambaSafety, we live the mission of safety, and we innovate. We invest in R&D and are continually building new products and services to serve that mission.  We think our job is to get there first. We have a proven track record of doing that time and time again. Our customers benefit from our ability to anticipate change, understand formerly hidden risks, participate in the discussion of the new model and prepare for a two-year look back.

It’s a choice to be proactive and get involved early to anticipate significant industry changes like IRT.  Hundreds of carriers are making that choice.  We think that’s a good thing for safety.

 

Question: Does IRT really improve upon legacy CSA?

It does, no question about it.

Let’s look at three of the defects of CSA as it exists today:

  1.  Safety Event Groups – Spiking Scores
  2. Disparate State Enforcement – Geography sways Scores
  3. A relatively small number of scored carriers – Less than 20% of carriers have sufficient data for any CSA Score

First, Safety Event Groups, as constructed today, group carriers in an attempt to compare similar carriers.  It has never worked well and has some problematic side effects.  Five of the seven Groups are based on inspection count.  A carrier can move from one group to another by virtue of just one inspection, even a clean inspection, and scores spike dramatically as a carrier moves up the ladder in Safety Event Groups. Related to this, a Carrier’s CSA Measure produces very different CSA Percentile Scores depending on what group they are in.  As a result, Carrier A with a BASIC Score of say 50% is not comparable to Carrier B with 50% in a different Group. 

IRT does away with Safety Event Groups in favor of a Risk Exposure Index.  IRT utilizes this Exposure Index which is created by blending Power Unit Count, Driver Count, VMT and Inspection Count to normalize carriers for comparison purposes. No groups to leap between and all carriers scores are comparable to each other.

Second, Law Enforcement Disparity has always plagued CSA.  CMV Enforcement on the New Jersey Turnpike is a very different animal than enforcement at the Texas border.  Our country is vast, diverse, and immensely challenging from an enforcement perspective. Enforcement knows their unique challenges and focuses on the specifics that they believe enhances safety in their own back yards. CSA does not formally recognize these challenges and punishes carriers who operate in targeted enforcement zones vs. carriers who do business elsewhere.

IRT is not about the frequency of violations.  IRT looks at patterns of violations; it’s not just counting them.  As a result, the wild swings we see in CSA Scores due to disparate enforcement are largely smoothed out. 

Third, CSA today cannot score a carrier with insufficient inspection data.  As a result, only about 100,000 carriers receive any CSA Score at all. IRT has a lower threshold for what constitutes sufficient data for it to evaluate and provides scores for almost 200,000 carriers.

There are other improvements that IRT brings in terms of a more scientific foundation, ability to adapt to changing patterns in the data and it is set up well for additional data to be added in the future.  We can hardly imagine the magnitude of varying types and amounts of data that trucking operations will generate in the future. IRT is uniquely capable of incorporating new types of data to make continuous improvements to our understanding of what constitutes an excellent Safety Culture in the Trucking Industry.

 

Steve Bryan

 

Happy Anniversary FMCSA – Meet The Parents

Happy February everybody, and Happy Anniversary!

Its been exactly 6 months (August 1 – Feb 1) since FMCSA launched their 24 month Crash Preventability Demonstration Program to see what impact it would have if FMCSA reviewed DOT Record-able crashes and made Preventable/Non-Preventable determinations for purposes of CSA crash scores. So happy 6-month (26 week) anniversary!

According to Match.com findings a 6-month relationship is important because this is when three key relationship milestones take place:

  • Revealing of one’s imperfections (24 weeks)
  • First argument (23 weeks)
  • Parental introductions take place (24 weeks)

So I ran some numbers FMCSA, its time to meet Momma.

Imperfection #1 – Permitted time period out of sync with CSA Scores

Lets start at the highest level.  Motor Carrier (CSMS) CSA scoring is based on the most immediate past 2 years of data including crashes.  In the two years prior to January 1, 2018, FMCSA recorded 307,732 crashes.  (MCMIS as of December snapshot).

All Crashes – Two Years

But FMCSA only allows crashes after June 1, 2017 to be submitted for reviews under its new program that started August 1, 2017.  So less than 25% of CSA crashes can be submitted at this time, it will only catch up with the 2 year CSA window, as the Demonstration Program comes to a close at the end of 24 months.

Crashes Qualifying for Review – 76,122 (24.7% of all Crashes)

Crash Locations – States (Oh look, Texas wins again!)

So as we sit here at the 6-month milestone, we can only judge the preventability project on the 1,149 submitted crashes out of 76,122 crashes that could possibly be submitted under the limitations put in place for this program, or 24.7% of all possible crashes. I never understood the reasoning behind limiting crash submissions to those since June, when FMCSA uses crashes back two years for scoring purposes. Seems fair to align those dates, but hey, we were early in the relationship and we didn’t want to have that first argument.

Imperfection #2 –  Transparency could be better

On August 1, as soon as the Demonstration Program launched, Vigillo started capturing all crashes submitted through DataQ’s.  FMCSA publishes the outcomes of crash reviews publicly, but cycles them through every 30 days, removing the previous month from public view.  There is no running total, so we created one.  We set up a daily capture and can now report that 1,149 crashes have been submitted for review by 588 different motor carriers since August 1.  That’s just 1.5% of the possible 76,122 that have been recorded in that same time period. This is starting to look an awful lot like the DataQ’s system we already dated and broke up with.  Lets get some reporting in there team, this is a demonstration program, lets see the results without having to do backflips. Motor Carriers, lets get busy, submit those crashes.

Imperfection #3 – Only certain predetermined categories can be submitted

FMCSA predetermined that only crashes in the following categories can be submitted for review:

  • When the commercial motor vehicle (CMV) was struck by a motorist driving under the influence (or related offense)
  • When the CMV was struck by a motorist driving the wrong direction
  • When the CMV was struck in the rear
  • When the CMV was struck while it was legally stopped or parked, including when the vehicle is unattended
  • When the CMV struck an individual committing, or attempting to commit, suicide by stepping or driving in front of the CMV
  • When the CMV sustained disabling damage after striking an animal in the roadway
  • When the crash was the result of an infrastructure failure, or falling trees, rocks, or debris
  • When the CMV was struck by cargo or equipment from another vehicle

And this is the breakdown now 6 months into the Program:

Actual Submissions by Crash category – 1,149 submitted

Actual Review Outcomes by Crash category – 596 Non-Preventable (52%)

Our first argument – Many clearly Non-Preventable Crashes are not being reviewed.

So to recap, Crashes going back two years should be permitted for submission, we need better and more transparent reporting from FMCSA, and with the huge investment in technology, including camera’s, there should not be predetermined categories of acceptable crashes.  It is impossible for anyone outside FMCSA to know how many crashes are being submitted and rejected out of hand because they are not in one of the pre-determined categories, and even less knowable, how many carriers don’t even try.

The link below is from a motor carrier who will remain anonymous, and it shows a crash that is clearly not preventable by the driver or motor carrier.  But it is a side collision, and FMCSA has predetermined that no side collision can be reviewed.  It is a Demonstration Program, lets demonstrate what is possible today with technology and video.  Until you do, I’m not sure how I feel about continuing our relationship, or having you meet my parents.

 

Steve

 

UPDATE: https://www.overdriveonline.com/fmcsa-clarifies-how-to-submit-crash-preventability-review-requests-for-pilot-program/?utm_source=daily_utm_medium=email_utm_content=02-07-2018_utm_campaign=Overdrive_ust_id=b5c8550d89cbb74bab233e6e8dc92bec