Social Security and Machine Driven Disability Determinations

Why is it so hard to get disability benefits nowadays? Social Security has not substantively changed any of its rules and there has been no change in the definition of disability. Yet, approval rates have plummeted to record lows over the last few years, even while Social Security continues to claim that its decisions are more accurate than ever. The answer to that question is probably very complicated but I believe it starts with SSA’s use of computer algorithms to evaluate claims and decisions.

There is a graph based on the 2015 SSA statistical report in the November 2016 NOSSCR Forum I found very interesting. It categorizes the reasons people win their disability claim into four categories: Meets level of severity of listings; equals level of severity of listings; medical and vocational factors considered; and other.

“Meets level of the severity of listings” refers to a set of regulatory criteria SSA has published for specific conditions. If a person meets those criteria they are automatically found disabled. Those criteria are commonly called the Listings.

“Equals level of severity of the listings” refers to conditions that may not specifically meet the criteria but are so similar that they can be found equivalent to a listed impairment. For example, SSA states that it will find that migraine headaches will be found to equal the listing for epilepsy in certain circumstances.

“Medical vocational factors considered” refers to what are commonly called the grid rules. The grid rules direct a finding of disabled or not disabled using age, education, and work skill attainment along with a person’s limitations. They generally apply only to people over the age of 50.

Both the listings and the grid rules have very specific requirements that are easy to evaluate for professionals familiar with Social Security’s rules and policies.

“Other” is not well defined in the statistical report but appears to refer to step 5 of the disability evaluation process. At this stage SSA is supposed to consider all of your conditions, whether severe or not, to determine your ability to do any work that may be available in the national economy. Decision making at this step necessarily involves a lot of judgement and generally requires the input of a vocational expert. It also requires an assessment of the consistency of a claimant’s description of limitations with the medical record as a whole. This is significant because the “other” category is the one that has seen the most change in recent years.

The numbers from the statistical chart go along pretty flat until 2012, when there are some very big changes. During this same period, SSA’s disability approval rate has dropped to historically low levels. To my knowledge, SSA has never offered an explanation for these drastic changes. There have not been any significant regulatory changes, policy changes, or personnel changes, that could account for such a drastic change. Most of us who practice Social Security law have been left scrambling to deal with the changes and to figure out why they may be occurring..

The data from the statistical report shows that the percentage of favorable decisions due to application of the medical/vocational factors has increased from a little over 40% to 55%; the percentage of favorable decisions attributable to meeting a listing has a very similar increase, from a little over 20% to 35%. That means 90% of all favorable decisions are attributable to either meeting a grid rule that directs disability or meeting a listing.

At the same time the other category had dropped from about 30% to about 5%.

Another chart that was included shows unsurprisingly that the primary reason why claimants are denied is that they are found “able to do other types of work,” a step 5 denial. But the percentage of cases with that finding has increased from a little over 30% to just under 40%. Interestingly, the big jump in these charts happened in 2011.

I think that is pretty much all you need to know if you want to know why SSA approvals have declined so much. SSA has basically eliminated step 5.

How did SSA accomplish such a thing?

I remember in about 2009 (maybe it was 2010) attending a NOSSCR conference where someone from SSA mentioned that they had a contract with IBM to develop a decision evaluation algorithm. The idea was that you could feed the computer data and a decision model to have it identify decisions that may have problems and then spend your time looking at those. Computers are really good at identifying specific rule applications like the grid rules and listings. I could easily imagine a computer that could read a set of medical records and find information relative to meeting a listing and then determine the likelihood the listing was met based on whether all the factors were discussed and how frequently. However, SSA has refused multiple requests by different parties to identify how the algorithm works, saying they don’t want to give anyone the tools to game the program. The program isn’t making decisions, just checking the work of the humans and providing training.

Gerald Ray, Deputy Executive Director, Office of Appellate Operations, discussed the program in detail in a pod cast in 2015. Mr. Ray takes issue with the premise that Social Security judges are experts. He sees the application of big data as a tool to refine not only the accuracy of the decisions but also sub-regulatory policy and training and instruction. He describes the process he took SSA through to develop a set of decision evaluation criteria and then to use those criteria to “train” decision makers to issue “better” decisions.  He says that SSA used over 2,000 data points to develop the model. SSA has not released those data points so we don’t know what they are, but it has stated the data is used to create a feed back loop so that decisions will be made to more closely match the model. In other words decision accuracy is determined by how closely a decision matches with a computer model of what a good decision is. The human element, the element most prominent at step 5 and in many cases most important to disability, has been removed. What once required judgement, is now simply a mechanical application of a secret computer algorithm. The closer you get to the algorithm the better your evaluations will be.

Leave a Reply

Your email address will not be published. Required fields are marked *