How machine learning can help alleviate the U.S. labor shortage

Experts have been debating the causes of the shortage of workers in the U.S. But one thing is painfully clear: There is a staggering disparity between the number of jobs available (over 10 million) and the number of workers looking for work (around 6 million).

In this short article, we’ll step back and take a look at how we got here, the multiple factors that have led to such a disparity, and some of the solutions being implemented to try to fight this problem. Notably, we’ll take a look at machine learning (ML) and how it is being used to alleviate both the causes and the effects of the labor shortage in the U.S.

The current U.S. labor shortage

According to the U.S. Chamber of Commerce, the labor force participation rate has dipped in recent years, dropping from 63.3% to 62.3%. While a 1% reduction in the number of able workers participating in the workforce might not otherwise present a huge nationwide problem, it’s coming after a pandemic that saw well over 30 million workers lose their jobs.

The industries that have been hit the hardest include leisure and hospitality, food service, durable goods manufacturing, education and health services. But there is virtually no sector of activity that hasn’t been affected.

What are some of the causes of the labor shortage?

The COVID-19 pandemic indeed shook up the labor market. Studies show that around a quarter of a million working-age people died from the disease, half a million have left the workforce due to lingering health effects from the virus, and a similar number of workers have gone directly from illness to retirement. 

This reduction in the workforce should have been compensated for by job-seekers looking to enter the market, but that hasn’t happened. Instead, the U.S. has seen a rise in the monthly quit rate across all sectors. In some industries, such as leisure and hospitality, the monthly quit rate exceeds 6%. Traditionally more stable sectors, such as business and professional services, still record an alarming quit rate of more than 3%.

Many workers have expressed a desire to continue working from home. This is a difficult expectation to meet for some industries, such as health services and manufacturing. But this shift in employee expectations only scratches the surface. At-work child care services, a shorter work week, better work-life balance and continuous training top the list of what employees are demanding from their employers, and companies are slow to catch up and adapt to the change in employee-employer dynamics. This partly explains why, although the nationwide hiring rate is far higher than usual, companies across all sectors are still left with millions of positions yet to be filled.

What is machine learning?

Although often used interchangeably with AI (artificial intelligence), ML is more precisely a subset or an application of AI. In simple terms, ML is the application of big data wherein machines (computers) use mathematical models to develop a new understanding without explicit instruction.

For example, image recognition is a widely used application of ML. With image recognition, computers are able to recognize and match faces (“tagging” posts on social media platforms) or identify cancerous growths in an x-ray.

ML is also widely used in the financial sector in what’s known as statistical arbitrage: Using algorithms to analyze securities in relation to set economic variables.

ML also allows computers to examine large datasets, identify causalities and correlations, and extrapolate from their predictions and likelihoods. Predictive insights help get the most out of data. Applications of this predictive capability are found in real estate pricing, product development and other spheres. Predictive analytics can also help job seekers and recruiters find better matches than they have been finding thus far.

How is machine learning helping with the U.S. labor shortage?

The current U.S. labor shortage combined with the alarmingly high quit rate has shown us that there is a problem: Workers are having a tough time finding jobs that suit them.

Increasingly, recruiters and job seekers alike are turning to advanced algorithms and statistical analysis of big data to help mitigate this problem.

ML has the capability to analyze large sets of data — in this case, workers who either quit or are relieved of their duties versus those who have staying power or are promoted — and identify the common attributes, characteristics and skills. With this understanding, recruiters can more quickly and more accurately filter out candidates who are not likely to succeed in the position they are applying for. The result is a faster and smoother job search that is far more likely to lead to positive results.

In addition to refining the matching process, ML has a positive impact on the speed and duration of the recruitment process. The excessively long time a job seeker spends applying for and then interviewing for a job they are not likely to get or be happy with can only serve to further exacerbate the job seeker. When faced with a crisis of unfilled positions and a high quit rate, we need job seekers who are enthusiastic about the recruitment process and not frustrated by it.

The evolution of the online job portal

Traditionally, an online job portal was where job seekers could peruse the available jobs in their location or sector of activity, read through the various descriptions and requirements and then take steps to apply for jobs. While that is still a staple of today’s online job portals, the more successful ones take things a few steps further.

Uploading a resume to an online job portal that uses ML, the job seeker can be directed and oriented toward jobs that best suit their skills and experience.

However, ML can do even more than that. Having the requisite skills and experience isn’t enough to guarantee that the available position will be a good fit. We need to take into account the job seeker’s personality and priorities. ML can also do just that. By having the job seeker fill out a questionnaire, take a personality test, or complete problem-solving tests that incorporate gamification, the online job portal that uses ML gains valuable insight into how the job seeker thinks and what kind of company or position they are more likely to be successful at.

In a nutshell

In the U.S., there are millions more job openings than people looking for work. And the high hiring rate can barely keep up with the staggering rate of workers quitting their jobs. Thanks to advancements in ML, computers can analyze large sets of data to identify causalities and correlations that can help recruiters and job seekers find matches that are more likely to be successful both in the short and the long term.

Gergo Vari is founder and CEO of Lensa, Inc.