In 2008 Thaler and Sunstein published their book, Nudge, advocating for a more nuanced approach to changing public behavior. Since then nudge theory has been quite popular but hasn’t created the revolution optimists had hoped.

Here is the core problem: people do not always act in their own best interest. Sometimes this affects only them, but often the negative impacts affect the people around them, their family, and even society as a whole. An obvious example is vaccinations.

There are many less-obvious examples, however. Poor health care decisions increase the cost of health care, which is a rapidly increasing burden on society. Poor financial decisions can leave people in debt, might cause them to default on those debts, and have an overall negative impact on the economy. We all share risks through insurance premiums and public costs.

And, we actually care about people. We are a social species and we do generally have empathy for others (unless they have been psychologically relegated to an out-group). It is also some people’s job to care about people.

Therefore, for various reasons, there are individuals and groups who care about changing other people’s behavior for their own good and for the good of society. This paternalism runs up against several obstacles.

The first is liberty (which, make no mistake, is a good thing). Consenting adults have the right to make poor decisions, and in a free and open society we recognize that we should tread lightly on the liberty of others. Any time society forces people into good behavior, the benefits have to be carefully weighed against the loss of liberty. Speed limits, seat belt laws, and helmet laws are examples of situations in which we favor safety over liberty (and even then such laws have their vocal critics).

The second obstacle is decision-fatigue. We live in an increasingly complex society, which means the average person is faced with an array of complex decisions on a regular basis. We all can’t be experts in health care, finances, car maintenance, food safety, nutrition, pyramid schemes and other frauds, travel safety, computer and identity threats, etc.

With the internet and mass media we are faced with a dizzying amount of information. Good information is mixed with bad. A lot of the information is just lazy and low quality, and a lot is ideologically biased. People, therefore, will tend to slip into simplistic narratives that give the illusion of simplifying their decision-burden.

So, any public information campaign, even with the best intentions, is just adding to this burden, and is competing with a lot of other sources of information, mostly misinformation.

The third obstacle is that it is just difficult to change people’s behavior. Behavior is the result of innate personality traits combined with a lifetime of habits and a complex web of cultural influences. There is no simple way to alter the resulting behavior.

The default method used by doctors, public service announcements, and advisers is the so-called “scared straight” method. Essentially this assumes that people are rational actors and if we just give them information they will make better-informed, and therefore better, decisions. Another way to look at this approach is that it assumes the problem is a deficit of information, and therefore filling the deficit is the solution.

As I discussed above, however, the problem is just as likely to be an excess of information, and psychological barriers that have nothing to do with information. We want people to say, “Really? Smoking causes cancer!” and then quit. That happens sometimes, but rarely.

Newer approaches include social norming or motivational interviewing. These use psychological pressures to force good behavior. Social norming is an approach public service announcements – instead of scaring people with statistics, tell them that their peers are engaging in good behavior. This seeks to leverage peer pressure toward good behavior.

Motivational interviewing is similar, but on an individual level. You ask people what their goals are and then discuss how to achieve their goals. If they do not engage in the recommended behavior, then they look inconsistent, and people tend to avoid this because it is socially embarrassing.

These methods are more effective than the scared straight approach, but not by much. It is enough to make them worthwhile, but still not what we would like.

This is where the Nudge comes in. Nudging, which some people call libertarian paternalism, seeks to take a minimalist approach that preserves free choice as much as possible. Nudging sometimes involves psychological manipulation, like social norming. It also makes use of default behavior.

For example, instead of requiring employees to opt in to a retirement plan, they are signed up by default and then have to opt out. They still have the free choice, but the default decision is the one deemed in their best interest.

Such nudging can take many forms. My cafeteria at work now offers you vegetables (like carrot sticks) as an accompaniment to your sandwich, rather than potato chips. You can still buy the potato chips if you want, but there isn’t someone offering to put a big helping onto your plate. Fast food restaurants can have their meal packages come with apple slices instead of fries. You could still buy the fries a-la-cart, but they aren’t offered to you, nor are you encouraged to “supersize”.

Nudging works, in that it does have an immediate effect on behavior. After a decade of further research, however, the limits of nudging are becoming more clear. In a recent review of the topic for Science News they discuss the many unintended consequences of nudging.

For example, reminding donors who gave to a charity in a previous year to donate again does result in a pulse of donations, but also results in a pulse of people unsubscribing from the e-mail list.

Signing up for a retirement plan by default does get more people to sign up, but sometimes they are defaulted into a plan that is not optimal for their situation, causing later regret.

Many of these issues are solvable with more sophisticated and evidence-based campaigns. Still, the data shows that nudging typically has short term behavioral changes only. It is no surprise that you cannot have a dramatic effect on people’s behavior which is based on years of habit, culture, and innate tendencies.

It is also perhaps not surprising that marketers are way ahead of the game of changing people’s behavior. While psychologists and policy makers are struggling to make small positive changes to behavior, the marketing and advertising worlds have already worked much of this out. The placement of products on store shelves, for example, is all about nudging. Commercials employ tactics of social norming and motivational techniques.

Marketing campaigns are about creating a culture in which their product is viewed as having a health-halo, or being part of a cool or rugged identity. Have you noticed that many commercials are very light on information but heavy on selling an image or an idea? The marketplace has already worked out how to influence people’s behavior – at least their buying behavior.

Entire movements and industries are based upon this. The organic food industry is a giant scam, for example, but they have manufactured a health halo out of misinformation, demonizing their competitors, stoking fears, and playing off the appeal to nature. This has worked.

Marketing campaigns, however, are viewed with special suspicion when they come from the government (perhaps they should be), or when we are being told that something is for our own good. Even if we try to adopt the knowledge of the marketing industry, getting people to engage in behavior that is in their own interest still hits psychological barriers.

There is no easy solution to this. Life is complex, and no one method will replace the effects of taking a knowledgeable and active role in taking care of every aspect of your life. There is no shortcut.

Policies to improve behavior, therefore, have to be equally complex. They need to consider human psychology, culture, unintended consequences, decision-fatigue, and many other factors. The most effective methods seems to be to change the culture itself, but this is incredibly hard and takes a long time. Any such efforts are also competing with many others who want to influence the culture to their own selfish or ideological ends.

As individuals we need to also take a complex approach. This means picking our battles, being sensitive to diminishing returns, finding good sources of information, and questioning our own beliefs.