Society has long tried to find ways to get us to do what we are told. From religious commandments to human rights. We have almost always had rules to guide our social lives. Now big data and technology can go one better.
In the novel 1984, society was controlled by a watchful controlling force called Big Brother to ensure everyone behaved as they were ordered. It was supposed to ensure harmony but created the opposite.
We live in a world with more surveillance cameras than ever before and rather than just monitoring a location; the cameras are now able to recognise and track us. It is not only the cameras, but your phone is also quietly tracking you everywhere you go. On the one hand, it helps you navigate, but it is also likely sharing your location with others many you don’t know who are either using it to target you with ads or determine who you are.
So when you hear China is testing a programme to give its citizens a social score based on their actions you would be forgiven for believing it has gone too far. More on that later, first consider how much you are being controlled already.
China wanted to find a way to reward positive actions and penalise actions that were not. Settling or not settling debt are actions that will earn a reward or get you in trouble. Not so strange given that credit agencies do that already, although they tend to only punish you for not settling the debt.
When you go shopping, you have a range of loyalty cards trying to get you to do certain things for a reward; especially to buy more even if you do get a discount. Some health insurance providers will reward behaviour that is considered healthy. Nothing wrong with that although there is always another level to attain and the rewards get harder to achieve over time.
So what is China looking to do and how bad is it?
The experiment which began in 2014 and is planned to roll out to the entire country by 2020 gathers data about many of your activities. Some are considered positive and would increase your initial social score of 600, sustained positive actions could see the number rise to over 900. Engaging in activities like getting a bad rating for service or spending extended time playing games may see the score drop to 300.
The score alone is an honour badge that those with high scores would be happy to share publicly. A low score would have the opposite effect and so push those individuals to stop doing things that reduce the score and do more things that will increase it.
The system was tested among a range of private companies and the Chinese State to determine how and what would serve as the best indicators. Efforts to reduce fraud, dishonesty and not settling debt are the key factors and while the system is primarily to encourage and reward good behaviour with low scores incurring no penalties, some actions which at the moment relate to poor debt repayment will see your ability to travel curtailed. Downgraded citizens may be prevented from booking high-speed rail tickets or airline seats. To regain the ability, you need to settle the debt and maintain a good score for some time.
In that respect, it is just like a traffic cop that by their presence on the road will reduce the chance you would break the law. Should you do so anyway, they will pull you over and, depending on what you did, issue a fine or arrest you.
South Africa is testing a system used in many other countries that see traffic infringements get the offender a fine and some points docked on their licence. Once a set number of points are docked during a year; your licence could be suspended. It is a labour intensive process though.
Technology could streamline it. Let’s assume all mobile phones are required to download an app to store your license which is issued by your municipality or the transport department. The app registers when you are moving and what the local road rules are for speed. Should the phone detect you are speeding, it could warn you to slow down. Fail to do so, and it could log the infringement, when and where it happened and how fast you were going. Do that too many times, and the app adds the docked point to your license. No cops needed.
This is a long-standing desire for tech companies to help people do more or discourage them from doing wrong using technology with the ability to scale and remove the need for human intervention. However, when the system no longer needs humans you can’t explain why you were speeding perhaps to escape a dangerous situation or for a medical emergency.
The intentions may be good, but systems like this almost always have unintended consequences or can be gamed by those who see an opportunity.
Here are a few examples to illustrate this. A US retail store wanted to reduce the time customers spent in queues. They offered a reward to cashiers to check customers' items faster to speed up the checkout process, and it did reduce the waiting times. However, because the focus was on speed when items did not scan the first time, cashiers would give the item away. The customer would not complain, the checkout speeds were good, but the stock losses began to climb. It was not easy to determine who, when and how items were being lost because employees and customers had an incentive to not disclose the issue with items that did not scan. Good idea, bad outcome.
Economist Stephen Levitt thought he could potty train his daughter with an incentive in the form of sweets. Initially, it worked just as he hoped. Whenever she used the potty he would give her some sweets. Three days later his plan ran into trouble. Rather than only using the potty when needed, his daughter would use it repeatedly. Not only had she learnt to use a potty, she had learned to control her bladder and her economist father.
The final warning against trying to create such a system without considering all the potential pitfalls is from the Netflix series Black Mirror. The episode called Nosedive paints the picture of what happens when we create unintended consequences to artificially get people to treat each other in a way to get a good rating. Capetonians may recognise that the episode was shot in Cape Town.
China will continue to test their system, and for all the potential good, you can expect to hear some horror stories too. But then they are not alone in trying to find a way to get us to behave better which in some instances will work and others not. These are powerful tools that will appeal to authoritarian governments keen to sustain their rule. The challenge is for us to keep our own watchful eye on the developments and only agree to implement options once all the safeguards and appeal options have been put in place.
Enjoy The Money Show, but miss it sometimes?
Get the best bits emailed to you daily, right after it ends:
Recommendedby NEWSROOM AI
For the last decade, the letter has tracked how the world has been improving.
Before Robocops we will need to deal with Robocallers.
New digital products intended to solve problems can also create new ones.
Plastic can be found everywhere on the planet, often as rubbish. How can we get rid of it?
Eskom is in big trouble, but not as much as the Pacific Gas and Electric Company.
A look back at past predictions and what it says about what to expect in 2019.
Economist Jee-A van der Linde looks at the impact cannabis export will have on South Africa's dwindling economy.
Naked Scientist Chris Smith explains.
Khabazela shares tweets and Facebook posts that have gone viral.
The Money Show’s Bruce Whitfield interviews Ian Moir, CEO at Woolworths.
A woman was attacked on her way to work earlier this month.
The Panorama, Welgelegen and Plattekloof Neighbourhood Watch has identified a new house break-in trend.