Society has long tried to find ways to get us to do what we are told. From religious commandments to human rights. We have almost always had rules to guide our social lives. Now big data and technology can go one better.
In the novel 1984, society was controlled by a watchful controlling force called Big Brother to ensure everyone behaved as they were ordered. It was supposed to ensure harmony but created the opposite.
We live in a world with more surveillance cameras than ever before and rather than just monitoring a location; the cameras are now able to recognise and track us. It is not only the cameras, but your phone is also quietly tracking you everywhere you go. On the one hand, it helps you navigate, but it is also likely sharing your location with others many you don’t know who are either using it to target you with ads or determine who you are.
So when you hear China is testing a programme to give its citizens a social score based on their actions you would be forgiven for believing it has gone too far. More on that later, first consider how much you are being controlled already.
China wanted to find a way to reward positive actions and penalise actions that were not. Settling or not settling debt are actions that will earn a reward or get you in trouble. Not so strange given that credit agencies do that already, although they tend to only punish you for not settling the debt.
When you go shopping, you have a range of loyalty cards trying to get you to do certain things for a reward; especially to buy more even if you do get a discount. Some health insurance providers will reward behaviour that is considered healthy. Nothing wrong with that although there is always another level to attain and the rewards get harder to achieve over time.
So what is China looking to do and how bad is it?
The experiment which began in 2014 and is planned to roll out to the entire country by 2020 gathers data about many of your activities. Some are considered positive and would increase your initial social score of 600, sustained positive actions could see the number rise to over 900. Engaging in activities like getting a bad rating for service or spending extended time playing games may see the score drop to 300.
The score alone is an honour badge that those with high scores would be happy to share publicly. A low score would have the opposite effect and so push those individuals to stop doing things that reduce the score and do more things that will increase it.
The system was tested among a range of private companies and the Chinese State to determine how and what would serve as the best indicators. Efforts to reduce fraud, dishonesty and not settling debt are the key factors and while the system is primarily to encourage and reward good behaviour with low scores incurring no penalties, some actions which at the moment relate to poor debt repayment will see your ability to travel curtailed. Downgraded citizens may be prevented from booking high-speed rail tickets or airline seats. To regain the ability, you need to settle the debt and maintain a good score for some time.
In that respect, it is just like a traffic cop that by their presence on the road will reduce the chance you would break the law. Should you do so anyway, they will pull you over and, depending on what you did, issue a fine or arrest you.
South Africa is testing a system used in many other countries that see traffic infringements get the offender a fine and some points docked on their licence. Once a set number of points are docked during a year; your licence could be suspended. It is a labour intensive process though.
Technology could streamline it. Let’s assume all mobile phones are required to download an app to store your license which is issued by your municipality or the transport department. The app registers when you are moving and what the local road rules are for speed. Should the phone detect you are speeding, it could warn you to slow down. Fail to do so, and it could log the infringement, when and where it happened and how fast you were going. Do that too many times, and the app adds the docked point to your license. No cops needed.
This is a long-standing desire for tech companies to help people do more or discourage them from doing wrong using technology with the ability to scale and remove the need for human intervention. However, when the system no longer needs humans you can’t explain why you were speeding perhaps to escape a dangerous situation or for a medical emergency.
The intentions may be good, but systems like this almost always have unintended consequences or can be gamed by those who see an opportunity.
Here are a few examples to illustrate this. A US retail store wanted to reduce the time customers spent in queues. They offered a reward to cashiers to check customers' items faster to speed up the checkout process, and it did reduce the waiting times. However, because the focus was on speed when items did not scan the first time, cashiers would give the item away. The customer would not complain, the checkout speeds were good, but the stock losses began to climb. It was not easy to determine who, when and how items were being lost because employees and customers had an incentive to not disclose the issue with items that did not scan. Good idea, bad outcome.
Economist Stephen Levitt thought he could potty train his daughter with an incentive in the form of sweets. Initially, it worked just as he hoped. Whenever she used the potty he would give her some sweets. Three days later his plan ran into trouble. Rather than only using the potty when needed, his daughter would use it repeatedly. Not only had she learnt to use a potty, she had learned to control her bladder and her economist father.
The final warning against trying to create such a system without considering all the potential pitfalls is from the Netflix series Black Mirror. The episode called Nosedive paints the picture of what happens when we create unintended consequences to artificially get people to treat each other in a way to get a good rating. Capetonians may recognise that the episode was shot in Cape Town.
China will continue to test their system, and for all the potential good, you can expect to hear some horror stories too. But then they are not alone in trying to find a way to get us to behave better which in some instances will work and others not. These are powerful tools that will appeal to authoritarian governments keen to sustain their rule. The challenge is for us to keep our own watchful eye on the developments and only agree to implement options once all the safeguards and appeal options have been put in place.
Enjoy The Money Show, but miss it sometimes?
Get the best bits emailed to you daily, right after it ends:
Recommendedby NEWSROOM AI
The World is both safer and more dangerous than it has ever been and these weapons could end human life on the planet.
Stem cells and 3D printing may give hope to those tearing their hair out trying to save theirs.
The World Wide Web is 25 years old. What was supposed to connect us has become more like a trap and we all helped it happen.
Technology is becoming the driver of human evolution. It took another step forward this month.
Solar activity could result in a rare and improbable event which could send civilisation back to the dark ages.
The likelihood of seeing a new Alphabet or Alibaba in the next decade is declining.
Only when consumer journalist Wendy Knowler chased Standard Bank did they admit wrongdoing that cost a client R41 000.
Plass senior researcher Dr Farai Mtero explains what the law says about evictions.
Association of Rural Advancement director Laurel Oettle praises ruling by ConCourt to uphold move to appoint a special master.
The inspiring South African choirU2's Beautiful Day at America's Got Talent quarter-finals.
Bruce Whitfield interviews Anthea Gardner, author of "Make Your Money Work for You: Think Big, Start Small".
Magda Wierzycka – Sygnia’s renegade CEO – on her attitude toward money (hopes and fears, successes and failures, etc.).
The reason for the leakage has been revealed, explains Table Mountain Aerial Cableway MD Wahida Parker.
W Cape Traffic Chief Kenny Africa says private security working with the department is helping curb the stonings on the highway.
Gift of the Givers fears two South African men who went missing in Vietnam may have fallen prey to an organ harvesting syndicate.