by James Rosone
Here we go, Dan thought.
Lou then opened a folder he had in front of him before continuing. “Prior to this meeting, I took some time to review your background and examine some of your recent work. I must say, your work is impeccable, and your academic credentials are impressive.”
Dan lifted his chin up a bit with pride at the compliment.
Lou continued, “When your family sent you abroad to receive your education, you left Shanghai to attend the University of Oxford’s department of computer science, where you graduated with top marks. You then went on to become a distinguished honor graduate in your master’s program in the same department.”
Lou eyed Dan for a moment before he added, “Your file says you turned down spots in PhD programs at both Oxford and MIT to pursue a PhD at Carnegie Mellon University in machine learning. Why did you make that decision?”
The question caught Dan off guard. No one had asked him that before. It was customary to get your PhD from a different university than your lower-level degrees. He also hadn’t told anyone he’d turned down the Oxford and MIT positions.
Dan replied, “It was a hard choice to make at the time. But I felt the artificial intelligence program at Carnegie was a better program. The person who would ultimately become my academic advisor was working on a special program for the American Department of Defense. What he shared with me about machine learning and what the future of AI would look like in ten years fascinated me. I wanted to learn from the best, and by all accounts, he was the best in the field, so I opted to pursue my PhD there, as opposed to MIT or Oxford.”
Grunting at the answer, Lou continued, “Your advisors and professors said you were by far one of the most gifted minds in the field of artificial intelligence they had seen. I hope you understand that was a big reason why we specifically recruited you to join our firm and my department. But we are also starting to face a problem with you that needs to be addressed.”
Crinkling his brow, Dan said, “A problem? What sort of problem? You said earlier my work is excellent.”
Lou waved off his concerns about work with his wrist. “This isn’t about your work performance, Dan. You are a gifted and exceptional employee. This is about your social credit score. Would you say you have struggled a bit socially since you have returned to China? You had, after all, been living abroad for nearly ten years. A lot has changed in that time.”
Dan tried not to chafe at the mention of his social credit score, which had taken a serious hit in the last four or five months. It was perhaps the most aggravating experience he’d had to deal with since he’d moved back to China—this recent hotel bookings denial being a case in point.
Dan sighed as he commented, “I apologize about my social credit score, Lou. I realize it has taken a hit the last few months. I will admit, I have been having some challenges readjusting to life in China. As you know, I lived in the UK and then the US for the last nine years. Things are obviously different there than they are here. I will try to do better.”
Lou appeared to be sympathetic, even if Joseph, Dan’s immediate supervisor, was not.
“Dan, the social credit score is becoming increasingly important in China. Right now, there is still some leeway in the system. There will come a time very soon when there will be no leeway. As it stands, your score is so low we cannot send you on a business trip for the company. As a matter of fact, with the new guidelines coming out in the near future, we would probably have to terminate you because you’d become blacklisted,” Lou explained calmly.
Lifting an eyebrow at this revelation, Dan immediately countered, “What? I’m sorry, Lou. What exactly have I done that would rise to that level? I’m the most knowledgeable person in the company when it comes to machine learning.”
Joseph then looked down at a folder Dan hadn’t noticed earlier, explaining in a softer tone than Dan had been expecting, “Dan, in the last ten months, you’ve received nineteen social infractions. Here are a few things that caused you to become labeled as a ‘problem.’ Three months ago, you made a reservation at a restaurant and then you failed to keep it.”
“Whoa, are you saying I could be blacklisted because I failed to keep a restaurant reservation? I tried to call them to cancel the reservation, but I had a problem with my cell phone,” Dan protested defensively.
Joseph nodded in agreement. “Yes, you were also three days late in paying your cell phone bill. They suspended your service until you paid it.” He held up a hand to forestall Dan saying anything else as he continued. “Two months ago, you left a derogatory comment on a computer science chat board about something utterly unimportant. Several people reported that comment and it was flagged for review.
“Later that same month, you were captured on a red-light camera in your Tesla, failing to yield to a traffic light. Three days ago, you received a citation for eating on the subway.”
Joseph then lowered the paper as he shook his head in disappointment. “These are only a few of the infractions you’ve received since you started here.” He paused for a second, letting some of that information sink in before he continued.
“Dan, none of these infractions is serious on its own. But when you add them all up, they become an issue. They become a pattern of behavior that needs to be addressed. That is why the social credit program was developed. Each infraction negatively affects your standing, until eventually, it flags you in the system. Once you are flagged, your social profile is reviewed. A reviewer then determines if you should be blacklisted until you are able to change your behavior.”
Dan sat there dumbfounded. He knew the social credit situation was starting to become a big deal, but he hadn’t paid much attention to it. His motto had been work hard and play harder—something he’d learned in America. Apparently, that wasn’t acceptable behavior here in China.
At this point, Lou stepped in. “Dan, we are telling you because we want to help you rebuild your social score so we can get you back on the right track.” Lou paused for a second before adding, “You are a brilliant man, and I want to have you work on a very special and sensitive machine learning program. But before I can do that, we need to help you break some bad habits you learned in America and rebuild your social score. To that end, I need you to understand this social credit system is serious, and very real. Second, if you understand that it functions like a game, it’ll be a lot easier for you to accept it and then find ways to manipulate it to your benefit.”
“Manipulate? How so?” asked Dan.
“OK, right now, your score is in the gutter. But there are easy ways for a person like yourself to get out. Unlike most people, you are paid a very healthy salary. If you begin to donate large sums of money to dozens of different charity organizations in Shanghai and around the country, those donations will be reported as positive social contributions. Doing this monthly will immediately boost your score. Second, if you donate blood once a month, it’ll be annotated as a positive contribution to society. Third, if you volunteer to assist certain organizations, those also get counted as positive engagements.”
“So, you’re saying I should start doing some of these things now and work on keeping my nose clean,” Dan replied.
Lou and Joseph both nodded.
Lou explained, “We have big plans for you, Dan. I want to get you back on the right track. Here is what we will do to help you. First, you will sit down with the same financial advisor both Joe and I use. He will get your financial house back in order. He’ll get your bills all set up on autopay and make sure you never get dinged for something as silly as a late payment.
“Second, we will arrange for you to teach a single class on computers and artificial intelligence at an underprivileged school. This will help you bring your score up as this school will give you a positive report each week. Third, we have identified eight charity organizations you will start providing money to on a monthly basis. This will generate eight additional positive interactions a month in addition to the teaching job.
&
nbsp; “If you follow this regimen we are outlining for you, we can get you removed from the “problem child” list within a couple of months. In three to four months, we’ll be able to get your score up to an acceptable level. In six months, you’ll be in the top tier like us. Then I can get you moved onto this new machine learning collaboration program. Is this something you are willing to do?” Lou asked.
Dan took in a deep breath. He’d honestly had no idea that his social score situation had warranted an actual intervention. The fact that they were this willing to help him, though, made him feel incredibly blessed to be working for Alibaba.
Dan looked the two of them in the eye. “I will. You have my word. I will do my best to stay clean and follow this plan.”
Joe and Lou both smiled, pleased that their intervention appeared successful.
*******
The BAT Laboratory
Shanghai, China
Standing before the most powerful men in China, Dr. Xi Zemin, the chief scientist for the BAT laboratory and arguably one of the world’s leading experts in machine learning, explained, “Mr. President, as an avid chess player, you understand the importance of being able to anticipate your opponent’s next move and the five or six moves after that. As Sun Tzu once said, ‘Do not engage an enemy more powerful than you. And if it is unavoidable, make sure you engage on your terms, not your enemy’s.’ I believe we have created a tool that can do just that.”
President Yao Jintao narrowed his eyes a bit as he countered, “That is a bold claim, Doctor. Please explain this a bit further.”
Xi nodded. “Mr. President, in the private sector, Alibaba, Amazon, Baidu, Netflix, and Google created a software algorithm that can anticipate, almost predict, consumer behavior. This has led to enormous economic growth and profitability in these organizations and within the broader economy. This is the power of machine learning.”
Xi continued, “When that technology is coupled with social media platforms, the economic growth and profitability become exponential. But, if we pair machine learning with a deep understanding of behavior analytics, we can create a weapon more powerful than any naval ship or stealth fighter our country or adversary can field.”
President Yao held a hand up. “Dr. Xi, perhaps you can explain this concept of machine learning you are talking about and how the government of China can leverage it to achieve our global ambitions.”
Smiling at the opportunity to expound further, Dr. Xi excitedly explained, “Yes, Mr. President. What we have been doing with machine learning in our social credit program is giving the algorithm access to the electronic data we have been collecting from the users on the system. By creating a series of what we call deep neural networks, which are computer programs that function and operate much like the human brain, we created an algorithm that learns more and more on its own. You see, our algorithm is in the process of learning how to automatically improve its assessments and understanding of human behavior through improved experiences.”
Before Dr. Xi could continue, someone from the Ministry of State Security interrupted to ask, “So what you are saying, Doctor, is that the machine you created is moving from the position of receiving and interpreting information to taking that information and making predictive analyses of future behavior based on past user actions.”
Xi excitedly nodded. “Essentially, yes. It’s kind of like learning how to ride a bicycle, if you will. In theory, a child or adult understands the concept of using their legs to push down on the pedals. They know if you do this, it will spin the wheels and that will in turn propel them forward. But once they sit on the bike and have to put that theory into practice, they now realize the importance of balance—a new concept they had not known, observed, or experienced when they watched other people riding a bicycle.
“You see, this concept of balance was something that could only be learned through experience. In the case of machine learning, we couldn’t preprogram certain functions. The algorithm needed to experience them in order to learn from them, much like a person does when they first learn how to ride a bicycle. At first, the person falls a few times. Each fall usually causes them to feel physical pain if they hurt themselves or emotional pain or embarrassment if they are laughed at. Those experiences are then integrated into the brain, which learns not to make those same mistakes as it tries again.
“This is what we call a neuro loop. The brain knows the basic functions, but now it is integrating new experiences into those basic functions, thus expanding them until the individual is able to ride the bike successfully. That is what we have been teaching our algorithm to do with the social credit system. It is absorbing all this information and then learning from it, identifying bad or corrupting behavior within society and then developing a system or process to be applied to the person to change the corrupt behavior into publicly acceptable behavior. This is an example of applied artificial intelligence.”
Interrupting him, this time one of the president’s advisors commented, “Let me know if I am understanding this correctly. The AI you created is using machine learning to become smarter, thus moving beyond the current tacit or narrow AI currently in use around the world? This is what allows your AI to better understand people and their behavior better, correct?”
Xi nodded. “Yes, that is correct. We built the basic algorithmic code—kind of like having a five- or six-year-old child. Now we need to allow it to learn so it can advance to the next level.”
The advisor responded, “You are implying that as your algorithm is fed more data, it will grow and learn. Its ability to observe human behavior will in time allow it to accurately identify ways to change or manipulate that behavior. If we provided the algorithm the desired outcome, it could design a series of actions to achieve those outcomes?”
Xi smiled at the advisor. He got it. “That is exactly what I am saying. The social credit system is the first step. What I am proposing we create is so much greater than this. Something that will revolutionize the world and humanity. If I may make another reference to Sun Tzu—he talks about the nine variations or nine contingencies one must plan for. I believe there is a tenth contingency he could not have been aware of during his day and age. That is machine learning and applied artificial intelligence.”
General Li Zuocheng, the head of the People’s Liberation Army, leaned forward in his chair. “Perhaps you can explain the military applications of this to me. How well could it develop war games against potential adversaries, and why would your AI be any more accurate than our human war planners? I ask not to suggest your AI does not have value, but a machine is not a human. It still does not fully understand human emotion or the irrationality of people.”
XI nodded. “That is a valid point, General. What we have done with the social credit program is given our machine a basic framework from which to understand people and human behavior. Let us look at it this way. The AI we want to build needs more than just a brain. It needs a knowledge base. It needs memories. We can create the smartest machine in the world, but if it doesn’t have memories of how humans reacted to situations or facts, it will never be able to really learn or anticipate future human behavior and thus how a person or adversary may ultimately respond.”
President Yao Jintao said, “This is why the social credit program was so fundamental to your AI project, isn’t it?”
Xi nodded but did not say anything right away.
The President continued, “But the social credit program will only take us so far. It will only understand what our people who live in our culture are able to think or how they react to things going on around them. How will you teach your machine to deal with the West if the only knowledge base you have to draw upon is our own society?”
A smile crept across Xi’s face. “Facebook’s founder, Mark Zuckerberg, realized that the real value of his company was its user data. Knowing what their users watched, shared, and commented on allowed them to create incredibly effective marketing products and services they could then
sell or allow users to promote and sell their own products across their platform. Mr. Zuckerberg also realized this data would only grow in value as more people joined his platform. Knowing this, he set out on a mission to bring free Wi-Fi to the world.
“Similarly, SpaceX’s Elon Musk wants to colonize Mars. However, space exploration is incredibly expensive. One way he’s working to solve this cost factor is to develop Starlink, a constellation of satellites that will provide high-speed internet to the world for a price. While his service isn’t free, he’s picking up where Zuckerberg left off. If we offer up DragonLink to the world for free, we will be able to monitor all the traffic that flows across it. That user information can then be fed directly into our servers, giving our AI even more data from which to learn.”
One of the President’s advisors commented, “You are talking about an incredibly expensive endeavor, Doctor. Do we even have the technology capable of doing all of this yet?”
“On our end, yes. We have most of what we need. What I need from the government is a large, secured place for us to build this super-AI and then the resources to build the brain for the machine,” Xi explained.
No one talked for a moment as all eyes drifted to the President. Ultimately, he would be the one to decide if the government would go all in on this.
President Yao steepled his fingers. “Sun Tzu also said, ‘The skillful employer of men will employ the wise man, the brave man, the covetous man, and the stupid man,’” he told Xi. “Which one of these categories you will ultimately fall into, we shall see over time. For the moment, you have convinced me of the value your proposal could bring to China. I will grant you the resources needed to build this machine under one condition. This AI you are creating—it needs to be designed with one and only one intent.”