LG: In-home AI will be decades away
LG's Dr Peter Pyungchul Kim talks about LG's AI initiatives, from how it can make existing appliances better over time to how to make people laugh at an air conditioner, as well as the future of AI for consumers.
At this year's CES, LG Electronics went all-in on its plans for AI across its product lines, showing off a range of TVs, appliances and (rather unfortunately for LG) a home robot that wouldn't respond to on-stage queries.
I recently had the chance to travel to Seoul to interview Dr Peter Pyungchul Kim, LG's Senior Vice President & Leader of AI around how its work to make appliances that aren't just better, but also notably smarter is progressing.
Outside of maybe a mobile phone, most of us don't update our home appliances on a yearly basis, which presents something of a challenge for LG if it wants to continue AI improvements year on year. However, if you're keeping a fridge or washing machine with ThinQ enabled, you should see that device get better over time, according to Dr Kim.
We want to change that, so that after you purchase it, and as you use it, it improves."
"One of the initiatives that we have with ThinQ is very unique to the appliance industry. Until now, when a customer purchases an appliance, the product itself starts to degrade," he said.
"We want to change that, so that after you purchase it, and as you use it, it improves. The value of the product actually goes up. That's something you already see from the service industry, and it's something we want to bring to our products. We call it asset evolution, and it's one of the three pillars we develop on."
That doesn't quite mean that your fridge will fix a busted coil for instance.
"Yes, there are mechanical parts that will degrade, but the value of the product doesn't just have to be the mechanical parts."
"For example, if you purchased a washing machine, it uses a certain amount of energy per cycle, or a certain amount of time to wash everything. But as you go on, it learns and improves so that the amount of energy and the time will reduce. That's what we want to deliver. Our ultimate goal is to have that to be achieved."
This isn't a small task, and it's one that LG's been working on for some time.
"In order to do that, we had to change some of the specifics of the product itself," he said.
"For example, right now, every appliance we make right now is Wi-Fi connected. That's a hardware change we had to make. A lot of other sensors are coming in as well so that this whole evolution could happen."
AI may well learn over time, but there's also the issue of delivering new software updates to deal with features, bugs and, of course, security issues.
"Right now, there are many areas to consider for updating consumer electronics. You have the hardware itself in the home, and other connected appliances there. You have a connection between them, and then you have the whole AI intelligence in the cloud itself. So we've put in security initiatives for all of these areas, and when we find there needs to be a feature or security update, we would be relying on a software update."
when security is broken, what we need to make sure is that privacy is protected."
LG's AI is cloud-based, which gives it a degree of control around the security of its machine learning, but there's a natural concern around the security of what are iOT devices in the home once our washing machines and fridges get smart.
"There are two distinctive issues here. One is the privacy issue, because when security is broken, what we need to make sure is that privacy is protected."
"There's a whole initiative we have that makes sure that privacy is protected. We have a compliance team that directly reports to the CMO. We're aware of the issues around GDPR compliance, there's a whole compliance team that's making sure that we're getting only necessary data from the customer."
"The other initiative we have is around security itself. It's essentially our technology platform. That's split across three areas."
"One is on the device, then there's the communication, and the other is in the cloud. We have practices that are applied to all three areas. The common practice we use, we call it LG Secure Development Lifecycle. It's our own way of making sure that every bit of code we develop is secure. It's to ensure that the code we're generating is actually secure."
LG has stated that it's working with open platforms for AI and sharing information with its partners. As an example, it has devices that operate using Amazon's Alexa client, while others use Google Assistant and in some cases, it's both, but it has also got its own AI initiatives under the broad ThinQ branding. Though that's not something that LG's looking to share out or licence to third-party hardware makers.
"We haven't thought about it. Although ThinQ itself is an open platform. With ThinQ we are collaborating with many partners."
"When we talk about open, we're talking about three different things. One is connectivity. We don't want to run into a situation where a customer purchases a lot of products from different manufacturers but they won't talk to each other. We want to address that through open connectivity. One of the initiatives we're participating in is the OCF; an industry-wide forum that's working towards an interoperable standard that covers all the equipment that a consumer might buy."
"Another is open platform. Because there's a lot of players outside who deliver other values than we do. Service values, search, all kinds of different kinds of value. As we go on, customers will expect more value than what we used to deliver through appliances, which are very specific to a task. But now people want to ask 'what's the value to your air conditioner', for example. There are a lot of needs we need to deliver that we're not at with our business. Those areas are where we're working with outside partners to deliver, because our customers want to have that."
"Open platform is the initiative we're working on with partners to deliver their value to our platform. Open partnership is in line with the open platform. Sometimes it's not a platform itself; it's just a service, so we are opening partnership with others. Those three different open areas are there so we can address customer needs."
Developing AI alongside iterating on existing products makes for a somewhat uneven development cycle, and one that Dr Kim says still has a long way to go.
"Predictive analytics is the most mature area of AI. It's been around under different names, such as data mining for around two decades now. So in terms of AI, say, it's pretty much ready for prime time for features such as energy savings."
"But there's a lot of sensors, and work to do with connectivity of those sensors to each other and the cloud to be developed. It's not there yet, and it's crucial to have it. Even for the sensors, we don't have in the market a smart sensor that tallies the exact amount of energy being used by a single washing machine in this moment. But there are a lot of sensors that can tell the amount over time, what the trend is."
"But to individually optimise the performance in the moment, it's got to tell that an appliance is on, and all the specifics around that. We don't have that, and there's lots of sensors to work on."
Prediction presumes appliances that will eventually know what we want to do before we want to do it. But still, predictions can be wrong, and there's work to be done there too.
"I don't think there's a clear measurement that we can tell whether this prediction was OK or out of bounds. When it comes to content prediction, people think that 80-90% is excellent. You never expected it, but it came up with what you like, right?"
"But if you're talking about, right now, our robot vacuum cleaner, it collects data within the machine about how the customer has been using it. What time of day, for what reason and for how long.
The customer goes on vacation, and the customer could ask the vacuum cleaner 'can you do the same that I used to do?' In that sense, it has to be pretty accurate. It's prediction based on prior history, and it has to be pretty accurate. It shouldn't be something out of range."
Buy the LG G7 ThinQ from Amazon AU
With the Qualcomm Snapdragon 845, dual wide angle lenses and AI to make your photos pop, the LG G7 ThinQ is LG's greatest smartphone yet.View details
Voice augments the experience, by making it more human."
Dr Kim is adamant that while there's plenty of AI work to do, the overall trend is positive for both new devices and how they'll improve over time.
"I expect and believe that AI will be very positive to the product as well as the service. In terms of product, there are 3 aspects of the AI that haven't been delivered yet, and we're working on where the industry will catch up."
"Firstly there's the user experience. Right now, you've got a device that you have to interact with. For human dignity it may be better if the machine can interact with me in a more human way. That's within the user experience area, and that's why we use a lot of voice agents for appliances. That's one area, even though we always have a backup plan, through touch and buttons. Voice augments the experience, by making it more human.
"The other aspect is that so far all appliances, they're very good at specific tasks. Washing machines can do your laundry. Air conditioners cool the air. But the customer's value is at a higher level, which is, let's say... the outside temperature is at one level, but as a customer you may be more interested in safety, convenience, energy usage, security and care. Those are some of the values you want to achieve. But appliances right now don't do that. That's where AI can fill the gap between those higher values and the specific tasks that the appliance can do. "
As far as the customer is concerned, it doesn't appear to be AI -- but it got better!"
"If you're talking about a washing machine, for example, for every cycle it has to drain the water, and then spin it, so it can drain all the water out before it starts the next cycle. There's a whole balancing act, and it's taken us more than 3 decades to develop it. I don't think it's perfect yet, it's still going on. But there are tons of AI disciplines we can bring in to improve and optimise the old technology – the controls and the mechanics. As far as the customer is concerned, it doesn't appear to be AI – but it got better!"
With ThinQ working in the cloud, it opens up concerns about data collection, but also possibilities for LG to deliver more creative updates to devices – alerting an air conditioner if there's a heatwave that may be working a little harder might be wise. That's not an area that LG's looking to get into right away, according to Dr Kim.
"Right now, we are collecting a lot of data from appliances. Mostly in the Korean market. But that's not good enough for us to come up with more creative uses. The first one is mostly around voice. Voice commands are very specific. In terms of intentions, filling that higher value gap, it's not in the data we're collecting."
In any case, LG's contention is that the most valuable data for machine-AI-control comes from its own R&D testing, rather than user data.
"That data isn't coming from customers, but mostly from the factory. When we build a product, and when we test it, we collect all the test data. Out of that, all the traditional mechanical engineering takes place, but we can bring the same data set to our AI optimisation. The amount of data we need there is completely different to other services."
LG's primarily using voice control technologies, sometimes in partnership with other companies, but it's not looking at distinct voice identification as a clear AI goal.
"We don't envision that we will recognise who the speaker is, but we have to recognise whoever is speaking. If we trained the AI engine so that it can recognise a certain gender, or certain voice, then we wouldn't be able to address every diverse situation. So we try to make it as broad as possible."
LG's own experience in its core home market bears witness to how important this is.
"For example, in Korea there are lot of different dialects and accents. So we brought the eastern part of the accent within our air conditioners, and people laughed at it!
"Because when we tested it, the standard accent, which is around Seoul, they liked it, but when we showed it with a different accent, the voice recognition rate was pretty bad. We haven't trained on their voice, so we did."
"It's a little bit different to recognise a single voice. That's voice identification. Voice identification can be useful. Maybe not for appliances, but maybe in the car, for instance. When you purchase a car, you might expect the settings to work individually for you. So maybe you'd expect the voice agent to recognise who is driving, and adjust accordingly."
"When we have a plan to deliver technology to a certain area, or a specific scenario, we have to respect our partners who specialise in this. Even for voice recognition we don't have any scenario where we need to identify users. The only thing we do is recognise what is said."
LG's current approach is cloud-based, and while he'd like to see that shift over time, it'll be a long time before we see true in-home AI, according to Dr Kim.
"There's a huge potential from AI, as well as limitations of AI. The current fundamental algorithm requires a huge amount of data, and a huge quantity of computing power that most customers can't afford. If they're going to have AI on a device, the current algorithm has a huge footprint.
For a smartphone, that's maybe OK. Autonomous driving, robot, technically, that might be OK."
"But when we're talking about iOT devices, appliances, that's not a level of computing power. The algorithm is not there yet where I don't need to send all the data to the cloud to train it, or the voice interaction to the cloud so they can understand. That technology is not there. One assumption that I would avoid as a customer are those limitations. AI cannot work without this huge data set, worked on in the cloud."
"I would expect that it would be decades before we'd see all that data worked on within a premises. But that's the ultimate target we're going for."
Alex Kidman travelled to Seoul as a guest of LG Electronics