The engineer who pioneered speech recognition now wants to use AI to track your toaster. Meet Sense, the buzzy startup that's raised $50 million to monitor every appliance in your home.

Benji Jones,Benji Jones

Feb 20, 2020, 23:19 IST

The engineer who pioneered speech recognition now wants to use AI to track your toaster. Meet Sense, the buzzy startup that's raised $50 million to monitor every appliance in your home.

Every electronic device in your home sucks down power in a slightly different and discernible way.
A pioneer in speech recognition, Mike Phillips cofounded Sense, a buzzy startup that makes a device that's able to tell those different electrical signals apart.
The new technology - which relies on machine learning, much like voice recognition - could save people money. It also gives them an awareness of what's going on in their homes, such as whether the coffee maker was left on.
Sense has raised about $50 million from big-name backers including Schneider Electric, Shell, and Energy Impact Partners.
Click here for more BI Prime stories.

You can't hear it, but there's a sort of chatter happening all around you, all the time, in the language of electricity.

Every electric appliance in your home - whether it's a washing machine or a lightbulb - draws energy from the grid in a slightly different way.

An incandescent light bulb, for example, pulls in a chunk of power when it turns on as the filament heats up, before stabilizing. A refrigerator, on the other hand, sucks down power in cycles.

Mike Phillips, a pioneer in speech recognition software, is betting there's value in measuring those signals - in interpreting that language.

Click here to subscribe to Power Line, Business Insider's weekly clean-energy newsletter.

In 2013, Phillips cofounded the startup Sense, which makes a device that uses artificial intelligence to tell you how much electricity each appliance in your home is using in real-time.

"It's kind of like doing speech recognition except with 30 people all talking at once," Phillips, who's also the company's CEO, said.

Sense has raised $50 million from big-name backers

The tech can save people money on their utility bills, he says. Plus, it gives users full awareness of what's happening inside their homes, such as if they've left the oven or coffee maker on.

Now, investors are piling on. The startup has raised about $50 million, from big backers including Schneider Electric, Shell, and Energy Impact Partners.

"If you know when your garage door is opening, if you know when your oven is on, that provides an unprecedented level of intelligence that never existed before in the home," Sameer Reddy, a partner at Energy Impact Partners, told Business Insider last year. "This level of home intelligence will become the gold standard within every new home."

From listening to people to listening to … toasters?

Imagine Apple's Siri, but 30 years ago. That might be similar to the technologies Phillips was working on at MIT in the late 80s and early 90s.

As a member of the Spoken Language Systems group, he helped develop technology that allowed users to perform basic computer tasks through speech, such as finding nearby restaurants.

"Instead of running on a mobile phone, we were running these systems on computers the size of refrigerators, and each spoken response took 20 minutes for the computer to recognize!" he wrote on LinkedIn.

In 1994, Phillips cofounded SpeechWorks - one of the first speech recognition companies in the world, which initially made software for call centers, he writes. Later, he cofounded Vlingo, which he says "created the first virtual assistants on mobile phones."

Both companies were eventually acquired by what is now Nuance Communications, leaving Phillips looking for another venture.

A speech-recognition expert wants to take on climate change

Phillips was concerned about climate change and wanted to apply his experience with consumer-facing products in another industry. Why not energy?

It was a good fit. Phillips knew consumers wasted energy in their homes. And he also knew the challenge of disaggregating a messy input of data into differentiated signals using machine learning.

"We take electrical signals and apply similar sort of machine learning on those electrical signals to figure out what your toaster is doing and what your microwave is doing," he said.

So how does Sense's technology actually work?

Sense samples current and voltage 'thousands of times per second'

The core offering of Sense is a $299 piece of hardware that fits inside a home's electric panel. Like a meter, it measures how much energy is flowing into your home - "but in a very detailed way," Phillips said.

"Electric meters send data to the utility every 15 minutes," he said. "We're sampling thousands of times per second."

Sense's software looks for finite patterns in your energy usage, which it then tries to match with a reference database, filled with the electrical signatures of various appliances. It's not unlike how 23andMe matches your genetic code to DNA in its database.

In an average household, Sense's technology detects about 12 appliances in the first month after its installed, the company says. After a year, it could recognize as many as 30.

"We're making use of the fact that different things in your home use power in slightly different ways," Phillips said.

Sense relays all that data to an app, allowing users to see how, exactly, they're using power in their homes, moment to moment.

One benefit of that added awareness is cost savings, Phillips says. Users can see which of their appliances are drawing down the most power and they can set goals for their monthly energy usage.

The app also shows how much energy idle appliances - such as microwaves that are plugged in but not running - are drawing down. There's not much data on how much energy they waste, but one 2015 study by the Natural Resources Defense Council suggests it's about a quarter of monthly usage.

Sense can tell you if you left your oven on, or if your garage door just opened

There's another benefit that Sense is now marketing: Home intelligence. Because garages, ovens, and even doors are increasingly electric, Sense can tell you when someone enters your home or if you left the oven on after you left.

"Having this detailed, real-time view about what's going on in your home is what's driving consumer engagement," he said. "Even though people care about energy, it's still a little bit boring for people."

Sense is also starting to focus on the repair and maintenance industry. Phillips says the technology could help homeowners figure out when an appliance, such as an AC unit, needs to be repaired, based on its energy signal.

In the years to come, the company will also be working on automation, he says, that would allow customers to draw energy from the grid when it's cheaper - or cleaner, as different kinds of energy are on the grid at different times of the day.

How big are the benefits, really?

Sense can save you money - but it's unclear how much. One recent pilot project involving the utility Alliant Energy and 100 households found that Sense could save customers up to $90 a year. The startup says anecdotally customers have saved much more.

As for accuracy, there seems to be some room for improvement, but customers appear satisfied overall, according to Amazon reviews. At the time of publication, the product has 4.2 stars based on 918 reviews.

"Cool device and very accurate at detecting overall energy usage," one customer wrote. "But individual device detection is hit and miss at best."

Sense says that as more and more customers sign up, the device will become more accurate.