Mystery Apple car suggests 3D street view is coming to Apple Maps

Apple is currently driving a tricked-out, super-data-sucking car all over the world, but no one really knows why.

Details about where in the world to find the Apple Maps car are here, but beyond promising that the data will be used to improve Apple Maps (which many people believe needs improving), there’s no information about what Apple is planning or what these cars can do.

Now that we’ve spotted one of these white giants on the road, thanks to the eagle eye of our own Sam Sheffer, we have questions.

This Tweet is currently unavailable. It might be loading or has been removed.

See all that stuff on the top of the car? What’s all that equipment for?

It occurs to me that the word "data" is super fuzzy and can mean anything from numbers to images to something more, which is why all the hardware bolted onto the roof is so interesting.

First, let’s take a step back.

Introduced more than five years ago, Apple Maps got off to an embarrassingly bumpy start with misidentified landmarks, flattened buildings, no transit maps, and generally bad directions.

It was so awful, Apple apologized.

Apple Maps has since gotten a whole lot better. The maps are detailed, with global coverage, 3D satellite imagery, public transit, and good integration with other mobile devices like the Apple Watch. I always use Apple Maps when I travel because:

1. I can rely on it.

2. I get a tap on my wrist for every turn.

When I tell people about this, thet stare at me like I have lobsters crawling out of my ears: “You rely on Apple Maps? It’s terrible.”

We agree to disagree.

I do, though, understand Apple Maps limits, and a big one is its lack of street view.

Google Maps has long taken you down to the ground floor of life, putting you at street corners, letting you virtually walk around the block. They build this incredible, stitched-together visual interface with an army of Google Street View cars, bicycles and people carrying around the 360-degree camera equipment. It’s hard to get lost when you can match what you’re seeing with Street View with what is in front of you.

It’s one of the fundamental difference between Google Maps and Apple Maps, but now I believe that’s about to change.

Take a good look at this car: I’ve grabbed a series of freeze frames from Sheffer’s video and made a composite image.

Original image has been replaced. Credit: Mashable

In addition to the small, white GPS antenna on top there are three wide-angle cameras in the center, two pointed at 45 degree angles toward the street level and one pointed up at the rooftops. Assume this triad is duplicated on the opposite side of the truck. There’s also a camera in the front and one on the back. That’s a clear sign that Apple is collecting street-level imagery, which could be used to build street view into Apple Maps.

But the cameras aren't the most interesting hardware. You see all those silver cylinders? Those are Velodyne LIDAR sensors. LIDAR (Light Detection and Ranging) is sophisticated 3D depth-sensing technology known mostly as one of the sensor techs in self-driving cars. Bouncing infrared light off objects can tell you the exact distance of the car in front of you, but also the distance to the bumper, windshield and roof. For a building, you get geo- and image-synced data on the distance to the curb, the front of the building, and the storefront sign jutting out above it. It’s a 3D map of your environment.

We debated in the office whether this was a skunk-works Apple self-driving car masquerading as an Apple Maps car, but I’ve come to the conclusion that the LIDAR is there to take Apple Maps to the next dimension.

If you combine a detailed 3D mesh -- the kind Lidar can build -- with all the imagery those cameras are grabbing, you build, at street level, something Google Maps Street View doesn’t have: true depth.

In Google Street View, the images are stitched together to realistic effect, but once you get close to any store front, building, or sidewalk, you realize that the imagery is all flat.

What Apple is building, in my opinion, is 3D street view.

3D street view opens up several possibilities. One of those is augmented reality Apple Maps. Apple is already building augmented reality into iOS 11 and handing ARKit to developers so they can build augmented reality abilities into their apps. If Apple Maps is AR-enabled this fall, any 3D object you have on your iPhone 8 could virtually live on a realistic street and interact with store fronts, curbs, street benches, and more.

All that imagery and 3D info will also make Apple Maps a powerful virtual reality platform. You could put on a VR headset, possibly attached to a the fancy new iMac Pro, and wander highly realistic streets. Someday soon, in Apple Maps, you may be able to virtually step under the awning at the Empire State Building.

This could be true, and it could be here as early as this fall. Or, Apple is simply testing self-driving car technology out in the open. But I'm going with door No. 1.

I’ve contacted Apple for comment and will update this post with their reply.