Connected Screens

Augmented Reality with Google Lens

13.06.2018 09:20

To mark the release or our new whitepaper on augmented and virtual reality (AR & VR) we have prepared a major update on augmented reality >> This is supplemented with the latest news about Google Lens, bringing you an up-to-date overview of relevant hardware and software developments, plus best practice examples.

The latest from Google Lens

Google has released a new version of Google Lens (beta phase). If you have one of the ten additional Android devices that integrate Google Lens into their own camera apps, you can access the feature simply by going to your camera and clicking on the icon at the bottom right.

When you point the smartphone camera at objects or sights, additional information about what you are viewing appears on the display. Google Lens also has a real-time finder that analyses what your camera sees before you tap the screen.

The tool will also process pixels through machine learning in order to offer more detail and provide relevant search tags.

What items can Google Lens identify?

Google Lens can identify clothing, books, buildings, plants, animals, and artwork. It will be able to give you shopping results for dresses it has identified, titles of books, landmark information, and details about paintings. You can also use it to identify the species or affiliation of an animal or plant.


The Lens will also soon be able to play a music video – for instance, if you have your camera aimed at a poster of a musician. Or it can show movie times when you are outside a movie theatre. With the "Smart Text" function, you can save the contact information from a business card or add details about an event from a flyer to your calendar.


Google Lens Smart Text function

The intelligent text selection scans words in documents and then lets you interact with them. Just aim the camera at a menu card, tap the screen, and you'll see explanations of the dishes. You can also create a picture of the inside of a book, highlight the text on the photo and then, just as you would expect from a messaging or editing app, open a menu where you can copy your selection elsewhere.



Google Lens & Maps

There is also a crossroads between Google Lens and Maps, incorporating Street View into AR to help users navigate in real time. Google has used AI (Artificial Intelligence) to continually improve both its maps and its news products.

With AI, Google Maps will soon provide Street View-style visuals, known as View Guides, which will gradually be superimposed over the smartphone camera into the real world. A pretty ingenious service! Google takes it one step further with the combination of the Google Assistant and Google Lens in Maps.

This allows you to pan across a street in any city and see pop-ups that mark restaurants and other places in real-time. According to Google, they've developed a new system for geolocalising objects in an environment called Visual Positioning System (VPS) that can handle everything from shop windows to street signs, allowing maps to be routed more accurately.



Our take at IQ mobile

The new features of Google Lens will be introduced in the next few weeks and mark a big step forward for augmented reality. AR is on its way to becoming a part of everyday life.

Google Lens continues to drive development in the field of image recognition. This year, apps will "know" what the camera "sees". It is still in the early stages – much like speech recognition was five years ago. We have all experienced how fast development goes, however. This service can quickly become a part of daily life. In this sense, such real-world scanning could accomplish something that AR glasses have failed to – so far.

Learn more about our thoughts on augmented & virtual reality here >>

Find out how customer interaction can be measured in virtual worlds here >>

A general overview of augmented and virtual reality in terms of hardware and software, including many examples, can be downloaded here >> (German only)


Stay up to date!