Menu

Colin Devroe

Reverse Engineer. Blogger. Chills easily.

Follow: @c2dev2, RSS, JSON, Micro.blog.

Retail AR

Dent Reality, a company looking to create practical applications for augmented reality, has released a video preview of their first offering Retail AR:

Retail AR can improve the customer experience and boost sales, by surfacing product details, displaying spatial information and navigating customers to relevant areas.

Be sure to watch the video. You’ll see its practicality immediately. I think the app will be useful on a phone, but imagine it built into some glasses? I hope more companies like Dent Reality spring up and I’m positive that they will.

Firefox Reality

Mozilla:

Mozilla has always been on the frontlines of virtual and augmented reality (see our work with WebVRWebAR and A-Frame), and this is a mixed reality browser that is specifically built to tackle the new opportunities and challenges of browsing the immersive web.

Me, in April 2017:

The way information is displayed is going to dramatically change within MR applications. How should a Wikipedia page on the honey bee be shown to a child wearing MR glasses while they are touring an apiary? Certainly this new wave of information layout should not be constrained to the resizable “windows” that we see in current demos but that we will see a rich set of layout and display tools that will make mundane information that the web currently hosts to come alive.

Mozilla sees this and they are skating to where the puck will be.

We’re in phase one of Augmented Reality

Apple just published a page dedicated to Augmented Reality in the latest versions of iOS. It is a good page overviewing some of the use cases we’re already seeing with AR.

This is just the beginning.

I’ve written about AR many times, so I won’t reiterate everything today. But look at these use cases and imagine them being accomplished, not within a small hand-held rectangle, but in full view using small glasses or — dare I even say it — contacts. It will be a while yet. But this first phase will set a lot of the UI, gestures, etc. that will be implemented in whatever the next generation of hardware is.

I still stand by this post. And the timeline in this post.

Richard MacManus’s tech predictions

Richard MacManus attempts to predict some things for 2018:

We’ll finally get a killer app for AR in 2018. Maybe hope springs eternal, but I’d love to see an AR app with real utility – not just a game like Pokemon Go.

I suppose it matters how you define “killer app”. For me a killer app would be when “most” people begin to use AR somehow. And, if that is the definition then I would say AR already had multiple killer apps. Pokemon Go, Snapchat, Instagram, Google Maps, all of these are excellent uses of AR and hundreds of millions of people use them.

Mixed reality is tough to segment and define. When I talk about AR I think I’m mainly talking about moving our computing experience away from our screens and into the real world. I would love to have my workspace no longer be tied to my work office. To have any size screen I want, wherever I need it. As I’ve written, I don’t think that will happen for another 9 years. But perhaps that is a different form of AR and AR has already “made it”.

I wonder what MacManus’s definition of “killer app” is?

A technology predication time capsule

Readers of my blog will know that I occasionally attempt to predict when certain technologies that I write about will hit the mainstream. While I’m very passionate about a few technologies, I try to temper that excitement with the experiences I’ve had, the wisdom that comes with age, and other factors. Usually, things take a little longer to happen than we’d like for the things we want to see most. And sometimes, sometimes, the things we want most never materialize at all.

For the purposes of this post, mainstream doesn’t mean critical mass but rather mass market adoption. With 7B+ people on the planet reaching critical mass is far easier than reaching mass market saturation. In other words, a company, product, or technology can reach sustainability and never truly hit the mass market. Examples: Tesla can succeed, be profitable, and have happy customers without the world moving on from fossil fuels. A company focusing on AI can make great livings, do compelling and challenging work, without every family having their own personal C-3PO.

Here are some stake-in-the-ground predictions on some of the most talked about technologies of our day. We’ll see in the next few decades if I was even close.

  • Legal, fully autonomous driving with no human assistance: Mid-summer 2026 – Even 9 years out there will still only be a few select vehicles that will fit into this category. There will still be humans driving on the road. And, only the most expensive cars will have all of these features. But, it will exist, be available to anyone, and be legal in the US. And I also believe there will be small fleets running in select cities for Lyft, Uber, and I believe Tesla will have a ride-share platform by this point. Also, don’t be surprised if Apple does too.
  • Bitcoin, or some crypto-currency, being widely transacted at small retail stores in the US: 2027 – If Square, or some other platform with high market saturation, turns on crypto for retail SMBs then we can say they accept this form of tender. But, I believe it will be 10 years before we see a decent number of daily transactions by consumers. I know, “decent” is relative so I’ll give it a number: $100,000,000 US dollar equivalent in a single month. This is roughly 25% of US monthly retail revenue as of September 2017. Side note: By this time we’ll see talk of the US dollar being converted to an all digital currency and, perhaps, transacted on its own blockchain.
  • Mixed Reality experiences used in everyday work environments: 2027 – Today we share links to web sites, documents on Google Drive, and flat or animated graphics to design and develop both soft and physical products. By 2025 many of these every day things will be accessible and even better experience within MR. I believe most businesses with digital assets will have multiple pairs of “glasses” or “goggles” that will allow team members to view or collaborate on these types of data. In other words, by 2030 rather than sending a child a link to Wikipedia to learn about our Solar System I believe we’ll be sending them MR experiences that they will consume using an augmented reality experience on a device other than a flat panel display. This happens today. But no where near mass market. And this industry has a long way to go. Even further than I previously thought.
  • Wireless internet takes over all cable based internet: 2029 – Most people in the US will connect to the Internet via wireless across all devices. And there will be no limitation on bandwidth usage.
  • Fully autonomous fleets replace individual car ownership: 2037 – Today US cities are plagued by traffic jams comprised of single occupant vehicles. Mass transit softens this but doesn’t solve the issue due to the convenience of a car. Ride sharing services have softened this even more and car ownership in urban areas is on the decline. By 2037 we’ll see massive reduction in individual car ownership in cities but also in the hinterlands as fleets of fully autonomous vehicles, combined with better mass transit, can care for the majority of transportation needs. I believe, however, families with at least 2 children will still have a single family-owned vehicle of some sort. Again, I’d like to put a number on this. So I’d say 15-25% less car ownership/use for individuals and commuters nationwide.
  • Mixed Reality replacing many conventional meat space locations/activities: 2050 – By 2050 the majority of children in the US will have the option to attend school in VR ala Ready Player One. Virtual classrooms will no longer be limited by federal budgets but will be designed to appear like cathedrals of learning.
  • (Because, why not?) An off-planet human civilization: 2175 – Humans will walk on Mars in the 2020s. And, perhaps, a small moon or Mars base will exist in similar fashion to today’s ISS in the 2030s. But a civilization, where people live, work, play, have children, and die peacefully etc. won’t exist on any other planet or moon (likely the Moon will have an established civilization prior to Mars). The reason I put this far-reaching prediction on this list is because I believe the excitement around a human footprint on Mars will lead to speculation about off-planet civilizations. But, we must all remember, we put a footprint on the Moon many, many decades ago and then just never went back. I do think that we’ll be mining objects near Earth much, much sooner. Even the Moon. But we’ll do that with robots and minimal human intervention.
  • Tweet editing – Never.

Check back in a few decades to see if I was even close.

Smartify

Smartify:

Smartify is a free app that allows you to scan and identify artworks, access rich interpretation and build a personal art collection in some of the world’s best museums and galleries.

Gunseli Yalcinkaya, writing for Dezeen, calls it “a Shazam for the art world”.

This is cool. Can’t wait to try it out.

/via Dave Mark at The Loop.

Oculus announcements galore

Be sure to check out the Oculus blog as they’ve announced a slew of new things. I’m still going through them all and will be sifting through it over the coming days. But, at first glance, I’m really excited to see this category of product move forward.

Here are the highlights for me (though there is a lot more)

 

Snap Art

Josh Constine for TechCrunch:

Snapchat plans to launch a new augmented reality art platform featuring pop artist Jeff Koons and others. It will allow art to be pinned to specific locations in augmented reality so users can see it when they hold up their phones in the right spot. Snapchat will solicit sign-ups from artists who want their art added to the platform.

Like Mirage except it will likely work well.

I’ll wait to see how this plays out but if it ends up being an Ello or Tumblr – meaning, users will feel the pressure to post nice things – with AR turned on it won’t be for me personally. But I can see it being a hit.

There is a video on Product Hunt.

 

Microsoft Windows Mixed Reality

In this video Tom Warren of The Verge uses some mixed reality headsets for Windows. Watching them I’m reminded just how far this industry has to go. I’d call much of what I see in this video very much beta-level hardware and software.

It has only been 5 months since I wrote the aforelinked piece and we’ve seen some major, major movement in this space during that time. Facebook, Apple, Google, and Microsoft have all thrown massive amounts of resources into mixed reality. I feel the pace of updates will only quicken over the next 24 months. By 2020 everything we see in this video will look ancient.

/via Dan Kimbrough on Twitter.

Mirage

Mirage:

A world living on top of reality.

I played around with the app this afternoon. It is very rough. Super frustrating to try and use. But I sincerely hope they continue to pull this thread. I hope to see a lot more of this type of thing over the next 36 months.

/via Andy Baio.

Tim Cook on ARKit

Tim Cook, in a recent quarterly earnings call for Apple, on ARKit:

One of the most exciting and most promising announcements from WWDC was ARKit, a new set of tools for developers to create augmented reality apps. It’s still early in the beta period, but it’s clear that ARKit has captured the imagination of our developer community. We think ARKit will help the most creative minds in the industry tap into the latest computer vision technologies to build engaging content. We believe AR has broad mainstream applicability, across education, entertainment, interactive gaming, enterprise, and categories we probably haven’t even thought of. With hundreds of millions of people actively using iPhone and iPad today, iOS will become the world’s biggest augmented reality platform as soon as iOS 11 ships.

And, later:

I could not be more excited about AR, and what we’re seeing with ARKit in the early goings. To answer your question about what category it starts in, just take a look at what’s already on the web in terms of what people are doing — and it’s all over the place. From entertainment to gaming, I’ve seen what I would call more small business solutions, I’ve seen consumer solutions, I’ve seen enterprise solutions. I think AR is big and profound. This is one of those huge things that we’ll look back at and marvel on the start of it. I think that customers are going to see it in a variety of ways. Enterprise takes a little longer sometimes to get going, but I can tell you there’s a lot of excitement already in there. I think we’ll start to see some applications there as well. It feels great to get this thing going at a level that can sort of get all the developers behind it. I couldn’t be more excited about it.

I feel the same way. ARKit is a foundational technology and the applications of it are going to be far reaching. And, no, my app isn’t based on ARKit.

See also.

Glass Enterprise

Jay Kothari, Project Lead for Glass:

Now the Glass product team is back at X, and we’ll be collaborating with the Google Cloud team and our partners to help customers across a variety of business sectors make the most of Glass. Together, we’re looking forward to seeing more businesses give their workers a way to work faster and in a more focused way, hands-free.

Glass Enterprise is a smart pivot by the Glass team. MR belongs at work and will have the greatest impact in these settings. More of this please!

WWDC 2017 recap

I wanted to take a few moments to jot down a comparison between my wish list for this year’s WWDC and what was announced. Also, towards the end, some quick thoughts on the surprises that were announced.

Here are my wishes, in order from the previous post, and whether or not we got them.

  • Shared iCloud Photo Libraries. Nope. It doesn’t appear so. I think if they had finished this they would have announced it.
  • iCloud data Family sharing. Yes! And, they gave us 2TB for the price of 1TB. So, a very good update here.
  • An all-new Mac App Store. Kinda? While they didn’t show this off, Phil Schiller did hint at it during John Gruber’s live interview with him and Hair Force One.
  • App Store demos. Nope. While the new iOS App Store looks very nice (and it getting great reviews all over the web) it didn’t include this.
  • App slimming. Not sure. I’ll wait for the public betas of iOS and macOS to determine if they’ve done any work in this area.
  • More Camera app filters. Yes! While the camera app may not have more filters built-in, the Photos app has tons of updates in this area. I’ll take it.
  • Apple Prime. Nope.
  • Rename iCloud Photo Library? Nope. But, not a big deal.
  • Apple Watch Series 3. Nope. Not yet. And the watchOS updates that were featured were lackluster. But, I think they were holding back for the event they’ll have in the fall.
  • Apple Photos improvements. Yes. Tons. I’ll wait until I get my hands on it to do a direct comparison with my wishes.
  • More iOS Extensions. Nope. I didn’t see much in this area mentioned, but I think they made up for it with the drag/drop features.
  • Siri. Nope. Read Manton’s post on this. He wrote what I was thinking.
  • Apple Maps accuracy updates. Nope. Not a single mention about Apple Maps that I saw. So, again, I’ll have to wait and see with the betas.

My last minute wish that I threw in was for driving mode. And that is a huge yes!

If we’re keeping score that’s like 8 nopes, 1 kinda, and 4 yeses. Which doesn’t seem like a good score but somehow I was very impressed with WWDC overall. I think we’re in for a great year of software updates coming from Apple.

Now, onto some of the surprises.

  • HomePod. While not a total surprise HomePod looks interesting. As a piece of hardware I really like it. As a device that allows you to access Siri I’m less excited because of how poor Siri is still. For example: Ask Siri “how far is New York City” compared to “Driving directions to New York City”. How can’t Siri answer the first question if it can answer the second? I refuse to believe that Apple isn’t staying up late nights to bolster Siri’s offering so hopefully we’ll see a massive improvement in Siri within calendar 2017 or 2018.
  • iMac Pro. What an incredible computer! My first Mac was an iMac and I bought Eliza an iMac somewhat recently. I really love all-in-one computers I just prefer to have a notebook myself due to working remotely, at work, in coffee shops, at a client’s, etc. If I were to buy a desktop computer for myself the iMac Pro would be it.
  • Macbook updates. I got my new Macbook Pro with Touchbar somewhat recently. But, these updates aren’t enough to make me regret my upgrade. They look solid though.
  • ARKit. As I’ve already noted, this will be huge.
  • New iPads Pro. The updates to the iPad (both software and hardware) are very, very good. Makes me wish I needed to upgrade.
  • iPad iOS features. Though it appears some of these could come to iPhone (or, perhaps the 10-year anniversary iPhone) – these features are amazing like drag and drop and the dock, etc. Pretty cool.

There are of course many things I haven’t mentioned but ll-in-all a solid week of Apple updates.

One last thing; recently Tim Cook has been quoted as saying that Apple is focused on autonomous driving (which we knew) but that they are focused on it as a category rather than a feature. Apple finds autonomy as an interesting area moreso than simply self-driving. I’m very interested to see how this idea manifests itself in future products.

Is VR overrated?

Kristopher B. Jones, an entrepreneur from near my neck of the woods, weighs in on VR in a recent Forbes piece debating the applicability of the technology:

I’m a strong believer that virtual reality is overrated, as it has limited applications outside of very specific industries. Industries like gaming and medical training are likely to see a boom from VR, while other industries such as food service, retail and finance with have limited to no applicability of VR. Much like Google Glass and 3-D television, the buzz will eventually die down.

In November of 2016 I said VR wasn’t ready. But that I thought it wasn’t far away. I was wrong because I was lumping VR in with a much larger mixed reality landscape. It wasn’t until I dug deep into mixed reality that I understood the subtle nuances between VR, AR, and the various other degrees of mixed reality experiences.

Kris likely understands this landscape even better than I do. He’s is right. VR will never be as big as the hype. In fact, I’d bank on it. However, “mixed reality experiences” (such as augmented reality) are popping up in every single app we currently have and will continue to do so. You already see it in Facebook, Instagram, Apple’s Clips app, even within the Uber app and Google Maps app. Facebook, Microsoft, and Google are already shipping platforms, frameworks, and APIs to help developers bring MR into their apps and services. And Google recently demonstrated an amazing technology called Lens that will be inside of Google Assistant and Google Photos soon. I also think the automobile dashboard and windshield is a huge future play for AR.

I don’t think Forbes or Kris lumps VR together with AR. But I do think that many consumers do. They think mixed reality is all about wearing huge goggles. It isn’t until you dig a little deeper that you see that mixed reality is all around us already. It’s already a hit. And it is just going to keep growing.

VR as a subset of MR is overrated. But, MR is far from overrated.

Build day 1 was great. I’m looking forward to day 2 as there are promises of Windows 10 and MR/AR news. Which, as you know, I’m interested in.

Observations on the computer-mediated reality landscape

Me wearing stupid looking VR goggles

The future won’t look this stupid. I promise.

For the past several months I’ve been doing research on computer-mediated reality (CMR) – that is, when what’s real is somehow changed, interrupted, distorted, or otherwise effected by a wearable computer.

This “ability” isn’t new and it is a nuanced superset of many different types including mixed reality (MR) (which I’m most interested in currently), augmented reality (AR), virtual reality (VR), and diminished reality (VR). These subsets, in turn, include many more subsets such as transreality (TRG), simulated reality (SR), and many more.

The more I have dug into this industry the more I’ve found how incredibly far reaching it is already and how much further it has to grow. Many of the applications in current use haven’t even hit the consumer market and others are hiding in plain sight – such as Pokemon Go, Foursquare, or even Google Maps. I’m willing to bet if you line 10 people up on the street and ask them if they use AR on a daily basis they’d say they didn’t think so. But if you inspect further I’m willing to bet they are and don’t even know it.

Here are some rather random observations I’ve made. Note that I’m mostly using the word “application” to mean how the technology is applied to a problem or situation rather than the typical use today as an “app” on your phone.

  • The common refrain today is VR vs. AR but upon inspection the industry is far, far more nuanced than that
  • MR is a hybrid of reality and VR which, to me, seems to have the largest number of applications for both businesses and consumers that has me interested the most
  • Pure VR applications will big a huge, huge market (esp in entertainment or recreational uses) but, to me, doesn’t have the broader applications that MR does
  • The industry supports a huge array of hardware and software to create the products that we see today and will see in the future. This means huge amounts of jobs. Think: Caves, HUDs (such as your bathroom mirror, your refrigerator,  your car’s windshield), head-mounted displays (such as glasses, goggles, phone holders, etc), tablets, phones, computers and hardware we haven’t even seen yet.
  • The way information is displayed is going to dramatically change within MR applications. How should a Wikipedia page on the honey bee be shown to a child wearing MR glasses while they are touring an apiary? Certainly this new wave of information layout should not be constrained to the resizable “windows” that we see in current demos but that we will see a rich set of layout and display tools that will make mundane information that the web currently hosts to come alive.
  • The pushback that Google Glass saw due to the possible “creepy” ways in which the technology could be used will disappear very quickly. Yes, I could have glasses on that could search for your LinkedIn profile while I’m looking at you. But, I could have done that later with an image on my phone too.
  • Speaking of which, facial recognition (and other machine learning algorithms that can find patterns and objects) are going to play an enormous role in the MR space. Expect many acquisitions in this space in the next 24 months.
  • As with any new platform the only way it will ever reach critical mass is if the applications (meaning, the apps, integrations, services that are supported) are plentiful. iOS’s biggest tentpole is the App Store. Mixed Reality applications for any platform (such as Hololens) need to be myriad before a business or consumer can truly invest in the platform. Microsoft claims, just one year in, that they have 150 apps. Depending on how they count the internal apps that companies like Japan Airlines have built exclusively for themselves, this could be an excellent day one offering.
  • MR could kill the computer display industry for businesses. (Though, this will take a very long time.)
  • Untethered devices, such as Hololens, will need to be affordable and have great battery life before the consumer market explodes. Likely something that will happen in 2018 or 2019.
  • Tethered devices, such as Oculus (though, this is VR not MR), will be valuable in gaming and enterprise contexts because it can rely on much more computing power coming from the connected hardware.
  • Think of tethered as “Pro” and untethered as “consumer” or “lite” for the most part.
  • The same way that tablet and mobile computers have revolutionized mobile computing (think: professionals on-the-go like visiting nurses, gas pipeline inspectors) and information transfer (think: pilots that used to have 100 books in the cockpit and now just have an iPad), MR will revolutionize contextual search and on-the-job training.
  • One of the complaints of MR is that you need to have a controller or move your hands all over the place to interact with the objects in your view. However, if you combine MR with what Elon Musk’s Neuralink (more here) will be making we’ll leapfrog how Tom Cruise used MR in Minority Report and move swiftly into the incredible territory of controlling virtual objects with our minds. Too future thinking? We’ll see.

This isn’t all that I’ve learned but are just some of the things I’m currently thinking about in this space. I’ll try to collect more tidbits under the CMR tag here on my blog.

I’m looking forward to following this industry as it matures and also supporting some at Condron Media. If you’re working on anything in this space please reach out to me.