Virtual reality, and augmented reality for data science
Virtual reality (VR) is a computer-generated simulation of a three-dimensional image or environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as virtual reality goggles. It is typically interactive, meaning the user can participate in the activity.
Companies are increasingly using VR to explore data. They use this to generate more insightful data visualizations and make it easier for the users to understand and interact with their data.
In addition, VR provides a more immersive experience for the users and makes it easier for them to understand complex information.
It also helps companies explain their products or services in an interactive way and provide them with better customer service.
Resources: https://business.penguinconnective.com/toolkits/xr-data-science/
About the guests
Jessica Spencer
A creative professional with a background in design and a deep understanding of business, strategy, and technology. With over 15 years of experience in a variety of industries, including technology, healthcare, consulting, financial services, engineering, and education, I’ve had the opportunity to work alongside business leaders to support day-to-day operations, guide initiatives, and provide a different perspective when outside-the-box solutions are needed. My curiosity for technology and love for learning have led me to become proficient in Microsoft Office, SharePoint, the Adobe Creative Suite, WordPress, and Salesforce.com. In recent years, I’ve enjoyed dabbling in AI/ML with IBM Watson and Python, as well as VR/AR/XR. Through the creation of infographics, animated videos, and other formats, I’ve found success communicating complex ideas in ways that are more easily understood.
James Intriglia
James is an independent Healthcare IT (HIT) Contract Consultant, specializing in developing and implementing functional testing programs to ensure compliance with business best practices and federal government regulatory requirements.
Transcript: AR and VR in data visualisation
Stelios: Hi, Evan. Welcome to the Data Scientist Podcast. I’m very happy to have here with us today, Jessica Spencer and James Triglia, and we’re going to have a very, very interesting conversation about VR, AR, and data visualization. Guys, would you like to go ahead and introduce yourselves? You’re going to do a better job with this event. I’m going to do.
Jessica Spencer: Sure. That sounds great. Well, thank you so much for having us today. I’m Jessica Spencer. So I have a background in marketing. I also have done a lot with just computer, different system administration. I also have done some sales analyst work. I have an MBA and I’m fascinated by technology. So I haven’t done a lot of projects actually in data visualization within virtual reality, but I am absolutely fascinated by what’s possible on some of the different trends that are going on right now. I know I’ve done like some studies and futurist trends through some independent study. And there there’s just so many neat things going on between virtual reality and artificial intelligence. And I think just the way that all of those can merge together to create new possibilities for us is pretty incredible.
So I really enjoyed looking into things that are possible through virtual reality. And I’ve worked quite a bit with Jim and Triglia also different projects. I know we’ve done some things within the medical and healthcare field as it relates to virtual reality and have also done some things more from an edutainment standpoint. So really trying to bring more of that entertainment feel to the education pieces and virtual reality to make it more engaging and immersive and everything. So Jim, do you want to introduce yourself?
Jim Triglia: Yes. And thank you Jessica and thanks Dr. Stelios for having us on your podcast. My name is Jim Triglia. I am basically a virtual reality consultant and analyst. I focus on VR applications, VR spaces for business professionals. That’s my scope of study. And my background is in computer science, engineering, and also business and I specialize in marketing. So that’s the entertainment I lead in. The way I like to market is to educate and do a soft sale. I think that’s one of the best marketing approaches. In my career, it spans over 30 years in computer science. One of the biggest developments was when I was introduced by a colleague who was a biostatistician into data visualization, and I have a background in data modeling as well. And Edward Tufty, we went to one of his seminars and it was, it changed my life as far as the power of visualizing data and what pictures could show that would never show up in words.
And I use this as an analyst. And I remember getting back long story short from this talk and doing an analysis from a biostatistician statistical point of view of a dataset, which was being used for an upcoming symposium, and finding real problems with the data from a visual perspective that we completely missed doing statistical data analysis on. But once I plotted it, I saw outliers that didn’t, 172-year-old patients, things like that, that should have been caught, but we didn’t. So virtual reality I think is a tremendous medium and has huge potential to be able to do data analysis that is not possible in the real world or even the 2D web world. That’s my belief.
Stelios: Okay. So I’m going to ask you a question that’s a little bit boring, but probably you saw it coming. So I like virtual reality as well, but like many users out there, like users are not like a casual user, which are reality, but occasional I’m concerned about things like getting dizzy the headsets and if I want to buy a headset for my own use, I guess the better one are quite expensive. So how democratized do you think this technology now is in your projects? Have you used like high-end expensive headsets or something which is more affordable? Like yeah, it would be great if necessarily you could explain what’s the current level of accessibility to this technology for the average user or business.
Jim: Yeah, great question. And because my clientele are entrepreneurs and IT professionals these are not wealthy people. And the startups have limited budgets like most businesses. So the tipping point for me was when Facebook now meta came out with the Quest 2, because that was a $399 headset, and that was a tipping point from a business perspective to me. So I own a Quest 2 headset. The majority of applications that deliver value are available on this headset through meta. On a global basis international basis. The contender would be the Pico headset, which you may have heard of. That’s actually not available in the United States, but it’s available worldwide and has been getting some very, very good reviews. As far as dizziness, I haven’t experienced that personally but I think a lot of that had gone away with the quality of the headsets and the technology that speeds to the process are increasing.
There were always reports and people that have problems with that, but I think they’re in the minority these days. As long as they have a computer. In the case of the Quest 2, it’s a standalone, but if they’re using like an HP Reverb, which is the other wired headset that I have, if they have a good computer that can keep up with the frame rates so that when they turn their head, everything sinks, there’s this. So I think that’s becoming less of an issue. And the majority of people on Quest 2 don’t really experience that from what I’ve read.
Jessica: Yeah, I think another thing that’s really cool is a lot of these experiences and a lot of the systems that you can use for virtual reality, you don’t have to do it through a headset. So a lot of them are accessible in 2D mode, which would be through just a computer. So you can still interact, engage with people, you still have access to the spatial audio, which lets you have like those, like one on one small group conversations within the same environment. So, and you can still interact with like, any of the objects that are in there. You’ll just use like your mouse keys to move around and that type of thing. So I think I know I can usually use my headset for maybe an hour at a time. I know battery with like the Oculus quest is still a bit of an issue.
So I think usually after about two hours or so, it’s about done for the day. So and I know as someone who wears glasses and can’t do contacts it’s a little bit trickier just because you do get that pressure like through your face and everything with the glasses. But yeah, I think definitely being able to access it though through the computer, and in some cases, you can even access environments on your mobile phone. So I think that makes it extremely accessible to people in a way where it’s not just headsets that allow you to engage.
Stelios: Uh huh, Uh huh. Interesting. And with regards to using VR for data visualizations, especially the healthcare, how have you seen this being applied? I also know that there are some links that you going to share with us later. For those of you who are listening or watching, the links are going to be in the description below, whether you’re Spotify, YouTube, or you’re watching this from a personal blog. But yeah, you could give us some examples. And also it would also great to understand whether the advantages of doing this over more traditional forms of visualizations or traditional dashboards.
Jim: We’d like to start. Yes.
Jessica: Well, Jim, from a healthcare standpoint, I’m kind of curious if you have any specific examples you want to share around that. I guess like, just kind of at a global level just to start out with that. I think the big advantages of doing it in virtual reality compared to, just a standard like Zoom presentation or something like that you really get more of that storytelling capability. And I think a lot of that is based around perspective. For instance, if you go into an environment and you are comparing, I don’t know, like the size of different skyscrapers, for instance, it’s one thing to look at a picture of it. It’s another thing to actually stand in front of it and actually visually be able to look up and see just that magnitude.
So I think that’s really interesting. So I know that a lot of folks say from like a decision-making standpoint you can do like AR overlays and stuff for that augmented reality where you can actually see real-time data against where you are located within like a geospatial framework. So I think that’s a huge advantage. I know they’re using that a lot within supply chain. I think that could also definitely be something that would be relevant to healthcare. Having that real-time access to data I could even see that relevant from like a surgical standpoint, you know kind of being able to see, read that like biometric data and everything real-time over certain parts of the body or see like a heat map or things like that I think could be very relevant.
Jim: Yeah, And from a medical perspective, Jess and I were actually chatting about this before the podcast. Edward Tufty, during his seminar talks about how a, I think it was an esthetician back in 1854, his name was John Snow. They had a collar outbreak that killed 600 Londoners, and they couldn’t figure out what the problem was. They were looking at raw data, numbers, addresses. And what Dr. Snow did is he plotted it on a map. And when he stepped back and looked at the plot, he realized that the deaths in the neighborhoods were centered around a particular well that was contaminated, and there’s now a pub on the site and to commemorate this, but that was a classic example of taking medical data in this case deceased patients and the redresses. And rather than looking at it as data and addresses, putting it on a map and saying, oh my God, there’s a concentration around this wellhead, and maybe the problem is the water, which is what it turned out to be.
Stelios: Yeah, actually, I’m familiar with this example, which I think is one of the best examples of data visualization. But to summarize, like your answers is probably that in visualization, especially in a VR setting, can be more engaging. But what I’m wondering is how accessible is it for the average data scientist or data analyst. So every, I’m a data scientist. Many people listening to this podcast, they’re data scientist, data analysts, computer scientists. We can assume that most of the audience is familiar with things like Excel. Maybe some people listening to this, they also know how to use Python or R and do some like simple charts, maybe even tools like Tableau. And I think really Tableau was a very, a key moment in the field of data analytics and visualizations because it’s really democratized and opened up the space to the extent that there was this new class of professionals who didn’t really know data science, but they could use Tableau to make good visualizations. So my question is, is there something similar for VR? Like, I’m a data scientist, if I’m going to use let’s say VR to create some cool 3D immersive visualizations for one of my projects, how would I go about doing it?
Jim: Yeah, that’s a great question. And what brought to mind, my son actually is in the field as well, introduced me to GS, GIST I don’t know what they, dot io, but is basically an open-source project that’ll take in dataset, many of them you great sets from publicly available data sources and automatically visualize them, and I believe it is and has some artificial intelligence behind it. So I’m looking for, I’m not aware of any of those applications, but the likelihood of them being out there is high. Because data analysis data analysts are huge in computer science. I would suspect that the useful tools are not going to so much be found in virtual reality spaces like Alt Space or Neos. They’re going to be found in virtual reality applications. That’s where you’ll, and you’ll have the capability within those spaces to not only do work and visualize data and play with it in a virtual reality space but also collaborate with colleagues. So that’s what and in the medical field, keep an eye on the clinicians. I work in the medical field as well, in the healthcare on electronic medical records. I have two colleagues. I have clinicians, and I have the business of healthcare folks. The clinicians are all over this. So I would keep an eye on clinicians in the virtual reality space that work with data that are doing data analysis research. Those folks will lead you to the applications you see.
Jessica: So I actually have been doing a little bit of research around this, and I’d love to, do you mind if I share my screen and I can kind of show you like…
Stelios: Sure, go ahead. I guess for those of you who are listening, just make sure to go to my blog, the data scientist.com actually, to make sure that you see this. But there are also going to be some links attached in the description.
Jessica: Awesome. And I will do my best to kind of describe what’s actually happening. So everyone has some idea of what’s going on. So this one I think, is fairly accessible to people, and it’s pretty fascinating what you can do. So I’m just going to this one is called Flow Immersive, and it’s flow immersive.com. They do have a free month promotional trial, but I just want to show you this, I think it’s like a TikTok or something that somebody did. And you may have to actually go on and watch it. I don’t know how this is going to show when I actually pull it up here, but, well, anyways, let me just talk through this. So what this shows is it shows the soccer player Messi, and he is, it shows him basically a scene behind him where he’s Kicking the ball toward the goal, and then there’s this guy that’s standing there at the goal, and it actually shows a data plot of all the different places.
And the goal that he’s kicked in over, I don’t know if it’s like the past, it looks like it is maybe over the past four years or something like that. And so it shows a data plot of every single, like, physical location that he’s kicked it in the goal so that you can actually kind of visualize like where those hot spots are. So I found that really fascinating. And there’s a lot of other things that you can do on this one. And I apologize. My screen is being a little bit slow. So anyway, that’s slow immersive, is what that one’s called. And I don’t think it was all that expensive either on a month-to-month basis, but there’s a lot that you can do in terms of plotting data. And then you’ll see, like, this guy right here. He’s actually standing behind it. And so it’s more of that augmented reality view, and he can actually turn the data and walk around it. So I think that’s pretty compelling from a storytelling standpoint. Right.
Stelios: No, absolutely. I really like the sports example because I was working for years, the of sports analytics. Yeah. I think that this type of visualization, especially VR, could probably make data more interesting for an audience which is traditionally not interested in data. So one of the problems in English football, when I was working with a club here, was that most of the people in the club, they were not really interested in data and, but obviously, if they’re going to wear a headset and I see something in 3D is it’s going to be much more cooler if you’re going to tell them, yeah, this is how you actually perform in the field, it’s going to be so much more engaging rather than just showing them some kind of 2D map and with all like different arrows and stuff. Like, I guess that’s one of the problems sometimes with data visualizations that unless you’re really into data, maybe you won’t find them as interesting. And I think that some of the best visualizations I’ve seen are visualizations that really make up this story, this narrative, and can be engaging for Perian, even if they don’t have a background in that domain.
Jessica: Exactly. And I think really, telling that story around the data and being able to compare to different views of data against each other, I think that’s something that’s really neat and virtual reality because you aren’t just working with, like, three computer monitors in front of you, which is still a lot, but you can actually have like an entire 360-degree view where you have different, like data sets that could show up, and then you could go in, and you could drag and drop one to another location. And it’s so easy. I mean, there’s definitely a wording curve with it just in terms of kind of how to navigate and move things around, but it’s so much easier to pull up this, like, massive amount of information and a single environment and have easy access to it to do comparisons and to be in that same environment with someone that’s across the world from you and have real-time conversations about it and interact with it. So I think that’s fascinating too, is just the ability to engage around data and have those like true conversations and experiences with people about how you really can create that information, that story around it.
Stelios: Yeah, absolutely. Jim, do you have any comments?
Jim: Yeah, one of the things that comes to mind is and I think you touched upon this in the beginning of the podcast, is the focus on applications. What I’m looking for is the ability to manifest in a virtual space, three-dimensional data visualizations based on data that’s fed into the application, and then allow the people in virtual reality through voice prompts. With this good artificial intelligence engaged to model that data to play with the data. This could be done as a group to do what IFS to go within the data. To literally travel inside the data to maybe get a macro view. So this is likely, again, I’m not aware of anything, but I don’t focus on data per se. I focus on process, but data is part of what I do, and I’d be willing to bet that there is an entire community out there that’s all over this as far as visualizing data and virtual reality applications.
Stelios: Yeah.
Jessica: Yeah. One thing we saw that was really cool, so Jim and I saw a demo of an environment where they had artificial intelligence that was worked into the VR environment. And they actually had these columns like throughout the setting that they would change colors based on natural language processing. So based on what people were saying within the environment, it was taking that information and making the columns change colors based on, like tone of voice based on the words that were being said in the context. So I thought that was fascinating. And what, like, so some potential outcomes I see from that is like biofeedback, for instance, what if you took a world like that and based on those biofeedback signals, you are able to change like the entire sky box of that environment.
So Sky Box is like the sky or the cityscape or whatever it is that’s kind of all around the world. It could be, I know we have a friend who has one that’s Barcelona, but you could probably change even the skybox to kind of relate to how people are feeling. So maybe it’s a color change, maybe it’s the whole environment itself just kind of changes, but I think that would be kind of fascinating visual feedback based on those body signals. So I think that’s a kind of cool application for where it could possibly go.
Stelios: Yeah, I guess the applications are countless, and it would be great if, I mean, for those of you who are listening, viewing in the description below, they’re going to be some more links for you to check out some more examples because I think maybe because it’s like, with data visualization in general, there’s so many different applications that’s to a large extent, and art as opposed to a size, I guess if you take VR or AR into account, this gives you more freedom, but obviously maybe many professionals don’t know what to do with this freedom. So I presume that in the near future will not see professionals specializing only this type of visualization. I don’t know what are your views on that. Have you had any interface with such professionals? Because it feels for now, there are things happening, but maybe they’re not really broken into the mainstream yet.
Jim: Well, I can give you another example. In aviation Microsoft’s flight simulator, which is now being used to not only train general aviation pilots on how to fly but also commercial pilots they’ve developed an interface into real weather a year or two ago, not too long ago. The neat part about the 40th-anniversary edition is they took. I’m a student pilot. I’m a student glider pilot. I fly fixed-wing gliders, which have no engine, so I’m completely dependent on airflow. What they did, which was genius, the data visualization is they took the weather information that had to do with thermal updrafts, and they give you an option to show it while I’m in the cockpit. So now I can see the airflows to steer my glider to it rather than looking for clouds and assuming they’re there, I can actually see it. That’s an amazing training aid for a student that’s trying to learn how to catch thermals. So I see a cloud. Hmm, there should be an updraft there. Now I can see if there really is based on real-time weather data. That’s amazing.
Stelios: This is actually Fascinating example. Is this like a, so this a VR flight simulators, so you’re talking about Microsoft’s call it product Now they’ve added this functionality in the product because I could easily imagine something like this existing in augmented reality as well, that’s why I’m asking?
Jim: Yes, yes. And that’s eventually where we’ll end up with. We’re going to eyeglasses that’ll both overlay data in our real world and then control __ and put us in a VR. But that’s actually the where the origin of virtual reality started in our own US Air Force in the United States in 1960. That’s how we use, we still do. We train our pilots, our aircraft, our fighter pilots. So that’s a real-world right now application that is in use in education and in pilot training.
Stelios: Yeah. Fascinating. Fascinating. Great. Okay. I think our time’s up, but this was a very interesting conversation. Any last words before we go?
Jessica: Thank y’all so much for having us, and it was a pleasure talking to you today.
Stelios: Yeah, likewise. I think it was very conversation and VR and AR it is like the greater theme is very interesting for me and many other people. I just believe it hasn’t broken into the next room yet, but every time I’ve been to an exhibition, and I’ve worn all of those headsets, I’ve been really impressed. So it’s great to see that there are more people into this area because, really I think the applications are countless. And I know there’s this vision of the metaverse, again in this blog the data scientist.com because we’re interested in web three and blockchain, also talk about the metaverse. But I think that these technologies have been around before the Metaverse and the have life outside of it. So while joining VR and blockchain and AI and all that, it’s pretty cool. There are many, many, many useful applications of VR and AR which have nothing to do with web three. So I think this is like a field in its own.
Jim: Yeah, I agree. And yes, thanks for having us on your show, and now that we’ve talked about the topic, you can be sure that we’ll all be of tripping over examples of good applications for virtual reality and visualizing data and information. So I’ll be happy to share those resources as they cross my path.
Stelios: Yeah, thanks a lot. Yes. For those of you who are listening, just take the description for some examples of VR and AR visualizations. And thank you, Evan for being here with us. Make sure to go to the data scientists.com for more content around data science AI, and blockchain. Thank you.
[END]