Connect with us

Technology

Multimodal AI promises transformative changes to health care delivery

The health-care industry is slowly adopting first-generation AI technologies, but with multimodal AI on the horizon, it’s time to get ready for what’s next

Published

on

Share this:

Multimodal AI is the next big thing in health care. Except the previous next big thing — regular ‘garden variety’ unimodal AI — hasn’t even fully arrived yet.

“In health care, AI is a promising technology that has been deployed in very small pockets here and there, but it has not broadly impacted health care in a meaningful way,” said Dr. Amol Verma, a physician, scientist, Assistant Professor in General Internal Medicine at St. Michael’s Hospital and the University of Toronto and the 2023 Temerty Professor of AI Research and Education in Medicine at the University of Toronto. 

“The vast majority of clinicians don’t really use AI in their clinical practice.” 

That fact doesn’t mean we’re not at the outset of a multimodal AI revolution in health care, with transformative changes including diagnostics, enabling remote care, and driving efficiencies in an industry not exactly known for them.

But before we get too far ahead of ourselves here — what exactly is multimodal AI?

Multimodal AI systems are capable of processing and synthesizing multiple forms of data in order to provide outputs that include decisions, recommendations, and predictions. 

Just like people.

“We are multimodal,” said Elham Dolatabadi, a data scientist and Assistant Professor at York University focused on machine learning in health care. “We can smell, see, and hear. These are all different modalities. Our brains integrate them to come to one decision.”

So, where a unimodal AI system in health care might only be able to assess MRI scans, a multimodal AI system could process the information from that MRI scan, along with clinical notes, lab tests, genomic data and real-time patient health tracking information, amongst other inputs. In this way, a multimodal AI system would replicate the way actual doctors process patient health information — by considering and analyzing multiple sources of data.

What multimodal AI could mean for health care

Multimodal AI systems could bring almost unfathomable computational power to health care, especially when it comes to diagnostics and predictive capabilities. This could mean a much more personalized and precise approach to health care for individual patients. Think earlier, more accurate diagnoses and better outcomes — all at scale. 

For example, a computer could process radiology images and incorporate that data into its predictions to make them more accurate. It could also incorporate actual conversations between a patient and physician and even assess clinical deterioration from the sound of a patient’s voice.

Multimodal AI could also unlock the potential of telemedicine. Right now, health care providers are limited to assessments based on their conversations and observations over video calls. But if a patient had sensory technology in their home capturing their personal data and feeding it into a multimodal AI system, that could change health care dynamics for millions of patients. 

“It could dramatically improve access to care for people who are far away from clinical care or who, for whatever reason, have a hard time getting out of their house,” said Dr. Verma.

Wait times could also be improved. For example, patients regularly face wait times for ultrasound imaging. That’s partly because operating an ultrasound machine is what’s called ‘operator dependent.’ A clinician needs to be properly trained to use it. But AI could actually help patients capture their own images if it was embedded in a device they could use themselves.

“It could change who can deliver health care, not just who receives it,” said Dr. Verma.

Ethical considerations remain central to AI deployment in health care

For many, the widespread use of AI in health care still seems risky. What about privacy? What about the risks of entrusting critical patient care decisions to AI? 

Shaina Raza is an Applied Machine Learning Scientist for Responsible AI at the Vector Institute, an independent, not-for-profit focused on AI research. Her work focuses on the ethics of responsible AI in public health.

“With critical decisions about the life or death of a person, the doctors should make those decisions,” said Raza. “It’s not appropriate for generative AI models to do that. But if AI is used to facilitate research or help the doctors’ decisions, that’s different. We can save hundreds or thousands of hours that way.”

Raza notes that ethical AI in health care is ultimately about humans creating the frameworks for the AI before it’s ever fully deployed. That’s how best to address issues like patient privacy and systemic biases. 

“Patient privacy is very sensitive. We need to de-identify or mask patient data before feeding it into the AI models.” said Raza. “We can also clean the data for biases before we feed it into the models, with what we call prompt engineering, the instructions we provide to the models.”

Once these sticky issues are addressed, multimodal AI will likely have revolutionary, positive impacts on our health-care system. But since the health-care industry is typically slow to adopt new technologies, that revolution may have to wait a few years.

This might actually be a good thing, according to Dr. Verma.

“Technology is deployed much more widely in the general society than it is in medicine,” said Dr. Verma. “We’re talking about a profession that still frequently uses fax machines. I can do more sophisticated things on my mobile device than I can for applications that are medical-specific. That’s a good thing because the stakes in medicine are very high — literally life and death. We don’t want to be deploying unproven technology so rapidly in that context.”

So while multimodal AI isn’t yet ready for broad deployment, the industry is still preparing for what’s next.

“Our aim is to include as many modalities in the models as possible, including images, text, electronic medical records, wearables, signals like ECG and EEG, and genomics,” said Dolatabadi, who is currently conducting research in multimodal learning with generative AI. 

“Then the models could be used for different applications. And once the models are built, hospitals or health-care organizations can fine tune them for their own applications and patient populations.”

How health-care organizations and providers can prepare for the age of multimodal AI

With multimodal AI a matter of when and not if, it’s incumbent upon everyone in the health-care industry to be prepared for the changes ahead. 

Dr. Verma outlines four key key areas of focus over the next few years: 

  1. Design the right systems to safely deploy AI technologies. “We need to be designing the right institutions, relationships and incentives. That means creating, effectively, centres of excellence with the infrastructure, data and skilled personnel that can monitor AI technologies. These centres would then connect to primary providers, who you can’t expect to assess whether an AI solution is working. That’s just not feasible or practical for them.”
  1. Create a plan for collecting critical data to avoid exacerbating system biases. “We know systems are prone to bias, so the second thing that must happen at a system level is creating a plan to collect data about patient race, language and gender, which we currently do not collect. If we don’t collect that information, we won’t know whether solutions are biased and will therefore perform poorly. We could end up exacerbating biases in our health-care system.”
  1. Building up AI expertise, skills and organizational capacity. “Organizations need to identify AI champions, either through recruitment or upskilling. They need people that have technical skills and skills related to the legality and ethics of AI. And they need people that have skills related to the change management aspects of AI implementation. There’s also the question of scale. Not every organization can do this. Big organizations should scale up and smaller organizations that can’t should partner with big organizations that can.”
  1. Get educated on AI technologies. “I think at the individual practitioner level, people basically just need to become more aware and educated through professional development. Some basic understanding of these technologies would be good.”

Despite all the complexity with AI, there are options for more people to live longer, healthier lives if multimodal AI is deployed across the health-care system. The fact that health-care organizations are still working to embed unimodal much less multimodal AI doesn’t change the trajectory. In the coming years, almost all of us will find our health care experiences enabled by AI in ways both obvious and hidden. It’s the inevitable next step.

Share this:

Business

How businesses can protect themselves from the rising threat of deepfakes

Dive into the world of deepfakes and explore the risks, strategies and insights to fortify your organization’s defences

Published

on

Share this:

In Billy Joel’s latest video for the just-released song Turn the Lights Back On, it features him in several deepfakes, singing the tune as himself, but decades younger. The technology has advanced to the extent that it’s difficult to distinguish between that of a fake 30-year-old Joel, and the real 75-year-old today.

This is where tech is being used for good. But when it’s used with bad intent, it can spell disaster. In mid-February, a report showed a clerk at a Hong Kong multinational who was hoodwinked by a deepfake impersonating senior executives in a video, resulting in a $35 million theft.

Deepfake technology, a form of artificial intelligence (AI), is capable of creating highly realistic fake videos, images, or audio recordings. In just a few years, these digital manipulations have become so sophisticated that they can convincingly depict people saying or doing things that they never actually did. In little time, the tech will become readily available to the layperson, who’ll require few programming skills.

Legislators are taking note

In the US, the Federal Trade Commission proposed a ban on those who impersonate others using deepfakes — the greatest concern being how it can be used to fool consumers. The Feb. 16 ban further noted that an increasing number of complaints have been filed from “impersonation-based fraud.”

A Financial Post article outlined that Ontario’s information and privacy commissioner, Patricia Kosseim, says she feels “a sense of urgency” to act on artificial intelligence as the technology improves. “Malicious actors have found ways to synthetically mimic executive’s voices down to their exact tone and accent, duping employees into thinking their boss is asking them to transfer funds to a perpetrator’s account,” the report said. Ontario’s Trustworthy Artificial Intelligence Framework, for which she consults, aims to set guides on the public sector use of AI.

In a recent Microsoft blog, the company stated their plan is to work with the tech industry and government to foster a safer digital ecosystem and tackle the challenges posed by AI abuse collectively. The company also said it’s already taking preventative steps, such as “ongoing red team analysis, preemptive classifiers, the blocking of abusive prompts, automated testing, and rapid bans of users who abuse the system” as well as using watermarks and metadata.

That prevention will also include enhancing public understanding of the risks associated with deepfakes and how to distinguish between legitimate and manipulated content.

Cybercriminals are also using deepfakes to apply for remote jobs. The scam starts by posting fake job listings to collect information from the candidates, then uses deepfake video technology during remote interviews to steal data or unleash ransomware. More than 16,000 people reported that they were victims of this scam to the FBI in 2020. In the US, this kind of fraud has resulted in a loss of more than $3 billion USD. Where possible, they recommend job interviews should be in person to avoid these threats.

Catching fakes in the workplace

There are detector programs, but they’re not flawless. 

When engineers at the Canadian company Dessa first tested a deepfake detector that was built using Google’s synthetic videos, they found it failed more than 40% of the time. The Seattle Times noted that the problem in question was eventually fixed, and it comes down to the fact that “a detector is only as good as the data used to train it.” But, because the tech is advancing so rapidly, detection will require constant reinvention.

There are other detection services, often tracing blood flow in the face, or errant eye movements, but these might lose steam once the hackers figure out what sends up red flags.

“As deepfake technology becomes more widespread and accessible, it will become increasingly difficult to trust the authenticity of digital content,” noted Javed Khan, owner of Ontario-based marketing firm EMpression. He said a focus of the business is to monitor upcoming trends in tech and share the ideas in a simple way to entrepreneurs and small business owners.

To preempt deepfake problems in the workplace, he recommended regular training sessions for employees. A good starting point, he said, would be to test them on MIT’s eight ways the layperson can try to discern a deepfake on their own, ranging from unusual blinking, smooth skin, and lighting.

Businesses should proactively communicate through newsletters, social media posts, industry forums, and workshops, about the risks associated with deepfake manipulation, he told DX Journal, to “stay updated on emerging threats and best practices.”

To keep ahead of any possible attacks, he said companies should establish protocols for “responding swiftly” to potential deepfake attacks, including issuing public statements or corrective actions.

How can a deepfake attack impact business?

The potential to malign a company’s reputation with a single deepfake should not be underestimated.

“Deepfakes could be racist. It could be sexist. It doesn’t matter — by the time it gets known that it’s fake, the damage could be already done. And this is the problem,” said Alan Smithson, co-founder of Mississauga-based MetaVRse and investor at Your Director AI.

“Building a brand is hard, and then it can be destroyed in a second,” Smithson told DX Journal. “The technology is getting so good, so cheap, so fast, that the power of this is in everybody’s hands now.”

One of the possible solutions is for businesses to have a code word when communicating over video as a way to determine who’s real and who’s not. But Smithson cautioned that the word shouldn’t be shared around cell phones or computers because “we don’t know what devices are listening to us.”

He said governments and companies will need to employ blockchain or watermarks to identify fraudulent messages. “Otherwise, this is gonna get crazy,” he added, noting that Sora — the new AI text to video program — is “mind-blowingly good” and in another two years could be “indistinguishable from anything we create as humans.”

“Maybe the governments will step in and punish them harshly enough that it will just be so unreasonable to use these technologies for bad,” he continued. And yet, he lamented that many foreign actors in enemy countries would not be deterred by one country’s law. It’s one downside he said will always be a sticking point.

It would appear that for now, two defence mechanisms are the saving grace to the growing threat posed by deepfakes: legal and regulatory responses, and continuous vigilance and adaptation to mitigate risks. The question remains, however, whether safety will keep up with the speed of innovation.

Share this:
Continue Reading

Business

The new reality of how VR can change how we work

It’s not just for gaming — from saving lives to training remote staff, here’s how virtual reality is changing the game for businesses

Published

on

Share this:

Until a few weeks ago, you might have thought that “virtual reality” and its cousin “augmented reality” were fads that had come and gone. At the peak of the last frenzy around the technology, the company formerly known as Facebook changed its name to Meta in 2021, as a sign of how determined founder Mark Zuckerberg was to create a VR “metaverse,” complete with cartoon avatars (who for some reason had no legs — they’ve got legs now, but there are some restrictions on how they work).

Meta has since spent more than $36 billion on metaverse research and development, but so far has relatively little to show for it. Meta has sold about 20 million of its Quest VR headsets so far, but according to some reports, not many people are spending a lot of time in the metaverse. And a lack of legs for your avatar probably isn’t the main reason. No doubt many were wondering: What are we supposed to be doing in here?

The evolution of virtual reality

Things changed fairly dramatically in June, however, when Apple demoed its Vision Pro headset, and then in early February when they were finally available for sale. At $3,499 US, the device is definitely not for the average consumer, but using it has changed the way some think about virtual reality, or the “metaverse,” or whatever we choose to call it.

Some of the enhancements that Apple has come up with for the VR headset experience have convinced Vision Pro true believers that we are either at or close to the same kind of inflection point that we saw after the release of the original iPhone in 2007.Others, however, aren’t so sure we are there yet.

The metaverse sounds like a place where you bump into giant dinosaur avatars or play virtual tennis, but ‘spatial computing’ puts the focus on using a VR headset to enhance what users already do on their computers. Some users generate multiple virtual screens that hang in the air in front of them, allowing them to walk around their homes or offices and always have their virtual desktop in front of them.

VR fans are excited about the prospect of watching a movie on what looks like a 100-foot-wide TV screen hanging in the air in front of them, or playing a video game. But what about work-related uses of a headset like the Vision Pro? 

Innovating health care with VR technology

One of the most obvious applications is in medicine, where doctors are already using remote viewing software to perform checkups or even operations. At Cambridge University, game designers and cancer researchers have teamed up to make it easier to see cancer cells and distinguish between different kinds.

Heads-up displays and other similar kinds of technology are already in use in aerospace engineering and other fields, because they allow workers to see a wiring diagram or schematic while working to repair it. VR headsets could make such tasks even easier, by making those diagrams or schematics even larger, and superimposing them on the real thing. The same kind of process could work for digital scans of a patient during an operation.

Using virtual reality, patients and doctors could also do remote consultations more easily, allowing patients to describe visually what is happening with them, and giving health professionals the ability to offer tips and direct recommendations in a visual way. 

This would not only help with providing care to people who live in remote areas, but could also help when there is a language barrier between doctor and patient. 

Impacting industry worldwide

One technology consulting firm writes that using a Vision Pro or other VR headset to streamline assembly and quality control in maintenance tasks. Overlaying diagrams, 3D models, and other digital information onto an object in real time could enable “more efficient and error-free assembly processes,” by providing visual cues, step-by-step guidance, and real-time feedback. 

In addition to these kinds of uses, virtual reality could also be used for remote onboarding for new staff in a variety of different roles, by allowing them to move around and practice training tasks in a virtual environment.

Some technology watchers believe that the retail industry could be transformed by virtual reality as well. Millions of consumers have become used to buying online, but some categories such as clothing and furniture have lagged, in part because it is difficult to tell what a piece of clothing might look like once you are wearing it, or what that chair will look like in your home. But VR promises the kind of immersive experience where that becomes possible.

While many consumers may see this technology only as an avenue for gaming and entertainment, it’s already being leveraged by businesses in manufacturing, health care and workforce development. Even in 2020, 91 per cent of businesses surveyed by TechRepublic either used or planned to adopt VR or AR technology — and as these technological advances continue, adoption is likely to keep ramping up.

Share this:
Continue Reading

Business

5 tips for brainstorming with ChatGPT

How to avoid inaccuracy and leverage the full creative reign of ChatGPT

Published

on

Share this:

ChatGPT recruited a staggering 100 million users by January 2023. As software with one of the fastest-growing user bases, we imagine even higher numbers this year. 

It’s not hard to see why. 

Amazon sellers use it to optimize product listings that bring in more sales. Programmers use it to write code. Writers use it to get their creative juices flowing. 

And occasionally, a lawyer might use it to prepare a court filing, only to fail miserably when the judge notices numerous fake cases and citations. 

Which brings us to the fact that ChatGPT was never infallible. It’s best used as a brainstorming tool with a skeptical lens on every output. 

Here are five tips for how businesses can avoid inaccuracy and leverage the full creative reign of generative AI when brainstorming.

  1. Use it as a base

Hootsuite’s marketing VP Billy Jones talked about using ChatGPT as a jumping-off point for his marketing strategy. He shares an example of how he used it to create audience personas for his advertising tactics. 

Would he ask ChatGPT to create audience personas for Hootsuite’s products? Nope, that would present too many gaps where the platform could plug in false assumptions. Instead, Jones asks for demographic data on social media managers in the US — a request easy enough for ChatGPT to gather data on. From there he pairs the output with his own research to create audience personas. 

  1. Ask open-ended questions

You don’t need ChatGPT to tell you yes or no — even if you learn something new, that doesn’t really get your creative juices flowing. Consider the difference: 

  • Does history repeat itself? 
  • What are some examples of history repeating itself in politics in the last decade?

Open-ended questions give you much more opportunity to get inspired and ask questions you may not have thought of. 

  1. Edit your questions as you go

ChatGPT has a wealth of data at its virtual fingertips to examine and interpret before spitting out an answer. Meaning you can narrow down the data for a more focused response with multiple prompts that further tweak its answers. 

For example, you might ask ChatGPT about book recommendations for your book club. Once you get an answer, you could narrow it down by adding another requirement, like specific years of release, topic categories, or mentions by reputable reviewers. Adding context to what you’re looking for will give more nuanced answers.

  1. Gain inspiration from past success

Have an idea you’re unsure about? Ask ChatGPT about successes with a particular strategy or within a particular industry. 

The platform can scour through endless news releases, reports, statistics, and content to find you relatable cases all over the world. Adding the word “adapt” into a prompt can help utilize strategies that have worked in the past and apply them to your question. 

As an example, the prompt, “Adapt sales techniques to effectively navigate virtual selling environments,” can generate new solutions by pulling from how old problems were solved. 

  1. Trust, but verify

You wouldn’t publish the drawing board of a brainstorm session. Similarly, don’t take anything ChatGPT says as truth until you verify it with your own research. 

The University of Waterloo notes that blending curiosity and critical thinking with ChatGPT can help to think through ideas and new angles. But, once the brainstorming is done, it’s time to turn to real research for confirmation.

Share this:
Continue Reading

Featured