非営利団体がClouderaとAIでデータを変革する方法

4 Min ReadWhen Brian Martin co-founded Rare Hopes NFP, a nonprofit focused on giving the public access to hypotheses for rare disease treatment, the organization needed a way to fulfill its purpose despite lacking the millions of dollars and resources of big pharmaceutical companies."For any nonprofit to be able to do this type of thing is generally an unreasonable proposition," Martin said in an interview at the Gartner Data & Analytics Summit in Orlando last week. He noted that the well-known nonprofit Every Cure, which seeks to use FDA-approved medicines to treat rare diseases, has raised about $76 million in funding, underscoring the significant capital needed for organizations with a similar mission.However, with Martin already having experience with the hybrid data and AI vendor Cloudera, he felt the vendor might be able to help Rare Hopes execute on its mission without the high costs that big pharmaceutical companies incur when releasing such hypotheses on rare diseases to the public. Martin did not disclose the amount Rare Hopes spends on using the Cloudera platform.Related:Databricks to Invest $850M in UK AI Operations"It's an opportunity to do something and to put that type of content in patients' and doctors' hands that we couldn't ever do without millions and millions of dollars," Martin said.The Cloudera EffectOne way Cloudera was instrumental in helping Washington, D.C.-based Rare Hopes fulfill its mission is that the nonprofit used the data and AI platform to gain insight from diverse types of data.With the platform, Rare Hopes was able to extract knowledge from research papers, medical images, and other documentation, identifying correlations and patterns that would have taken years to discover, Martin said. Using Cloudera, Rare Hopes created data pipelines that processed unstructured data, such as scientific papers, and transformed it into structured data. Using a tool in Cloudera called PySpark (for building data engineering and machine learning pipelines), Rare Hopes can extract knowledge from scientific data, transform that information from unstructured to structured, and then use the transformed data in tools and platforms outside Cloudera or run analysis and find correlations between concepts such as a disease and a drug. Rare Hopes brings the hypothesis back into the Cloudera platform and continues to conduct further studies. In that case, Rare Hopes uses a large language model (LLM) to generate an analysis or hypothesis that the organization will present to the public."That data information knowledge, insight, wisdom and impact chain, that's a pretty well-established hierarchy," Martin said. "We use Cloudera to automate that base part, that human axis, that wisdom link, to deliver the impact."Related:Nvidia Aims to Bolster HPC With AcquisitionCloudera and ModelsAs for generative AI models, Rare Hopes is not committed to any specific model.For its part, Cloudera does not require its customers to use a specific model. However, the vendor has integrated Nvidia NIM microservices into its infrastructure, enabling it to deploy and manage LLMs. Nvidia NIM microservices is a suite of prebuilt, packaged containers that include an AI model, inference engines, standard APIs, and other tools enterprises need to deploy AI models."Cloudera doesn't make a model and sell it to you," said David Dichmann, vice president of product marketing and evangelism at Cloudera. "Choose your model, choose your model well, and we recognize you want freedom of choice. Use the right model for the right use case. Do not try to fit everything into one kind of model."Rare Hopes also recognizes that because different models work better for different tasks and applications, it is important to have access to a range of models. Model choice in Cloudera is an added benefit to the nonprofit, Martin said. The nonprofit does not have to build the infrastructure to access the models, provide them with data, and then bring the results back into the Cloudera platform. Related:In AI Play, IBM Acquires Data Streaming Provider Confluent"The Nvidia NIM infrastructure gives us the ability to run some of that stuff directly natively," Martin said.While Cloudera already helps Rare Hopes save a significant amount of time by helping deliver different hypotheses on various diseases to the public by publishing its research and white paper findings, the nonprofit is now looking at how to monitor changes to the data when a new research paper is published."How do we handle different change events within those pipelines to know what the different downstream effects are?" Martin said. "Those types of things save an immense amount of time because instead of rerunning the entire process over again every time there's new data, we can run incremental processes to analyze the changes and the differences."About the AuthorNews Writer, AI BusinessEsther Shittu brings four years of expertise covering artificial intelligence technologies and industry trends. As co-host of the "Targeting AI" podcast, she talks to thought leaders and practitioners exploring critical AI developments. Previous to AI Business, she wrote for several publications including the New York Daily News, Bklyner and the Brooklyn Daily Eagle. When she's not diving deep into the world of AI, she spends her time on passion projects and raising her three daughters.