(Bloomberg) -- Paul Allen's artificial intelligence institute is putting together a coalition including Microsoft Corp., Google, Baidu Inc. and the Gates Foundation to share technology and ideas to help scientific researchers and academics find and take advantage of the latest discoveries and information.
Called the Open Academic Search project, the goal is to aid researchers by having the companies, institutes and nonprofits involved make their AI and analysis tools open-source, or freely available to other groups to use and tweak. The project seeks to empower researchers, doctors and professors to use the latest discoveries amid a sea of new work and data that's being created too rapidly for anyone to keep track.
The Allen Institute for Artificial Intelligence is trying to make the advancement of academic search and analysis a collaborative process, rather than a competitive one. Companies already freely distribute some of their tools and let others use them, but it's still not as open and cooperative as it could be, said Oren Etzioni, chief executive officer of the Seattle-based institute.
"For us it's a call to action to say, `hey, you've got nothing to lose and everything to gain, put some of these building blocks in open source and let's work on it together and it's a win-win,'" Etzioni said.
New papers, discoveries and research are being generated at such a torrid clip -- one 2014 study found that scientific output doubles every nine years -- that it's become impossible for doctors, scientists and people in various fields to keep up, let alone to apply those new findings to their own work. Teaming up and opening the groups' separate AI and analysis technologies to be used together and in novel ways is an important step toward using these advances to realize the full potential of academic research, because the organizations and companies involved are each working on different pieces of the AI puzzle.
"For scientific research we are running into a situation where we are actually losing knowledge every day," said Kuansan Wang, a managing director for outreach innovation at Microsoft Research, who is on the advisory committee for the new group. "We have so many publications, venues and discoveries and all they are coming online fast and furious and the capability for any single human to be able to process all these new discoveries has been challenging."
For its part, the Allen Institute, created by the Microsoft co-founder, has something called Semantic Scholar, which uses artificial intelligence techniques to find the most relevant papers on a topic or find the most important citations in a paper. It is training AI-based systems to read papers and learn from them so the systems can provide an overview of a topic or sort through the most important citations, rather than having a human having to do all that analysis. Microsoft Academic, meanwhile, is training its machine systems to be "well-read" in 50,000 different fields and is working on a project with Tsinghua University in China to help keep track of which authors are which, even when they have the same name -- a common challenge for researchers with Asian names, Wang said.
Google Scholar has been around since 2004, providing a way to find papers, and Baidu Scholar is the most widely used academic search engine in China, with more than 3 billion searches a year. Baidu Scholar Product Manager Bing Cao and Google Developer Advocate Dan Brickley will also serve on the new group's advisory committee, along with professors from the University of Washington and the University of California at Berkeley. For now, many of the open-source resources contributed to the project come from material that has already been available publicly from each of the companies and organizations involved, but organizers expect material to be added in the future.
As part of the project, Baidu later this year will open up some Baidu Scholar metadata and user-behavior data for interested parties to study and implement, and will promote the project as part of Baidu's other partnerships in China in the hope of bringing more research institutions and publishers on board, Cao said.
The project is important tor the Bill and Melinda Gates Foundation because the nonprofit has a policy of providing open access to the reams of data generated by its studies on world health. These kinds of tools will aid both foundation researchers and anyone else who wants easier access to the information.
The Allen Institute and the companies involved here have much more experience in "tools for sifting through large volumes of data and text in ways we don't do at the foundation and, frankly, don't have the expertise," said Richard Wilder, associate general counsel at the foundation and one of the lawyers who worked on its open-access policy.