Paul X. McCarthy on networks to find experts, identifying authorities, computational social science, and latent knowledge

“Productivity is ultimately one of the greatest predictors of success in all fields. It doesn’t matter whether you’re an artist or a scientist, or whatever you’re doing, productivity is a key marker to long-term success.“

– Paul X. McCarthy

About Paul X. McCarthy

Paul is CEO of data science and research startup League of Scholars, which works with a wide range of organizations including Nature and News Corporation, and the co-founder of a number of other ventures, He is an Adjunct Professor at U of NSW and Honorary Research Fellow at Western Sydney University, and the author of Online Gravity, a successful book on how technology is rebooting economics.

Website: Paul X. McCarthy

LinkedIn: Paul X. McCarthy

Twitter: Paul X. McCarthy

Facebook: Paul X. McCarthy

Instagram: Paul X. McCarthy

Books

Online Gravity

What you will learn

• How to identify experts or stars in a field of study within their networks (03:34)
• How to ask simple questions to uncover their hidden expertise (06:24)
• How to find an expert in your network that you should be listening to (09:14)
• How to get even more granularity in finding experts (11:00)
• How to identify credible and authoritative sources (15:29)
• To what degree can we infer credibility from an expert’s network centrality (17:49)
• Why purpose leading to clarity and focus is key to thriving on overload  (19:09)
• Why productivity is a key marker to long term success (21:00)
• Why sharing insights from information is crucial to network formation (24:20)

Episode resources

• Albert-László Barabási
• Six Degrees of Separation
• Marshall Kirkpatrick
• Proceedings of the National Academy of Sciences (PNAS)
• Rise And Fall Of Rationality In Language
• Life in the network: the coming age of computational social science
• Evolution of diversity and dominance of companies in online activity
• The Science of Science
• Unsupervised word embeddings capture latent knowledge from materials science literature

Transcript

Ross Dawson: Paul, it’s wonderful to have you on the show.

Paul McCarthy: Thanks, Ross.

Ross: Paul, I’d like you to tell us about the League of Scholars and what the underlying principles are, and how it helps you and others to thrive on overload?

Paul: League of Scholars is a global startup that looks at researchers and research analytics worldwide, and the basis of League of Scholars is that individuals are the key to the success of the research. In recent years, in the last couple of decades, there’s been a global rise in the rankings of universities and other research institutions worldwide. There are now three large global ranking systems, the ShanghaiRanking, the Times Higher Ed, and the QS ranking. All people interested in the university sector are aware of these and very acutely aware of the rankings game between institutions in terms of how they’re perceived in terms of their institution’s reputations.
What we’ve realized is while these rankings are useful, there are a lot of drawbacks to them. They’re not very up to date. Often, they include things like Nobel Prize winners, whose work is 20-25 years plus years ago, and often the rankings don’t change very much each year, so most rankings have hovered at the top and that hasn’t changed significantly in the last couple of decades, the elite list of organizations. What’s not so visible, I guess, is information about individuals. That granular and timely information is about what the League of Scholars is about, about uncovering the individuals in science, engineering, health, but also in other areas, in humanities, in social sciences, trying to understand who are the leaders in these individual more specific fields but also looking at tomorrow’s leaders and the emerging stars.

Ross: What’s the basic principle underlying how it is you identify these stars in these fields?

Paul: We use a variety of traditional bibliometric techniques. For those unfamiliar with the research world, research impact is citations; it’s the number of times that work has been cited by other scholars in peer-reviewed journals and publications. We use those traditional measures of bibliometrics but also predictive measures. We use machine learning to try and understand who is most likely to have the greatest impact in the future, especially for early career and mid-career people. As inputs, there’s a variety of measures that are known to be predictive of future impact. One of those, of course, is your peer network, that idea of who are your co-authors, what are your current co-authors, and how fast the school of fish you’re swimming with now is, is one way to think about it.

Ross: Of course, you can just go on Google Scholar and see the number of citations of a particular scientist from their papers and so on but that’s a pretty crude measure, so how does the network aspect overlay that to identify who’s most well regarded in the field?

Paul: What we do is we look at their co-author networks. There’s a range of network analytics approaches we use to understand the influence of their network, both their direct co-authors and also their co co-authors, so we build this analysis into the inputs to machine learning algorithms, which then go on to predict scientists’ or other academics’ likely future impact.
We’re looking at other things. Citation patterns vary radically between fields and across disciplines so you can’t compare the citation impact of scholars in different fields. You need to compare like with like, so we take that into account too, and also the stage and age of people. The quality of the venues that they’re publishing is important too. Early on in one’s career, there’s not a lot of data so it’s quite difficult to see, just with the untrained eye, to distinguish between people but there are signals in the data. There is information that can be used to predict things like the quality of the venue, the co-authors, how many co-authors are outside your institution, and a bunch of social publishing metrics.

Ross: This idea you’ve mentioned to me of the expert’s expert.

Paul: Yes.

Ross: I’d love to hear about where you’ve come across that idea and how you apply that in both League of Scholars and also more generally how you are keeping across the information.

Paul: Yes. This idea was introduced to me by a colleague, Doris Field Tanner. I think you may know Doris, she’s an expert in network analytics. She explained to me that some of her previous work showed that you can discover an expert in any field by asking a series of simple questions iteratively to your peers. One can do it oneself. In a very simple sense, if you’re looking for information about a restaurant in another city that you’re not familiar with, you might ask someone who lives there. Then they might not be much of a foodie, and you might ask them to ask who would they know in their city that’s a restaurant.
Obviously, it becomes a bit more complicated if you’re looking to understand quantum machine learning, for example, you might think of someone you know who’s a scientist in your field, and then ask them to ask, who do they know in their sphere, who’s the greatest authority in quantum computing and then quantum machine learning and other specialization, a really hot field that’s emerging now. They may know people in their sphere and so on. We know from the work of network scientists like Barabasi and others that the six degrees of separation storied in the Fred Schepisi’s film is very true and is shrinking, so there is a path between us and most other people on the planet, which is quite short, and there’s an easy way of identifying that through intuitive crowdsourcing approach.

Ross: Marshall Kirkpatrick, who has also spoken to us on Thriving on Overload used the expert’s expert frame for his platform Little Bird to identify influences. It’s also interesting to look at your network, social network analysis, one of the classic techniques is the snowball where everybody asks who all should be included and such sort of building out the scope of the group and t