Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Michael Galkin
Foundation Models in Graph & Geometric Deep Learning Original
Foundation Models in language, vision, and audio have been among the primary research topics in Machine Learning in 2024 whereas FMs for graph-structured data have somewhat lagged behind. In this post, we argue that the era of Graph FMs has already begun and provide a few examples of how one can use them already today. This post was written and edited by Michael Galkin and Michael Bronstein with significant contributions from Jianan Zhao, Haitao Mao, Zhaocheng Zhu.
Graph & Geometric ML in 2024: Where We Are and What’s Next (Part II - Applications) Original
State-of-the-Art Digest Following the tradition from previous years, we interviewed a cohort of distinguished and prolific academic and industrial experts in an attempt to summarise the highlights of the past year and predict what is in store for 2024. Past 2023 was so ripe with results that we had to break this post into two parts. This is Part II focusing on applications, see also Part I for theory & new architectures.
Graph & Geometric ML in 2024: Where We Are and What’s Next (Part I - Theory & Architectures) Original
State-of-the-Art Digest Following the tradition from previous years, we interviewed a cohort of distinguished and prolific academic and industrial experts in an attempt to summarise the highlights of the past year and predict what is in store for 2024. Past 2023 was so ripe with results that we had to break this post into two parts. This is Part I focusing on theory & new architectures, see also Part II on applications.
Foundation Models for Knowledge Graph Reasoning
Let's define the setup more formally: KGs are directed, multi-relational graphs with arbitrary sets of nodes and relation types Graphs arrive without features, that is, we don't assume the existence of textual descriptions (nor pre-computed feature vectors) of entities and relations. Given a query (head, relation, ?), we want to rank all nodes in the underlying graph (inference graph) and maximize the probability of returning a true tail.
ULTRA: Foundation Models for Knowledge Graph Reasoning
What’s new in Graph ML? One model to rule them all Training a single generic model for solving arbitrary datasets is always a dream for ML researchers, especially in the era of foundation models. While such dreams have been realized in perception domains like images or natural languages, whether they can be reproduced in reasoning domains (like graphs) remains an open challenge. In this blog post, we prove such a generic reasoning model exists, at least for knowledge graphs (KGs).
Graph Machine Learning @ ICML 2023
Whatâs new in Graph ML? Recent advancements and hot trends, August 2023 edition Magnificent beaches and tropical Hawaiian landscapes đ´did not turn brave scientists away from attending the International Conference on Machine Learning in Honolulu and presenting their recent work! Letâs see whatâs new in our favorite Graph Machine Learning area. Thanks Santiago Miret for proofreading the post.
Graph Machine Learning @ ICML 2023
CLIP and its descendants have become a standard staple in text-to-image models. Can we do the same but for text-to-protein? Yes! ➡️ Xu, Yuan, et al present ProtST, a framework for learning joint representations of text protein descriptions (via PubMedBERT) and protein sequences (via ESM).
Neural Graph Databases
The literature on neural methods for complex logical query answering (aka, query embedding) has been growing since 2018 and the seminal NeurIPS work of Hamilton et al on Graph Query Embedding (GQE). GQE was able to answer conjunctive queries with intersections and predict missing links on the fly. GQE started the whole subfield of Graph Machine Learning followed by some prominent examples like Query2Box (ICLR 2020) and Continuous Query Decomposition (ICLR 2021).
Neural Graph Databases
What's New in Graph ML? We introduce the concept of Neural Graph Databases as the next step in the evolution of graph databases. Tailored for large incomplete graphs and on-the-fly inference of missing edges using graph representation learning, neural reasoning maintains high expressiveness and supports complex logical queries similar to standard graph query languages. Neural Graph Databases: What and Why?
Neural Graph Databases
What's New in Graph ML? We introduce the concept of Neural Graph Databases as the next step in the evolution of graph databases. Tailored for large incomplete graphs and on-the-fly inference of missing edges using graph representation learning, neural reasoning maintains high expressiveness and supports complex logical queries similar to standard graph query languages. Neural Graph Databases: What and Why?
Graph ML in 2023: The State of Affairs
Last year, we observed a major shift in KG representation learning: transductive-only approaches are being actively retired in favor of inductive models that can build meaningful representation for new, unseen nodes and perform node classification and link prediction. In 2022, the field was expanding along two main axes: 1ļøā£ inductive link prediction (LP) 2ļøā£ and inductive (multi-hop) query answering that extends link prediction to much more complex prediction tasks.
The State of Affairs
STATE OF THE ART DIGEST 2022 comes to an end and it is about time to sit down and reflect upon the achievements made in Graph ML as well as to hypothesize about possible breakthroughs in 2023. Tune in 🎄☕ The article is written together with (Stanford University), Zhaocheng Zhu (Mila & University of Montreal). We thank Christopher Morris and Johannes Brandstetter for the feedback and helping with the Theory and PDE sections, respectively.
Denoising Diffusion Generative Models in Graph ML
Let’s recapitulate the basics of diffusion models using the example of the Equivariant Diffusion paper by Hoogeboom et al using as few equations as possible 😅 Input: a graph (N,E) with N nodes and E edges Node features often have two parts: z=[x,h] where x ∈ R³ are 3D coordinates and h ∈ R^d are categorical features like atom types (Optional) Edge features are bond types Output: a graph (N,E) with nodes, edges, and corresponding features Forward diffusion process q(z_t | x,h): at each time...
Denoising Diffusion Generative Models in Graph ML
What's new in Graph ML? The breakthrough in Denoising Diffusion Probabilistic Models (DDPM) happened about 2 years ago. Since then, we observe dramatic improvements in generation tasks: GLIDE, DALL-E 2, Imagen, Stable Diffusion for images, Diffusion-LM in language modeling, diffusion for video sequences, and even diffusion for reinforcement learning.
Graph Machine Learning @ ICML 2022
Knowledge graph reasoning has long been a playground for GraphML methods. In this year’s ICML, there are quite a few interesting papers on this topic. As a trend of this year, we see a significant drift from embedding methods (TransE, ComplEx, RotatE, HAKE) to GNNs and logic rules (in fact, GNNs are also related to logic rules). There are four papers based on GNNs or logic rules, and two papers extending the conventional embedding methods.
Graph Machine Learning @ ICML 2022
What's New in GraphML? International Conference on Machine Learning (ICML) is one of the premier venues where researchers publish their best work. ICML 2022 was packed with hundreds of papers and numerous workshops dedicated to graphs. We share the overview of the hottest research areas 🔥 in Graph ML.
GraphGPS: Navigating Graph Transformers
Structural and positional features aim at encoding a unique characteristic of each node or edge. In the most basic case (illustrated below), when all nodes have the same initial features or no features at all, applying positional and structural features helps to distinguish nodes in a graph, assign them with diverse features, and provide at least some sense of graph structure.
Navigating Graph Transformers
Recent Advances in Graph ML In 2021, graph transformers (GT) won recent molecular property prediction challenges thanks to alleviating many issues pertaining to vanilla message passing GNNs. Here, we try to organize numerous freshly developed GT models into a single GraphGPS framework to enable general, powerful, and scalable graph transformers with linear complexity for all types of Graph ML tasks.
Inductive Link Prediction in Knowledge Graphs
Starting a new Inductive Link Prediction Challenge 2022 Since very 2011, the area of representation learning over Knowledge Graphs has been dominated by one task: transductive link prediction. Is it still relevant in 2022? 🤔 Rather not. Transductive setup: training and inference is performed on the same graph. Inductive: inference is on the new graph. Colored arrows represent different edge types (relations). Question marks denote edges to predict.
Graph ML in 2022: Where Are We Now?
Thoughts and Theory, State of the Art Digest It’s been quite a year for Graph ML — thousands of papers, numerous conferences and workshops… How do we catch up with so many cool things happening around? Well, we are puzzled as well and decided to present a structured look at Graph ML highlighting 🔥 trends and major advancements. The image was generated by ruDALL-E with a prompt “graphs floating in space”.
Graph ML in 2022: Where Are We Now?
Thoughts and Theory, State of the Art Digest It’s been quite a year for Graph ML — thousands of papers, numerous conferences and workshops… How do we catch up with so many cool things happening around? Well, we are puzzled as well and decided to present a structured look at Graph ML highlighting 🔥 trends and major advancements. The image was generated by ruDALL-E with a prompt “graphs floating in space”.
Graph ML in 2022: Where Are We Now?
Thoughts and Theory, State of the Art Digest It’s been quite a year for Graph ML — thousands of papers, numerous conferences and workshops… How do we catch up with so many cool things happening around? Well, we are puzzled as well and decided to present a structured look at Graph ML highlighting 🔥 trends and major advancements. The image was generated by ruDALL-E with a prompt “graphs floating in space”.
Knowledge Graphs in Natural Language Processing @ ACL 2021
State Of The Art, Summer’21 Welcome to the third iteration of our regular overview of NLP papers around Knowledge Graphs, this time published at ACL 2021! What will be (or has been) trending this year that you wouldn’t want to miss? 👀 Background photo by Milkovi on Unsplash, adapted by Author Neural retrieval continues to be one of the fastest-growing and hottest 🔥 topics in NLP: it now works with billions of vectors and indices of the scale 100+ GB.
NodePiece: Tokenizing Knowledge Graphs
Compositionality & Graphs Mapping each node to an embedding vector results in enormously large embedding matrices. Is there a way to have a fixed-size vocabulary of tokens akin to “subword units”? This blog post was co-authored with Etienne Denis and Paul Wu and is based on our paper “NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs”. Language models have a fixed-size vocabulary and expressive encoder.
Machine Learning on Knowledge Graphs @ NeurIPS 2020
What’s new for KGs in Graph ML? Your guide to the KG-related research in NLP, December edition NeurIPS is a major venue covering a wide range of ML & AI topics. Of course, there is something interesting for Graph ML aficionados and knowledge graph connoisseurs 🧐. Tune in to find out! This year, NeurIPS had 1900 accepted papers 😳 and 100+ among them are on graphs. Plus take into account several prominent workshops like KR2ML, DiffGeo4ML, and LMCA.
Machine Learning and Its Impact on Different Industries
Machine Learning raises many questions: is it a complicated word to describe the same computer programs that we have been using for decades? Or is it a mystical computer capable of learning everything? More importantly, why is this of interest to your business? Light on this small revolution at the crossroads of big data and artificial intelligence The most concrete way to define machine learning is to compare it with traditional computer programming.
Representation Learning on RDF* and LPG Knowledge Graphs
GNNs Beyond Triples Hyper-relational KGs encode much more knowledge than triple KGs. We adopt recent advances in Graph ML and propose a GNN encoder for such graphs along with a new benchmark. Knowledge graphs (KGs) are a cornerstone of modern NLP and AI applications — recent works include Question Answering, Entity & Relation Linking, Language Modeling, Information Extraction, and even playing text RPGs with Reinforcement Learning.
Knowledge Graphs in Natural Language Processing @ ACL 2020
State of the Art Mid 2020 This post commemorates the first anniversary of the series where we examine advancements in NLP and Graph ML powered by knowledge graphs! 🎂 1️⃣ The feedback of the audience drives me to continue, so fasten your seatbelts (and maybe brew some ☕️): in this episode, we are looking at the KG-related ACL 2020 proceedings! In this context, questions are sent against structured sources like SPARQL-based KGs or SQL databases (other query languages are not that prominent).
Knowledge Graphs @ AAAI 2020
The first major AI event of 2020 is already here! Hope you had a nice holiday break 🎄, or happy New Year if your scientific calendar starts with a conference (which means NY comes from NYC).
Machine Learning on Graphs @ NeurIPS 2019
If you still had any doubts — it’s time to admit. Machine Learning on Graphs becomes a first-class citizen at AI conferences while being not that mysterious as you might have imagined 🧙. Let’s check out the goodies brought by NeurIPS 2019 and co-located events! 2D embedding of a tree in a hyperbolic space.
Knowledge Graphs & NLP @ EMNLP 2019 Part II
Here is the second part of the review of knowledge graph related papers from EMNLP 2019. In this part, we’ll talk about Question Answering over Knowledge Graphs, NLG from KGs, Commonsense reasoning with KGs, and some old school Named Entity/Relation Recognition & Linking. Let’s start 🚀 Question Answering (QA) enjoys a growing traction from the NLP communinty.
Knowledge Graphs & NLP @ EMNLP 2019 Part I
Hey there! 👋 The review post of the papers from ACL 2019 on knowledge graphs (KGs) in NLP was well-received so I thought maybe it would be beneficial for the community to look through the proceedings of EMNLP 2019 for the latest state of the art in applying knowledge graphs in NLP. Let’s start! Source: https://github.com/roomylee/EMNLP-2019-Papers EMNLP 2019 enjoys ever-growing number of submissions and accepted papers.
The Mushroom Effect or Why You Need Knowledge Graphs for Dialog Systems
In the past projects aimed at building infotainment goal-oriented dialog systems we observed a series of unexpectedly funny responses 🍄. Having enjoyed a pun, our investigation led to more serious conclusions. In this article, I’d like to share our findings and discuss some recently delivered goodies from IJCAI 2019 which took place August 10–16th in Macao.
Knowledge Graphs in Natural Language Processing @ ACL 2019
Hello, ACL 2019 has just finished and I attended the whole week of the conference talks, tutorials, and workshops in beautiful Florence! In this post I would like to recap how knowledge graphs slowly but firmly integrate into the NLP community 😉 ACL 2019 was enormous — 2900 submissions, 660 accepted papers, more than 3000 registered attendees, and four workshops with about 400 attendees (well, workshops are bigger than some international CS conferences).
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Michael
Contact Michael, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack