Abstract:Knowledge graph embedding research has mainly focused on learning continuous representations of entities and relations tailored towards the link prediction problem. Recent results indicate an ever increasing predictive ability of current approaches on benchmark datasets. However, this effectiveness often comes with the cost of over-parameterization and increased computationally complexity. The former induces extensive hyperparameter optimization to mitigate malicious overfitting. The latter magnifies the importance of winning the hardware lottery. Here, we investigate a remedy for the first problem. We propose a technique based on Kronecker decomposition to reduce the number of parameters in a knowledge graph embedding model, while retaining its expressiveness. Through Kronecker decomposition, large embedding matrices are split into smaller embedding matrices during the training process. Hence, embeddings of knowledge graphs are not plainly retrieved but reconstructed on the fly. The decomposition ensures that elementwise interactions between three embedding vectors are extended with interactions within each embedding vector. This implicitly reduces redundancy in embedding vectors and encourages feature reuse. To quantify the impact of applying Kronecker decomposition on embedding matrices, we conduct a series of experiments on benchmark datasets. Our experiments suggest that applying Kronecker decomposition on embedding matrices leads to an improved parameter efficiency on all benchmark datasets. Moreover, empirical evidence suggests that reconstructed embeddings entail robustness against noise in the input knowledge graph. To foster reproducible research, we provide an open-source implementation of our approach, including training and evaluation scripts as well as pre-trained models in our knowledge graph embedding framework (<a class="link-external link-https" href="https://github.com/dice-group/dice-embeddings" rel="external noopener nofollow">this https URL</a>).

Machine-Learning Kronecker Coefficients

Data-scientific study of Kronecker coefficients

All Kronecker coefficients are reduced Kronecker coefficients

Dominant Z-Eigenpairs of Tensor Kronecker Products are Decoupled and Applications to Higher-Order Graph Matching

Kronecker Factorization for Speeding Up Kernel Machines.

Predicting root numbers with neural networks

Predicting the cardinality and maximum degree of a reduced Gröbner basis

Kronecker Coefficients For (Dual) Symmetric Inverse Semigroups

Combinatorics on several families of Kronecker coefficients related to plane partitions

Fast Kronecker product kernel methods via generalized vec trick

On the Kronecker product of Schur functions of square shapes

Rational Maximum Likelihood Estimators of Kronecker Covariance Matrices

On the growth of the Kronecker coefficients

Machine learning for moduli space of genus two curves and an application to isogeny based cryptography

A New Method of Kronecker Product Decomposition

Covariance Estimation in High Dimensions Via Kronecker Product Expansions

Kronecker Decomposition for Knowledge Graph Embeddings

On the Expressive Power of Kernel Methods and the Efficiency of Kernel Learning by Association Schemes

Some unimodal sequences of Kronecker coefficients

The Kronecker-clique Model for Higher-Order Clustering Coefficients

Analysis and Approximate Inference of Large Random Kronecker Graphs