In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the second matrix. The resulting matrix, known as the matrix product, has the number of rows of the first and the number of columns of the second matrix. The product of matrices A and B is denoted as AB.[1]
Matrix multiplication was first described by the French mathematician Jacques Philippe Marie Binet in 1812,[2] to represent the composition of linear maps that are represented by matrices. Matrix multiplication is thus a basic tool of linear algebra, and as such has numerous applications in many areas of mathematics, as well as in applied mathematics, statistics, physics, economics, and engineering.[3][4]Computing matrix products is a central operation in all computational applications of linear algebra.
Strassens Matrix Multiplication 4x4 Example Pdf Download
Download Zip: https://urlgoal.com/2vFApw
In most scenarios, the entries are numbers, but they may be any kind of mathematical objects for which an addition and a multiplication are defined, that are associative, and such that the addition is commutative, and the multiplication is distributive with respect to the addition. In particular, the entries may be matrices themselves (see block matrix).
Historically, matrix multiplication has been introduced for facilitating and clarifying computations in linear algebra. This strong relationship between matrix multiplication and linear algebra remains fundamental in all mathematics, as well as in physics, chemistry, engineering and computer science.
Matrix multiplication shares some properties with usual multiplication. However, matrix multiplication is not defined if the number of columns of the first factor differs from the number of rows of the second factor, and it is non-commutative,[10] even when the product remains definite after changing the order of the factors.[11][12]
For example, if A, B and C are matrices of respective sizes 1030, 305, 560, computing (AB)C needs 10305 + 10560 = 4,500 multiplications, while computing A(BC) needs 30560 + 103060 = 27,000 multiplications.
The definition of matrix product requires that the entries belong to a semiring, and does not require multiplication of elements of the semiring to be commutative. In many applications, the matrix elements belong to a field, although the tropical semiring is also a common choice for graph shortest path problems.[13] Even in the case of matrices over fields, the product is not commutative in general, although it is associative and is distributive over matrix addition. The identity matrices (which are the square matrices whose entries are zero outside of the main diagonal and 1 on the main diagonal) are identity elements of the matrix product. It follows that the n n matrices over a ring form a ring, which is noncommutative except if n = 1 and the ground ring is commutative.
A square matrix may have a multiplicative inverse, called an inverse matrix. In the common case where the entries belong to a commutative ring R, a matrix has an inverse if and only if its determinant has a multiplicative inverse in R. The determinant of a product of square matrices is the product of the determinants of the factors. The n n matrices that have an inverse form a group under matrix multiplication, the subgroups of which are called matrix groups. Many classical groups (including all finite groups) are isomorphic to matrix groups; this is the starting point of the theory of group representations.
Since matrix multiplication forms the basis for many algorithms, and many operations on matrices even have the same complexity as matrix multiplication (up to a multiplicative constant), the computational complexity of matrix multiplication appears throughout numerical linear algebra and theoretical computer science.
In our paper, published today in Nature, we introduce AlphaTensor, the first artificial intelligence (AI) system for discovering novel, efficient, and provably correct algorithms for fundamental tasks such as matrix multiplication. This sheds light on a 50-year-old open question in mathematics about finding the fastest way to multiply two matrices.
This operation is used for processing images on smartphones, recognising speech commands, generating graphics for computer games, running simulations to predict the weather, compressing data and videos for sharing on the internet, and so much more. Companies around the world spend large amounts of time and money developing computing hardware to efficiently multiply matrices. So, even minor improvements to the efficiency of matrix multiplication can have a widespread impact.
For centuries, mathematicians believed that the standard matrix multiplication algorithm was the best one could achieve in terms of efficiency. But in 1969, German mathematician Volker Strassen shocked the mathematical community by showing that better algorithms do exist.
In our paper, we explored how modern AI techniques could advance the automatic discovery of new matrix multiplication algorithms. Building on the progress of human intuition, AlphaTensor discovered algorithms that are more efficient than the state of the art for many matrix sizes. Our AI-designed algorithms outperform human-designed ones, which is a major step forward in the field of algorithmic discovery.
First, we converted the problem of finding efficient algorithms for matrix multiplication into a single-player game. In this game, the board is a three-dimensional tensor (array of numbers), capturing how far from correct the current algorithm is. Through a set of allowed moves, corresponding to algorithm instructions, the player attempts to modify the tensor and zero out its entries. When the player manages to do so, this results in a provably correct matrix multiplication algorithm for any pair of matrices, and its efficiency is captured by the number of steps taken to zero out the tensor.
For example, if the traditional algorithm taught in school multiplies a 4x5 by 5x5 matrix using 100 multiplications, and this number was reduced to 80 with human ingenuity, AlphaTensor has found algorithms that do the same operation using just 76 multiplications.
From a mathematical standpoint, our results can guide further research in complexity theory, which aims to determine the fastest algorithms for solving computational problems. By exploring the space of possible algorithms in a more effective way than previous approaches, AlphaTensor helps advance our understanding of the richness of matrix multiplication algorithms. Understanding this space may unlock new results for helping determine the asymptotic complexity of matrix multiplication, one of the most fundamental open problems in computer science.
With AlphaTensor, DeepMind Technologies has presented an AI system that is supposed to independently find novel, efficient and provably correct algorithms for complex mathematical tasks. AlphaTensor has already identified a new algorithm with which matrix multiplications can be carried out faster than before, as the research team explains in a paper published in the magazine Nature. The team gives the factor as a ten to twenty percent acceleration compared to previous standard methods.
The team had translated the problem of finding useful algorithms for matrix multiplication into a single-player game. In the game, the "board" is a three-dimensional tensor (number range) that records how far an algorithm is from the correct result in each case. The player tries to modify the tensor so that the entries in it are "zeroed out". The possible moves are fixed and correspond to algorithmic instructions. If the player makes a good play, the result is a provably correct algorithm for matrix multiplication (for any pair of matrices). The efficiency is measured by the number of steps that were necessary to lead the tensor to zero.
According to the DeepMind team, the results presented hold out the prospect of a significant acceleration of numerous computing operations in computer science and AI, at least according to the authors of the blog post and the technical article. Above all, they are initially intended to be a contribution to complexity research. It should be noted that the paper deals specifically with matrix multiplication, a simple algebra operation that underlies many processes in IT and the digital world.
Markus Bläser is Professor of Computational Complexity at Saarland University in Saarbrücken, Germany, and is considered an expert in algebraic algorithms and complexity. According to him, the theoretical understanding of the complexity of matrix multiplication as well as the development of fast, practical algorithms is of great interest, since matrix multiplication serves as the basis of numerous operations and is widely used in applied mathematics, computer science, but also in engineering. According to Bläser, the present work of the DeepMind team finds some new upper bounds for the multiplication of small matrices using machine learning methods. This contributes to the theoretical understanding of matrix multiplication.
Holger Hoos, Professor of Machine Learning at Leiden University and Chair of AI at RWTH Aachen University, also describes matrix multiplication as a fundamental operation for computer science applications. A practically relevant acceleration of matrix multiplication would therefore be "of considerable importance". In this context, he considers the Deep Reinforcement Learning method used by DeepMind to be new and interesting. Overall, however, he dampens the euphoria about the results of the study, which in his eyes are "easy to overestimate".
The automatic design of algorithms based on machine learning is not an entirely new development. The approach has been researched for more than ten years and has already led to new algorithms - among others for the "notoriously difficult Boolean satisfiability problem" in propositional logic, which is studied intensively in computer science. This problem has important applications in the verification and testing of hardware and software. Hoos knows other examples from the design of algorithms for solving mathematical optimisation questions from industry and science; mixed integer programming comes to his mind as a minefield. The application of automatic algorithm design to matrix multiplication is new, and the methodological approach seems promising. However, Hoos sees "no signs yet of a breakthrough in the field of automatic algorithm design". A practical distinction is probably significant here: research into improving matrix multiplication algorithms has so far concentrated primarily on methods that yield mathematically provable accelerations for arbitrarily large matrices. The algorithms found in this way are almost insignificant in practice, since their advantages "only become apparent for astronomically large matrices", as Hoos objects. 2ff7e9595c
Comments