It has finally been revealed how AI-hidden thoughts in deep neural networks make decisions thanks to a groundbreaking research method developed by scientists.
How data is categorized into groups makes it more reliable and safer for real-world applications like healthcare and self-driving cars. It also brings us closer to understanding artificial intelligence in the proper context.
Understanding AI processing steps.
Deep neural networks mimic how the human brain processes information. Despite this, it has been a longstanding challenge to understand how these networks make decisions.
To understand this, researchers at Kyushu University have developed a new method to interpret how deep computational networks classify and organize data into categories.IEEE Transactions on Neural Networks and Learning Systems published their findings to improve AI’s accuracy, reliability, and safety.
Humans solve puzzles layer by layer, just as deep neural networks do. The input layer is the first layer where data is collected. Next, hidden layers analyze the data in stages. Edges and textures are detected as simple features. Early hidden layers identify simple features, such as edges or textures, similar to identifying individual puzzle pieces. A deeper layer combines these features to recognise more complex patterns, such as distinguishing between a dog and a cat.
Transparency in AI Decision-Making
We can see what’s happening at the surface of these hidden layers but can’t see what’s happening inside,” says Danilo Vasconcellos Vargas, an Associate Professor at Kyushu University. Whenever AI makes mistakes, even something as small as pixel changes can trigger a serious problem. To ensure AI is trustworthy, it’s essential to understand how it makes its decisions.”
Limitations of Current Visualization Methods
To visualize AI’s information organization, methods rely on simplifying high-dimensional data into 2D or 3D images. These methods allow researchers to observe how AI categorizes data points – for example, grouping cats with other cats and separating them from dogs. This simplification, however, has critical limitations.
We lose important details and fail to see the whole picture when simplifying high-dimensional information into fewer dimensions. In addition, this method of visualizing how the data is grouped makes it challenging to compare different neural networks or data classes,” says Vargas.
Introducing the k* Distribution Method
This study developed a new method that more clearly visualizes and assesses how well deep neural networks categorize related items.
As part of the model, each inputted data point receives its own “k* value,” which indicates the distance to the nearest unrelated data point. It suggests a well-separated data point (such as one far from any other dogs), while a low value suggests possible overlap (such as one in which dogs are nearby). As a result of looking at all the data points within a class, such as cats, this approach provides a detailed breakdown of how the data is organized.
As a result of our method, the higher-dimensional space is retained, so no information is lost. According to Vargas, the first and only model can accurately depict the ‘local neighbourhood’ around each data point.
Impact and Applications of the New Method
The researchers found deep neural networks sort data into clusters, fractures, or overlaps. An AI can sort the data well when similar items (e.g., cats) are grouped closely while unrelated items (e.g., dogs) are separated. Fractured arrangements, however, indicate that similar items are scattered over a large area. Moreover, overlapping distributions occur when unrelated items are placed in the same space, making classification errors more likely.
A well-organized warehouse stores similar items together, making retrieval more manageable and more efficient. When mixed items are harder to find, increasing the risk of selecting the wrong one.”
AI in Critical Systems and the Future
Medical diagnostics and autonomous vehicles, which require accuracy and reliability, increasingly rely on artificial intelligence. Researchers and lawmakers alike can use it to identify potential weaknesses or errors in AI’s organization and classification of information.
Besides supporting legalization processes, this provides valuable insights into how AI “thinks.” Researchers can refine AI systems by identifying the root causes of errors to make them more accurate and robust to handle blurry or incomplete data and adapt to unexpected situations.