ReLU-DCT and LASP: Real-valued frequency-domain activations and adaptive pooling for DCT-Based CNNs

Alshareef, Ibrahim Yousef, Rahman, Ab Al-Hadi Ab, Shaameri, Ahmad Zuri, Khan, Nuzhat, Rusli, Mohd Shahrizal, Hassan, Mohamed Khalafalla, Manzak, Ali and Marsono, Muhammad Nadzir (2026) ReLU-DCT and LASP: Real-valued frequency-domain activations and adaptive pooling for DCT-Based CNNs. Knowledge-Based Systems, 343. ISSN 1872-7409

Full text not available from this repository.

Abstract

Convolutional Neural Networks (CNNs) operating in the frequency-domain have shown promise for enhanced computational efficiency and improved signal interpretability. However, existing spectral CNNs frequently rely on complex valued transforms and lack effective formulations for nonlinear activation and pooling within the spectral space. This paper introduces a fully real-valued Discrete Cosine Transform (DCT)-based CNN framework that incorporates two key innovations: (1) ReLU-DCT, a DC-preserving activation function (AF) specifically designed for the DCT domain, and (2) Layer Adaptive Spectral Pooling (LASP), a lightweight mechanism that dynamically regulates frequency retention across network depth. Together, these components preserve low frequency semantics, mitigate spectral distortion, and stabilize learning dynamics in real valued spectral CNNs. The proposed approach is implemented on LeNet-5 and VGG7 architectures and evaluated using the MNIST and a custom 94-class ASCII character dataset. The proposed method achieves up to 98.44% accuracy, representing a 3.23% point improvement over the baseline DCT model (95.21%), achieving near-parity with spatial CNNs while significantly reducing computational and memory costs through real-valued spectral processing. The primary contribution lies in recovering accuracy losses without sacrificing the efficiency advantages of spectral processing. The current evaluation focuses on grayscale datasets (MNIST and ASCII-94). Extending the framework to more complex and color datasets such as CIFAR-10 and Tiny ImageNet is identified as important future work.

Item Type:	Article
Uncontrolled Keywords:	Spectral convolutional neural networks; Discrete cosine transform; Activation function design; Layer adaptive spectral pooling; Frequency domain deep learning
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Staff Research and Publications
Depositing User:	Tamara Malone
Date Deposited:	06 May 2026 11:49
Last Modified:	06 May 2026 11:49
URI:	https://norma.ncirl.ie/id/eprint/9298

Actions (login required)

View Item