Improving perturbation-based explanation methods with application to face recognitionLu, Yuhang; Ebrahimi, Touradj
doi: 10.1117/12.3030872pmid: N/A
In recent years, significant advancements in face recognition have been witnessed thanks to the rapid development of artificial intelligence. Despite remarkable performance, predictions made by such techniques tend to be challenging to explain. Considering their wide applications in security-sensitive areas, it is essential to fully understand the decision-making process of AI-based face recognition techniques and make them more acceptable to society. Many studies have been dedicated to offering visual interpretation for face recognition systems’ decisions, such as generating similarity and dissimilarity saliency maps. One of the most promising approaches is based on the perturbation mechanism, which has demonstrated exceptional performance in highlighting similar regions between two matching face images. However, this type of method has shown to be less effective in identifying the dissimilar regions, which are particularly critical in the decision-making process for two nonmatching face images. Therefore, this study focuses on the specific problem of the perturbation-based mechanism when applied to the explainable face recognition task. In particular, we first thoroughly analyze the limitation of a perturbation-based method in generating dissimilarity saliency maps. Then, a new regularization technique is proposed to alleviate this problem, followed by experiments to validate its effectiveness.
Neural networks in 3D printing error detectionNasretdinov, Ruslan R.; Tolstoba, Nadezhda D.; Bodrov, Kirill Yu.
doi: 10.1117/12.3029741pmid: N/A
The article addresses the importance of timely detection of errors occurring during 3D printing. One of the 3D printing methods chosen is FFF (Fused Filament Fabrication). Sources are provided where one can familiarize themselves with the main errors that occur during 3D printing. A dataset dedicated to the detection and correction of 3D printing errors using neural networks has been found. The results of training neural networks with ResNet50 and EfficientNet architectures on the found dataset are presented.
Online-learned graph transforms for adaptive block size intra-predictive codingLu, Wen-Yang; Pavez, Eduardo; Ortega, Antonio; Zhao, Xin; Liu, Shan
doi: 10.1117/12.3034041pmid: N/A
Current video coding standards, including H.264/AVC, HEVC, and VVC, utilize discrete cosine transform (DCT), discrete sine transform (DST), to decorrelate the intra-prediction residuals. However, these transforms often face challenges in effectively decorrelating signals with complex, non-smooth, and non-periodic structures. Even in smooth areas, an abrupt transition (due to noise or prediction artifacts) can limit their effectiveness. This paper presents a novel block-adaptive separable path graph-based transform (GBT) that is particularly adept at handling such signals. This new method focuses on adaptively modifying the block size and learning GBT to enhance the performance. The GBT is learned in an online scenario using sequential K-means clustering, where each available block size has K clusters and K GBT kernels. This approach allows the GBT to be dynamically learned for the current block based on previously reconstructed areas with same block size and similar characteristics. Our evaluation, integrating this method with H.264/AVC intra-coding tools, shows significant improvement over the traditional H.264/AVC DCT in processing high-resolution natural images.
Towards immersive multimedia over 5G mobile networksRodrigues, Rafael; Klungre, Øyvind S.; Pinheiro, Antonio M. G.; Pajo, Jane Frances; Perkis, Andrew; Ebrahimi, Touradj
doi: 10.1117/12.3032237pmid: N/A
Emerging 5G technologies bring various new opportunities for the media sector. In particular, they allow the incorporation of ultra-high resolution video formats and immersive augmented, virtual, and extended reality content into low-latency streaming applications while providing a reliable and high-quality user experience. In this paper, we focus on streaming 8K immersive content in a transmedia scenario and validate the feasibility of efficient, cost-effective solutions by measuring the added value brought by the latter. This is done using various key performance indicators in the framework of a European innovation project called 5GMediaHUB, with a scenario focusing on Interactive Digital Narratives. The main story is presented linearly to the user on the first screen, relying on an encoded video stream with marked frames. Each marked frame prompts the user to interact with the story on a secondary screen. The user is finally immersed in a Virtual Reality experience by completing a quiz on the second screen. The quality of the transmitted video is a critical requirement in this scenario, as well as a very low jitter and packet loss. Overall, a very low latency is required to allow effective and satisfactory interaction.
Forward error correction for low-latency transmission of JPEG XS video streamsRichter, Thomas; Foessel, Siegfried
doi: 10.1117/12.3027984pmid: N/A
JPEG XS is a lightweight, low-latency image coding standard for transmission of video streams over IP; sim- ilar to many other coding standards, it does only define the codestream syntax and the decoding algorithm, but not the transport mechanism. For this, IETF defines in RFC 9134 an RTP payload format for JPEG XS for transmission over UDP packets. As long as encoder and decoder are part of the same local area network, error-free transmission can be assumed, but this does not hold for wide-area or wireless networks were packet loss may disrupt the transmission. While multiple protocols exist to ensure error-free transmission in such networks, many of them depend on packet retransmission which impacts the latency of the overall system and by that one of the unique design aspects of JPEG XS. In this work, we equip a JPEG XS encoder and decoder with foreward error correction according to SMPTE ST 2022-5 which is able to mitigate such errors while also keeping the latency under control. We report on the outcome of an experiment where we measured image quality as a function of error rate and also provide estimates on the additional latency due to the error correction layer.
On the performance of learning-based image compression as source coding for JPEG DNAUpenik, Evgeniy; Lazzarotto, Davi; Testolina, Michela; Ebrahimi, Touradj
doi: 10.1117/12.3031848pmid: N/A
Using DNA molecules for data storage presents a compelling solution to the ever-growing demand for efficient and sustainable data storage systems. DNA offers notable advantages in terms of storage density, longevity, and energy efficiency. This has made the development of effective coding and compression techniques for DNA-based storage a critical research area within signal processing. One particularly challenging aspect is the encoding of multimedia content, such as images, for storage in DNA. JPEG DNA, a recent standardization effort led by the JPEG Committee, addresses this challenge by integrating both source and channel coding. The source coding focuses on data compression, while the channel coding ensures error resilience and accommodates the biochemical constraints of the DNA medium. In this paper, the impact of integrating learning-based source coding into the JPEG DNA framework is explored. This study reveals promising improvements in performance by replacing traditional image compression techniques with a learning-based approach, highlighting the potential for further advancements in DNA-based data storage.
On evaluating quality of experience in mixed immersive communication scenariosUpenik, Evgeniy; Lazzarotto, Davi; Mange, Robin; Bello, Javier; Iturrioz, Kepa; Martínez, Anjo; Ebrahimi, Touradj
doi: 10.1117/12.3031211pmid: N/A
This paper introduces a methodology for evaluating quality of experience in mixed immersive communication systems. It focuses on assessing the impact of advanced 3D capture techniques, immersive eye-sensing light field displays, and efficient compression mechanisms in mixed setups where terminals offer different imaging modalities. The paper methodically investigates the influence of the above technologies on user experience in various communication scenarios. By employing a quantitative assessment, the primary objective is to develop methods for comprehensive evaluation of how such immersive technologies affect perceived visual quality, presence, engagement, and overall satisfaction and preference compared to traditional video communication techniques. The experimental design incorporates a series of tests where participants interact through state-of-the-art immersive communication setups, followed by detailed feedback sessions to gauge their experiences. Through this approach, the study seeks to uncover the nuances of user satisfaction in immersive environments and identify the key factors that enhance the overall quality of peer-to-peer communication.
AI-based content-aware encoding at scale utilizing hardware resources in video ASICs for data centerKim, Jongju; Kim, Sijung; Oh, Syehoon; Jeon, Minyong
doi: 10.1117/12.3031558pmid: N/A
This paper presents a method to simplify content-aware encoding for streaming services using AI, to enhance viewing experience and efficiency in bandwidth-limited environments. Unlike traditional encoding, which uses fixed bitrates, this AI-driven approach optimizes the bitrate based on the content's complexity, significantly reducing the necessary computational steps and bitrates. It is achieved by predicting an optimized Adaptive Bitrate (ABR) ladder through minimal encoding steps and lightweight analysis, resulting in substantial bitrate savings and streamlined workflow. The approach also fits well with the trend of adopting video ASICs in data centers, further enhancing cost-effectiveness and scalability.
Automated retinal disorders classification: leveraging digital image enhancement techniques and deep learning on OCT imagesKargar Nigjeh, Mahdi; Kargar Nigjeh, Mahsa; Umbaugh, Scott E.
doi: 10.1117/12.3028346pmid: N/A
Retinal Optical Coherence Tomography (OCT) plays a pivotal role in diagnosing ocular disorders by providing detailed imaging of retinal layers. However, the analysis process remains time-consuming, posing a challenge to its widespread use. This study investigates the integration of Artificial Intelligence (AI) to streamline the analysis of OCT images. Employing Deep Learning (DL) models—VGG16, ResNet18, DenseNet—transfer learning, and data augmentation, the research aims to enhance OCT images, optimize disease recognition, and accurately classify CNV (Choroidal Neovascularization), DME (Diabetic Macular Edema), DRUSEN, and NORMAL pathologies. The dataset undergoes preprocessing, resizing, and enhancement to refine the images. The DenseNet model achieved the highest test accuracy of 92.41% after 25 epochs, demonstrating its potential in efficiently diagnosing ocular pathologies through OCT images.