Menelaos PhDs thesis

Sparse Reconstruction for high-resolution inverse SAR imaging

The main focus lies on innovative signal processing methods for millimeter wave radars to overcome the systems limitations in bandwidth or in range resolution. In this work disjoint frequency bands with relatively small individual bandwidths will be fused by using compressed sensing methods to gain a high overall bandwidth andthereby a very fine range resolution. For the investigation two suitable table measurement setups are available at FHG FHR, which can be used and modified for data acquisition.

Goal-directed compressive radar imaging

One important characteristic of sparsity-driven compressive radar imaging is that it can preserve and enhance features in the scene that are potentially important for automated interpretation of the scene for a particular high level goal such as object recognition or scene classification. However, so far only very low-level features have been used in such imaging methods, and this has been an “open-loop” process, without explicitly considering the final information extraction objective. Through this individual research project, we aim to develop new compressive radar imaging methods that explicitly take into account and adapt to the final decision-making goals.

Deep Depth from Defocus (Deep DFD) for near range and in-situ 3D exploration

Acquiring 3D geometry of the scene is essential for a series of applications like scene understanding, navigation, robotics etc. Among the existing approaches, those using passive 3D sensing are more cost-effective and allow the use of compact, standard systems. Two hints for depth estimation by passive sensing are the blur and geometrical deformations due to perspective representation. The depth can be inferred from blur, the approach being known as Depth from Defocus (DFD). Despite the considerable progress made on this subject in the recent years, the DFD used solely remains severely limited by the ambiguity with respect to the focal plane and dead zone. In parallel to DFD, machine learning methods relying on geometric deformation have been developed. The success of DL in a series of applications has leveraged this approach and methods for inferring depth from defocused images were proposed. The results certify that the joint use of structural and blur information overcomes current limitations of single-image DFD.

The main objective of the thesis is to propose, develop and test DL based methods that exploit jointly the naturally arising blur and the structural information to obtain more accurate 3D maps. To this end, the PhD student will propose an architecture for the neural network and implement it using free DL libraries. The network will be trained on pairs of true depth maps and images from existing databases enriched with new ones. The network validation will be carried on indoor images taken with a DSLR camera (an additional sensor will be used to obtain the ground truth) and on in the wild images.

Learning with adversarial samples for EO multi-spectral images

Adversarial samples became popular in the area of DL, where they have been defined as input samples subtly modified to cause a machine learning misclassification. Generative Adversarial Networks (GAN) are a DL approach for generating artificial examples that plausibly could be drawn from a certain data category. GANs architecture is composed of two networks.  The first is a generator, the second a discriminator. The generator is learning a generative model close to the data model that afterward is used to generate artificial data samples. The discriminator compare the generated and actual data in order to force the learning of a good data representation.

In the case of EO multi-spectral images, the adversarial samples may occur “naturally”. Sensor artifacts like LSB stripes or saturation are at the origin of such samples. In addition, EO multispectral data contains physical information. Thus, the adversarial information shall represent it in a consistent meaningful model.

This project aims to provide solutions for DL for EO multi-spectral images in the presence of naturally occurring adversarial samples and also considering their physical nature and models. The expected results are a scientific and methodological paradigm for: 1) The concept of adversarial samples for EO multi-spectral images, 2) Build of a database of specific adversarial samples and algorithms to generate them, 3) Study of the effects of adversarial samples for the case of DNN applied to EO multi-spectral images and design of specific DNN paradigms to alleviate the sequels of adversarial samples. 

Deep learning for SAR data in presence of adversarial samples

Classification of SAR image data remains a challenge. Major difficulties include the scarcity of available data, and the difficulty of semantically interpreting the SAR backscatter signal. Linked to those problems, there are no large-scale, SAR-derived image databases for remote sensing image analysis and knowledge discovery. Furthermore, while optical image classification has seen a breakthrough with the advent of DL methods that require Big Data, SAR-based systems have so far not experienced the same progress, likely because of not enough data associated training labels is available. The nature of the adversarial samples occurring spontaneously depends on the sensor type. In SAR data, for instance, the effect of strong scattering or the model of image formation and physical processes behind need very specific methods for adversarial samples. Given the specific nature of these samples, the solutions to avoid their insertion in the training sets or to alleviate their effects must be tailored accordingly.

The main objective of this project is to give solutions for deep learning with spontaneous adversarial samples in the case of SAR data. Scientific and methodological dissertation and publications are expected, with novel findings highlighting: 1) Transformation methods for a relevant SAR data representation, in order to avoid insertion of adversarial samples, 2) Design of DNNs for SAR data classification in order to achieve a given invariance to spontaneous adversarial samples, 3) Projections of features when learning the semantic axes for 3D visualization such to contextually disambiguate the meaning and to ensure a consistent training.