Machine learning

Fabio Del Frate received from the University of Rome “Tor Vergata” a Master Degree in Electronics Engineering (1992) and the Ph.D. degree in Computer Science (1997). In 1995-1996 he was Visiting Scientist at Massachusetts Institute of Technology, and from 1998 to 1999 he was with European Space Agency. He then joined the University of Rome “Tor Vergata”, where he is currently Associate Professor of Remote Sensing and Applied Electromagnetism in various Master and PhD Programs. On those topics, he has been lecturer at several European Universities, as well as principal investigator / project manager in several European Space Agency (ESA) and Italian Space Agency (ASI) funded projects, leading research activities in the field of Artificial Intelligence applied to satellite data for Earth Observation (referred to as “EO” in the following).

Q2. What is Machine Learning?

Well, I would say that machine learning is a possible, I guess at the moment the most used one, technical process to generate artificial intelligence inside a machine. We’ve seen already that there are  two main actors: the data and the model, more exactly a mathematical model. The challenge is to find the parameters of the mathematical model so that it can represent the relationships among the data. Everything relies on setting-up a learning process based on the data themselves. We may have different types of models, for example neural networks or support vector machines and, consequently different algorithms to learn the model. The final goal however is always the same: to extract knowledge from the training data used in the training process and to apply this extracted knowledge to new data. By the way, the evaluation of how good the performance of the model is on the new data, that is data not considered during the training phase, is very important. It says to us how much the model is applicable in the real world, how much it is able to generalize. 

In any case, the quality and the quantity of the data used during the learning phase are crucial factors for a successful performance. In particular, the quantity of data needs to be statistically significant to provide the relevant features of the phenomenon we want to investigate. Another important key issue is to decide how complex our mathematical model should be. It would be a serious problem if we choose a too simple model to look for very subtle relationships among the data but, at the same way, if the rules underlying the data are less complicated, also the structure of the model can be lighter, sometime…. less is more, as we say, where less is referred to the  number of parameters characterizing the model. 

By the way, the model complexity  has also to do with the time we need to train the model and with the amount of data which is necessary for the implementation of the complete learning process.

Q4. What is Supervised Learning? Can you give some examples of applications that use it?

Well I would say that Supervised learning is the most common sub-branch of machine learning today. Supervised learning is a form of learning where, for a given input, a “true” outcome is available and this truth can be taught to the model. In other words, as a teacher or a supervisor, for each input given to the model we know what the output of the model should be and we want the model to incorporate the general rule. Typically, this happens by minimizing what is called an error or a cost function, which represents the distance between the desired output values and the actual output values generated by the model during the training phase. Actually, supervised learning can be assumed to be successful when the final overall error over a set of examples, which have not been used during the training phase, is under a predefined threshold. The training phase can be very long, but once we have concluded it properly, we have a very powerful tool, basically operating in real time, for the processing of the new data. Supervised learning can be useful in many fields. A rather massive application regards image classification. In this case the true data are data labelled by human image-interpretation. The classification can be performed at various levels: pixel level, patch level, object level. The level of classification guides the choice of the input that we want to give to the model. More in particular, supervised learning is now very much applied in Earth Observation where we want to make estimation of bio-geophysical parameters from data collected by satellites. An interesting example regards precision farming in agriculture. If we want to monitor the biomass of a crop in order to optimize the amount of fertilizer to be used, we can exploit satellite measurements taken on that crop. However, especially at the microwave region of the electromagnetic spectrum, the extraction of vegetation parameters is not trivial because the measurement is also, let’s say, contaminated by the underlying soil, so AI supervised models can definitely be one of the tools which allow us to retrieve that part of the information in the measurement more connected to the vegetation, discarding the one more conditioned by the soil.

Q6. There are various groupings of Machine Learning algorithms, e.g. regression, classification, and so forth. Can you explain their relevance and difference between them?

There are various possibilities to group machine learning algorithms. I think that the main one relies on the type of the mathematical model. More in particular, as mentioned before, we have models for supervised training and models for unsupervised training. Regarding the former, among the most important ones we have neural networks, deep or shallow, support vector machines, decision trees, ensemble techniques. For the unsupervised models we can mention k-means, principal component analysis, auto-associative neural networks, self-organized maps, and so forth. 

But as your question was pointing out, another possibility to group machine learning algorithms addresses the distinction between regression and classification. Once a certain input is provided to our mathematical model, the difference between regression and classification mainly relies on the type of output we’re interested in. In the case of classification, the output will be a label, a class. For example, let’s consider again a satellite application where remote sensing is used for meteorology. Let’s therefore assume we have to make a meteorological analysis, and we want to decide which pixel in an image of the sky has to be associated to a cloudy sky and which pixel of the image to clear sky, so that the final purpose is to produce a map where we represent with the blue color the clear sky pixels and with the grey color the cloudy pixels. We say in this case that we want to obtain a classification map. For the regression, basically, we want that our mathematical model, as a response of its data processing, computes, or estimates, a number, more in general a real value number. So, if we consider the same scenario set on meteorology, we want to know from the remotely sensed data how much is the humidity or the precipitation rate, real numbers, in a given area. However, it is interesting to note how a regression problem can be translated into a classification problem. This is what happens if I say: within this range of precipitation values, so between these two numbers, I have the class “low precipitation”, in this other interval or range I have “moderate precipitation”, and so on. In this case I map ranges of values into classes.

Thank you, Prof. Fabio Del Frate, we really appreciated. It has been a clear & interesting snapshot of Artificial Intelligence features and relevant applications.