Compiled and Edited by Prof. Dr. N. Alper TAPAN

In our previous article, we briefly discussed the roots of machine learning (ML), which is a part of artificial intelligence dating back to the early 20th century, and briefly discussed also different machine learning techniques used, and how machine learning is applied in various fields. In this article, we will provide an example of an ML application in the field of direct alcohol fuel cells from our previous study [1] (This study was the first of its kind on application of  machine learning on direct alcohol fuel cells although some data mining tools were applied on cathode materials of solid oxide fuel cells and mechanical behavior of polymer electrolyte fuel cells  [2,3]) and demonstrate how machine learning is used to explore hidden and useful information from collected data or in another way of saying, how machine learning is used in data mining. In fact, these kind of data oriented strategies with open ended and growing database was aimed to help researchers for their future directions about fuel cells. 

In our previous study [1], since observations are needed in order to train machine learning algorithm, database representing direct alcohol fuel cell (DAFC) performance was constructed by collecting representative 12 years of experimental polarization data (4682 observations) with selected (most frequently analyzed) features of DAFCs such as 41different catalytic, material and operational variables.  At this point,  it is necessary to point out that database construction is a critical step in machine learning because biased observations or observations not representing studies from various sources could lead to misleading model results which can not be generalized.

If the complex interactions between material, catalytic, operating variables in DAFCs are  considered , it seems that it was almost impossible to reach a solution by either modeling and experimental works alone. Therefore by the combining the experience from the previous experimental works and ML techniques such as classification tree and artificial neural network, these complex relationships hidden in the previous observations was modelled to be used as suitable tools for future and more focused experimental studies.  Therefore , we were able to determine the following tasks for DAFCs by these ML techniques:

-Routes to maximum power densities (high performances) by classification tree

-Prediction of polarization behavior (characteristic and critical behavior of the system) by ANN with given operational and catalytic variables 

-Determination of dominant variables on the performance of DAFC by ANN.

Now lets briefly examine the two ML techniques used in this case study:

Classification tree

As we have mentioned in the previous montly issue ,classification tree is a supervised machine learning strategy. It gives hidden relationships between different classes of features after the model is trained and tested with a randomly selected set of observations [4]. There are many tools available like libraries in Python, ML toolbox in Matlab, packages in R  or  Rapidminer software etc. used for training and testing of decision tree models.

Basically, during execution of classification tree, observation sets are split recursively in the leaf nodes based on the desired purity criteria with best selected features at the root and branch nodes of the tree. Different algorithms were developed for construction of decision trees like ID3, CART, C4.5, C5.0 algorithms [5,6,7] that have different performance advantages  like high classification speed, strong learning ability and simple construction.

The purity criterions (which is a hyperparameter in decision tree modelling) generally used in the softwares for evaluating the performance of decision trees are gini diversity index, twoing, deviance, information gain and node error. The performance of decision tree is also dependent on the right choice of purity criterion and different purity criterion could result in different classification accuracy.

The flow diagram of CART (classification and regression tree) decision tree algorithm for binary tree (meaning every branch has two children) can be represented as six steps shown below:

    1) Examine all possible binary splits,

    2) Select a split based on selected optimization criterion,

    3) Check if the split meets the hyperparameters like # of observations in the leaf node 

    4) Apply the split

    5) After the binary split,  apply step 1 to 4 for child nodes,

    6) Check the stopping rule: Is the node pure ?, Are there fewer observations than minimum # of parent observation(hyperparameter selected)?, Do the child nodes have fewer observations than MinLeaf observations (hyperparameter selected)?

Other than selection of purity criterion, the other most important factor in construction of optimum decision tree is overfitting and underfitting regions. If decision tree learner construct a very big tree of many leafs in order to perfectly fit the training data, the model would not be able to generalize the behaviour of model and would give high testing error (or low testing accuracy from test data: % correct prediction of classes) although training error would be low.  Therefore the tree should be prunned to avoid overfitting of model and minimize testing error which could be achieved by setting a constraint for minimum number of samples at a leaf node or setting a maximum depth of the tree [5].

In our study [1] after determining testing errors for different sizes of prunned trees for maximum power density in DAFC, optimum size of tree with 31 leaf nodes was decided. In order to see the performance of decision tree , testing and training classification accuracies were compared for different classes of power density as shown in Fig. 1 below.