An LSTM-Based Neural Network Architecture for Model Transformations

We propose to take advantage of the advances in Artificial Intelligence and, in particular, Long Short-Term Memory Neural Networks (LSTM), to automatically infer model transformations from sets of input-output model pairs.

1. An LSTM-Based Neural Network Architecture for Model Transformations Loli Burgueño , Jordi Cabot, Sébastien Gérard MODELS’19 Munich, September 20 th , 2019

2. 2

3. Artificial Intelligence Machine Learning - Supervised Learning: 3 Input Output Training Transforming ML Input Output ML Artificial Intelligence Machine Learning Artificial Neural Networks Deep Artificial Neural Networks

4. Artificial Neural Networks Graph structure: Neurons + directed weighted connections Neurons are mathematical functions Connections are weights Adjusted during the learning process to increase/decrease the strength of the connection 4

5. Artificial Neural Networks The learning process basically means to find the right weights Supervised learning methods. Training phase: Example input-output pairs are used (Dataset) 5 Dataset Training Validation Test

6. Artificial Neural Networks Combine two LSTM for better results Avoids fixed size input and output constraints 6 MTs ≈ sequence -to- sequence arch

7. Architecture Encoder-decoder architecture + Long short-term memory neural networks 7 Encoder LSTM network Decoder LSTM network Input Model Output Model

8. Architecture Sequence-to-Sequence transformations Tree-to-tree transformations Input layer to embed the input tree to a numeric vector + Output layer to obtain the output model from the numeric vectors produced by the decoder 8 Input Tree Embedding Layer Encoder LSTM network Output Tree Extraction Layer Decoder LSTM network Input Model Output Model

9. Attention mechanism To pay more attention (remember better) to specific parts It automatically detects to which parts are more important 9 Architecture Input Tree Embedding Layer Encoder LSTM network Output Tree Extraction Layer Decoder LSTM network Attention Layer Input Model Output Model

10. Pre- and post-processing required to… represent models as trees reduce the size of the training dataset by using a canonical form rename variables to avoid the “dictionary problem” 10 Model pre- and post-processing Input Model (preprocessed) Input Tree Embedding Layer Encoder LSTM network Output Tree Extraction Layer Output Model (non-postprocessed) Decoder LSTM network Attention Layer Input Model Output Model Preprocessing Postprocessing

11. Preliminary results Class to Relational case study 11

12. Model representation 12 MODEL ASSOC OBJ c Class ATTS isAbstract name false family OBJ a Attribute ATTS multivalued name false surname OBJ dt Datatype ATTS name String att c a ASSOC type a dt

13. Preliminary results Correctness Measured through the accuracy and validation loss 13

14. Preliminary results 14 Performance How long does it take for the training phase to complete?

15. Preliminary results Performance How long does it take for the training phase to complete? 15 2. How long it takes to transform an input model when the network is trained?

16. Limitations/Discussion Size of the training dataset Diversity in the training set Computational limitations of ANNs i.e., mathematical operations Generalization problem predicting output solutions for input models very different from the training distribution it has learn from Social acceptance 16

17. An LSTM-Based Neural Network Architecture for Model Transformations Loli Burgueño , Jordi Cabot, Sébastien Gérard MODELS’19 Munich, September 20 th , 2019