Lashing force prediction model with multimodal deep learning and AutoML for stowage planning automation in containerships

Lee, Chaemin; Lee, Mun Keong; Shin, Jae Young Article Lashing force prediction model with multimodal deep learning and AutoML for stowage planning automation in containerships Logistics Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Lee, Chaemin; Lee, Mun Keong; Shin, Jae Young (2021) : Lashing force prediction model with multimodal deep learning and AutoML for stowage planning automation in containerships, Logistics, ISSN 2305-6290, MDPI, Basel, Vol. 5, Iss. 1, pp. 1-15, https://doi.org/10.3390/logistics5010001 This Version is available at: https://hdl.handle.net/10419/310126 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ logistics Article Lashing Force Prediction Model with Multimodal Deep Learning and AutoML for Stowage Planning Automation in Containerships Chaemin Lee 1,* , Mun Keong Lee 2and Jae Young Shin 3   Citation: Lee, C.; Lee, M.K.; Shin, J.Y. Lashing Force Prediction Model with Multimodal Deep Learning and AutoML for Stowage Planning Automation in Containerships. Logistics 2021,5, 1. https:// dx.doi.org/10.3390/logistics5010001 Received: 30 October 2020 Accepted: 16 December 2020 Published: 28 December 2020 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/ licenses/by/4.0/). 1Total Soft Bank Ltd., Busan 48002, Korea 2Maersk Singapore Pte. Ltd., Singapore 089763, Singapore; [email protected] 3Logistics Engineering Department, Korea Maritime & Ocean University, Busan 49112, Korea; [email protected] *Correspondence: [email protected] or [email protected] Abstract: The calculation of lashing forces on containerships is one of the most important aspects in terms of cargo safety, as well as slot utilization, especially for large containerships such as more than 10,000 TEU (Twenty-foot Equivalent Unit). It is a challenge for stowage planners when large containerships are in the last port of region because mostly the ship is full and the stacks on deck are very high. However, the lashing force calculation is highly dependent on the Classification society (Class) where the ship is certified; its formula is not published and it is different per each Class (e.g., Lloyd, DNVGL, ABS, BV, and so on). Therefore, the lashing result calculation can only be verified by the Class certified by the Onboard Stability Program (OSP). To ensure that the lashing result is compiled in the stowage plan submitted, stowage planners in office must rely on the same copy of OSP. This study introduces the model to extract the features and to predict the lashing forces with machine learning without explicit calculation of lashing force. The multimodal deep learning with the ANN, CNN and RNN, and AutoML approach is proposed for the machine learning model. The trained model is able to predict the lashing force result and its result is close to the result from its Class. Keywords: lashing force; containership; stowage planning; multimodal deep learning; AutoML; ANN; CNN; RNN 1. Introduction 1.1. Consideration of Stowage Planning Stowage planning is a highly complex process with the goal to achieve cost efficiency and safety of crews and containership at the same time. This is done by ensuring that containers are loaded in the appropriate places on the containership, with consideration of the infrastructure limitation of all terminals in the round trip port rotation of the subject vessel, container composition to be loaded at each terminal, necessary segregations of the dangerous goods cargo, adherence to navigation visibility requirement, maximum number of cranes that can work concurrently, fulfilment of special stowage requirements from shippers, safety of containers and vessels, such as stability, strength, lashing, etc. In Figure 1, more considerations are categorized by the goal of stowage planning. Logistics 2021,5, 1. https://dx.doi.org/10.3390/logistics5010001 https://www.mdpi.com/journal/logistics Logistics 2021,5, 1 2 of 15 Logistics 2021, 5, x FOR PEER REVIEW 2 of 15 Figure 1. Categorization of stowage planning evaluation. 1.2. Literature Review Ding [1] and Avriel [2,3] studied and developed heuristic algorithms for automated stowage planning in order to reduce a number of restows/shifts which are categorized as “Low Cost”. Low [4] proposed and developed a system with consideration for crane intensity and a number of rehandles (restows/shifts), which are categorized as “Low Cost”, and stability, categorized as “Safety”. Ambrosino [5] studied the master bay plan problem (MBPP) with the Linear Programming model and also presented a heuristic approach to relax and solve the combinatorial optimization problem. This MBPP is related to “Space Utilization” of “High Revenue” and “Low Overstow” and “Low Restow/Shift” of “Low Cost”. Korach [6] studied an efficient mathematical programming technique within a heuristic framework for the slot planning problem which is categorized as “High Revenue”, especially for the “DC HC (normal or high cubic container) mix under deck” case. Rahsed [7] applied a rule-based greedy algorithm to solve the unnecessary Restow/Shift movement which is related to the “Low Restow/Shift” of “Low Cost” category. Shen [8] introduced the Deep Q-Learning Network (DQN) as a model to solve the stowage planning problem, and this study showed the possibility to apply Machine Learning, Deep Learning or Reinforcement Learning for stowage planning. The introduced features are mostly under “High Revenue” and “Low Cost”. Rathje [9] introduced the new lashing rule of Germanischer Lloyd (GL) to offer containership operators more flexibility in on-deck container stowage without compromising safety. However, there is no study on Lashing Forces, taken into consideration stowage planning automation for the “Safety” category despite the importance of lashing forces on large containerships. 1.3. Lashing in Containership As the containerships become larger, container stacks on deck become higher. Today, the largest containerships in the world can carry as many as 23,964 Twenty-foot Equivalent Units (TEUs) [10]. Back in the 1950s, the first generation of containerships had only two tiers on deck. Today, containerships are routinely carrying containers on deck up to eleven (11) tiers high. As such, lashing on containers becomes increasingly important. The lashing is the securing arrangements onboard to prevent containers from moving from their places or falling off into the sea when the vessel is in motion, especially during rough weather. Its effectiveness is measured by the magnitude of the various forces that act on Figure 1. Categorization of stowage planning evaluation. 1.2. Literature Review Ding [ 1 ] and Avriel [ 2 , 3 ] studied and developed heuristic algorithms for automated stowage planning in order to reduce a number of restows/shifts which are categorized as “Low Cost”. Low [ 4 ] proposed and developed a system with consideration for crane intensity and a number of rehandles (restows/shifts), which are categorized as “Low Cost”, and stability, categorized as “Safety”. Ambrosino [ 5 ] studied the master bay plan problem (MBPP) with the Linear Programming model and also presented a heuristic approach to relax and solve the combinatorial optimization problem. This MBPP is related to “Space Utilization” of “High Revenue” and “Low Overstow” and “Low Restow/Shift” of “Low Cost”. Korach [ 6 ] studied an efficient mathematical programming technique within a heuristic framework for the slot planning problem which is categorized as “High Revenue”, especially for the “DC HC (normal or high cubic container) mix under deck” case. Rahsed [ 7 ] applied a rule-based greedy algorithm to solve the unnecessary Restow/Shift movement which is related to the “Low Restow/Shift” of “Low Cost” category. Shen [ 8 ] introduced the Deep Q-Learning Network (DQN) as a model to solve the stowage planning problem, and this study showed the possibility to apply Machine Learning, Deep Learning or Reinforcement Learning for stowage planning. The introduced features are mostly under “High Revenue” and “Low Cost”. Rathje [ 9 ] introduced the new lashing rule of Germanischer Lloyd (GL) to offer containership operators more flexibility in on-deck container stowage without compromising safety. However, there is no study on Lashing Forces, taken into consideration stowage planning automation for the “Safety” category despite the importance of lashing forces on large containerships. 1.3. Lashing in Containership As the containerships become larger, container stacks on deck become higher. Today, the largest containerships in the world can carry as many as 23,964 Twenty-foot Equivalent Units (TEUs) [ 10 ]. Back in the 1950s, the first generation of containerships had only two tiers on deck. Today, containerships are routinely carrying containers on deck up to eleven (11) tiers high. As such, lashing on containers becomes increasingly important. The lashing is the securing arrangements onboard to prevent containers from moving from their places or falling off into the sea when the vessel is in motion, especially during rough weather. Its effectiveness is measured by the magnitude of the various forces that act on containers, comparing against their limits and displayed in a percentage. Each Classification society has a slightly different method of measurement. Logistics 2021,5, 1 3 of 15 In severe sea conditions, as well as in the case of improperly stowed containers and overweight containers, these forces may become excessive, causing, for example, failure of twist locks or collapse of lower-stacked containers. Consequently, whole container stacks may collapse and go overboard, which is not just an economic issue but also an issue for safe passageway, as these containers may be floating on the sea surface. Besides this, deck containers may be loaded with dangerous goods. Thus, containers going overboard also pose significant environmental implications [ 11 ]. According to the report of the World Shipping Council, the industry loses as many as 10,000 containers a year at sea [12]. The calculation of lashing forces is one of the important aspects in terms of cargo safety as well as slot utilization. It is solely dependent on the Classification society (Class), that the ship is certified under. Unlike other calculations such as Stability, Strength and DG (Dangerous Goods) check, the lashing calculation formula is not published and differs from Class to Class (e.g., Lloyd, DNVGL, ABS, BV, and so on). The stowage planner has to rely on the Class-certified Onboard Stability Program (OSP) to ensure his/her stowage plan is lashing compliant as described by the process in Figure 2. Logistics 2021, 5, x FOR PEER REVIEW 3 of 15 containers, comparing against their limits and displayed in a percentage. Each Classification society has a slightly different method of measurement. In severe sea conditions, as well as in the case of improperly stowed containers and overweight containers, these forces may become excessive, causing, for example, failure of twist locks or collapse of lower-stacked containers. Consequently, whole container stacks may collapse and go overboard, which is not just an economic issue but also an issue for safe passageway, as these containers may be floating on the sea surface. Besides this, deck containers may be loaded with dangerous goods. Thus, containers going overboard also pose significant environmental implications [11]. According to the report of the World Shipping Council, the industry loses as many as 10,000 containers a year at sea [12]. The calculation of lashing forces is one of the important aspects in terms of cargo safety as well as slot utilization. It is solely dependent on the Classification society (Class), that the ship is certified under. Unlike other calculations such as Stability, Strength and DG (Dangerous Goods) check, the lashing calculation formula is not published and differs from Class to Class (e.g., Lloyd, DNVGL, ABS, BV, and so on). The stowage planner has to rely on the Class-certified Onboard Stability Program (OSP) to ensure his/her stowage plan is lashing compliant as described by the process in Figure 2. Figure 2. Typical process of lashing verification during the stowage planning. 1.4. Machine Learning in Lashing of Containership As a trend of Machine Learning (ML), especially for Deep Learning nowadays, the idea is that ML can fulfil the needs of stowage planners to get the lashing force values. Instead of calculating the lashing forces by navel architecture engineering, this study proposes multimodal deep learning with ANN, CNN and RNN to train machines to predict the lashing forces. As illustrated in Figure 3, the idea is that the stowage plan, consisting of stowage (e.g., container weight, height, slot position, etc.), condition (e.g., GM, Wind Speed, Roll Angle), and containership structure (e.g., bays, rows, tiers), is given to one of the appropriate ML models, trained per each Class (e.g., DNVGL, ABS, Lloyd, BV, etc.), and its ML Model predicts and returns Lashing Forces as a result during the stowage planning in the stowage planning tool. Without relying on OSP, stowage plans can be generated within the same system swiftly, making lashing forces compliant. This study proposes the Multimodal Deep Learning [13] model with AutoML [14–16] approach to predict Lashing Forces as a part of the process of stowage planning automation. Figure 2. Typical process of lashing verification during the stowage planning. 1.4. Machine Learning in Lashing of Containership As a trend of Machine Learning (ML), especially for Deep Learning nowadays, the idea is that ML can fulfil the needs of stowage planners to get the lashing force values. Instead of calculating the lashing forces by navel architecture engineering, this study proposes multimodal deep learning with ANN, CNN and RNN to train machines to predict the lashing forces. As illustrated in Figure 3, the idea is that the stowage plan, consisting of stowage (e.g., container weight, height, slot position, etc.), condition (e.g., GM, Wind Speed, Roll Angle), and containership structure (e.g., bays, rows, tiers), is given to one of the appropriate ML models, trained per each Class (e.g., DNVGL, ABS, Lloyd, BV, etc.), and its ML Model predicts and returns Lashing Forces as a result during the stowage planning in the stowage planning tool. Without relying on OSP, stowage plans can be generated within the same system swiftly, making lashing forces compliant. This study proposes the Multimodal Deep Learning [ 13 ] model with AutoML [ 14 – 16 ] approach to predict Lashing Forces as a part of the process of stowage planning automation. Logistics 2021,5, 1 4 of 15 Logistics 2021, 5, x FOR PEER REVIEW 4 of 15 Figure 3. Illustration of idea. 2. Lashing Force Prediction with Multimodal Deep Learnings 2.1. Idea and Process The process of stowage planning automation with lashing force prediction is depicted as follows. 1. As part of auto stowage planning process, the stowage planning tool slots containers on deck. 2. The conditions, as input parameters for lashing force prediction, are set from both the stability result (e.g., GM, Draft, Trim, etc.; subject to Class), calculated by the stowage planning tool and inputted values (e.g., Wind Speed, Roll Angle, etc.; subject to Class) by the stowage planner. 3. The stowage system requests the lashing force result for one of the embedded lashing force prediction models, trained per each Class. 4. The model returns the lashing force percentage for each lashing component. 5. If any of the returned lashing force values is greater than 100%, the stowage planner or stowage planning tool changes the containers with lighter ones and repeats from step no. 2. A 10,000 TEU containership, belonging to one of biggest shipping lines, has been chosen and her real life, fully loaded stowage, especially On-Deck for the last port of region, has been selected as the input to train the above-mentioned model, as depicted in Figure 4. Almost all Rows in each On-Deck of Bay are fully loaded up to capacity. The lashing force percentage in this stowage is close to 100%. The Classification society is ABS and the lashing rule is the In-House Lashing Rule. Figure 3. Illustration of idea. 2. Lashing Force Prediction with Multimodal Deep Learnings 2.1. Idea and Process The process of stowage planning automation with lashing force prediction is depicted as follows. 1. As part of auto stowage planning process, the stowage planning tool slots containers on deck. 2. The conditions, as input parameters for lashing force prediction, are set from both the stability result (e.g., GM, Draft, Trim, etc.; subject to Class), calculated by the stowage planning tool and inputted values (e.g., Wind Speed, Roll Angle, etc.; subject to Class) by the stowage planner. 3. The stowage system requests the lashing force result for one of the embedded lashing force prediction models, trained per each Class. 4. The model returns the lashing force percentage for each lashing component. 5. If any of the returned lashing force values is greater than 100%, the stowage planner or stowage planning tool changes the containers with lighter ones and repeats from step no. 2. A 10,000 TEU containership, belonging to one of biggest shipping lines, has been chosen and her real life, fully loaded stowage, especially On-Deck for the last port of region, has been selected as the input to train the above-mentioned model, as depicted in Figure 4 . Almost all Rows in each On-Deck of Bay are fully loaded up to capacity. The lashing force percentage in this stowage is close to 100%. The Classification society is ABS and the lashing rule is the In-House Lashing Rule. Logistics 2021,5, 1 5 of 15 Logistics 2020, 4, x FOR PEER REVIEW 5 of 17 Figure 4. Stowage of 10,000 Twenty-foot Equivalent Unit (TEU) containership. 2.2. Feature Extraction and Engineering 2.2.1. Containership Structure As illustrated in Error! Reference source not found., the structure of containerships is well standardized because the container itself is standardized with several dimensional types (e.g., commonly 20 ft or 40 ft in length and normal or high cubic in height). Generally, one Hatch consists of two physical 20 ft bays (depicted Bay 25 and Bay 26) and one logical 40 ft bay (depicted Bay 26). This means that two 20 ft containers or one 40ft can be stacked in one slot. The bay consists of Rows and Tiers as in the table, and each square is called Slot. One Bay is divided into Under Deck and On Deck, and the lashing is needed On Deck only. There is a big number of Slot differences in the Bay between the small and large containerships. Typically, the size of the dimension needs to be fixed in order to train the machine, therefore, the maximum size of the dimension is defined by 26 Rows (horizontal) and 13 Tiers (vertical) which are able to accommodate the largest containership in the world. Since lashing forces are independent per each Hatch with the given ship level condition, such as GM, in this study, one dataset is defined by 1 Hatch (3 Bays) and On Deck. Each slot is presented by three-dimension array 𝑆 where; 𝑏 𝑖𝑠 𝐵𝑎𝑦 𝑖𝑛𝑑𝑒𝑥 { 0 , 1 , 2 } 𝑟 𝑖𝑠 𝑅𝑜𝑤 𝑖𝑛𝑑𝑒𝑥 { 0 , 1 , ⋯ 24 , 25 } 𝑡 𝑖𝑠 𝑇𝑖𝑒𝑟 𝑖𝑛𝑑𝑒𝑥 { 0 , 1 , ⋯ 11 , 12 } （1） Figure 4. Stowage of 10,000 Twenty-foot Equivalent Unit (TEU) containership. 2.2. Feature Extraction and Engineering 2.2.1. Containership Structure As illustrated in Figure 5, the structure of containerships is well standardized because the container itself is standardized with several dimensional types (e.g., commonly 20 ft or 40 ft in length and normal or high cubic in height). Generally, one Hatch consists of two physical 20 ft bays (depicted Bay 25 and Bay 26) and one logical 40 ft bay (depicted Bay 26). This means that two 20 ft containers or one 40ft can be stacked in one slot. The bay consists of Rows and Tiers as in the table, and each square is called Slot. One Bay is divided into Under Deck and On Deck, and the lashing is needed On Deck only. There is a big number of Slot differences in the Bay between the small and large containerships. Typically, the size of the dimension needs to be fixed in order to train the machine, therefore, the maximum size of the dimension is defined by 26 Rows (horizontal) and 13 Tiers (vertical) which are able to accommodate the largest containership in the world. Since lashing forces are independent per each Hatch with the given ship level condition, such as GM, in this study, one dataset is defined by 1 Hatch (3 Bays) and On Deck. Each slot is presented by three-dimension array Sbrt where; b is Bay index {0, 1, 2} r is Row index {0, 1, · · · 24, 25} t is Tier index {0, 1, · · · 11, 12} (1) Logistics 2021,5, 1 6 of 15 Logistics 2020, 4, x FOR PEER REVIEW 6 of 17 Figure 5. Presentation of containership structure in Bays. 2.2.2. Features Lashing forces are calculated with container stacking profiles and containership structures. The following six features are proposed to represent the factors that influence lashing forces, illustrated in Error! Reference source not found.. For 𝐹 = {𝑓(1),𝑓(2), 𝑓(3), 𝑓(4), 𝑓(5), 𝑓(6)}:  Physical slot availability in each slot, 𝑆, from fixed maximum dimension. If available set 1, otherwise 0.  Weight of container in each slot, 𝑆. Generally heavier containers stack in lower slots to be stable.  Height of container in each slot, 𝑆. Generally a lower height is more stable.  Slot Highof Lashing Bridge Fore Side. Higher lashing bridge gives a safer lashing force value in general.  Slot High of Lashing Bridge Aft Side. Higher lashing bridge gives lower lashing force value in general.  Deck Level where the deck starts as compared to other Bays. For example, the Sunken Bay has lower lashing force values because it is one level lower than other normal Bays. In addition to the container stacking profiles on each Bay, there are vessel conditions that influence lashing forces. The following three conditions are extracted and modelled as auxiliary ANN for the multimodal modelling.  GM (Metacentric Height)—this is the result condition when stability is calculated.  Wind Speed. Figure 5. Presentation of containership structure in Bays. 2.2.2. Features Lashing forces are calculated with container stacking profiles and containership structures. The following six features are proposed to represent the factors that influence lashing forces, illustrated in Figure 6. For F={f(1),f(2),f(3),f(4),f(5),f(6)}: • Physical slot availability in each slot, Sbrt , from fixed maximum dimension. If available set 1, otherwise 0. • Weight of container in each slot, Sbrt . Generally heavier containers stack in lower slots to be stable. •Height of container in each slot, Sbrt. Generally a lower height is more stable. • Slot Highof Lashing Bridge Fore Side. Higher lashing bridge gives a safer lashing force value in general. • Slot High of Lashing Bridge Aft Side. Higher lashing bridge gives lower lashing force value in general. • Deck Level where the deck starts as compared to other Bays. For example, the Sunken Bay has lower lashing force values because it is one level lower than other normal Bays. In addition to the container stacking profiles on each Bay, there are vessel conditions that influence lashing forces. The following three conditions are extracted and modelled as auxiliary ANN for the multimodal modelling. •GM (Metacentric Height)—this is the result condition when stability is calculated. •Wind Speed. •Roll Angle. Logistics 2021,5, 1 7 of 15 Logistics 2021, 5, x FOR PEER REVIEW 7 of 15 Figure 6. Example of extracted features. 2.2.3. Lashing Force As illustrated in Figure 7, there are 10 lashing force components per each Row and two of them, the Lashing Special Corner H Forces and Lashing Special Corner V Forces, are not applicable for this containership. These 10 values are answer labels for train and test data. • Corner Cast Compression; • Corner Casting; • Corner Post Compression; • Lashing Rod Tension; • Lashing Special Corner H Forces (not applicable for this ship); • Lashing Special Corner V Forces (not applicable for this ship); • Longitudinal Racking; • Pull Out; • Shear; • Transverse Racking. Figure 7. Lashing force components. Figure 6. Example of extracted features. 2.2.3. Lashing Force As illustrated in Figure 7, there are 10 lashing force components per each Row and two of them, the Lashing Special Corner H Forces and Lashing Special Corner V Forces, are not applicable for this containership. These 10 values are answer labels for train and test data. •Corner Cast Compression; •Corner Casting; •Corner Post Compression; •Lashing Rod Tension; •Lashing Special Corner H Forces (not applicable for this ship); •Lashing Special Corner V Forces (not applicable for this ship); •Longitudinal Racking; •Pull Out; •Shear; •Transverse Racking. Logistics 2020, 4, x FOR PEER REVIEW 7 of 17  Roll Angle. Figure 6. Example of extracted features. 2.2.3. Lashing Force As illustrated in Error! Reference source not found., there are 10 lashing force components per each Row and two of them, the Lashing Special Corner H Forces and Lashing Special Corner V Forces, are not applicable for this containership. These 10 values are answer labels for train and test data.  Corner Cast Compression;  Corner Casting;  Corner Post Compression;  Lashing Rod Tension;  Lashing Special Corner H Forces (not applicable for this ship);  Lashing Special Corner V Forces (not applicable for this ship);  Longitudinal Racking;  Pull Out;  Shear;  Transverse Racking. Figure 7. Lashing force components. Figure 7. Lashing force components. Logistics 2021,5, 1 8 of 15 2.2.4. Dataset Fully loaded stowage for thelast port of region is selected as the input to train as depicted in Figure 4. The lashing force percentage in this stowage is close to 100%, so this stowage is used as a baseline dataset. Since the prediction model is supervised learning, the label is needed for every dataset and the label comes from OSP. Therefore, over 100,000 training datasets are generated by interface between Stowage Planning Tool and OSP represented both in the following strategy and in Table 1: •11 different realistic vessel conditions; • For each condition, random weight variance in 10%, 15% and 20% for each container onboard; •A total of 21 different Hatches as the different stacking profile. Table 1. Training and test data. Condition GM Wind Speed Roll Angle 10% Variance 15% Variance 20% Variance Total 104,786 1 2.00 30.00 22.00 4578 6636 2982 14,196 2 1.80 29.00 21.50 2520 2583 5796 10,899 3 2.10 28.00 21.00 2611 2708 2856 8175 4 2.40 27.00 20.50 2898 2580 2503 7981 5 2.70 26.00 20.00 3066 2646 4662 10,374 6 3.00 25.00 19.50 3141 2710 2559 8410 7 3.30 24.00 19.00 2594 3149 2581 8324 8 3.60 23.00 18.50 2541 2541 2552 7634 9 3.90 22.00 18.00 2568 2705 2734 8007 10 4.20 21.00 17.50 2566 2791 3799 9156 11 4.50 20.00 17.00 2478 3191 5961 11,630 2.2.5. Modeling In this study, Multimodal Deep Learning is applied with Artificial Neural Network (ANN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). It is common nowadays to adopt multimodal to predict results more accurately; for instance, Video–Audio input to recognize human emotion [ 17 ]. First of all, the vessel conditions are used as the auxiliary input of the ANN. General deep learning network, as ANN, is adopted for the stowage plan because each slot position itself can be considered as a meaningful feature. In addition, the Bay structure, presented in the Stowage Planning Tool—as illustrated in Figure 5—is already very similar as an image, i.e., 26 × 13 pixels with six features as channels; therefore, CNN is adopted as one of the inputs. Additionally, the lashing force is affected by the adjacent Rows because the outer row can protect the wind force to the inner row. This means that the sequence of the stacked container might impact the lashing force values for the next rows. This is the reason why the RNN model is used in this study. The features described in Section 2.2.2 are used for all ANN, CNN and RNN models, except for the auxiliary model. As illustrated in Figure 8, the first auxiliary input is one dimension to accommodate the ship conditions, GM, Wind Speed and Roll Angle. The second ANN input is four dimensions to represent Bays, Tiers, Rows and Features. The third input is three dimensions to represent Tiers, Rows and Bays × Features as Channels of the CNN input. The last input is two dimensions to represent Tiers and Bays × Rows × Features as nodes of the RNN input. Logistics 2021,5, 1 15 of 15 4. Low, M.; Xiao, X.; Liu, F.; Huang, S.Y.; Hsu, W.J.; Li, Z. An automated stowage planning system for large containerships. In Proceedings of the International MultiConference of Engineers and Computer Scientists, Hong Kong, China, 17–19 March 2010; pp. 17–19. 5. Ambrosino, D.; Sciomachen, A.; Tanfani, E. Stowing a containership: The master bay plan problem. Transp. Res. Part A Policy Pr. 2004,38, 81–99. [CrossRef] 6. Korach, A.; Brouer, B.D.; Jensen, R.M. Matheuristics for slot planning of container vessel bays. Eur. J. Oper. Res. 2020 ,282, 873–885. [CrossRef] 7. Rahsed, D.M.; Gheith, M.S.; Eltawil, A.B. A Rule-based Greedy Algorithm to Solve Stowage Planning Problem. In Proceedings of the 2018 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), Macau, China, 16–19 December 2018; pp. 437–441. 8. Shen, Y.; Zhao, N.; Xia, M.; Du, X. A Deep Q-Learning Network for Ship Stowage Planning Problem. Pol. Marit. Res. 2017 ,24, 102–109. [CrossRef] 9. Rathje, H.; Abt, D.; Wolf, V.; Schellin, T.E. Route-specific container stowage. In Proceedings of the PRADS 2013, Changwan City, Korea, 20 October 2013. 10. Wikipedia. Available online: https://en.wikipedia.org/wiki/List_of_largest_container_ships (accessed on 24 August 2020). 11. Wolf, V.; Darie, I.; Rathje, H. Rule development for container stowage on deck. In Proceedings of the Third International Conference on Marine Structures—MARSTRUCT, Hamburg, Germany, 28–30 March 2011; Volume 1, pp. 715–722. 12. World Shipping Council. Containers Lost at See—2017 Update. Available online: https://www.worldshipping.org/industryissues/safety/Containers_Lost_at_Sea_-_2017_Update_FINAL_July_10.pdf (accessed on 24 August 2020). 13. Ngiam, J.; Khosla, A.; Kim, M.; Nam, J.; Lee, H.; Ng, A.Y. Multimodal deep learning. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, WA, USA, 28 June–2 July 2011. 14. Gijsbers, P.; LeDell, E.; Thomas, J.; Poirier, S.; Bischl, B.; Vanschoren, J. An open source AutoML benchmark. arXiv 2019 , arXiv:1907.00909. 15. He, X.; Zhao, K.; Chu, X. AutoML: A survey of the state-of-the-art. Knowl.-Based Syst. 2021,212, 106622. [CrossRef] 16. Real, E.; Liang, C.; So, D.R.; Le, Q.V. Automl-zero: Evolving machine learning algorithms from scratch. arXiv 2020 , arXiv:2003.03384. 17. Tran, D.; Bourdev, L.; Fergus, R.; Torresani, L.; Paluri, M. Learning Spatiotemporal Features with 3D Convolutional Networks. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 4489–4497.