Ultrasonic hydrogen production from water

MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX ECOLE POLYTECHNIQUE MSc&T STEEM Academic Year 2022-23 POIROT Victoria INTERNSHIP REPORT Ultrasonic Hydrogen production from water NON CONFIDENTIAL REPORT Referent teacher: Alexandre STEGNER Internship tutor: James KWAN Internship dates: 01/04/2023 – 30/09/2023 MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Declaration of Academic Integrity Hereby, I, Victoria Poirot, confirm that: 1. The results presented in this report are my own work 2. I am the author of this report 3. I have not used the work of others without clearly acknowledging it, and quotations and paraphrases from any source are clearly indicated Victoria Poirot, 18/09/2023, MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Aknowledgment First and foremost, I would like to extend my gratitude to my tutor, James Kwan, for his invaluable guidance, unwavering support, and expert supervision throughout my internship. His mentorship has been instrumental in the success of this project. I would like to express sincere thanks to EDF UK for funding my internship. Their financial support made this opportunity possible, and I am grateful for the investment they made in this project. I would also like to warmly thank Antoine Jerusalem for bridging the connection with EDF UK, and the Maison Française d’Oxford. I want to acknowledge the contribution of Lillian Usadi, with whom I had the pleasure of working. Her collaboration was enriching and significantly contributed to the progress of our research. Finally, I want to acknowledge the BUBBL lab for providing the data on which I was able to work. These data were crucial for the completion of this study. I must also extend my thanks to the entire Kwan Research Group for fostering a supportive and stimulating environment in which I could work. MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Abstract This article focuses on the production of hydrogen using ultrasound, exploiting acoustic cavitation. Two forms of cavitation, stable and inertial, are being studied to understand their impact on hydrogen production. The main objective is to predict cavitation behaviour by analysing acoustic and video data, using both supervised and unsupervised machine learning approaches. The results show that prediction of cavitation behaviour is possible, paving the way for a more efficient design of sono-chemical reactors for hydrogen production, by improving their performance and resistance. This in-depth understanding of cavitation will help to shape the future of ultrasonic hydrogen production, offering reactors that perform better and last longer. MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Contents 1. Presentation of the lab ......................................................................................................... 6 2. Objective of the study ......................................................................................................... 7 3. Data acquisition .................................................................................................................. 9 4. Supervised Cavitation classification ................................................................................. 10 4.1. Manual classification ..................................................................................................... 10 4.2. Supervised Classification with Principal Component Analysis (PCA) ......................... 11 4.2.1. Analysis of the reduced dimensions ........................................................................ 15 4.3. Cavitation state prediction ............................................................................................. 16 5. Unsupervised classification .............................................................................................. 19 5.2. Video pre-processing ..................................................................................................... 20 5.3. Training of the autoencoder ........................................................................................... 20 5.4. Clustering from the reconstructed data .......................................................................... 21 6. Perspectives ....................................................................................................................... 22 7. Conclusion ........................................................................................................................ 23 8. References ......................................................................................................................... 24 9. List of figures .................................................................................................................... 25 MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX 1. Presentation of the lab The Physical Acoustics Lab (PACLAB) is a leading research facility specializing in the application of cavitation for sonochemistry. PACLAB's research encompasses a wide range of applications, including wastewater treatment, CO2 reduction, hydrogen (H2) production, and various catalytic reactions. A significant emphasis of PACLAB's research portfolio is on the investigation of the physical, chemical, and biological effects of acoustic cavitation. This includes the study of sonochemistry, as well as the exploration of single and multi-bubble sonoluminescence. Furthermore, PACLAB's research extends to the realm of photo-acoustics and acousto-optics, where the intricate interaction between light and sound is examined. The lab's active research projects span across diverse fields, encompassing biology, chemistry, and medicine. PACLAB's dedication to cutting-edge research positions it as a hub for pioneering breakthroughs with the potential to address real-world challenges. MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX 2. Objective of the study The production of hydrogen through ultrasonics harnesses the phenomenon of cavitation. Ultrasonic waves induce high-energy chemistry by causing acoustic cavitation, which involves the formation, growth, and explosive collapse of bubbles in a liquid. During the collapse of these cavities, the bubbles reach extremely high temperatures of around 5000°C, pressures of approximately 500 atmospheres, and have a lifespan of just a few microseconds [2]. The shockwaves generated by cavitation result in high-speed inter-particle collisions, powerful enough to break apart molecules, hence their use in hydrogen production. There are two primary types of cavitation: stable and inertial. Stable cavitation occurs when the pressure and temperature conditions generated during bubble collapse remain relatively constant and moderate. Unlike inertial cavitation, stable cavitation bubbles do not collapse explosively. Instead, they go through a cycle of formation, growth, oscillation, and contraction without spectacularly imploding. This process is generally less energetic than inertial cavitation. Stable cavities are often used in applications such as ultrasonic cleaning, particle dispersion, or the release of encapsulated compounds. In these cases, stable cavitation is preferred as it generates fewer equipment and environmental damages. In contrast, inertial cavitation is characterized by extreme conditions during bubble collapse. This violent implosion generates shockwaves and intense shear forces, making inertial cavitation highly effective at breaking chemical bonds in the liquid. Applications of inertial cavitation are often associated with processes requiring high energy. When inertial cavitation bubbles collapse, they trigger more potent chemical reactions due to the extreme temperature and pressure conditions. These two types of cavitation lead to different hydrogen production outcomes. The goal of this study is to better understand these cavitation types to make precise predictions of hydrogen production rates using ultrasonics. Since cavitation is a phenomenon challenging to model using conventional physics laws, we will employ data science. From these predictions, we can gain insights into the conditions that favor higher production rates. A deeper understanding of cavitation behavior will directly enable the design of more efficient and robust sonochemical reactors. In fact, inertial cavitations have the advantage of releasing more energy, thereby promoting chemical reactions, but they can also potentially damage the sonochemical reactor. Therefore, it is imperative to control the occurrence of both types of cavitation to design sonochemical reactors that are both efficient and durable. To achieve this, we will initially create a predictive model of cavitation behavior (inertial or stable) based on acoustic data and videos provided by the BUBBL lab. Using this model, we can better study and comprehend the acoustic noise in ultrasonic hydrogen production, thus optimizing the sonochemical efficiency of H2 production. MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Below are the main stages of our research, and in red, the part studied in this report: Figure 1: Flowchart of the key steps of the model MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX 3. Data acquisition The dataset utilized in this study was provided by the BUBBL Lab. The data acquisition process primarily revolved around the Grand Unified Bubble experiment (GUB), designed to capture high-speed video recordings and passive acoustic maps (PAM) of bubbles subjected to various conditions, involving alterations in amplitude and frequency within a simplified experimental setup. In the GUB experiment, an air bubble was introduced through a lipid membrane and transported through a tube using a pressure gradient. As the bubble reached the focal zone of the high-intensity focused ultrasound (HIFU) transducer, the transducer initiated controlled interactions with the bubble. Simultaneously, the passive acoustic mapping (PAM Verasonics) machine was employed to record the acoustic emissions generated by the bubble. The PAM system captured noise data in an array format, allowing for precise bubble localization and the acquisition of emissions from individual, isolated bubbles within the experimental setup. This data acquisition methodology provided a comprehensive dataset for our analysis, offering valuable insights into the behavior of bubbles under varying experimental conditions. Here is a diagram of the experimental set-up: Figure 2: Diagram of the experimental set-up for data acquisition MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Figure 10: First factorial plan with a gradient of color for the frequency of max intensity This graph validates our initial hypothesis. It illustrates that stable cavitations exhibit their most intense harmonics at higher frequencies (between 7 and 10MHz), whereas inertial cavitations manifest more substantial power intensity at lower frequencies (between 2 and 5MHz). This confirmation underscores the practicality of initially assessing cavitation states by observing their spectrograms, particularly by identifying the frequency at which power intensity (in dB) reaches its maximum. 4.3. Cavitation state prediction In order to apply machine learning, and thus predict the state of the cavitation based on its spectrogram, we will first plot the time averaged spectrogram on the first factorial plan. This means that each spectrogram will correspond to one point on the graph, as we can see in the graph below: MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Figure 11: Time averaged first factorial plan with classification Using this graph, we will apply various machine learning methods and assess their performances. The objective remains the same: predicting the cavitation state based on its spectrogram. To ensure the robustness of our assessments, we employed cross-validation. Cross-validation is a statistical method involving the division of the dataset into several subsets, known as "folds," and evaluating the model's performance using these folds iteratively. In essence, it simulates learning and evaluating the model on multiple distinct subsets of data, reducing overfitting risk and providing a more reliable estimate of machine learning method performance. The results obtained through cross-validation enable us to objectively compare the different methods we tested and determine the one that offers the best performance for our cavitation state prediction task based on the spectrogram. Here is a brief description of the methods employed: 1. Support Vector Machine (SVM): SVM is a widely used supervised learning technique for classification and regression. Its primary goal is to find a hyperplane that optimally separates different data classes, maximizing the margin between classes while minimizing classification errors. SVMs are effective for processing highdimensional datasets and can handle both linear and non-linear data through specific kernels. 2. K-Nearest Neighbors (KNN): The KNN algorithm is a simple and intuitive supervised learning method. It operates on the principle that similar data points are generally close to each other in the feature space. KNN considers the k nearest neighbors of a test point and assigns the majority class among these neighbors to the test point. It is sensitive to the choice of k and can be used for both classification and regression. 3. Logistic Regression: Logistic regression is a commonly used supervised learning technique for binary classification. Contrary to its name, it addresses classification MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX problems rather than regression. Logistic regression models the probability that an observation belongs to a particular class using a logistic function. It is relatively simple to understand and implement and can also be extended to multiclass classification problems. 4. Random Forest Classifier: The Random Forest classifier is an ensemble learning technique that combines multiple decision trees to improve accuracy and reduce overfitting. Each tree is trained on a random subset of training data and features. Subsequently, predictions from each tree are aggregated to produce the final prediction. Random Forests are robust, suitable for high-dimensional data, and can handle complex datasets. 5. Artificial Neural Network (ANN): Artificial Neural Networks (or ANN) are a form of deep learning inspired by the human brain's functioning. They consist of interconnected layers of neurons that process and transform data. Neural networks can be used to solve a wide range of tasks, including classification, regression, and pattern recognition. Their complexity and ability to learn complex representations make them a powerful method, although their training may require a substantial amount of data and computational resources. The performance of each method is summarized in the table below: Method Prediction performance using cross validation Support vector machine (SVM) 55.3% K-Nearest neighbours (KNN) 58.2% Logistic regression 79.3% Random Forest Classifier 82.5% Artificial Neural network 85% Table 1: Comparison of Prediction Performance with Cross-Validation for Different Machine Learning Methods The results indicate that the Artificial Neural Network demonstrated the highest prediction performance at 85%. It is therefore the method of choice for future use of the model on experimental data. MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX 5. Unsupervised classification The method we are about to discuss eliminates any arbitrary decisions. Up until now, spectrogram processing has been automated using PCA, but video processing has been manual. We had to process them one by one and determine whether the cavitation was more stable or inertial. This procedure is time-consuming and introduces bias into the classification. By utilizing unsupervised classification, we overcome this bias and make the model training automatic and robust. To achieve this, we will use autoencoders with the cavitation videos and acoustic noise spectrograms as input data. Autoencoders are artificial neural network models that play a crucial role in dimensionality reduction and data compression while preserving their structure and important features. When used in the context of unsupervised classification, as in our case, autoencoders become powerful tools. The fundamental principle of an autoencoder is to take raw data as input and compress it into a lower-dimensional latent representation, called the coding space. This latent representation should capture crucial information from the input. Then, the autoencoder attempts to reconstruct the original data from this latent representation. To do this, it employs a decoder that transforms the latent representation into an output that closely resembles the initial input, as illustrated in the diagram below. Figure 12: Diagram explaining how an autoencoder works [1] Training an autoencoder involves minimizing the difference between input data and output data, compelling the model to learn a compact and informative representation of the input. This compression ability of autoencoders is valuable for reducing the dimensions of complex data like videos while retaining their essential content. In our case, autoencoders will allow us to automatically process videos and acoustic data by capturing key cavitation characteristics in a latent representation. This eliminates the need for manual and subjective analysis, thus reducing potential bias. By employing unsupervised MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX classification with autoencoders, we make our model training process more automatic and robust, which is essential for efficiently analyzing a large volume of video data. 5.2. Video pre-processing To ensure the extraction of relevant information from the videos, we will extract two key pieces of data: the number of bubbles over time and the area of the primary (largest) bubble over time. In this data extraction process, each video undergoes an image processing procedure, including contrast adjustment, adaptive binarization, contour detection, and calculation of the area of detected shapes. These data are then graphically represented as time series curves, enabling in-depth analysis of area variations and the number of bubbles in different videos. This process ensures precise extraction and clear visualization of the essential features of the studied videos. Below is an example of data extracted from the video available here : Figure 13: Extracted data from a cavitation video; Area and number of bubbles as a function of time 5.3. Training of the autoencoder To begin, we prepared the input data by normalizing and transforming them into feature vectors. These data encompass both the information extracted from the videos and the acoustic noise data. Next, we merged these data into a single dataset and converted them into tensors for efficient manipulation. The subsequent step involved training the autoencoder, with the objective of learning to reconstruct the input data. To achieve this, we used Mean Squared Error (MSE) loss as a measure of reconstruction quality and the Adam optimizer to fine-tune the autoencoder's parameters. MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX The data encoded by the autoencoder were considered crucial as they contained essential features from the original data. These encoded features were then used for data clustering using the K-Means clustering algorithm. Below is an example of data reconstructed using the autoencoder: Figure 14: Original vs reconstructed data; Area of the bubble as a function of time 5.4. Clustering from the reconstructed data After successfully training the autoencoder, which learned to reconstruct the input data, we proceeded to the clustering phase. The encoded data produced by the autoencoder played a crucial role as they contained essential features extracted from the original data. These encoded features, often residing in a lower-dimensional space, served as a simplified representation of underlying patterns within our dataset. To form clusters and gain insights into inherent data structures, we employed the K-Means clustering algorithm. K-Means divided the encoded data into distinct groups based on their proximity within this reduced feature space. This clustering process allowed us to group similar observations. We determined the number of clusters to be 2. Indeed, our dataset is too small to reliably identify more clusters. Here is the result obtained: MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX Figure 15: Clustering of the experiences projected on the encoded features Comparing these results with the manual categorization we conducted earlier, we achieved a 76% similarity rate. Our exploration of unsupervised classification utilizing autoencoders shows promise in overcoming bias introduced by manual classification. These autoencoders facilitate automatic capture of essential cavitation characteristics from videos and acoustic data. Our video preprocessing methods ensure precise data extraction, while K-Means clustering effectively groups data. These initial findings suggest a robust and automated approach to cavitation analysis. 6. Perspectives We have now established an efficient and automated model training method. Using videos and acoustic noise data, we can predict bubble behavior. With a larger dataset of videos and acoustic data, autoencoders can enhance the reliability and precision of cavitation categorization by identifying more clusters. This opens the possibility of predicting aspects such as aspect ratio evolution and fragmentation. Autoencoders will allow us to label each acoustic noise data, enabling predictions of cavitation behavior based on their spectra. Additionally, supervised classification using PCA can be explored. PCA's linear nature, unlike the non-linear encoded features, will allow us to study the reduced dimensions of PCA and the spatial arrangement of different clusters, providing a direct link to spectrogram appearances. With this high-performing model, we can advance to the next stage: making predictions regarding hydrogen production. By analyzing spectrograms under various production conditions and comparing them with chemical reaction outcomes, this model will enhance our MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX understanding of ultrasonic hydrogen production. Ultimately, it will enable the design of efficient and resilient sono-reactors for hydrogen production. 7. Conclusion We have been able to demonstrate the feasibility of predicting cavitation behaviour using both supervised methods, such as PCA, and unsupervised techniques, in particular autoencoders. This advance in our understanding of cavitation opens the door to further optimisation of sono-chemical reactors, by improving their efficiency and robustness. By eliminating the need for manual classification that is prone to bias, we have developed a model that can automatically categorise cavitation behaviour. This approach will allow us to increase the accuracy of our predictions as we accumulate more data. The future of this research promises to be even more exciting. By continuing to accumulate video and acoustic data, we will be able to refine our model to predict even more complex aspects of cavitation behaviour, which will have a direct impact on the optimisation of sono-chemical reactors. Ultimately, this study brings us one step closer to achieving more efficient and sustainable ultrasonic hydrogen production reactors. It opens up exciting opportunities to shape the future of clean, renewable hydrogen production. MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX 8. References [1] https://www.compthree.com/blog/autoencoder/ [2] Sonochemistry, Kenneth S. Suslick, Available here [3] GPU-accelerated study of the inertial cavitation threshold in viscoelastic soft tissue using a dual-frequency driving signal, Tatiana Filonets and Maxim Solovchuk, available here [4] Origin of the broad-band noise in acoustic cavitation, Kyuichi Yasui, available here MASTERS OF SCIENCE AND TECHNOLOGY ECOLE POLYTECHNIQUEF 91128 PALAISEAU CEDEX 9. List of figures Figure 1: Flowchart of the key steps of the model ..................................................................... 8 Figure 2: Diagram of the experimental set-up for data acquisition ............................................ 9 Figure 3: Manual extraction of harmonics from acoustic data ................................................. 10 Figure 4: Manual cavitation classification ............................................................................... 11 Figure 5: Example of a spectrogram for one experiment ......................................................... 12 Figure 6: Histogram of the explained variance ........................................................................ 12 Figure 7: First factorial plan ..................................................................................................... 13 Figure 8: First factorial plan with manual classification .......................................................... 14 Figure 9: Linear decomposition of dimension 2 ...................................................................... 15 Figure 10: First factorial plan with a gradient of color for the frequency of max intensity ..... 16 Figure 11: Time averaged first factorial plan with classification ............................................. 17 Figure 12: Diagram explaining how an autoencoder works [1] ............................................... 19 Figure 13: Extracted data from a cavitation video - Area and number of bubbles as a function of time ...................................................................................................................................... 20 Figure 14: Original vs reconstructed data; Area of the bubble as a function of time ............... 21 Figure 15: Clustering of the experiences projected on the encoded features ........................... 22