DNS over HTTPS Detection Using Standard Flow Telemetry

Received 6 April 2023, accepted 3 May 2023, date of publication 12 May 2023, date of current version 26 May 2023. Digital Object Identifier 10.1109/ACCESS.2023.3275744 DNS Over HTTPS Detection Using Standard Flow Telemetry KAMIL JERABEK 1, KAREL HYNEK 2,3, ONDREJ RYSAVY 1, AND IVANA BURGETOVA 1 1Faculty of Information Technology, Brno University of Technology, 601 90 Brno, Czech Republic 2CESNET z.s.p.o, 160 00 Prague, Czech Republic 3Faculty of Information Technology, Czech Technical University in Prague, 166 36 Prague, Czech Republic Corresponding author: Kamil Jerabek ([email protected].cz) This work was supported in part by the Ministry of Interior of the Czech Republic (Flow-Based Encrypted Traffic Analysis) under Grant VJ02010024; in part by the Grant Agency of the Czech Technical University in Prague under Grant SGS23/207/OHK3/3T/18; and in part by the Technology Agency of the Czech Republic (Context-Based Encrypted Traffic Analysis Using Flow Data) under Grant FW03010099. ABSTRACT The aim of DNS over HTTPS (DoH) is to enhance users’ privacy by encrypting DNS. However, it also enables adversaries to bypass security mechanisms that rely on inspecting unencrypted DNS. Therefore in some networks, it is crucial to detect and block DoH to maintain security. Unfortunately, DoH is particularly challenging to detect, because it is designed to blend into regular HTTPS traffic. So far, there have been numerous proposals for DoH detection; however, they rely on specialized flow monitoring software that can export complex features that cannot be often computed on the running sequence or suffer from low accuracy. These properties significantly limit their mass deployment into real-world environments. Therefore this study proposes a novel DoH detector that uses IP-based, machine learning, and active probing techniques to detect DoH effectively with standard flow monitoring software. The use of classical flow features also enables its deployment in any network infrastructure with flow-monitoring appliances such as intelligent switches, firewalls, or routers. The proposed approach was tested using lab-created and real-world ISP-based network data and achieved a high classification accuracy of 0.999 and an F1 score of 0.998 with no false positives. INDEX TERMS DNS over HTTPS, DoH, machine learning, detection, classification, network monitoring, network flows. I. INTRODUCTION Domain Name System (DNS) is one of the core elements of the modern internet that allows translation between human-friendly domain names and IP addresses. Since the DNS is transferred unencrypted, anyone on the path can intercept, analyze and misuse the domain names for user surveillance. Government-operated mass DNS surveillance programs such as QUANTUMDNS or MORECOWBELL proved to be very effective [1] and thus triggered concern about users’ privacy. Therefore, DNS over HTTPS [2] (DoH) and other encrypted DNS approaches have been proposed as a natural response to protect users’ privacy. The associate editor coordinating the review of this manuscript and approving it for publication was Mahdi Zareei . On the other hand, mass DNS encryption also brings additional challenges. Visibility into DNS is crucial for many security systems. DNS traffic analysis can reveal many security threats, such as the presence of malware, policy violations, or data exfiltration using DNS tunneling [3]. There are many computer networks, such as corporate networks or restricted institutional networks, where DNS inspection is justifiable and necessary as an additional level of security protection. Detection of encrypted DNS in the network traffic has become essential since it might indicate malicious behavior [4]. Hynek et al. [3] described multiple types of malware that currently leverage DoH to gain its command and control infrastructure stealthfully. Moreover, DoH is also weaponized in various attack vectors by Red Teams. 50000 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME 11, 2023 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry DoH is widely misused due to its stealthiness in the network. Compared to other encrypted DNS approaches, DoH is designed to blend into regular Hypertext Transfer Protocol Secure (HTTPS) traffic, making its reliable detection challenging. According to Garcia et al. [5], DoH cannot be reliably detected by blocklists of Internet Protocol (IP) addresses and domain names due to multiple small and private DoH resolvers on the internet. Therefore, researchers already use the statistical properties of connections in combination with Machine Learning (ML) to recognize DoH and achieve an accuracy of more than 99%. The high accuracy of these detectors comes with the requirement of additional information about the connections, such as individual packet lengths [6] or median and mode of packet sizes [7], [8]. Extraction of these traffic features is usually not supported by standard network monitoring tools that can be deployed on high-speed networks, which significantly limits the possibility of their deployment. Additionally, survey [3] also identified the difficulty of detecting single query DoH, which needs to be addressed. Since DoH creates a DNS Application Programming Interface (API) over HTTPS, short connections have very similar characteristics to other HTTPS API requests. Moreover, other no-payload parameters, such as HTTP headers, or the size of the HTTP/2 preface, influence the connection shape much more significantly than the DoH payload itself, making the classifier fit specific server settings [9]. Nevertheless, previous studies did not focus on short DoH connection [3] problems, even though they significantly influence the accuracy of DoH detection in the real world. Given that none of the existing proposals provide a satisfactory solution for reliable DoH detection in the real environment. Particularly, they use hard-to-compute features and ML-based classifiers with low but still unacceptable false positive rates—with 1000 flows per second, even 99% false positive rate produce in multiple misclassifications per second and possibly overwhelming the security personnel with a high amount of false alarms [10]. Therefore, this study proposes a novel DoH detector that can work with the standard flow data source, making it deployable to almost any flow monitoring infrastructure (from local area networks to high-speed backbone lines) while still maintaining high accuracy over 99.9%. The standard flow statistics can be obtained from all NetFlow capable appliances, such as switches1, firewalls2, or specialized flow monitoring probes such as Flowmon3. To overcome the information-limited standard flow telemetry data source, a combination of heterogeneous classifiers is used—IP-based, machine learning, and active probing detectors. The industry-proven IP filtering is based on the dynamic list of identified DoH servers, automatically adjusted when the new DoH server is identified in the 1https://www.cisco.com/c/dam/en/us/td/docs/security/stealthwatch/ netflow/Cisco_NetFlow_Configuration.pdf 2https://www.cisco.com/c/en/us/td/docs/security/asa/special /netflow/asa_netflow.html#bidirectionalflows 3https://www.flowmon.com/en/products/appliances/probe monitored communication. The ML-based DoH classifier is used to identify candidate DoH servers by performing an online analysis of flow data, which are then confirmed by the active probing detector resulting in almost zero false positives. Moreover, the proposed multi-classifier approach can detect even short DoH connections, which are neglected by existing works. The contributions of this paper can be summarized as follows: 1) This paper explores the properties of short DoH connections which pose the main limitation of state-ofthe-art DoH detectors. The findings were then used for false positives reduction, which is essential for production-grade threat detectors. 2) This paper proposed a novel DoH detector that can correctly operate with standard flow telemetry, making it deployable to almost any monitoring infrastructure ranging from small local area networks to large Internet Service Provider (ISP) based infrastructures connecting millions of devices. 3) This paper proposed a feedback detection system that effectively mitigates the disadvantages of using limited network telemetry data. 4) Compared to previous studies, this paper evaluates the novel detector on the additional independent dataset containing traffic from more than 11 different resolvers, simulating the behavior of the same network making the evaluation more realistic due to significant inconsistencies in DoH resolvers’ behavior [9]. Moreover, the evaluation is also made on real-world data captured from a completely different network to analyze the robustness of the approach. This paper is organized as follows: Section II summarizes related works. Section III provides background knowledge about DoH, flow monitoring, and datasets used for classifier training and overall system evaluation. Section Vintroduces the concept of the proposed detector. Section VI evaluates the whole detector and presents its performance. Section VII discusses the detection results and its limitations. Finally, Section VIII concludes this article. II. RELATED WORK Since its proposal by RFC8484 [2] in 2018, DoH was intensively studied from various perspectives. The survey performed by Hynek et al. [3] summarizes the DoH research and divides it into four main perspectives: 1) DoH performance measurements, 2) Research on DoH adoption, 3) Privacy perspective, and finally, 4) DoH security research part where DoH detection belongs. The first mention of the necessity of DoH detection was in 2020 by Bumanglag and Kettani [4] in their survey about the impact of mass DoH deployment. In the same year, Bushart and Rossow [21] used a list of DoH resolver IP addresses to recognize DoH in their traffic fingerprinting approach. In addition, Garcia et al. [5] studied the completeness of VOLUME 11, 2023 50001 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry TABLE 1. Comparison of related work metrics. The abbrevations stands for: NF - Number of required statistical features; DDataset; SNumber of DoH Servers in the dataset; SFE - Short flow elimination before passed to ML model; DoHBrw -CIRA-CIC-DoHBrw-2020 dataset [11]. publicly available DoH blocklists. According to their conclusions, IP-based DoH blocklisting is not currently efficient due to the incompleteness of publicly available blocklists. One of the first DoH detection approaches that used teletraffic engineering to recognize DoH was proposed by Vekshin et al. [6]. They studied the shape of the DoH traffic using flows extended for information about the first 30 individual packets — packet lengths, packet times, Transmission Control Protocol (TCP) flags, and direction. They extracted 18 discriminatory features from those extended flows that proved efficient in DoH recognition, and their detector achieved an accuracy of 99.6%. DoH detection was also studied by MontazeriShatoori et al. [7]. They use extended flows with 28 traffic features along with machine learning to distinguish DoH from regular HTTPS traffic. They evaluated multiple machine learning approaches, and the best one achieved an F1 score of 0.993. Moreover, they published the CIRA-CIC-DoHBrw-2020 [11] dataset used during their study, which became the de-facto standard dataset for DoH detection. Banadaki [8] designs his method using the CIRA-CICDoHBrw-2020 dataset, which achieves an accuracy of 100%. His approach utilizes machine learning algorithms with timerelated features. However, he also uses IP addresses and ports in its feature vectors, which is criticized by Behnke et al. [16]. Behnke et al. then improved the Banadaki proposal by further exploring his feature vector. They removed overfitting features (IP addresses and ports), further reduced statistically insignificant features, and finally measured several trained models. Their improved approach achieved a high F1 of 0.998. Following studies that also used CIRA-CIC-DoHBrw2020 dataset made by Casanova and Lin [15], Jha et al. [14], Wu et al. [22], Zebin et al. [20] and Mitsuhashi et al. [17] also performed well in DoH detection tasks achieving over 99% accuracy or 0.99 of F1 score. The majority of DoH detection studies use CIRA-CICDoHBrw-2020 for evaluation; however, there are also works that created their own dataset. A recent approach by Nguyen and Park [18] proposed a Transformer Neural Network to detect DoH. They used custom datasets and CICFlowmeter to extract 29 features and achieved F1 score of 0.99. Even though all mentioned studies claimed very high accuracy, there is still a significant limitation preventing their mass deployment. Either they are packet-based or require specialized data sources — flows extended for information about individual packets as in the case of Vekshin et al. [6] or non-standard time-related features as created by DoHLyzer tool and provided within CIRA-CIC-DoHBrw-2020 dataset or by CICFlowmeter, which are then used by several researchers [7], [8], [15], [16], [17], [18], [22]. These approaches require features such as median or mode of packet lengths, which cannot be computed on the running sequence. It means that the monitoring apparatus needs to hold the information about all packets in the memory. The computational and memory complexity prevents its deployment on monitoring infrastructures where hardware resources are limited— router or high-speed monitoring probes. The only approach which relies on standard telemetry data was proposed by Konopa et al. [19], who trained a standard multilayer forward neural network to distinguish between DoH, regular HTTPS, and HTTP. Nevertheless, it achieved only 94.4% accuracy— significantly lower compared to other proposals. Table 1summarizes previous flow-based detection methods. Compared to the majority of approaches, the proposed method uses standard telemetry data obtained from all flow-monitoring devices while achieving similar or better accuracy. The flow-monitoring approach is standard and widely deployed. For example, the Steinberger et al. [23] survey showed that 70% of network operators have devices supporting flow export. Moreover, the proposed method uses only features that can be computed on running sequences, making it deployable to any flow monitoring infrastructure, even those monitoring high-speed ISP networks. The only approach that uses standard flow telemetry data is from Konopa et al. [19]. However, its classification performance is relatively low, which prevents its deployment in 50002 VOLUME 11, 2023 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry practice—one misclassification occurs every 20 flows. The proposed method outperforms Konopa et al. [19] by almost five percentage points. Moreover, we are the first flow-based approach that uses more than ten different DoH resolvers, which is crucial for trustworthy evaluation due to the entirely different behavior of some DoH implementations [9]. III. BACKGROUND This section contains the necessary background information about DoH protocol and flow-based network monitoring. A. DOH PROTOCOL DoH [2] uses HTTPS secure communication to transfer DNS messages protecting the domain name resolution against eavesdropping. The DoH is designed and recommended to be used in cooperation with HTTP/2 [24] to maintain its efficiency due to the possibility of sending multiple concurrent requests. The protocol utilizes the features such as creating multiple streams within a single TCP connection and multiplexing using separated frames to transmit headers and data. Nevertheless, despite the RFC recommendations, some DoH resolvers only support HTTP/1 [5]. DNS messages, either queries or responses, are encoded in DNS wire format [25], which is a compact binary representation where the average size of each message payload is about a hundred bytes [26]. Also, DoH uses only GET and POST methods for queries. DNS queries are transmitted to a resolver in the body of HTTP POST methods encoded using DNS wire format or embedded in the query string of the GET method encoded as Base64. To identify DNS data in the HTTP stream, string application/dns-message should be used for Content-Type header in all DoH messages and in Accepts header field for queries. Except for new headers, possible cache-control options, and the body data format, RFC 8484 does not define any additional specialties that should be included in the application messages. However, the amount of headers is not limited, and DoH clients and servers are free to include more headers in DoH messages as necessary. The considerable leeway in DoH specification leaves space for many server behavior discrepancies. According to previous research [9], DoH traffic shape significantly depends on the client and server implementation, HTTP headers, and used protocol. Currently, DoH is used by default in all major browsers, such as Chromium4based browsers or Mozilla Firefox, for users in the US5. Additionally, DoH is already supported on the OS level in all major operating systems6. 4https://blog.chromium.org/2020/05/a-safer-and-more-private-browsingDoH.html 5https://www.zdnet.com/article/mozilla-enables-doh-by-default-for-allfirefox-users-in-the-us/ 6https://www.inspire2rise.com/how-to-use-dns-over-https-on-windowsandroid-mac-ios.html TABLE 2. Flow record features available across multiple Flow Export protocols. B. NETWORK MONITORING The flow-based network monitoring approach is one of the prevalent approaches [27], mainly due to its flexibility and broad support by networking hardware [23]. The diagram of typical flow monitoring infrastructure is shown in the left part of Figure 1and consists of network probes or network devices capable of flow export distributed throughout the monitored network. The probes observe the ongoing traffic and create statistical flow records, which are then sent to a single collector device, where the analysis and post-processing are performed. Flows record can be defined as aggregated information about packets transferred through the network that shares some common properties [27]. The common properties are called flow-key and usually consist of IP addresses, ports, and transport protocol. Besides the flow key, there are no standardized features that each flow record needs to carry. Flow traditionally contains the number of transferred bytes and packets [28]. These standard flow features can be extended for information from an application layer (such as HTTP headers, DNS payload, or Transport Layer Security fingerprints) or statistics of the aggregated communication. However, these extended flow fields are not standard and vary widely based on monitoring infrastructure [27]. The flows are transmitted between probes and collectors using flow export protocols. Multiple standard flow export protocols are currently in use [27]—NetFlowV5 [29], NetFlowV9 [30], and IPFIX [31]. While NetFlowV9 and IPFIX support templates, and thus it is possible to customize the flow features highly, NetFlowV5 has fixed records and does not allow the addition of flow features [29]. NetFlowV5 thus highly limits the feature availability; however, according to Hofstede et al. [27], it is still the most used flow export protocol. Table 2shows features that are considered standard flow features [27] and are supported by all currently used flow export protocols. According to Hofstede et al. [27], flow records usually describe only unidirectional communication. Thus the bidirectional network communication is split into two records, aggregating packets in a single direction. When necessary for additional analysis, the bidirectional flows can be created from two unidirectional by applying flow stitching on the collector. VOLUME 11, 2023 50003 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry TABLE 3. Overall information about used datasets containing the number of used DoH resolvers, number of DoH flows, and the number of Non-DoH flows. IV. DATASETS High-quality datasets are crucial for designing a reliable and accurate network classifier. The de-facto standard dataset created for DoH detection is called CIRA-CIC-DoHBrw2020 [11], and it is used by the majority of related works (see Section II). The CIRA-CIC-DoHBrw-2020 contains DoH and regular HTTPS traffic in the form of extended flow records and pcap files. The traffic, either DoH or regular HTTPS, was generated by Firefox and Chrome browsers querying only four DoH resolvers at varying network speeds. Nevertheless, the behavior of DoH resolvers and browsers significantly differ as described in previous research [9] (see Section III-A); thus, the four resolvers in CIRA-CICDoHBrw-2020 cover just a fraction of DoH traffic shapes that can be observed on the internet. To overcome the limited number of DoH resolvers in the CIRA-CIC-DoHBrw-2020 dataset, the DoH Network Traffic dataset [12] is also used. This dataset contains both DoH and web session HTTPS traffic resembling the characteristics of a real-world environment. The HTTPS traffic was generated by accessing web pages taken from the Majestic Million list7. The DoH queries are generated against 11 different DoH servers making the dataset more comprehensive. Packet captures (pcap files) from both CIRA-CICDoHBrw-2020 and DoH Network Traffic datasets were merged into one Design dataset, which was used for detector design. The packet captures were then processed by NetExp tool8, aggregating the packets into standard bidirectional flow records. The whole proposal was evaluated using completely different datasets to make a trustworthy and independent evaluation. For this purpose, the generated and real-world parts from the huge collection of DoH datasets [13] were used. The generated part named Evaluation dataset was created similarly to CIRA-CIC-DoHBrw-2020 and DoH Network Traffic; nevertheless, it was created in a different time and contained traffic towards more DoH servers; thus, its use for evaluation simulates the system’s real-world deployment in the same environment. The second part represents capture from a real-world network called Real World dataset which simulates deployment of the method in a different environment and helps evaluate the robustness and sensitivity to data drift. The summary of all used datasets is in Table 3. 7https://majestic.com/reports/majestic-million 8https://github.com/kjerabek/netexp Algorithm 1 Flow Records Labelling Input: list_of _flows,rules,classifier,length Output: setting of flow.doh 1: for all flow in list_of _flows do 2: if flow.dst_ip is in rules.doh then 3: flow.doh ←true 4: else if flow.dst_ip is in rules.non_doh then 5: flow.doh ←false 6: else if flow.length ≤length then 7: flow.doh ←false 8: else if not classifier(scaling(features(flow))) then 9: flow.doh ←false 10: else 11: correct_result ←doh_query(flow.dst_ip) 12: if correct_result then 13: flow.doh ←true 14: rules.doh.add(flow.dst_ip) 15: else 16: flow.doh ←false 17: rules.non_doh.add(flow.dst_ip) 18: end if 19: end if 20: end for V. THE PROPOSED DOH DETECTOR Since the detector is constrained to use traditional flow telemetry with the standard features (see Section III-B), it cannot solely rely on a statistical machine-learning method as that would result in a high number of false positives. Instead, the deployment of a heterogeneous detection approach was chosen. The detector utilizes three different types of detection—IP-based, ML-based, and active probing detection. The detection pipeline uses a feed-forward loop, where a reliable but resource-consuming verification step creates a blocklist/allowlist, which is then utilized by a fast IP-based detector. To limit the number of active probes, an ML-based classifier was deployed, which selects the DoH-suspicious flows that are worth verifying. Using three different classifiers, the proposed approach overcomes each detector limitation: The obsolescence of IP lists is mitigated by active verification and continuous updates, the ML inaccuracy is mitigated by active verification, and the resources needed by active verification are minimized by IP list and ML prefiltration. The processing pipeline of the proposed detection system is described by Algorithm 1and further depicted in Figure 1labeled as DoH Detector and can be divided into six following steps: Flow Stitching is an optional step needed to be deployed into flow monitoring infrastructure with unidirectional flow records. Since the proposed approach requires bidirectional flow records, which are still not widely adopted by flow monitoring infrastructures [27], their reconstruction from two unidirectional records is needed. 50004 VOLUME 11, 2023 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry FIGURE 1. Holistic view of the proposed approach with the DoH detector design in context of the monitoring infrastructure. IP-based Detection uses knowledge acquired from previous IP inspections to recognize DoH resolvers. In the absence of any previous knowledge about the destination IP address, the flow is forwarded to the next step. Filtration step performs pre-filtration of flows. When the flow is too short for reliable DoH detection, it is directly labeled as non-DoH. Other flows proceed to the following step. Further information about this step is provided in Section V-D. Classification uses machine learning to detect DoH traffic. Moreover, this step also performs feature extraction. A detailed description of this step is provided in Section V-C. Verification step uses active DoH queries to the suspected DoH resolver to confirm its DoH resolution capability. When verified, the DoH resolver’s IP address is stored in the DoH blocklist, which is then used in the filtration step. This step is further described in Section V-E. IP Rule Extraction processes the detection results and maintains a blocklist/allow list which is then used by the IP-based detector. Each component is described in the following sections, their description follows the order as they are depicted in Figure 1except for the classification and filtration. Thenecessity of the filtration step arises from the limitations posed by the ML classifier and only standard NetFlow telemetry data. Hence it is described after the classification. Measurement and demonstrations in the following sections are made on the design dataset. The evaluation and real world datasets are held exclusively for the final evaluation of the whole system (see Section VI). A. FLOW STITCHING Flow stitching performs the conversion of unidirectional flow records into bidirectional. Flow stitching is a standard process that can reconstruct one bidirectional flow record from two TABLE 4. Bidirectional flow record after application of flow stitching. The fields denoted in italic are statistical features and thus can be used for detection. Other features are specific to the dataset and should not be used. unidirectional records. It is often placed or implemented on the collector. Multiple open-source tools are capable of flow stitching, such as BiFlow Aggregator9or Cisco Joy10. The features of bidirectional flows created by stitching are shown in Table 4. These flow records are then directly forwarded to the IP-based detection stage. Naturally, the flow stitching adds some latency to the detection pipeline depending on the aggregation time window of the stitching tool. The aggregation windows should be set for particular monitoring infrastructure and its overall latency and jitter. Nevertheless, since flows from both directions are going to be exported at a similar time, the flow stitching aggregation time window could be very small in the order of seconds; hence similar latency would be added. 9https://github.com/CESNET/Nemea-Modules/tree/ master/biflow_aggregator 10https://developer.cisco.com/codeexchange/github/repo/cisco/joy VOLUME 11, 2023 50005 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry B. IP-BASED DETECTION The IP-based detection uses previously acquired knowledge about server IP addresses, maintained by IP Rule Extraction and the database, to prevent unnecessary active verification. It directly detects DoH or regular HTTPS by observing the servers’ IP address field in the flow. When there is no prior knowledge about the servers’ IP address, the flow is forwarded to the next stage. C. CLASSIFICATION The classification step uses a machine-learning classifier for an additional selection of DoH-suspicious flows. The description of the classification step is provided before filtration since the ML classifier showed unsatisfactory results (but comparable with previous proposals) when used with short connections and only NetFlow telemetry data. This limitation is then mitigated by the filtration step, which is described afterward. The machine-learning-based classifier relies solely on the statistical properties of the flows to learn the DoH traffic shape. This section describes the process of creating the machine learning model for DoH detection. 1) FEATURE SELECTION Identifying discriminative features from the incoming flow data is one of the essential tasks during the detector design. The selected features directly influence the detector’s performance. To determine features properly, the analysis of the Design dataset was performed. Moreover, the information from previous research [9] has also been used. The following DoH characteristics can be observed that can discriminate DoH from other HTTPS traffic: Observation 5.1: The DoH, in comparison to non-DoH HTTPS traffic, transmits fewer data in requests and responses. Observation 5.2: The DoH connections last longer, containing many short transactions, creating an overall higher number of smaller packets in the flow. Observation 5.3: DoH in browser creates multiple streams over the same HTTP/2 connection and uses multiplexing for faster DNS resolution. Observation 5.4: The packet size variance of DoH is lower compared to other HTTPS traffic. Observation 5.5: The DoH connections are more symmetrical, than regular HTTPS in terms of transferred packets and bytes in each direction. The statistical bidirectional flow feature set is relatively limited, as seen in Table 4. Instead of using features directly, all possible pairs were created by applying division, leaving us with 21 features—ratios of the original features. Consequently, a feature reduction step was performed by calculating the pair-wise Pearson correlation coefficient. The feature reduction removed features that showed near-perfect correlation—their Pearson correlation coefficient was higher than 0.9 [32]. TABLE 5. Selected features together with their Gini importance. The abbreviation stands for: C→Srepresents Client-to-Server direction, S→Crepresents Server-to-Client direction, sc-bytes/sc-packets represents number of transferred bytes/packets in S→Cdirection, cs-bytes/cs-packets represents number of transferred bytes/packets in C→Sdirection. The final feature set is listed in Table 5with the corresponding importance based on the computed Gini index. The final features are also related to traffic observations. The most essential feature #1 and also feature #3 capture the DoH behavior from observations 5.1 and observations 5.2. Feature #4 captures the flow symmetry property from observations 5.5. observations 5.3 is captured by feature #2. Unfortunately, used features cannot describe observations 5.4 since it cannot be computed or approximated from traditional flow data. 2) MACHINE LEARNING CLASSIFIER Given the features identified in Section V-C1, several classification algorithms were applied to the dataset. Python programming language was employed using Scikit-Learn11 and XGBoost12 libraries for learning classification models. Random Forest, K-Nearest Neighbors (KNN—in this case, 4NN was used, which was chosen as the best performing during the hyperparameter tuning phase), Naive Bayes classifiers, and popular boosted algorithms XGBoost and AdaBoosted Decision Trees classifiers were utilized. The detailed description of these ML-based algorithms, which are commonly used in networking tasks [33] is provided by Han et al. [34]. The standard metrics for unbalanced datasets was used to determine the best-performing algorithm for DoH detection. In particular, precision (1), recall (2), and F1 measure (3) were calculated13. The primary measure is the F1 measure showing the classifier’s overall performance. However, the precision metric is also important since practical applications in network monitoring aim to reduce the number of false positives. precision =TP TP +FP (1) recall =TP TP +FN (2) F1=2·precision ·recall precision +recall (3) The process of algorithm evaluation followed a standard machine-learning classifier design. First, the Design dataset 11https://scikit-learn.org 12https://xgboost.ai 13Abreviations have following meaning: True Positives (TP), False Positives (FP), False Negatives (FN), True Negatives (TN) 50006 VOLUME 11, 2023 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry TABLE 6. Performance metrics of classifiers. TABLE 7. Confusion matrix for XGBoost classifier. (see Table 3) was divided into the train (75%) and test (25%) parts using stratified sampling. The train part was used for hyperparameter tuning using cross-validation and training the classifier, and the validation part was reserved for classifier evaluation. Then the features within each part were standardized using Z-score normalization (also called standardization) since the feature values fall into different ranges. Feature normalization was applied, particularly for features #1, #2, and #3 from Table 5, which contain values ranging from 0 to hundreds. Each algorithm requires setting proper input parameters, also called hyperparameters. The hyperparameter tuning of all algorithms was done experimentally using a grid-search technique and 5-fold cross-validation on the train part. Models with the best hyperparameters were then trained on the whole train part and evaluated on the test part. Table 6 presents performance results computed using standard metrics numerically. Most algorithms performed well, capable of achieving an F1 score of higher than 0.95, except for the Naive Bayes. Among them, the best performance is achieved by boosting algorithms. The XGBoost algorithm had the best performance and achieved F1=0.973 with the following hyperparameters: maximum depth of seven and subsample of 0.9. The detailed performance of the best-performing XGBoost classifier for DoH recognition can be seen in Table 7. Although the F1 of 0.973 can be considered high accuracy, it may still not be accurate enough for deployment on the computer network due to a high number of predictions per second. According to Hofstede et al. [35], the Czech national research and educational network create up to 10000 flows per second—the same number of predictions per second would be made by the classifier when deployed on this network. Therefore, the classifier would produce around 2.6 false positive14 detections per second (9360 false positive detections per hour). Generating such a large number of misclassifications would overwhelm the administrators and the system, causing such detection not to be beneficial. Hence, 14FP/(FP+FN+TP+TN) = 0.0026 FIGURE 2. Stacked histogram of misclassified flows. FIGURE 3. Schema of HTTPS (DoH) flow with all connection prefaces highlighted. there is a need to push the detection system toward more accurate detection. Further investigation revealed that short flows are the main cause of reduced accuracy. Figure 2shows the number of false positive detections depending on the number of packets in flows. It can be seen that misclassification happens more often when flows have less than 100 packets. To reduce false positives, the elimination of shorter flows is needed since those cannot be reliably recognized by the machine-learning classifier that uses standard flow information. D. FILTRATION The DoH connection (similarly to other HTTPS connections) involves a TCP handshake, Transport Layer Security (TLS) handshake, HTTP/2 preface, application data transfer phase, and TCP termination, as shown in Figure 3. Each phase amounts to several packets exchange. Based on the measurement [9], it was determined that one DNS request and response over HTTP/2 requires (on average) 28 packets (HTTP/1 requires on average only 17 packets15), with only two packets carrying the actual DNS payload. The shorter flows might not contain any DoH queries at all. Browsers usually create multiple connections to the target server to optimize the loading process. The browser contacts 15Lower number of packets is caused by missing novel HTTP/2 features such as connection preface, multiplexing and HTTP header and payload transmission separation. VOLUME 11, 2023 50007 K. Jerabek et al.: DNS Over HTTPS Detection Using Standard Flow Telemetry FIGURE 4. Minimum number of packets in flow hyperparameter tuning. the server with several connections—at least one connection is then used, and the rest are failed or terminated16. Such connections do not transmit any application data. Hynek et al. [3] mentioned short DoH connections as challenging. Particularly the single-query DoH produces flows with similar statistics as other short API calls. The influence of many HTTP/2 preface packets compared to the small portion of DoH data is high and complicates reliable DoH detection. Therefore, the determination of the minimal number of packets in the flow carrying enough DoH packets is needed to recognize DoH from other HTTPS traffic reliably. 1) DETERMINING THE MINIMAL NUMBER OF PACKETS THRESHOLD There are multiple ways to determine the minimal number of packets in flow needed for reliably distinguishing DoH from regular HTTPS traffic. One of the approaches would be to add the threshold to hyperparameters and find the best value during the hyperparameter tuning phase. Another way would be to use an unsupervised clustering method to group samples automatically and measure the ability to separate samples into two classes by measuring the intra-class similarity. Both approaches were used. At first, the experiments with threshold tuning were performed. The best-performing XGBoost algorithm was used, and the minimum packet threshold was incrementally increased. The model was retrained each time the threshold was raised. As expected, the model’s accuracy increases with a higher threshold, as shown in Figure 4. The model reaches first stable performance around 116 packets and then maintains similar accuracy. In the second approach, K-Means [34] clustering method was used with k-means++ initialization to determine the minimal packet lengths. It was done by using the algorithm’s 16The number of parallel connections created by a browser is not specific for DoH and can often be configured [36]. FIGURE 5. Purity score of the converged clusters based on minimum packets in flows. ability to group samples into a predefined number of kclusters (in this case k=2, the same as the number of classes) based on their similarity without the need to label the samples. Applying the algorithm to all flows without filtration would lead to a different capability of distinguishing the samples reliably than when applied to filtered flows. The dataset without filtration was clustered and then the required number of packets in flows was increased. The intra-class similarity was measured by calculating the purity score of the resulting clusters. Since the dataset is highly imbalanced in favor of the non-DoH class, random undersamplingwas appliedbefore each clustering to balance the dataset and correctly compute the purity score. Undersampling rather than over-sampling was chosen since there were enough data to work with and it works only with real data records without the need for any data augmentation. The results are depicted in Figure 5. It can be seen that the minimal number of packets in flow highly influences the DoH recognition capability of the clustering method. More packets in flows make the recognition more reliable, which is consistent with previous observations. The 90% of purity index of both clusters is reached when flows have at least 112 packets. In addition, the estimation of the average number of packets in the DoH connection needed for fetching one website was performed. The DoH connections of the top 10000 visited websites from the Majestic Million dataset were investigated. The average number of unique domain references by these websites was 20, leading to an equal number of DNS resolutions. Measured in terms of packets, this gives roughly 120 packets in a DoH flow corresponding to one website visit. All experiments showed satisfactory DoH detection performance with similar values ranging from 112-116 packets, while flows with more packets are generally more reliably recognized. It was decided to set the filtration threshold 50008 VOLUME 11, 2023