Cyber-attackers manipulating sensor data can deceive cyber-physical systems (CPSs), potentially leading to hazardous conditions in physical plants. A physics-based Anomaly Detection System (ADS), such as the CUSUM, aims to detect ongoing attacks by comparing sensor signals with those generated by a model. Given the inevitability of noise, physics-based methods are necessarily threshold-based, which can result in both false positives and undetectable stealth attacks. As a result, attacks can remain undetected while impacting the state of the system, leading to potentially large deviations with respect to the desired behavior. It is, therefore, of interest to be able to assess the worst-case impact of such attacks on the system state, depending on the selected ADS threshold. Although the tradeoff between sensitivity and false positive rate in tuning the ADS threshold is well-understood, sensitivity is a concept that is not uniquely defined. In this paper, we introduce a metric called transparency that uniquely quantifies the effectiveness of an ADS in terms of its ability to prevent state deviation. While existing research presents optimization-based approaches for designing optimal zero-alarm stealth attacks, an unaddressed challenge remains to detect more sophisticated multi-alarm attacks. These hidden attacks can generate alarms at a rate comparable to nominal conditions induced by the system noise. Leveraging the concept of transparency, we analyze the conditions that require the inclusion of multi-alarm scenarios within the context of worst-case impact assessments. Furthermore, we propose an optimization problem specifically designed to identify multi-alarm attacks. In our formulation, multi-alarm attacks are derived by relaxing the constraints of a zero-alarm attack problem. Our findings reveal that multi-alarm attacks can cause a greater state deviation than zero-alarm attacks, emphasizing their critical importance in the security analysis of control systems.