Aviation has a remarkable safety record ensured by strict processes, rules, certifications, and regulations, in which formal methods have played a role in large companies developing commercial aerospace vehicles and related cyber-physical systems (CPS). This has not been the case for small Unmanned Aircraft Systems (UAS) that are still largely unregulated, uncertified, and not fully integrated into the national airspace. However, emerging UAS missions interact closely with the environment and utilize learning-enabled components (LECs), such as neural networks (NNs) for many tasks. Applying formal methods in this context will enable improved safety and ease the immersion of UASs into the national airspace. We develop UAS that interact closely with the environment, interact with human users, and require precise plans, navigation, and controllers. They also generally leverage LECs for perception and data collection. However, the impact of ML-based LECs on UAS performance is still an area of research. We have developed an advanced simulator incorporating ML-based perception in highly dynamic situations requiring advanced control strategies to study the impacts of ML-based perception on holistic UAS performance. In other work, we have developed a WebGME-based software framework called the Assurance-based Learning-enabled CPS (ALC) toolchain for designing CPS that incorporate LECs, including the Neural Network Verification (NNV) formal verification tool. In this paper, we present two key developments: 1) a quantification of the impact of ML-based perception on holistic (physical and cyber) UAS performance, and 2) a discussion of challenges in applying these methods in this environment to guarantee UAS performance under various Neural Net (NN) strategies, executed at various computational rates, and with vehicles moving at various speeds. We demonstrate that vehicle dynamics, rate of perception execution, the design of the controller, and the design of the NN all contributed to total vehicle performance.