Recommendations for Implementing Knowledge Testing for Recreational UAS Operators Background
Section 349 of the Reauthorization Act, enacted in October 2018, directed the FAA, “in consultation with manufacturers of unmanned aircraft systems, other industry stakeholders, and community-based organizations, [to] develop an aeronautical knowledge and safety test, which can then be administered electronically” to operators of unmanned aircraft systems (UAS) used for recreational operations.
The UAST’s mission is to bring together stakeholders to develop and recommend safety enhancements to promote safe UAS operations in the national airspace. Congress intended that the FAA consult with stakeholders on the development of a knowledge test for limited recreational operations. In this paper, we provide a high-level set of concepts to assist the FAA in developing this test, recommendations on the test administration process, and insights that could also be useful to other entities implementing their own UAS pilot knowledge testing.
The Purpose of Testing
The UAST strongly supports the implementation of knowledge testing as a means to ensure that operators understand pertinent rules essential to safe operation. UAS technology is widely available and affordable to consumers without prior remote pilot training. Therefore, in order for FAA to achieve the goal of widespread voluntary participation, a reasonable balance must be struck between rigor and relaxed standards with regards to the core knowledge that should be imparted on test takers and the means by which the test is administered. A testing process or content that is overly challenging, expensive, time-consuming, or cumbersome, will undermine the goal of broad compliance.
In Section 349(g), Congress clarified that the test should be “designed to adequately demonstrate an operator’s (a) understanding of aeronautical safety knowledge; and (b) knowledge of Federal Aviation Administration regulations and requirements pertaining to the operation of an unmanned aircraft system in the national airspace system.” Here, we provide the following recommendations for how to fulfill these goals:
First, we note that there are relatively few “regulations and requirements” pertaining to UAS operated pursuant to Section 349. These requirements are listed as “limitations” in the statute, for example: visual line of sight requirements, registration requirements, altitude limits, the role of community-based organizations, operations at a fixed site, and the process for operations in controlled airspace. These limitations can each be the subject of a test question because they are fundamental to operations under the recreational operations statute. Special emphasis areas should include knowledge of operational limitations for flight near other aircraft, altitude limits, flight over people, damage to property and injury reporting, and the need for prior authorization before flying in certain controlled airspace. Other specific FAA requirements that apply to Section 349 operations can be added as test questions in the future, to include remote identification equipage requirements and Low Altitude Authorization and Notification Capability (LAANC) process (once it becomes available to recreational operators). The test should be focused on the most important things recreational operators need to know, and expressed in a manner that encourages a comprehensive understanding of the training material.
Second, any other questions on the test relating to “aeronautical safety knowledge” should be germane to recreational consumer operations and focused on principles central to the safe operation of a UAS pursuant to Section 349. We advise against including questions that are not germane to the operation of UAS for recreational purposes, such as questions about sectional charts, METARs, air traffic communication procedures, and principles of aerodynamics. The test should not include topics relating to flight scenarios or concepts recreational operators are unlikely to encounter, for example Class A airspace or the difference between a TCA and a TRACON (although it would be reasonable to discuss risks UAS may pose to visual flight rule (VFR) operations in certain categories of airspace). New tools such as the B4UFly app have made the interpretation of complex aeronautical charts unnecessary. Accordingly, questions should be designed to ensure that operators are aware of these tools and know where to find them.
Overall, it is important for test-takers to have the sense that the material and questions they encounter on the test are actually important to safe operations, so they feel their time is well-spent to the point that they will encourage fellow operators to be compliant with the new testing requirements.
We recommend the testing platform be consolidated with the FAA’s online UAS registration platform to provide a “one stop shop” experience, and that the FAA support other potential test facilitators, such as community-based organizations and manufacturers to administer the test, as contemplated by Congress. Creating these multiple pathways to taking the test will enhance compliance rates and may lead to better remote pilot decision-making in public spaces where UAS are flown.
This one-time test should be designed to take a relatively short amount of time, from start of educational steps, to the successful completion. A lengthy test may cause test-takers to grow frustrated and decide to terminate their efforts. It is also important for test-takers to be able to remember everything they learn and apply it to their operations. Therefore, the number of actual questions should be designed both to satisfy Congress’ goal to “adequately demonstrate” knowledge without introducing an onerous testing burden. The FAA should retain an instructional specialist, if not available in house, to assist in the development of the test and ensure the content is valid, reliable, fair, and takes into account the range of age groups taking it. This specialist will help develop the length of the exam and a passing score that is in line with the goals of the assessment. There should be a minimum level of performance required and a remediation mechanism sometimes referred to as “errorless learning” in educational communities.
The questions and the answer options need to be randomly selected and balanced according to a weighting system (different weights for different content categories) for each exam iteration, so that no two exams are identical, yet they still address core knowledge areas.
Testing should incorporate sufficient safeguards to prevent against fraudulent test taking, while also recognizing that Congress states that this test could be administered “electronically.”
The testing process is an opportunity to not only evaluate knowledge, but also provide additional educational information to UAS operators that can further enhance safety. For example, UAS operators should be reminded to inspect their UAS prior to flight to ensure it is in good working condition, and not to fly over unprotected people. The UAST, B4UFly app, and Know Before You Fly contain these types of advisories, and we recommend incorporating them into the testing interface, or including UAST content as “tips” on the certificate that is generated after passing the test. We also recommend that the test process refer test-takers to the FAA-recognized community-based organizations for ongoing knowledge and educational opportunities.
The UAST thanks the FAA for the opportunity to provide input to this important legislatively mandated mechanism to promote safe and responsible behavior in the growing community of recreational UAS operators.
The UAST welcomes any feedback by UAS operators, manufactures and service providers regarding this recommended guidance. Continuous improvement is a goal for any SMS, and continually improving this important guidance is a goal of the UAST.
The Unmanned Aviation Safety Team (UAST) advocates the use of Safety Management Systems (SMS) for Unmanned Aircraft Systems (UAS) operations because using an SMS is a structured means for individuals and/or organizations to make informed safety risk management decisions. An SMS provides an accountable way to manage safety before a system failure occurs. In addition, an SMS can deliver increased confidence in risk controls through structured safety assurance processes. Finally, the safety promotion framework within an SMS can be used to build and support a sound safety culture.