Sedative medications are widely used in intensive care unit (ICU) patients. Structured assessment of sedation and agitation is useful to titrate sedative medications and to evaluate agitated behavior, yet existing sedation scales have limitations. We measured inter-rater reliability and validity of a new 10-level (+4 “combative” to −5 “unarousable”) scale, the Richmond Agitation–Sedation Scale (RASS), in two phases. In phase 1, we demonstrated excellent (r = 0.956, lower 90% confidence limit = 0.948; κ = 0.73, 95% confidence interval = 0.71, 0.75) inter-rater reliability among five investigators (two physicians, two nurses, and one pharmacist) in adult ICU patient encounters (n = 192). Robust inter-rater reliability (r = 0.922–0.983) (κ = 0.64–0.82) was demonstrated for patients from medical, surgical, cardiac surgery, coronary, and neuroscience ICUs, patients with and without mechanical ventilation, and patients with and without sedative medications. In validity testing, RASS correlated highly (r = 0.93) with a visual analog scale anchored by “combative” and “unresponsive,” including all patient subgroups (r = 0.84–0.98). In the second phase, after implementation of RASS in our medical ICU, inter-rater reliability between a nurse educator and 27 RASS-trained bedside nurses in 101 patient encounters was high (r = 0.964, lower 90% confidence limit = 0.950; κ = 0.80, 95% confidence interval = 0.69, 0.90) and very good for all subgroups (r = 0.773–0.970, κ = 0.66–0.89). Correlations between RASS and the Ramsay sedation scale (r = −0.78) and the Sedation Agitation Scale (r = 0.78) confirmed validity. Our nurses described RASS as logical, easy to administer, and readily recalled. RASS has high reliability and validity in medical and surgical, ventilated and nonventilated, and sedated and nonsedated adult ICU patients.