Abstract
Rapid advances in the development and use of artificial intelligence (AI) is having a profound effect both on organizations and society at large. While already used extensively in organizations to promote rational decision making, its emergence is also giving rise to profound ethical concerns. As such, we argue that organizational research has a crucial role to play in promoting beneficial development and use of this technology. Given the rapidly increasing autonomy and impact of AI systems, we draw attention to the fundamental importance goal setting and value alignment in determining the ethical desirability of outcomes from this development. Importantly, we claim that goal setting necessitates ethical considerations that are not amenable to technology. Further, while the pursuit of goals can be partially delegated to AI, challenges relating to the representation of goals and (ethical) constraints imply that human involvement is crucial in preventing unforeseen consequences. Finally, we discuss issues relating to the malicious misuse and heedless overuse of AI, arguing that the importance of human agency and inclusion in decision making in fact increases with adoption of the technology due to an escalating scale and impact of decisions made. Given the profound impact of these decisions on numerous stakeholders, we suggest that organizational research stands to contribute to the relevance, comprehensiveness, and integrity of AI ethics in the face of this revolutionary technology.